The last time Google did a media run about Deepmind finding bugs, it related to a vulnerability on an dev branch that hadn’t been deployed yet (and was not likely to have been with the vulnerability).
I don’t think anyone is suggesting that it is impossible for an LLM to find any vulnerabilities?
But right now we are specifically discussing the costs of a breach, and your post that I responded to specifically relied on a bug not being identified a person.
The discussion isn’t whether an LLM can identify bugs, it’s whether it can do so in a useful way. In the single previous example, it was not useful.
But similar to the last time, it is likely that the limited utility will only be known until well after the breathless reporting on how amazing AI is
In the example you provided, it found a vulnerability, which is useful, but they didn’t point it at production code. The vulnerability might have been found by other tests and code reviews or it might have not been. The question of whether it’s valuable or not really depends on what sort of code we’re talking about and what the cost of missing a vulnerability would be.
All I’m saying here is that AI is just another tool that helps find bugs. People here freaking out over the idea that there might be legitimate uses for AI is kind of hilarious to be honest.
We don’t know, however of this is security related issues then it doesn’t matter. The cost of a breach would be obviously higher.
compare to the cost of humans finding them the normal way, not whatever breach you’re imagining.
Clearly the humans didn’t find them the normal way, because they wouldn’t be there to be found otherwise would they?
We don’t know the details yet. Maybe they have a great new tool; perhaps they picked projects that are not maintained so well.
It will be awesome if they found bugs in curl, not so good to show if they picked my project.
What they did will be revealed in time
I’m sure we’ll get more info in due time.
Yes, hopefully in a couple of weeks
The last time Google did a media run about Deepmind finding bugs, it related to a vulnerability on an dev branch that hadn’t been deployed yet (and was not likely to have been with the vulnerability).
So it found a vulnerability in the code it was given. 🤷
I don’t think anyone is suggesting that it is impossible for an LLM to find any vulnerabilities?
But right now we are specifically discussing the costs of a breach, and your post that I responded to specifically relied on a bug not being identified a person.
The discussion isn’t whether an LLM can identify bugs, it’s whether it can do so in a useful way. In the single previous example, it was not useful.
But similar to the last time, it is likely that the limited utility will only be known until well after the breathless reporting on how amazing AI is
In the example you provided, it found a vulnerability, which is useful, but they didn’t point it at production code. The vulnerability might have been found by other tests and code reviews or it might have not been. The question of whether it’s valuable or not really depends on what sort of code we’re talking about and what the cost of missing a vulnerability would be.
All I’m saying here is that AI is just another tool that helps find bugs. People here freaking out over the idea that there might be legitimate uses for AI is kind of hilarious to be honest.