Across those runs, regression-hunt produced 25% fewer false-positive findings than the baseline, with a 95% confidence interval of 14% to 38%.
What were the baseline numbers of false positives? No mention? Silly me, expecting some rigor in numbers that make AI look good
I checked the Scapy repo linked, no mention of how many false positives there were before and after. The only concrete numbers are
Eleven search runs across 670 agent sessions reported 471 crashes and bugs, which deduplicated to 265 distinct bugs. Of those, 129 went to the maintainers (92 private reports, 37 pull requests); as of Sep 14, 2026, 80 patches had been merged (85% of patches) and 89% of reported issues were resolved.
So, of the 265 bugs, 236 were fixed. Those are the only concrete numbers we got.
2
Your AI agent fixed the bug, but what else did it change? | Spyke
1 reply
What were the baseline numbers of false positives? No mention? Silly me, expecting some rigor in numbers that make AI look good
I checked the Scapy repo linked, no mention of how many false positives there were before and after. The only concrete numbers are
So, of the 265 bugs, 236 were fixed. Those are the only concrete numbers we got.