Code review used to be the only way to catch a certain kind of bug. That meant a human auditor had to read thousands of lines of code, trace the possible paths of untrusted input, and reason about whether a function was doing exactly what its name promised. The process was slow, expensive, and reliant on the few people who could think like both a developer and an attacker.
An automated system called NOVA now appears to have changed that equation. The system read the source code of 3,915 open-source projects over two months and produced 14,090 confirmed vulnerabilities, with every finding passed through a validation pipeline designed to separate real exploitability from static-analysis noise. The research was carried out by vulnerability researchers at Palo Alto Networks’ Unit 42. When they later compared the output against public vulnerability records, only 85 findings matched anything that had already been documented. Most of those had been published two to eight weeks after NOVA had already found them.
The raw count is not the part that should worry security teams. Machines have been finding bugs at scale for years, notably since Google launched OSS-Fuzz in 2016. By August 2023, that project had helped identify and fix more than 10,000 flaws across 1,000 projects. What has changed is the kind of bug.
Why fuzzing is no longer enough
Fuzzing works by throwing malformed input at a program and waiting for something to break. If a crash happens, the bug is usually in a class of weakness that is easy to identify and hard to fix: memory corruption, integer overflow, null dereference, or another flaw that leaves a visible trace. Fuzzers are less useful for bugs that do not stop execution, because those vulnerabilities are invisible to the crash-based signal that fuzzing depends on.
NOVA’s findings show exactly how far beyond fuzzing a modern automated system can go. In the taxonomy used by the researchers, memory and calculation issues represented 557 findings, or 4.0% of the total. Even when resource-management and denial-of-service issues were added, the categories most friendly to traditional fuzzing totaled only 1,121 findings, or 8.0% of the overall number.
“The other 92 percent of the findings are bugs that never announce themselves with a crash,” Xu Zou, SVP of Cloud Delivered Security Services at Palo Alto Networks, explained. Those bugs exist in categories where nothing visibly breaks. Access control flaws, where the software checks who you are but not whether you are allowed to do what you asked. Path traversal, code injection, prototype pollution, server-side request forgery. Catching them means reasoning about what the code was supposed to do.
That is why code review used to be the only way to catch these bugs. A developer looking at a codebase could see a missing authorization check, notice that a parameter flows directly into a filesystem operation, or understand that an object can be polluted through a recursive merge. A fuzzer never sees any of that, because the program keeps running and no error is produced.
How vulnerability profiles differ by language
The distribution of NOVA’s findings varied by language in a way that tracks how each ecosystem is built. C and C++ codebases produced clusters of memory-safety issues, reflecting the absence of automatic bounds checking. JavaScript and TypeScript projects produced a high number of injection, prototype pollution, and server-side request forgery findings, consistent with the way those languages are used in web applications that process untrusted input. PHP, Java, and Python applications were dominated by broken access control, the recurring weakness of applications that have to track many different kinds of users and roles.
These language-specific patterns should be useful for both developers and security teams. A single security checklist applied to every project will inevitably miss the weaknesses that are most likely to appear in a given stack. A C++ service and a Node.js API may both be vulnerable, but the paths to exploitation are completely different. Knowing the vulnerability profile of the language and framework is a first step toward focusing manual review and automated scanning where it will have the most impact.
The supply-chain multiplier
NOVA’s output included a significant supply-chain component. Across the package ecosystems examined, the pipeline produced 5,421 supply-chain findings. Of those, 1,280 were flaws in dependency packages themselves. The remaining 4,141 were downstream exposures, where the application reaches a vulnerable dependency in a way that could be exploited.
The ratio is the number to hold onto: roughly 1,300 flaws in packages generated more than 4,000 downstream exposures. And 2,776 of those downstream exposures were validated with a working proof of concept launched from the downstream application, not merely inferred from a dependency graph. Software composition analysis tools can flag the presence of a vulnerable version. Here, the research team established that the flaw could actually be triggered from where the application sits.
This distinction matters because the security industry has become reliant on dependency scanners that report every transitive dependency matching a CVE-tagged version. That approach produces a high volume of alerts, many of which are not exploitable in the context of a particular application. NOVA’s validation pipeline was designed to cut through that noise by proving the chain of calls from the application to the vulnerable code. The result is a smaller but far more actionable set of supply-chain exposures.
Even so, the presence of thousands of validated downstream exposures in only a few thousand projects suggests that the open-source ecosystem has a systemic problem. One flaw in a widely used package can ripple outward to hundreds or thousands of applications that never directly touch the vulnerable code but call an intermediate library that does.
The patch window has collapsed
One of the most sobering takeaways from the NOVA report is about the speed of exploitation. The security industry has long operated on the assumption that a discovered vulnerability can be patched before it is widely exploited. The report challenges that assumption directly.
“Our experience with NOVA highlights a clear structural change: the patch window has collapsed,” Zou wrote. The industry average for deploying a traditional patch, by his team’s reckoning, is 55 days. An attacker does not need the latest frontier model to benefit from any of this. Once a fix ships, the diff between the old version and the new one narrows the search to a handful of lines. Working backward from a patch to an exploit is a far cheaper problem than finding the bug cold.
Faster discovery means more patches, and more patches means more starting points for exploit development. The very automation that helps defenders find vulnerabilities also gives attackers a richer set of clues. In this environment, a 55-day patch cycle is less a window and more an invitation.
No CVEs, no way to check
Unit 42 says it is working with open-source maintainers and with vulnerability clearinghouses, naming Lightwell and Akrites, to get the flaws fixed upstream. What it has not done is name a single affected project, publish a CVE identifier, or say when any of the 14,090 flaws were reported to the people who maintain the code. That makes it impossible for anyone outside the process to know whether a project they depend on is affected or whether a fix is already in progress.
The lack of disclosure details is not necessarily a criticism. There are good reasons to withhold public vulnerability data until patches are available. But the absence of a timeline also means the public cannot evaluate whether the effort is moving quickly enough, or whether the 14,090 findings are being triaged in an order that matches real-world severity.
How the numbers are distributed
The headline total can also be misleading if read as a perfectly even spread across the open-source world. The findings were highly concentrated. Four of the six ecosystem groups covered fewer than 100 projects between them, yet produced more than half of the findings. Large, dense codebases yield many vulnerabilities at once, so the total is not 14,000 separate surprises across thousands of independent projects. It is a handful of big applications giving up a lot at once, plus a long tail of packages giving up a little.
The concentration has practical implications. Remediation can begin with a small number of large projects and produce an outsized reduction in total risk. At the same time, the long tail means there is no point at which the work can be considered finished; new code and new dependencies appear continuously.
Severity depends on the ruler
The research also illustrates how much the chosen scoring system shapes the final risk picture. Under CVSS version 3.1, somewhat over a quarter of the findings rated as high or critical. Under CVSS version 4.0, just under two fifths rated that way. The same bugs produce different severity readings depending on the scoring model. That is not a contradiction; it is a reminder that severity scores are designed as decision-support tools, not absolute truths. The underlying vulnerabilities are the same, but the arithmetic used to communicate their risk changes the impression an executive or an engineer takes away.
Source: Help Net Security News