Is NVD Dead? RIP NVD!
The Inception and Evolution of NVD, Current Challenges, Future of NVD, and the Way Forward for the Cybersecurity Industry!
Anyone who has worked in cybersecurity or specifically into Vulnerability Management would have come across not only the Common Vulnerabilities and Enumerations (CVE), but also the NIST National Vulnerability Database (NVD).
I had covered about the CVE in my previous article, and the focus of this article is the NVD. I touch upon the origins and evolution of the NVD over time, its significance in vulnerability management, recent developments affecting NVD in the past few weeks, reasons these developments raise concerns, and the industry’s reaction to these events.
What is NVD?
The National Vulnerability Database (NVD) is a standards based vulnerability management database by the U.S. National Institute of Standards and Technology (NIST). The database provides a standardized framework for collecting, assessing, and cataloguing information about security vulnerabilities found in computer hardware and software.
The NVD works closely with the Common Vulnerabilities and Exposures (CVE) system. CVE assigns unique identifiers to vulnerabilities, and the NVD uses these identifiers to provide more detailed information about each vulnerability, including how to protect against it. The database is a crucial tool for cybersecurity professionals, as it helps them stay informed about known vulnerabilities, assess risk to their systems, and take appropriate actions to safeguard against potential threats.
While both NVD and CVE work in close collaboration, these are both separate programs, NVD being managed by the National Institute of Standards and Technology (NIST), and the CVE List being managed by The MITRE Corporation.
Inception and Evolution of NVD
The Information Technology Lab at NIST created Internet Category of Attack Toolkit (ICAT) in 1999, a catalogue of initial attack scripts and vulnerabilities. The ICAT was later rebranded as National Vulnerability Database (NVD) in 2005.
The establishment of NVD was a significant step in improving the cybersecurity infrastructure by providing a standardized and accessible platform for the collection and dissemination of vulnerability data.
NVD enriches the CVE List with risk and impact scoring using the Common Vulnerability Scoring System (CVSS), and provides other references and metadata, such as, patch information, affected products, security checklist reference, and Security Content Automation Protocol (SCAP) mappings.
Since its establishment in 2005, the National Vulnerability Database (NVD) has undergone significant evolution to enhance its capabilities in providing comprehensive vulnerability management data.
By adopting the Security Content Automation Protocol (SCAP) and Common Weakness Enumeration (CWE) since 2007, NVD has been able to automate aspects of vulnerability management and categorize vulnerabilities as specific software or system weaknesses, the adoption of Common Product Enumeration (CPE) since 2008 has helped with the structured naming scheme to identify vulnerable systems, software, and packages. The adoption of various tools and protocols over the years has ensured that the NVD is not just a repository of information but also a tool for proactive cybersecurity management that remains accessible and useful to a global level, beyond just the U.S. government organizations.
The full timeline since inception and evolution over time is published here.
Challenges and Recent Activities
Historically, the NVD has been very consistent on enriching the CVE data as shown in the graph below.
Source: Anchore blog post
The graph illustrates the correlation between the number of CVE IDs published (Green) and the NVD enriched records (Red) between 2005 and 2023.
News announcement— On February 13, 2024, the NVD made the following announcement on their website:
Source: https://nvd.nist.gov/general/news/nvd-program-transition-announcement
and a banner started appearing on the NIST website with the following notice:
Source:
https://nvd.nist.gov/
Upon reviewing the news announcement and the preceding notice above, nothing seemingly unusual stands out to the casual observer. One might infer that NIST is implementing enhancements to the NVD, suggesting that the current disruption could merely be a temporary hiccup before services return to normal.
However, the NVD dashboard published on the NIST website tells a completely different story.
Source: https://nvd.nist.gov/general/nvd-dashboard
The numbers in the table above indicate that the enrichment of CVE data has been ~50% since the beginning of this year, dropped below 50% in February 2024, and has been significantly lower at ~6% in March 2024, marked as ‘Last Month’ and ‘This Month’ respectively in the table above.
The decline in CVE Record enrichment is highlighted further by Anchore engineers in the graph below.
Source: Anchore blog post
The graph clearly indicates that the NVD has not only slowed down but has almost stopped processing the CVE Records since around mid-February 2024. This graph also confirms the numbers in the table above, which is published on the NIST website.
This situation presents a significant concern and should not be taken lightly, especially given the global reliance on the crucial NVD data for understanding vulnerability severity rating (CVSS), vulnerability categorization (CWE), and to identify vulnerable products (CPE) for vulnerability management programs.
Reactions from Industry and Cybersecurity Experts
There has been considerable speculation among industry professionals regarding the current state of NVD, and rightly so, with many cybersecurity experts sharing their analysis of the NVD’s recent data enrichment slowdown.
Many experts have offered their perspectives, seeking to understand the reasons behind the slowdown, and some of the main reasons cited include —
An explosive growth of cybersecurity vulnerabilities reported each year since 2017 putting strain on resources to conduct analysis.
Potential budget constraints within NIST affecting the NVD program.
NIST’s contract coming to an end with the contractor working on the NVD program.
Potential internal politics around vulnerability standards such as CPE, and PURL etc.
Some of these speculations could be attributed to a perceived lack of transparency and communication from NIST.
The initial comprehensive analysis by Anchore engineers, as mentioned above, prompted further investigations by others in the industry.
For instance, Jay Jacobs from the Cyentia Institute provided an analysis as shown in the images below, comparing the status of the analyzed CVEs to those pending analysis in the NVD for the first three months of 2023 and 2024.
Source: Analysis conducted by Jay Jacobs
As per the images below, Jay Jacobs’ subsequent analysis illustrate the stagnation in the enrichment of four key fields by the NVD for each CVE, with noticeable flattening in the trend lines starting mid-February 2024.
Source: Analysis conducted by Jay Jacobs
The cybersecurity community has responded through the creation of an image below to demonstrate the pivotal role of the NVD in the broader vulnerability management ecosystem.
Source — Patrick Garrity via LinkedIn. Image Origin — xkcd.com
In light of the current challenges, some have proposed transitioning to alternative vulnerability databases, such as the GitHub Advisory Database or the OSV open-source vulnerability database, among potential others. How these alternatives will fill the void left by the NVD remains to be seen.
Moreover, some of the cybersecurity organizations are doing a great job by striving to bridge the gap by providing community-driven resources. An exemplary initiative is VulnCheck NVD++, a free community service that enhances published CVEs with automated CPE enrichment.
Implications to Cybersecurity
For now, the lack of CVE Record enrichment presents a significant challenge as many cybersecurity products depend on the NVD data for identifying vulnerabilities in software products, and providing risk scores for prioritization of vulnerability remediation efforts.
With the new vulnerabilities still being identified and the lack of associated metadata to identify specific vulnerable products, organizations are left in the dark regarding which of their products and systems are affected by particular vulnerabilities, leaving a potential gap in their vulnerability management programs and exposing them to heightened cybersecurity risk.
A Ray of Hope, or is It?
The cause of the recent disruptions to the NVD or the rationale behind the proposed consortium, as mentioned on the NIST website, remains unclear.
At the VulnCon 2024, NIST has finally spoken up and addressed the community, conveying that despite current challenges, the NVD has no plans to shut down and continues to operate, especially in processing critical vulnerabilities in the short term. NIST also revealed plans to establish a consortium, promising to share further details in the forthcoming weeks. However, this announcement did not fully reassure many industry professionals at the conference.
The consortium’s formation and its operational specifics are not shared by NIST at the moment. However, the primary goal seems to be to bring together expertise and resources from various stakeholders to improve the NVD’s capabilities. This collaborative approach could involve industry experts, cybersecurity organizations, and other governmental bodies, leveraging their collective knowledge to address the NVD’s challenges.
This situation raises several questions — such as the nature and the membership of the proposed consortium, the operating model and the modifications it might bring, and the potential delays the cybersecurity industry might face in vulnerability analysis during this transition period.
Dan Loranc from Chainguard has taken a step further by drafting an open letter to the U.S. Congress and Secretary of Commerce, emphasizing the severity of the situation and its repercussions on the cybersecurity industry.
The effectiveness of these initiatives and their eventual impact on the NVD and the broader cybersecurity landscape remains to be seen.
Final Thoughts
This situation raises the question —
“Will the NVD ever be able to recover from the current situation?”
“Have we gone back to the time before the availability of NVD data, and if so, what alternative databases could we depend on should the NVD not recover?”
There are many unresolved questions that demand both short and long-term answers. These are the questions that every cybersecurity professional is currently contemplating and striving to answer.
Only the time will tell how the things turn out! Let’s hope there’s light at the end of the tunnel.