by the Open Source Security Foundation (OpenSSF) Securing Critical Projects Working Group
This document presents the “Critical Open Source Software Projects”, aka, “Critical Projects”, as of 2023, which represents a point-in-time analysis from 2022-June 2023. This identifies a set of open source software (OSS) projects that have been determined to be highly important by the Open Source Security Foundation (OpenSSF) Securing Critical Projects Working Group through the process described below. This set is intended as a starting point for conversation about Open Source projects that are integral to the Internet, whose health should be monitored, and who are potential candidates for additional investment.
The purpose of the “Set of Critical Projects'' is to help guide the open source community in determining highly important open source projects that have been identified through research and discussion as critical with the Securing Critical Projects Working work group. These include projects under OSI-approved licenses that have such extensive use that vulnerabilities (unintentional or malicious) in that software could cause widespread problems, both in significance or breadth.
This is not a list, it is a set. This set is listed in alphabetical order by project name; there is no implied ordering within the set. It was challenging to determine what was or was not in the set; ranking between these projects would have been even more difficult and was unnecessary for our purposes.
This set of projects are projects that the working group has deemed critical. It is not a set of “top” or “most” critical projects. The working group has not considered or evaluated all projects in existence. We have strived to cast a wide net and incorporate a variety of sources for discovering projects. Any omission of a project may be due to the project not being considered, and is not an assertion that the project is not critical.
Please note that this is not a list of “OSS projects in trouble”. In fact, being on this list is a sign of success, as it indicates that many are depending on that project’s software.
The overall process used in creating the set included two primary steps:
- Find projects for consideration and gather data.
- Evaluate projects in a working group meeting.
There are many ways to discover "critical" projects, so the Securing Critical Projects WG combined the results of several different analyses (aka the "Selection Criteria"). The analyses ("selection criteria") for identifying candidate critical OSS projects included:
-
OpenSSF Criticality Score: A top OpenSSF criticality score value. This metric prefers projects that are extremely active. Such projects are likely to be important (at least to the participants). However, note that widely-used but less-active projects will score low here (a limitation others have pointed out). Also, it currently only considers GitHub-hosted projects. Due to the nature of the score, we used it to discover and identify projects for consideration, and factored it into our discussion, however we didn’t consider a low score a reason for a project to not be considered critical.
-
Census II Program: This is Harvard Census II of Free and Open Source Software — Application Libraries report. This uses data from multiple SCAs and dependency data to identify widely-used OSS. This analysis tends to emphasize lower-level application libraries that are depended on, transitively, by many. We also used the preliminary Census Program II document until the final version was available.
-
OSTIF Managed Audit Program: Programs OSTIF has recommended for audit. These were selected earlier from research sources, focusing on securing the most critical projects. You can see the OSTIF Managed Audit Program (MAP25)
-
Featured Google Project: Featured on Google Open Source page and widely adopted.
-
Featured Microsoft Project: Featured on Microsoft Open Source page and widely adopted.
-
Featured Linux Foundation Project: Featured on Linux Foundation Project page.
-
Secure Supply Chain Tool: Directly related to supply chain security (as identified by the WG)
-
Survey Response: Response to public survey
-
Language implementation: Identified by community as a widely-used language implementation
-
Community Addition: Separately identified by the community as important.
-
Widely Covered: If software has been previously attacked & it made headlines, it must be critical enough to attack.
The selection criteria used to find projects above was sometimes used as an evaluation factor to support a project being critical. Values like download counts, or criticality score are generally in support of a project’s widespread or important use. We also knew to not consider an absence of these things as reasoning for a project to not be critical, for example a project not on GitHub wouldn't have a criticality score, but that would not be held against the project.
Generally in classes of software (ex: databases), we would set a high bar for ubiquitous use, and only include a top number of those types of projects. Another factor discussed was the exposure that project or type of software has in a typical software system. For example, an http server or load balancer is typically exposed directly to the internet, while a basic library may be separated by many layers from untrusted input.
Using a combination of supporting metrics, known widespread use, and system criticality the working group came to a consensus for each package to be considered critical or not.
Limitations of this Set: We used these things as initial input. It wasn’t a matter of considering it comprehensive, we just used different inputs to the best of our abilities within the time frame we worked with. We understand that many other factors can influence or determine a project’s criticality, and that our set would not be comprehensive.
Every method for identifying critical OSS projects has its strengths and weaknesses. We believe the combination of quantitative analysis, combined with human review by a group, is better than only applying one analysis approach. For example, a high criticality score tends to emphasize very busy projects; human review can remove projects that are busy but for whatever reason are less critical (e.g., because they are not widely depended on or because vulnerabilities in it are unlikely to cause significant harm). Also, some projects are very important yet not active; by using other measures (not just the OpenSSF criticality score) we can still identify them.
The set represents a snapshot in time and was based on the proposed projects and analysis at the time of creation. We have no doubt that other OSS projects will be added to the critical OSS projects list over time. There are millions of OSS projects; all are important to someone, and it is challenging to review those millions to find which ones are widely depended on. If you're interested in helping to do that, please join the working group.
The set gravitated to OSS projects related to server-side Internet functionality, not other regimes, such as embedded, edge, safety critical, life critical, financial, etc. that may utilize Open Source software. Those areas were not excluded intentionally, and would be welcome to be considered for inclusion in the set.
Being a set of only Open Source projects, only the OSI approved license version of a project was considered for inclusion when there were multiple versions of a project. For example, some projects have an Open Source licensed version and a parallel commercially licensed version. Although the non-OSI version might be widely used or “critical”, if the Open Source version was not widely used, the project would not be deemed critical.