Project Level Metrics
For a given open source project, here are some ways of measuring the health of the project, thereby determining Dependency Risk.
What To Measure
Committer Strength
Number of individual committers (within a given recency).
GitHub provides some indication of this for each project. First on the front page of the project it gives a committer count:
Secondly on the Insights/Pulse page (for example here) you can see the activity for each committer, like so:
See:
- Backstage: This builds a graph of the tech landscape by consuming other golden-sources of data, such as GitHub, Artifactory, Jenkins, JIRA, etc. This gives you a really good high-level view of the whole landscape. More and more sources are getting backstage plugins to allow them to connect their data.
- The GitHub GraphQL API.
- Cauldron.io: Open source ecosystem analytics
Committer Diversity
Diversity of committer organisations
If all the committers to a project hail from the same organisation, this might represent a single-point-of-failure if the organisation defunds the initiative.
Stars
GitHub and GitLab both allow you to "Star" projects you like. The number of stars helps as an indicator of all-time popularity but doesn't show whether a project is still_popular.
Issues
It can be useful to review the open issues for a project, as well as comparing this with the number of closed issues. A healthy project like the one below will have vastly more closed issues than open ones.
Badges
Projects will often advertise "badges" on their README Pages. This can be used as another signal for determining Dependency Risk
Of particular interest are badges such as:
- OpenSSF: This is a checklist of security behaviours.
- FINOS Active: indicates the project is now mature in terms of quality and community.
Foundations like Apache don't rely on badges, but instead rename their projects when active. Therefore Incubator-Pegasus will become simply "Pegasus" once it reaches Active state.
Release Cadence
For an open source / inner source project, how quickly are changes incorporated and released? If this can be done automatically, this is often (but not always) a sign of a mature CI / CD Pipeline.
For GitHub projects, details of the releases and tags are readily available, both summarised on the front page:
Broken down more thoroughly in the /tags and /releases sub-pages:
... and available programmatically via GraphQL.
Total Cost of Ownership (TCO)
How does using open source software compare with buying off-the-shelf? The costs are very different.
See:
- How useful are ‘proprietary vs. open source’ TCO studies? - from linux.com.