OpenChain Releases Metrics to Evaluate Source Code Scanning Tools

Recently, OpenChain Project has released a community contribution from Ibrahim Haddad which covers metrics that can be applied to evaluate source code scanning tools. Source code analysis tools, also referred to as Static Application Security Testing (SAST) Tools, are mainly designed to analyze source code and/or compiled versions of code to help find security flaws or vulnerabilities.

This has been a timely topic, given the amount of discussion around the OpenChain global community around automation, and the initial release of this document is intended to encourage discussion about what a final, official OpenChain reference document could contain.

OpenChain Releases Metrics to Evaluate Source Code Scanning Tools

There are numerous open-source and proprietary compliance tools in the market. How do you decide which one is best for you?

Here are a number of metrics that you can refer to when evaluating source code scanning tools

  • Size of the knowledge base against which scanned code is being compared
  • Frequency of updates to the knowledge base – how often does the tool provider update the knowledge base to keep up with the pace of open source development
  • Speed of scans for the same loads
  • Supported deployment models – cloud, on-premise, hybrid
  • Ability to identify origin and license of snippets – many tools do not provide such support and are only capable of identifying whole open source components, others have poor support
  • Ability to auto-identify open source snippets in scanned code flagging their component of origin and license – saving endless hours on manual labor
  • Support for vulnerability discovery – is the tool capable of identifying vulnerable code that was copy/pasted from one component into another? Or simply just able to identify vulnerabilities found in their original components
  • Ability to represent and manage the end-to-end review and approval process directly from within the tool via a self-defined workflow
  • The total cost of ownership – which include the yearly license cost, training cost, cost of customizations (workflow, features, integration, etc.), cost of servers required for your specific install and Internal sys admin support for your install
  • An intuitive UI – easy and inviting to use – minimizing the learning curve and making it less of a chore
  • Support for APIs and a CLI that you can interconnect with your CI/CD environment for ease of integration with existing development and build systems
  • Ability to use the tool for M&A transactions without restrictions on the use of the tool as part of the licensing agreement
  • Support for different audit methods – several methods exist
  • Programming languages agnostic – the tool should be able to process any source code regardless of the programming language
  • Support for SPDX – discovering licenses declared using SPDX identifiers and exporting scan results in SPDX format
  • Ability to represent company policies and apply them on scanned code triggering specific actions depending on the license of the scanned code and related policies

Original Author:
Ibrahim Haddad

PDF of this Metrics

Metrics to Evaluate Source Code Scanning Tools PDF

Access all Versions and Contribute via GitHub

Need help in identifying and eliminating vulnerabilities in your source code? This is what Lyra Infosystems specializes in. Contact us to know how we can help.

Leave a Reply

Back to top