Brought to you by:
Enterprise Strategy Group | Getting to the Bigger Truth™
ESG TECHNICAL VALIDATION
Identify and Remediate Application Vulnerabilities with Coalfire ThreadFix
A DevSecOps Automation Platform for Risk Reduction
By Tony Palmer, Senior Validation Analyst
FEBRUARY 2022
Introduction
This ESG Technical Validation examines the Coalfire ThreadFix application security (AppSec) vulnerability management platform that helps organizations prioritize remediation efforts with real-time intelligence that measures vulnerability risk. The report includes results of remote validation of ThreadFix.
Background
According to ESG research, 60% of organizations currently develop and deploy production cloud-native applications, and another 27% plan to develop and deploy them in the next 12 months. However, just like legacy applications, managing an organization’s cloud-native cyber-risk has become more difficult, as attackers gain sophistication and knowledge through experience, developing stealthy attacks targeting weakly protected environments.
Thus, as shown in Figure 1, organizations face the reality of a diverse range of cloud-native cybersecurity incidents, ranging from the 27% that have suffered from lateral movement of malware to cloud workloads to the 24% that experienced “zero day” exploits that leverage unknown vulnerabilities in the cloud-native code stack and the 17% that faced exploits that took advantage of known vulnerabilities.
Figure 1. The Reality of the Diverse Range of Cloud-native Cybersecurity Incidents
Which of the following cybersecurity incidents, if any, has your organization experienced in the last 12 months related specifically to cloud-native applications and infrastructure? (Percent of respondents, N=383, multiple responses accepted)
Source: Enterprise Strategy Group
When asked about the functional capabilities of cloud-native app security controls, 25% cited software vulnerability scanning of production containers and server workloads as one of the most important. 41% of organizations said that one of their top security control implementation priorities is to automate the introduction of controls and processes via integration with their software development lifecycle and continuous integration and continuous delivery (CI/CD) tools.
The ThreadFix Platform
ThreadFix is a platform for the security practice focused on application vulnerabilities, both on-premises and in the cloud. Coalfire recognizes that the risk inherent in application vulnerabilities isn’t just from defects in source code, but also from the infrastructure that is hosting that source code. In addition to integrating with more than 50 security testing, compliance, and defect tracking tools, ThreadFix also integrates with three of the most popular network vulnerability scanners—Tenable, Nexpose, and Qualys—to integrate traditional security vulnerabilities.
ThreadFix is designed to enable an organization’s security team to integrate all the tools that they run their programs around, gather data from internal or third-party penetration tests, and automatically merge and deduplicate it so they get a single list of the actual issues within their code bases that need to be addressed.
Figure 2. Veritas InfoScale and Hitachi Vantara Storage
The ThreadFix remediation cycle, shown in Figure 2, takes data from automated scans and manual tests, normalizes and merges vulnerabilities found, performs analysis and flags false positives, sorts by multiple criteria and reports them out, tracks resolution activities, monitors developer activities, and closes out false positives found in previous scans.
ESG Technical Validation
ESG performed remote evaluation and testing of ThreadFix. Testing was designed to demonstrate how ThreadFix collects vulnerability data from various scan tools, consolidates vulnerability data into a central location, and provides sophisticated context around the overlap—where multiple, disparate tools find and report on the same vulnerability in different ways. ESG also looked at how ThreadFix communicates priorities to the development and security teams in their own tool sets to maximize productivity and reduce risk.
Vulnerabilities
ESG tested in a simulated development environment with dozens of applications; multiple, heterogeneous vulnerability scanners; and multiple CI/CD tools.
ESG Testing
First, ESG looked at the ThreadFix dashboard. The dashboard is customizable through a collection of reports and filter sets ThreadFix makes available. As seen in Figure 3, this dashboard highlights key metrics like most vulnerable applications, a comparison of ingested scans, a list of open source components with known vulnerabilities, and a trend report.
Figure 3. The Coalfire ThreadFix Dashboard
The Vulnerabilities report is sorted by severity and provides insight into dependencies and versions that an organization should be most concerned about in practice. ESG was able to drill down into one of the most severe, Apache Tomcat 6, and immediately discern the exact vulnerability (see Figure 4).
Figure 4. Vulnerable Software Components
While the ThreadFix dashboard provides a useful overview of what’s going on in an environment at a high level, ESG drilled down into the Portfolio view for a look at teams and applications. Teams are a logical collection of individual application assets within ThreadFix and Applications are the collection of application assets being scanned (see Figure 5).
Figure 5. The ThreadFix Portfolio
The Application list shows a high-level view of what vulnerabilities are in each application, overall policy status, and when the app was last scanned and assessed. Policies are user-configurable and can be assigned to individual applications or entire teams of applications. ThreadFix can also be configured so that applications inherit the policy based on the metadata applied to it, so that whenever a policy tag is applied to an application in the future, the application will inherit that policy and ThreadFix will report on its success or failure.
It’s important to note that application assets are completely customizable based on an organization’s requirements. As an example, a parent application may leverage many microservices. If an organization is concerned with scanning and tracking each microservice individually over time, it would create an application asset for each microservice. If that level of granularity isn’t needed, a single application asset can be created for the parent, and all the microservices will be identified as a part of the parent app when connecting scan tools to ThreadFix.
Why This Matters
In ESG’s 2022 Technology Spending Intentions Survey, 46% of organizations report that IT has gotten more complex over the last two years for multiple reasons. The changing cybersecurity landscape (37%), the increase in the number and type of applications used by employees (33%), and new data security and privacy regulations (32%) all impact application development and can decrease productivity significantly while increasing risk.
ESG validated that Coalfire ThreadFix addresses these challenges by providing organizations with an environment that is tailored to their needs and requirements that enable policy-based automation to simplify vulnerability management while reducing risk.
Application Details
Next, ESG looked at the application detail page for a deeper look at what is currently of concern within a specific application. This view (see Figure 6) reflects the latest information based on all the scan data that has been uploaded.
Figure 6. The ThreadFix Portfolio-Application Detail
In the upper right is the list of the top 10 vulnerabilities, and at the bottom left, the vulnerabilities are normalized to the Miter CWE (common weakness enumeration) list that those vulnerabilities are part of, sorted by severity. Note: CWE is a community-developed list of software and hardware weakness types with detailed descriptions and guidance used for weakness identification, mitigation, and prevention efforts. In the upper left, that vulnerability trending chart shows the number and severity of vulnerabilities over time. In this case, a steady downward trend can be seen as vulnerabilities have been resolved.
ESG expanded a vulnerability—Cross-site Scripting. Each of the large rectangles represents an instance of the vulnerability, while the grey tags inside the listing show which scanners identified it. In this case, seven separate scanners found this vulnerability, but ThreadFix automatically identified them as the same vulnerability and merged them into a single instance.
Figure 7. The ThreadFix Portfolio-Application Detail
Drilling down into the specifics of one instance, ESG saw the additional information that was provided by the scanning tools, including a data flow from Checkmarx showing what code paths were executed in order to find this particular vulnerability and recommendations on how to resolve it from Fortify. Users can dig even deeper into details of specific findings; ThreadFix retains the raw content from scanners because there is often value in it, but it’s available on demand, as needed.
Why This Matters
ESG asked organizations which considerations will be most important in justifying IT investments to their business management team over the next 12 months. Improving cybersecurity (44%) and increasing productivity (32%) were among the most-cited responses.
In the context of application development and security, ESG validated that ThreadFix provides a solution that offers insight and controls that can reduce risk without increasing complexity or hindering productivity. When needed, highly granular visibility into the underlying data is always available.
Scanner and Defect Tracker Integration
Next, ESG examined how ThreadFix integrates with scanners and defect trackers. Scanner integration gets the data into ThreadFix to be normalized and analyzed, while defect tracker integration is used for both communicating discovered defects to the developers and communicating back to SecOps. The security team can use the UI to see how developers are moving defects through their process. ThreadFix uses a “trust but verify” model. When a developer moves a task to “fixed” in their issue tracking system, ThreadFix identifies that the defect has been flagged as resolved but does not move it off the defect list until a scan confirms the action. If it is not resolved, ThreadFix has reporting for the SecOps team that a vulnerability identified as “fixed” by development has failed validation.
ESG Testing
For most of the tools that Coalfire customers use, ThreadFix has remote provider integration, meaning that Coalfire has built a native back-end API connection between ThreadFix and server- and cloud-based vulnerability scanners. When creating a new provider, users simply select which platform to connect to, enter credentials, and perform any other tasks that are required by that tool—if anything—and ThreadFix establishes the connection.
Figure 8. ThreadFix Scanner Integration
Organizations can take the projects as identified by the scanning tool and persistently map those to applications in ThreadFix in a many-to-one scenario so that ThreadFix will always pull in data from all appropriate sources, whether automatically on a set schedule or on demand.
Figure 9. Mapping Projects to Applications
Finally, we looked at how ThreadFix integrates with defect trackers. Organizations can set up profiles to prefill known data. This enables organizations to tailor the output to their needs. As an example, an organization might want to always capture the CWE value and a short description of the vulnerability as a replaceable value so that any time they create a defect that can be prefilled, they always know what data is required and how a particular project requires its data. ThreadFix identifies those fields in red.
Figure 10. ThreadFix Defect Tracker Integration
Organizations have the flexibility to customize how ThreadFix integrates to their workflow. They can require that recommendations be included in every ticket description they create, or they can choose to just link them back to ThreadFix, creating the ticket for tracking purposes but requiring the actual work be done in ThreadFix. This is all configurable with the description template, and it's something that many ThreadFix users choose to leverage for a much faster way of getting the data they want to the developers with the end goal of automating as much as possible.
Why This Matters
ESG asked organizations which tasks would be the highest priorities to implement in the context of application and infrastructure security. Automating controls and processes via integration between their software development lifecycle and CI/CD tools was the most-cited response (41%).
ESG validated that by merging and normalizing vulnerability scans and communicating that data to both development and security teams, ThreadFix gives organizations actionable insights into vulnerabilities that can be automated and completely integrated into DevOps and SecOps workflows.
The Bigger Truth
Organizations have rapidly shifted to modern cloud-native application environments. And there’s a strong correlation between cloud-native applications and extensive usage of newer application development methodologies and processes. Among the 60% of organizations that currently develop and deploy cloud native apps, 54% extensively employ DevOps methodologies, and 50% extensively employ agile software development methodologies.
In the face of the continuing onslaught of cyber-attacks, organizations are expanding DevOps to include security—what is now known as DevSecOps—with 36% of organizations currently looking to identify and remediate software vulnerabilities before deployment to production and another 44% planning to integrate software vulnerability management in the next 12-24 months.
ThreadFix is designed to address these challenges by enabling security teams to integrate all the vulnerability scanning tools that they run, gather data from internal and/or third-party penetration tests, and automatically merge and deduplicate it so they get a single list of the real vulnerabilities in their AppDev ecosystem that need to be addressed.
ESG validation testing confirmed that Coalfire ThreadFix communicates with various scan tools, collects/consolidates vulnerability data into a central location, and provides sophisticated context, which is then communicated to the development and security teams via their own tool sets. It’s important to note that everything in the UI is automatable. ThreadFix is focused on not obfuscating risk for their users. ThreadFix merge logic is very specific to avoiding “false positive merges,” meaning that everything that merges should be based on defensible data that shows that the merged vulnerability is definitively the same problem.
Most vulnerability management solutions in the market are focused on prioritization and triage, which are big problems, but organizations are facing a much larger challenge. Specifically, how can they manage this data in the workflow. ThreadFix gives organizations the ability to integrate CI/CD and their management flow with vulnerability integration and normalization. This enables ThreadFix to give actionable insights, helping organizations understand what needs to be fixed now and what can wait until later, with no ambiguity.
The results that are presented in this document are based on testing in a controlled environment. Due to the many variables in each production environment, it is important to perform planning and testing in your own environment to validate the viability and efficacy of any solution.
Public cloud infrastructure adoption is nearly ubiquitous and ESG is seeing a continuing shift to cloud-first policies. ESG research revealed that 77% of applications that are on-premises today are candidates to move to the public cloud within just five years. With the continuous integration and deployment of cloud-native security controls, DevSecOps automation platforms like ThreadFix are now mission-critical to achieving today’s security-first coverage and scalability requirements in complex environments with multiple vulnerability scanning platforms.
DOWNLOAD PDF REPORT
This ESG Economic Validation was commissioned by Zscaler and is distributed under license from ESG.
All trademark names are property of their respective companies. Information contained in this publication has been obtained by sources The Enterprise Strategy Group (ESG) considers to be reliable but is not warranted by ESG. This publication may contain opinions of ESG, which are subject to change from time to time. This publication is copyrighted by The Enterprise Strategy Group, Inc. Any reproduction or redistribution of this publication, in whole or in part, whether in hard-copy format, electronically, or otherwise to persons not authorized to receive it, without the express consent of The Enterprise Strategy Group, Inc., is in violation of U.S. copyright law and will be subject to an action for civil damages and, if applicable, criminal prosecution. Should you have any questions, please contact ESG Client Relations at 508.482.0188.
Enterprise Strategy Group | Getting to the Bigger Truth™
Enterprise Strategy Group is an IT analyst, research, validation, and strategy firm that provides market intelligence and actionable insight to the global IT community.