Redefining How to Measure the Success of Your Vulnerability Management Program

Could your team be wasting its time reporting vulnerability metrics that don’t matter? Security teams often fall into the trap of reporting  security metrics that don’t actually matter to the business. In this post, we’ll discuss which vulnerability risk management metrics matter and which ones don’t, and how to communicate them effectively to the business.

Moving away from security metrics that aren’t helpful

Metrics like a count of unpatched vulnerabilities or number of assets assessed may sound important at first glance but are of very little use to non-technical stakeholders and the greater business. First, because most executives, board members, and other teams don’t understand this metric in context and, second, it’s not actionable to them.

Another metric we often see security teams lean on is CVSS score. While it’s a baseline metric to understand a vulnerability, it needs to be taken into consideration alongside metrics like malware exposure, exploit exposure, and vulnerability age to be useful in actually prioritizing vulnerabilities for remediation. CVSS alone simply doesn’t offer enough context and can leave you with thousands of “critical” vulnerabilities to fix, which isn’t helpful to your security team nor is it easily communicable to other business counterparts.

The third category of metrics are those outside your security team’s control, such as reporting on an aggregate risk score. Wait… what? That’s how I’ve been doing this for years. Yup—but stick with me on this—what if right before you go into your board meeting to report your security metrics there is a huge Patch Tuesday? Your aggregate risk score is going to spike, suggesting that your team has fallen down on the job. However, you probably have become even more effective than the last time you reported metrics to the board. Aggregate risk score simply isn’t indicative of the efficacy of your program.

Reports on these metrics often don’t give enough insight, may make issues seem worse than they are, and can paint security in a bad light. This makes it difficult to secure a budget if your metrics don’t adequately articulate the real problems on hand or the real scope of an issue. Reporting these metrics to other business units can lead to breaks in communication, budget decreases, and poor rapport.

All this is to say, measuring and reporting on the wrong metrics isn’t beneficial to security nor the organization at large. But reporting on the right metrics that will not only give you a better handle on your vulnerability risk management program, but also relate back to the greater business goals can make all the difference. It might even help you get a raise.

How to report on business metrics that matter

The first step is to start thinking about (at least) two distinct categories of metrics and tying them together. First, you have Key Risk Indicators (KRIs)—these are the operational metrics you are going to use to run your security program. Second, you have Key Performance Indicators (KPIs)—these are the metrics that the business uses to measure its effectiveness overall, with security and IT being one small (yet critical) piece of that overall picture; hopefully, you can get these from your organization’s leadership. Now you have to map your KRIs in a clear and defensible way to your organization’s KPIs.

For example, if your organization has a sales team, there is undoubtedly a KPI measuring their productivity. As a security team, we have a goal to remove obsolete operating systems. Now let’s map the relationship by asserting that when employees have an up to date operating system they are more productive. It’s true, obsolete operating systems and software can slow down productivity due to system malfunctions and other inconveniences. Now when you report the value of your security program to the business you can say:

“We’ve helped the organization achieve its goal of increasing sales team productivity 5% this quarter by progressing towards our security team goal of removing all Windows 7 workstations by the end of Q1 2020.”

Now non-technical executives, boards, and counterparts across the business can understand why your efforts are valuable.

Another KRI example for a security team could be reducing the  time-to-remediate vulnerabilities. The related business KPI is the cost of IT man-hours. If your vulnerability management solution is helping to decrease the time it takes to complete these repetitive tasks, then the business goal of decreasing the cost of IT labor is achieved.

A third security team goal is to decrease  actively exposed vulnerabilities. While your security team knows what this means, this number often has no meaningful translation for the rest of the company. The business, however, does care that critical systems stay functioning. An effective security team could report the achievement of the following desirable KRI: “95% of critical vulnerabilities were patched on payroll-related systems within 7 days in Q4 2019.” This directly translates to the business goal of making sure payroll always goes out on time.

By taking a security-focused KRI metric and mapping it to a KPI business metric, it can be understood by all other non-technical counterparts, relay the importance of security to the business as a whole, and garner better relationships, trust, and budget.

To effectively report these metrics, you’ll want to show a comparable month-over-month percentage improvement that shows how you’re making progress. It’s also important to consider the audience in which you are sharing these metrics, as they should be specific and defined for each audience so they resonate and have importance.

Creating goals, SLAs, and reports within your vulnerability management program

Within Rapid7’s vulnerability risk management solution, InsightVM, security teams can gain clarity into the risk across the ecosystem, extend their influence across the organization, and see shared progress with the security team and the colleagues in operations and development.

InsighttVM’s Goals and SLAs feature can help you reduce overall risk and improve the security of your environment, something every business unit can be on board with. You can set service level commitments like, “80% of critical vulnerabilities will be patched on payroll-related systems within 7 days”. InsightVM will help you visualize and report how you are doing towards that goal, month over month. Now you can have conversations like “In Q3 2019 we achieved our SLA for critical patched on payroll systems 70% of the time. Because of increased investment and team training, we achieved a 95% adherence to this SLA in Q4 2019.” This shows your efficacy and clearly exhibits how you are protecting the business’ most important systems and data. Goals and SLAs can also ensure that things like compliance and various industry-specific standards are monitored and met.

When you’re ready to roll up your regular report to the C-suite, InsightVM has a robust Executive Report that shows high-level, month-over-month trends specific to business leaders who don’t need to get into the weeds of things like the security and IT teams would. The report lets leadership see how the vulnerability management program is performing without inundating them with data.

Redefine your vulnerability management metrics with InsightVMTo be successful in communicating about security, collaborating with other teams, and securing and maintaining a healthy budget, it’s necessary that you translate security objectives into business objectives. InsightVM’s robust metrics and reporting enable you to track the right metrics and create audience-specific reports so you can communicate the value of security faster and more effectively. Interested in giving it a spin? Sign up for a free trial today.