Read the original article: Enterprise Cybersecurity Measurement
On Feb. 4, 2021, the New York State Department of Financial Services issued guidance on the cyber insurance market to foster more robust industry approaches to “managing and reducing the extraordinary risk we face from cyber intrusions.” Critical elements of that guidance include expectations that insurance companies should “rigorously measure insured risk” and “incentivize the adoption of better cybersecurity measures by pricing policies based on the effectiveness of each insured’s cybersecurity program.” This follows a separate recommendation last year from the Cyberspace Solarium Commission to establish a Bureau of Cyber Statistics. The commission envisioned that the bureau would be “the government statistical agency that collects, processes, analyzes, and disseminates essential statistical data on cybersecurity, cyber incidents, and the cyber ecosystem to the American public, Congress, other federal agencies, state and local governments, and the private sector.”
A key impetus behind both initiatives is to foster cyber metrics that bring transparency, accuracy and scalability to the way that industries understand, manage and communicate around cybersecurity risk. Transparency refers to and includes clear traceability in mapping defensive countermeasures and countermeasure effectiveness from threat and impact. Accuracy means that metrics present an authentic representation of threat, countermeasure value and countermeasure effectiveness. And scalability points to methodologies that should generate similar results when consistently applied in a decentralized ecosystem.
In earlier offerings, Paul has written several posts that looked at the problem of evaluating cyber metrics from a general perspective and also inquired and assessed whether a set of external metrics that would be a good proxy predictor for the security of an enterprise might be available. Adam’s more practical work in developing a security risk management framework for private enterprises has been designated under the Safety Act, and forms much of the basis for our review here.
Our review here examines a more “traditional” vision of evaluating security. Most ideas about cyber metrics conceive of the assessment as one that works from the inside-out. The thought is to conduct an internal analysis of an enterprise based on some combination of assessments of its governance, its processes and the ways in which it implements security solutions. To a large degree, this is an attractive and familiar construct. It is modeled on how policymakers and government officials normally think about assessing other areas of concern such as the environmental compliance, health, or safety of an enterprise. In regulatory America, any enterprise would be experienced with this audit, assessment or compliance construct. Is such a construct feasible and practical for cybersecurity?
Broadly speaking, our intent in conducting this sort of internal assessment is to allow an enterprise to manage, mitigate and monitor its cyber risk—that is, to implement a set of safeguards accurately aligned with reasonably foreseeable threats and to know that these safeguards are operating as intended. Nobody thinks that eliminating all risk is feasible, so the objective here and with any system of measurement is to enable an organization to answer critical questions about the enterprise’s ability to manage cyber risk effectively. From this perspective, an organization should be able to: reflect on its own business model, and identify reasonably foreseeable threat actor groups who might have an interest in the organization based on that business model; identify how those threat actors could actually compromise their operational environment; and assess whether their existing security approach provides reasonable coverage against this kind of threat tradecraft.
An organization armed with this situational awareness might then ask a series of architectural, engineering and operational questions: how it should weigh tradeoffs in alternative security investments under consideration; whether the security countermeasures in place actually work; and, of course, whether the organization is prepared to respond effectively in the event of a compromise. If our goal is to measure security performance, it may be enough to quantify risk “likelihood” as against a predefined set of impacts in some meaningful and intuitive manner. If our goal is to justify security investment, we need an additional layer that quantifies risk in terms of dollars.
In this paper, we want to describe, in some detail, exactly how such an inside-out measurement system might work in practice. First, we’ll describe, generically, what it means to do a risk assessment of an enterprise. Second, we’ll explain the MITRE ATT&CK framework—a knowledge-based tool that allows a company to understand the tactics, techniques and procedures (TTPs) that an adversary might use (the overarching thesis of the analysis is that TTPs are hard for an adversary to change, so understanding them is the key to understanding the threat to an enterprise). Third, we’ll describe how knowledge of a threat model allows an enterprise to assess whether the countermeasures being deployed are in sync with those threats, and thus what vulnerabilities may be in place. Fourth, we’ll examine how this kind of mapping effort gives an enterprise the ability to conduct an internal evaluation of its own security, providing a valid internal measurement that is reproducible and auditable. And finally, the paper concludes with some considerations about the utility of this form of measurement more broadly, identifying mechanisms needed for this to scale while maintaining accuracy and transparency.
Defining the Metric
Our goal is simple: to describe a transparent, accurate, scalable and generally agreed upon metric of cyber security performance or, if you prefer a corollary, a similar measure of the degree to which an enterprise has reduced its cyber risk.
When we speak of risk reduction for an enterprise, our focus is on the traditional risk assessment conceptual framework. In this construct, risk is a function of the likelihood of a harm occurring (a given threat source exploiting a potential vulnerability) and the degree of harm (the resulting impact or consequences of that adverse event on an organization or entity). Risk evaluation can take place at a strategic level or, as for many cyber enterprises, at a tactical level. What we describe here is a tactical security assessment that is informed by strategic-level knowledge.
The initial challenge for an enterprise is to define more precisely the nature of the threat. That is, to collect information that will allow the enterprise to assess the likelihood of some event occurring. As Figure 1 below shows, threat information can take many forms, each of which has utility in differing contexts. Some of the information is about the threat actor, some about the nature of the threat and some about its operation—in effect, the who, what, when, where, why and how of the threat. Put another way, threat information is a combination of assessing intent and capability that allows one to assess: who can harm us, who wants to, and how they would go about it?
Figure 1 illustrates that threat information can take many different forms.
The last decade has seen a proliferation of cybersecurity guidance and large increases in cyber spending—over $123 billion globally on products and services in 2020—and yet large, market-leading companies continue to be victimized by major attacks. Why? Because, in addition to constant challenges on cyber hygiene, defining the universe of threats that could target an enterprise is opaque and often focuses on techniques around initial access—for example, phishing emails, exploiting public-facing applications, brute forcing valid account passwords and using USB sticks. These potential threats are, in turn, different from methods an organization might use to defend itself assuming an adversary has an initial foothold inside the organization.
Similarly, most security guidance is controls-focused but provides little direction on how to map threats to control choices, and usually provides very limited guidance on how to link threats with controls assurance and testing. Security vendors have historically offered poor characterizations of tool coverage against threat techniques, and defenders often struggle in tuning tools, leading to a false sense of security.
Here we describe a more nuanced approach, one that is informed by a structured knowle
[…]
Read the original article: Enterprise Cybersecurity Measurement