How to construct an effective security controls evaluation

<p>I once received an ad from a company that promised to lower home energy costs by conducting a free energy audit. The audit, it said, could be done over the phone — no home visit — and would require absolutely “zero questions asked” — i.e., about our current energy use, heating and cooling systems, insulation or anything else.</p>
<p>It struck me as objectively ridiculous. How can you reach a fact-based, evidence-driven conclusion without at least measuring something?</p>
<p>I bring this up because I see CISOs promising something similar with their <a href=”https://www.techtarget.com/searchsecurity/tip/How-to-develop-a-cybersecurity-strategy-Step-by-step-guide”>security strategies</a>. Namely, they say they can manage their security controls in the absence of important contextual knowledge, without information about control efficacy — let alone efficiency — and, in some cases, without any operational performance data at all. Yet, just like the information-free “energy audit,” this approach undermines decision-making. Missing information means we pay more for an outcome that diminishes our control, makes no impact on reducing risk and yields poorer security overall.</p>
<p>By contrast, better measurement reduces risk. Contextualized performance information helps us understand how well controls perform relative to each other, which in turn makes investments more efficient and improves how we manage and operate those controls.</p>
<p>Let’s take a look at how to better measure security controls, how to use the data collected to best effect and why a security controls evaluation matters in the first place.</p>
<section class=”section main-article-chapter” data-menu-title=”Multiple angles of security control evaluation”>
<h2 class=”section-title”><i class=”icon” data-icon=”1″></i>Multiple angles of security control evaluation</h2>
<p>To start, it’s important to realize there are multiple dimensions, or vantage points, from which to measure controls. And there are countless ways to measure control performance. The three I’ve found most helpful to measure are:</p>
<ol class=”default-list”>
<li><b>Effectiveness.</b> Does the control work?</li>
<li><b>Maturity.</b> How reliable is the process supporting the control?</li>
<li><b>Efficiency.</b> How does the control perform economically?</li>
</ol>
<p>The first area is perhaps the easiest to intuitively understand. Effectiveness assesses how well the control performs at its intended task. Is it implemented? Does it work? Is it appropriately scoped? Does it cover the portions of the environment we need it to?</p>
<p>If you were to conduct a compliance audit against a set of controls — for example, something like the controls in ISO/IEC 27001:2022 Annex A, <a target=”_blank” href=”https://csrc.nist.gov/pubs/sp/800/53/r5/upd1/final” rel=”noopener”>NIST Special Publication 800-53</a> or specific controls required by a regulatory framework such as PCI DSS — this is the lens that would magnify most of the evaluation. In addition to measuring whether the control exists or not, though — as you would for a regulatory compliance audit, for example — you also want to account for how well it performs. The specifics of this will vary based on the individual control. Some systems might involve comparing rates of false/true positives versus false/true negatives; others might measure remediated versus unremediated issues, for example, quarantined malware versus unquarantined.</p>
<p>The second dimension is the maturity of the implementation or processes that support the control’s operation. Different processes — even those designed to achieve the same or similar outcomes — can have different levels of maturity. Consider two separate approaches to a single task — for example, <a href=”https://www.techtarget.com/searchnetworking/tip/5-principles-of-the-network-change-management-process”>change management</a>. One company might use a disorganized process for oversight, while another uses a well-documented, quantitatively measured one. Even if these processes perform equivalently, the more mature process has advantages that the less mature one does not — for example, resilience to adverse events such as personnel attrition or process failure. This leads to, in aggregate, more predictable security outcomes.</p>
<p>How might you measure maturity? There are whole frameworks devoted specifically to this. For example, the <a href=”https://www.techtarget.com/searchsoftwarequality/definition/Capability-Maturity-Model”>Capability Maturity Model</a> defines five levels of maturity:</p>
<ol class=”default-list”>
<li><b>Initial.</b> Unpredictable, ad-hoc,

[…]
Content was cut in order to protect the source.Please visit the source for the rest of the article.