Nobody told you how to measure security behavior. That's not your fault.

Dear renegades, mavericks, and deviants,

A few months ago, I found myself chatting to a long-time contact who runs a security awareness, education, and culture program at their org’. Over a decade in the field, sharp, well-read, and genuinely respected by peers. I can't remember how we got onto it, but at one point, they said to me, "Oz, no one’s ever explained how to measure security behavior, I've just been working it out myself."

There was a heavy pause before they said it, almost as if they were trying to work out whether to trust me with a secret. The reality is this isn't the first time I've heard some version of this, and each time I'm struck less by the admission itself and more by the apparent relief that follows it.

The three camps

I've come to realize that there are broadly three groups of people in this field when it comes to measuring security behavior.

The quietly lost. They've never really had the conversation. Not because they're not capable, but because nobody in their career ever sat them down and walked them through it. They measure what is measurable and hope it’s the right thing.

The silently ashamed. They know they don't fully get it. But in a field where confidence is currency, saying "I'm not sure how to do this" feels like a professional risk. So they say nothing, and the gap gets wider.

The confidently wrong. They think they have it figured out. They've been doing it a certain way for so long that their method has become their truth. They aren’t looking for different answers, because, as far as they're concerned, they already have them.

Most of us have spent time in all three camps. Often without a torch.

The definition trap

The single thing that trips everyone up is the definition itself.

Most organizations think they're measuring security behavior. They're not. Or, at least not as much as they think. They're actually mixing the measurement of awareness, intent, and engagement with the occasional security behavior. This is pretty common. Many track things like:

How people feel about security
How many colleagues engaged with this month’s security comms
How many people volunteered to become security champions
Whether the security intranet page saw a spike in visits after an all-hands email

These things might be interesting. They might even be useful. But they're not actually security behaviors.

A security behavior is an action or practice by an individual that directly protects information, systems, or assets from unauthorized access, damage, or threats. It is not a habit, an attitude, or a training completion rate.

Let’s look at the difference…

"Employees are more security aware" is not a behavior.
"This person reported a suspicious message" is a behavior.

"Colleagues engaged with our security comms this month" is not a behavior.
"This person only shares sensitive information on an approved application" is a behavior.

"Twelve people volunteered to be security champions" is not a security behavior measurement. Neither is "sixty percent of staff attended the awareness webinar."

Those are signals of engagement, at best. They tell you almost nothing about whether anyone is actually doing anything differently in terms of security behavior.

If it isn't a single specific action or response, or it doesn’t directly protect information, systems or assets, you're measuring something else entirely.

And measuring the wrong thing doesn't just give you an inaccurate picture, it gives you a false sense of confidence in a picture that doesn't exist.

Six methods of measurement

Once you're clear on what you're measuring, you need to decide how.

There are six main methods, and none of them is a silver bullet on its own.

Objective measures are the most reliable. System telemetry, computer logs, and technical integrations. This tracks what people are actually doing, not what they say they do.

The catch: It requires data access, technical infrastructure, and openness and transparency with your workforce about what's being collected and why. Skip that conversation, and you have a data ethics problem before you have a measurement program.

Self-report asks people to describe their own behavior through surveys or interviews. It’s useful, accessible, relatively easy to run, and tells you something real. But it’s also limited by social desirability bias (people answering how they think they should, not as they actually behave). Treat it as indicative rather than definitive, and design questions that are specific and time-bounded.

Proximate measures assess intent and motivation. Useful for understanding why people behave a certain way, but less useful as evidence of behavior itself. Believing intent equals action is like measuring how much someone wants to exercise and concluding they're fit.

Scenarios present realistic fictional situations and ask how someone would respond. Because it’s a simulation, people tend to be more candid. They work best when grounded in situations people would feasibly actually encounter.

Observational data involves watching people in their natural work environment. Rich and unfiltered when done well, but limited to visible behaviors, which excludes a lot of what matters, and leaves it prone to observer bias. This requires trained observers to avoid recording what they expect to see.

360-degree feedback gathers peer observations from colleagues about a person's visible security behaviors. It works best in physical team environments, but becomes harder to run (reliably) at scale in hybrid or distributed workplaces.

The reality is you need a mix. Objective measures tell you what is happening. Self-report and proximate measures help explain why. Scenarios and observation surface things that telemetry misses. Where possible, use multiple methods and triangulate.

Navigating the noise

A few critical factors will always affect the validity of your data:

Psychological safety: If people fear punishment for mistakes, they'll hide their behavior rather than exhibit it naturally. That distorts everything downstream.
Organizational disruption: Major company changes introduce noise that's hard to separate from the effect of your interventions. Use control groups alongside intervention groups to isolate the true signal, even in a messy environment.
Data blind spots: If you don't have clean access to the data sources that reveal actual behavior, you'll have blind spots. Own it. Be entirely honest about what your data can and can't tell you.

Good measurement isn't about waiting for perfect conditions; it's about deeply understanding the conditions you have.

From awareness to engineering

Get the measurement right, and you stop doing ‘security awareness’.

You move into behavioral engineering. You're running interventions with clear baselines, tracking change over time, controlling for confounders, and actually knowing (and proving) whether what you're doing is working.

That's a different job. A far more valuable one. And it starts with being honest about what you're measuring.

Become the person in the room who actually knows

The CybSafe team works with security professionals building exactly this kind of practice. If you want to talk through where your program sits today, and what rigorous, robust measurement could look like for your organization, we're easy to find.

👉 Book time with the team

P.S. If this felt like something you needed to read but couldn't have asked for, send it to one person you think feels the same way. Chances are they'll appreciate the support.