Patching

The Finding Is Never Clever

The vulnerabilities that get real systems broken into are almost never
zero-days. In engagements, the way in is overwhelmingly a thing that
had a patch available, sometimes for months, sometimes for years, that
nobody applied. The industry number that keeps showing up is that
around 60% of breaches exploit a known vulnerability for which a fix
already existed. The attacker did not out-think anyone. They scanned,
found a version behind on updates, and used the exploit that was
published the day the patch came out.

This is the uncomfortable shape of the problem. Patching is the
security control with the clearest evidence base of anything in the
field. Do it well and you remove the majority of how you actually get
breached. And almost nobody does it well, not because they do not know
they should, but because the doing is genuinely hard in ways the advice
"just patch your systems" completely ignores. This post is about why it
does not happen, and what a system that actually patches looks like.

Why It Does Not Happen

Patching fails for reasons that are structural, not lazy, and pretending
they are laziness is why so much patching advice bounces off reality.

Patching breaks things. This is the real one. A patch changes running
software, and changed software sometimes stops working with everything
around it. Anyone who has been burned by an update that took down a
production service learns caution, and that caution hardens into "we
patch during the quarterly maintenance window after full testing." That
instinct is not wrong, it comes from real scars, but it collides head-on
with a threat landscape where the gap between a patch being published
and the exploit being weaponized is now measured in days, sometimes
hours. A thirty-day test cycle against a seventy-two-hour exploitation
timeline is a losing race, and the caution that came from protecting
uptime becomes the thing that gets you breached.

There are too many patches to apply them all. A vulnerability scanner
hands you ten thousand findings before lunch. Nearly eighteen hundred
new CVEs were registered in a single week this year. No team can act on
that volume, so they triage by severity, and severity, it turns out, is
the wrong signal, which is the next problem.

The scoring misleads. Most teams prioritize by CVSS, the 0-to-10
severity score, and chase every 9.8. But CVSS measures how bad a
vulnerability would be if exploited, not whether anyone is actually
exploiting it. Only about three-tenths of one percent of new
vulnerabilities ever cross into active exploitation. Chase every 9.8 and
you burn the team out on bugs nobody is attacking while a genuinely
dangerous 7.5 that is being used right now sits unpatched because it
ranked twelfth on the spreadsheet.

And ownership is unclear. In most organizations, whose job is it to
patch the load balancer, the base image, the forgotten VM that a
project spun up two years ago and still runs? The answer is often nobody
specifically, which means it happens when someone remembers, which means
it does not happen.

The Signal That Cuts Through: KEV

The single most useful shift in patch management is to stop
prioritizing by how bad a bug could be and start prioritizing by
whether it is actually being exploited. The tool for this is CISA's
Known Exploited Vulnerabilities catalog, the KEV.

The KEV is a list of vulnerabilities with confirmed exploitation in the
wild. Not theoretical, not high-scoring, actually being used by real
attackers right now. It is a small fraction of all CVEs, which is
exactly what makes it useful: it turns ten thousand findings into the
handful that matter this week. A bug on the KEV list is being used to
break into systems today, and its CVSS score is irrelevant to that
fact. A KEV-listed 8.9 that attackers are actively exploiting is a
five-alarm fire; a 9.8 that no one has ever exploited and for which no
public exploit exists can wait.

The mature version uses three signals together. CVSS tells you how bad
it would be. EPSS, the Exploit Prediction Scoring System, tells you how
likely exploitation is. And KEV tells you it is already happening. A bug
that clears all three, high impact, high predicted likelihood, and
confirmed in-the-wild use, goes to the front of the queue ahead of
everything. KEV is the one you cannot ignore, because it has removed all
the uncertainty: the debate about whether this one is worth the risk of
patching is over, the attackers already decided for you.

Tiers, Not One Rule

"Patch everything immediately" is not a policy, it is a wish, and it
ignores that patching has a cost. The workable approach is tiered: match
the urgency to the actual risk, so the genuinely dangerous things move
fast and the rest move at a sustainable pace.

The shape most teams converge on: anything on the KEV list or an
actively exploited zero-day on an internet-facing system gets patched in
a day or two, because that is a fire. The same class on internal systems
gets a week, because the attacker has to get inside first. A high-severity
bug with a public exploit but no confirmed exploitation gets a couple of
weeks. Everything else rides the normal maintenance cadence, and the
genuinely low-risk stuff can wait for the next scheduled window or a
system upgrade.

The point of the tiers is not the exact numbers, which vary by
organization and by what a system is exposed to. The point is that not
everything is an emergency, and treating everything as one guarantees
the real emergencies drown. A tier system lets you move fast on the few
things that warrant it precisely because you are not trying to move fast
on all ten thousand.

Note the pattern in those tiers: exposure drives urgency as much as
severity. The internet-facing box gets the tight deadline; the same bug
buried behind three layers of internal network gets more room, because
the attacker has more work to do to reach it. Where a system sits
matters as much as how bad the bug is.

Automate the Boring Majority

Here is the resolution to the "patching breaks things" fear that
actually works: automate the low-risk patches so completely that they
stop being a decision, and reserve human attention for the ones that
genuinely need judgment.

On Debian and Ubuntu, unattended-upgrades applies security updates
automatically, and for the overwhelming majority of security patches,
minor updates to libraries and daemons that do not change behavior, this
is simply the right default. The fear of an auto-update breaking
production is real for major version bumps and for the handful of
components your system is tightly coupled to, but it is misplaced for the
steady stream of small security fixes that make up most of the volume.
Those you want landing automatically, because the alternative is them
landing never.

The model that works is: automatic security updates for the base OS and
the broad low-risk majority, so that stream takes care of itself and the
default state of the system is current. Then a deliberate, tested,
human-in-the-loop process for the things that are genuinely risky to
update, the database engine, the load balancer, the framework your
application is pinned to, the major version jumps. Most of your patching
should require no human at all. The scarce resource, human review and
testing time, gets spent only where it earns its keep.

This is the same idea as everywhere else in operations: make the safe
thing the default and the automatic path, so that doing the right thing
is what happens when nobody is paying attention. A system that patches
itself for the boring 95% is a system where the 5% that needs a human is
small enough that a human actually gets to it.

You Cannot Patch What You Do Not Know You Have

The failure underneath all the other failures is inventory. You cannot
patch a system you forgot exists. The forgotten VM, the container image
built two years ago and never rebuilt, the appliance in the corner
running firmware from a vendor that no longer exists, the dependency
buried three levels deep in your build, these are where the unpatched,
exploitable software actually lives, and they are unpatched precisely
because nobody is tracking them.

So the foundation of patching is not a patching tool, it is knowing what
you run. An inventory of your systems, their versions, and their
dependencies is what turns "are we vulnerable to this KEV entry" from a
frightening unknown into a query you can answer in minutes. Software bills
of materials, dependency scanning, and asset inventories are not
compliance paperwork; they are the thing that lets you know, when the
next actively-exploited bug drops, whether it is in your estate at all.
Without that, every new critical vulnerability is a scramble of "do we
even run that," and the honest answer is usually "we are not sure," which
is the worst possible position on a seventy-two-hour clock.

The Point

Patching is the least glamorous control in security and the one with the
best evidence that it works. Most breaches ride in on a vulnerability
that had a fix available, which means most breaches were preventable by
the boring discipline nobody gets credit for. It does not happen because
patching genuinely breaks things, because there are too many patches,
because severity scores point at the wrong ones, and because nobody owns
the forgotten systems where the risk actually lives.

The way through is not heroics. Prioritize by what is actually being
exploited, using the KEV catalog, not by raw severity. Tier your response
so the internet-facing fires move in hours and the rest move sustainably.
Automate the boring low-risk majority so the system stays current without
anyone deciding to make it so. Reserve human testing for the genuinely
risky updates. And under all of it, know what you run, because you cannot
patch what you have forgotten you own. None of that is clever, and that is
the point: the attacks are not clever either, and boring discipline is
what stops them.