NIST 800-53 compliance automation
Roughly a third of an 800-53 baseline is mechanically verifiable, another third is verifiable only with evidence a human interprets, and the rest is organizational fact. Automating the first third well is worth more than automating all three badly.
NIST SP 800-53 Rev 5 gives you a catalog. A moderate baseline tailored for a real DoD system runs to a few hundred controls, and each of those decomposes into individually testable assertions. That is the scale problem, and it is why the phrase “800-53 compliance automation” gets used so freely.
But controls are not uniform, and the useful first move is to stop treating them as if they were.
Three kinds of control
Sort your baseline into these three buckets before you automate anything. The sorting takes an afternoon and it changes what you build.
Mechanically verifiable. The assertion is a fact about a running system that a machine can check. Session lock timeouts, audit record content, cryptographic module configuration, account lockout thresholds, patch currency. Most of the configuration-management and audit families live here, and this is where DISA STIGs do most of their work.
Evidence-backed, human-judged. A machine can gather the evidence and confirm it exists and is current. Whether the evidence satisfies the assertion is a judgment. Incident response capability, contingency plan testing, security awareness training completion, supply chain risk processes. Automation collects and tracks; it does not determine.
Organizational fact. The assertion is about how the organization is arranged. Roles assigned, agreements in place, policies approved, personnel screened. There is nothing to scan. What automation offers here is currency tracking — when was this last reviewed, and by whom — which sounds unambitious and is where a surprising number of findings come from.
The mistake that sinks automation programs is building for the first bucket and then claiming coverage of all three. The claim is discovered during assessment, at the worst possible moment.
Automate at the CCI level, not the control level
This is the single most consequential structural decision, and it is the one most non-DoD tooling gets wrong.
A control statement decomposes into Control Correlation Identifiers — individually testable assertions derived from the control text. AC-2 is not one determination. It is a set of them, and they can have genuinely different outcomes: automated account management working, inactive account disabling not implemented.
If your system models a control as a single record with a single status, you cannot represent that, and you will resolve it by picking the worse status and writing the nuance into a comment field. That comment field is now load-bearing, unqueryable, and the reason your reporting is wrong.
Model the CCI. Roll up to the control for reporting. Never the other way round.
What automation should actually do
Four jobs, in rough order of return on effort.
Ingest and normalize. ACAS and Nessus, SonarQube, ZAP, Grype, Trivy, Gitleaks, Semgrep, Dependency-Check, CycloneDX SBOMs, STIG checklists. Each speaks its own dialect. Normalizing them into one finding shape is unglamorous work with an immediate payoff, because every downstream capability depends on it.
Correlate to controls. A finding is only useful to RMF once it is attached to the assertions it bears on. A critical CVE on a public-facing component bears on flaw remediation and vulnerability monitoring, and the correlation should be a stored relationship rather than something a person re-derives every cycle.
Preserve decisions. A false positive judged once should stay judged across every subsequent upload of the same scan. This is the capability practitioners value most and vendors mention least, because re-adjudicating four hundred findings you already dismissed is the specific experience that makes people hate their tooling.
Track currency. Which determinations are stale? Which evidence is older than its refresh interval? Which controls were assessed against an architecture version the system has since moved past? These are queries against a record, and they are impossible without one.
What automation should not do
Make determinations. A satisfied or not-satisfied decision is an authorization judgment. It has a person’s name on it and it should stay that way. A machine that decides a control is satisfied has made an authorization decision without authority, and every downstream artifact inherits that problem invisibly.
Write control narratives unsupervised. A language model will produce a fluent implementation statement for AC-2 with no knowledge of your system. It will read well. It will describe a system that may not exist. Model output belongs in a proposal that carries the passage or fact it was drawn from, accepted or rejected by a person.
Invent identifiers. Models can generate plausible control, CCI and STIG identifiers that do not exist. This is worth filtering server-side before anyone sees it, because a well-formed reference to a non-existent CCI is genuinely hard to catch by eye.
The tailoring problem nobody automates
Baseline selection is the easy part: FIPS 199 categorization gives an impact level, the impact level gives a baseline. Any tool can do that.
Tailoring is where the real work is, and it is almost entirely undocumented in most programs. Every control that is not applicable, every compensating control, every parameter value chosen, has a rationale — and that rationale is exactly what an assessor asks about.
On most programs the rationale exists in someone’s memory and in a comment column. When they leave, the tailoring becomes unexplainable, and the next assessment starts by re-litigating decisions that were made correctly three years ago.
Capture the rationale as a fact attached to the tailoring decision. It is the cheapest piece of automation in this entire article and it prevents a category of expensive argument.
The honest scope of what you get
If you automate the first bucket properly, correlate findings to CCIs, preserve adjudication decisions and track currency, here is what changes:
- Scan-to-finding-to-POA&M stops being manual and stops losing traceability.
- Stale determinations become visible instead of being discovered at assessment.
- The same false positives stop consuming assessor time every cycle.
- The SSP and SAR can be generated from current state rather than assembled from memory.
Here is what does not change: somebody still has to make the determinations, write the narratives that are genuinely narrative, and defend the tailoring. That is the job. Automation makes it possible to do the job well rather than doing it badly at three in the morning.