Handbook sections

Undo a hardening run: restore points

Last reviewed

Pavois photographs the prior state before it converges, and can put it back. Measured on a live host: 597 of 620 controls returned to their exact prior state, and the 23 that did not are named. Here is how it works, and what it honestly cannot undo.

"How do I undo this?" is the question that decides whether a remediation ever touches production. Among the tools that remediate, none answers it. OpenSCAP hands you a generated bash script and no inverse. Canonical's guidance for the USG is not "snapshot first", it is blunter: "Always run the DISA-STIG hardening scripts on fresh installations of Ubuntu." The CIS Build Kits apply and do not unapply. Lynis and CIS-CAT Assessor do not remediate at all, so they have nothing to undo. That is not a criticism of them, it is the state of the art, and it is why most operators never apply a remediation to a production host.

The idea is not new, and it has been tried. Bastille Linux shipped RevertBastille around 2003 and died with the project; CalCom sells one-click rollback in a commercial hardening suite; HardeningKitty documents a backup-and-restore flow on Windows. What is missing from the open-source, norm-anchored Linux scanners is an inverse that the tool itself owns.

(If your machines are NixOS or rpm-ostree, you already have a better answer than this page: use it. Everything below is for the machines that are not.)

Pavois can answer it, for one specific reason: the plan is declarative, so what the run will touch is knowable BEFORE it runs.

What a restore point is

Before harden apply converges a single resource, it photographs the prior state of everything the compiled recipe names:

  • the content of every file the recipe is about to write;
  • the list of files that did not exist yet: a rollback deletes those, rather than restoring a ghost;
  • the installed/absent state of every package it will install or remove;
  • the enabled/active state of every service it will enable, disable or mask.

That lands in a local directory, and the apply prints where:

./pavois harden apply --target user@host --key ~/.ssh/id --sudo-prompt --yes plan.yml
# pavois: 📸 restore point → restore-points/user_203-0-113-10-20260714-1505
#   (undo with: pavois harden rollback restore-points/user_203-0-113-10-20260714-1505 --yes)

Pass --no-restore-point to skip the photograph. You lose the rollback; the flag exists so the choice is yours and explicit.

Undoing

./pavois harden rollback restore-points/user_203-0-113-10-20260714-1505 \
  --key ~/.ssh/id --sudo-prompt --yes

Without --yes it prints what it would do and stops: how many files it restores, how many it deletes, how many packages and services it returns, and the list of remediations it cannot undo. Read that list before typing --yes.

The rollback restores the files verbatim, deletes the ones Pavois created, purges the packages it installed (and autoremoves what their removal orphans), returns the services, and reloads the daemons. That last step is not a detail: a restored sshd_config that nobody reloaded has rolled back nothing. Pavois audits effective configuration; it restores effective configuration.

What it restores, measured

On a fresh debian 12 (Debian cloud image, 14 July 2026), a full 268-item apply, then a rollback. This is a DIFFERENT run from the campaign shown on the sample report, which is a converged golden build (kernel + partitions): a rollback needs a single, un-converged apply to undo, so the numbers below are not the campaign's and should not be compared to them.

Controls passing Evaluated
Reference (untouched host) 258 527
After the apply 495 564
After the rollback 274 533

The denominator moves, and that is not sleight of hand: a control is only evaluated when it applies. Installing auditd makes thirty auditd controls applicable that were skipped before; purging it makes them skippable again. Reading the pass count alone would be meaningless, which is why the measure below is not a pass count.

The measure is per-control identity: of the profile's 620 controls, how many end in the same state (pass / fail / not-applicable) as before the apply?

Measure Result
identical state (pass / fail / not-applicable) 597 of 620, that is 96%
same verdict (failing or not failing) 604 of 620, that is 97%

The stricter number is the one to hold us to. The 23 controls that differ are published, one line each, as a CSV, and they fall into four groups:

Group Count What it means
the hardening survived the rollback 12 it was not undone: a /tmp already mounted, a live sysctl value the kernel keeps until reboot, a chmod the restore point never captured
a control became applicable 7 the run created /var/log/auth.log and /var/log/syslog; deleting the config does not delete the logs, so controls that were skipped before now apply, and pass
the rollback introduced a failure 3 findloop-no-dirs-unowned-by-root, findloop-no-files-unowned-by-user (purging a package deletes its system user, and its files become unowned) and misc-all-apparmor-profiles-enforced
a control stopped being applicable 1 journald-forwardtosyslog

Read the third row twice. Three controls that PASSED before the apply FAIL after the rollback. A rollback is not a null operation: undoing an installation leaves traces, and on those three the host ends up slightly worse than it started. Nothing on this page is worth much if it hides that.

And the honest caveat, before anyone else makes it: this is Pavois measuring its own rollback with its own scanner. The reason to believe it is not the number, it is that the 23 exceptions are named, published, and each one is something you can go and check on the host yourself.

What it cannot undo, and says so

Most of the divergence is inherent to undoing an installation, not a defect. Each affected remediation is named in the manifest, under irreversible, before you confirm:

What Why it cannot be undone
a filesystem the run mounted (/tmp as tmpfs) /etc/fstab is restored, but a rollback does not unmount a live filesystem under a running system
a chmod driven by find / the files it touched cannot be enumerated in advance, so they were never captured
the system user a purge deletes files that user owned become unowned; the host is not bit-identical
the tool a removal takes away removing apparmor-utils removes aa-status, the very command a control checks with
a kernel build, a repartition a config file is not a kernel, and restoring /etc/fstab does not un-partition a disk

A rollback is a safety net, not a time machine. It is worth exactly as much as its honesty about the difference.

"Why not just take a snapshot?"

Take the snapshot. This does not replace it, and the page says so twice on purpose.

But a snapshot rolls back the whole machine, to a point in time. It also throws away everything else that happened since: the logs, the database writes, the packages your colleague installed, the incident you were debugging. On a VM you own, at 2am, that is fine. On a server that has been serving traffic for six hours, it is not a rollback, it is an outage with data loss.

A restore point rolls back one hardening run, and nothing else. The machine keeps running, keeps its logs, keeps the six hours of work. That is a different tool for a different moment, and the honest comparison is:

Snapshot Restore point
What it undoes the entire machine, to a point in time one harden apply, and nothing else
What it costs everything that happened since (and, on a plain VM snapshot, downtime; LVM, ZFS and btrfs snapshot hot) a reload of the affected daemons
What it needs a hypervisor, a filesystem with snapshots, disk nothing but the plan you already have
Where it works a VM you control a VM, a bare-metal host, a container host
Coverage 100%, by construction 96%, measured, with all 23 gaps named

"Puppet and Chef are convergent: just change the manifest back"

They are, and you can. But convergence is not an inverse: nothing in Puppet, Chef or Ansible captured what the value was before your change, so "changing it back" means knowing what it was: and for 268 settings across 45 files, on a host you did not build, you do not know. You would be authoring the inverse by hand, from memory, on a machine you have just modified.

Pavois does not ask you to remember. It photographed it.

The rule this feature enforces on the rule base

A remediation that installs a package from inside an exec (apt-get install -y acct) is invisible: it does not appear in the plan, the operator never opts into it, and a rollback cannot undo it. Three rules did that, and a rollback left acct and sysstat behind on a host it had "restored".

They now declare the package (requires_package), and a linter (exec-installs-package) refuses the pattern. A remediation that hides what it does cannot be undone, so it is not allowed to hide.

FAQ

Where does the restore point live, and does it contain secrets? It is staged on the target (0600, owned by the account you connect with, never world-readable), then fetched back to the control host and left in restore-points/: that is the copy harden rollback reads. It contains the content of the config files the run is about to modify, so treat both copies as sensitive: delete the staging copy on the target if that matters to you, and keep the local one where you keep secrets.

Can I roll back a run I did months ago? Yes, if you kept the restore point directory: it is self-contained (a manifest plus a tarball). It restores the state as it was when the photograph was taken, which is not the same thing as the state today. Read the manifest.

Can I roll back one control? Not today. The restore point is per-run. The finer-grained answer is to make the run itself finer: harden plan is opt-in per rule, so apply what you understand, in the size you can reason about.

Should I stop taking snapshots then? No. Take the snapshot. A 96% rollback is a safety net for a run that went wrong, not insurance against everything that can go wrong on a Linux host.

Does a rollback restore only what the apply changed? It restores the FILES the apply was going to write, verbatim, to their content at photograph time. That is file granularity, not change granularity: if a colleague edited one of those same files between the apply and the rollback, their edit is overwritten too. Roll back soon, or read the manifest's file list first.

Key takeaways

  • harden apply photographs the prior state before it converges, and harden rollback puts it back: files, packages, services, and a daemon reload so the restoration is EFFECTIVE.
  • Measured on a live host: 597 of 620 controls returned to an identical state (96%), and the 23 that did not are published, one line each. Three of them PASSED before and FAIL after: a rollback is not a null operation.
  • What cannot be undone is named in the manifest before you confirm: a mount already made, a find-driven chmod, the user a purge deletes, the tool a removal takes away.
  • Among the tools that remediate: OpenSCAP, Ubuntu's USG, the CIS Build Kits, ansible-lockdown: none ships the inverse of its own remediation (Lynis and CIS-CAT Assessor do not remediate, so they have nothing to undo). Bastille Linux tried in 2003 and died; CalCom sells it commercially. Pavois ships it, and the reason is structural: a declarative plan makes the blast radius knowable before anything runs.

Be clear about what that does NOT claim. Where you have snapshots at the filesystem or image level: LVM, ZFS, btrfs, a VM snapshot, NixOS generations, rpm-ostree rollback: those are strictly better than this and you should use them: they are exact, total, and they do not depend on anyone having enumerated anything. A restore point exists for the machines that do not have them, and to tell you, before it converges, what the run is about to touch.

Next steps