Operating Proofroom client stacks
This guide is for the operator: whoever runs the client’s server. That is the client’s own IT, or a reseller or we on
the client’s behalf if the client chooses. Proofroom runs on the client’s own server. Each client gets a
stack: its own containers, database, files, keys, domain and licence. proofroomctl creates, updates, backs up,
restores and removes stacks; you never need docker compose by hand.
AI assistants (MCP): see AI assistants (MCP).
1. What a stack is
| Directory | /srv/proofroom/stacks/<name>/ |
| Files | compose.yml (from the release, replaced by update; do not edit), .env (image, port, memory, database passwords), app.env (the app’s settings, including MASTER_KEY), licence/licence.json, backups/, db-init/ |
| Containers | compose project proofroom-<name>: app and db |
| Data | Docker volumes proofroom-<name>_pgdata (database) and proofroom-<name>_evidence (evidence files, encrypted with MASTER_KEY) |
| Address | the app listens on 127.0.0.1:<port> (from 41000), behind Caddy (or your own proxy) at the client’s domain |
| Memory | app 1 GB, database 512 MB by default (--app-memory, --db-memory at create; resize changes them later) |
A stack name is 2 to 31 lower-case letters, digits or dashes, starting with a letter. proofroomctl only ever touches
proofroom-* containers, networks and volumes, /srv/proofroom, /etc/caddy/proofroom.d/ and the patch report’s
script and units (section 10). Other applications on
the same server are left alone. Every command logs what it changes. Exit codes: 0 done, 1 failed, 2 wrong usage.
See before you change: simulate. Put -s (or --simulate, or --dry-run as in install.sh) before any command:
sudo proofroomctl -s update acme. It prints each change the command would make as a numbered step (files, systemd
units, containers, the database, Caddy, calls to the licence server that change something), with passwords, keys and
tokens shown as ****, and changes nothing. The checks that only read run for real (the system, free memory and disk,
where the domain points, the licence file, versions, whether another command is running), so the plan is the one for
this server, and the exit code is the one the command would end with: 0 it would succeed, 1 it would fail, after the
same message.
Every command and option is described by sudo proofroomctl help <command>, by man proofroomctl (installed by
init and each update) and in the command reference, all written from the same source. On a minimised
Ubuntu image without man, proofroomctl help has the same text.
The commands in this guide are run from an ordinary admin account with sudo. Signed in as root on a server without
sudo (Debian)? Leave out sudo.
2. The server
- Ubuntu 24.04 or Debian 12, and an account that can use
sudo(root also works). - Docker Engine with the compose plugin:
curl -fsSL https://get.docker.com | sudo sh(or your distribution’sdocker.ioanddocker-compose-v2packages), pluscurlandopenssl. - Ports 80 and 443 free (Caddy takes them), unless you use your own reverse proxy.
- Memory: 8 GB recommended (2 GB is the minimum, enough for one small stack). About 1.5 GB per stack of the default
size, plus 1 GB for the system;
inittells you how many fit, andresizegives a busy stack more (§5). - For each client, a DNS record for its domain pointing at the server, before you create its stack (Caddy gets the TLS certificate as soon as the site is added).
3. Getting proofroomctl
The release image is public, so no registry login is needed:
sudo docker run --rm --entrypoint cat ghcr.io/proofroom-team/proofroom:<version> /app/release/proofroomctl | sudo tee /usr/local/sbin/proofroomctl.new > /dev/null \
&& sudo chmod 755 /usr/local/sbin/proofroomctl.new && sudo mv /usr/local/sbin/proofroomctl.new /usr/local/sbin/proofroomctl
sudo proofroomctl help
The download goes to a new file first, so a failed download never leaves you without a working proofroomctl.
Each release carries its own proofroomctl. After a successful update, proofroomctl puts that release’s copy in place
of itself (§6), so you get it once, here. (From a private registry, log in first:
sudo docker login <registry host> -u <user> --password-stdin < <token file>.)
If your security team asks what is in the image, each release has a CycloneDX SBOM: SBOM and vulnerability scanning.
4. Preparing the server: init
sudo proofroomctl init
For a private registry: sudo proofroomctl init --registry R --registry-user U --registry-token-file F.
It checks the server (operating system, Docker, memory), logs in to the registry if given a token, installs Caddy from Caddy’s own
package repository if it is not there, writes /srv/proofroom/ctl.env and installs the daily patch report (section
10) and updates by button (section 6). Run it again at any time; it changes only what is missing. sudo proofroomctl init --refresh installs or refreshes only the patch report and updates by button, on a server set up before they existed.
| Option | |
|---|---|
--registry R |
the image repository (default ghcr.io/proofroom-team/proofroom) |
--registry-user U, --registry-token-file F |
log in; - reads the token from standard input. The token never appears on a command line |
--own-proxy |
do not install or touch Caddy. For each stack, proxy the client’s domain to http://127.0.0.1:<port> (printed by create) with a request body limit of at least 110 MB, and terminate TLS yourself |
--licence-server URL |
where stacks check in (default https://licence.proofroom.com); "" for servers without internet access |
--update-url URL |
where the server reads the newest release’s version every 4 hours (default https://proofroom.com/releases/latest.json); "" turns the check, the dashboard notice and Update now off on this server |
If Caddy was already installed, init adds one line to /etc/caddy/Caddyfile (import /etc/caddy/proofroom.d/*.caddy)
and keeps the rest; a copy from before is kept as Caddyfile.before-proofroom. Every Caddy change is checked before
Caddy reloads, and undone if Caddy refuses it, so your other sites keep running.
5. A new client: create
You get a licence file from us (<id>.proofroom-licence.json). It names the client’s domain and holds the stack’s
check-in secret: keep it like a password.
sudo proofroomctl create acme --domain compliance.acme.example --licence ~/lic_….proofroom-licence.json --admin-email it@acme.example
It checks the licence (signature and domain) with the release’s own image, generates the stack’s passwords,
AUTH_SECRET and MASTER_KEY, picks a port, starts the stack, waits until it is healthy, creates the first admin and
prints their one-time setup link (valid 24 hours; send it to them securely), and adds the Caddy site. A lost or
expired link: sudo proofroomctl admin-link NAME --email E prints a new one (it also recovers an admin locked out of their
authenticator). Add --name "Full Name" to set the person’s display name, on a new or an existing account; names can
also be changed in Admin → Users, and each person changes their own on My profile. Options:
--version V (default: proofroomctl’s own version), --app-memory 1g, --db-memory 512m, --master-key-from FILE
(section 8).
A firm licence (one licence whose users are shared by an audit firm’s client sites) comes as
<id>.proofroom-firm-licence.json, for every client site under the firm’s domain. create with it registers the new
site with the licence server first: the domain must be one level below the firm’s (client7.auditfirm.com) or listed
on the licence, and seats must remain. The site gets its own install secret; the firm token in the file stays with
you, not on the stack. It needs Proofroom 1.10 or later and a server that can reach the licence server.
Settings such as single sign-on go in the stack’s app.env (each setting is described in the install guide); then sudo proofroomctl restart <name>. Email is set in Admin → Settings → Email, or in app.env,
which wins when it sets MAIL_TRANSPORT. If you later remove the MAIL_* lines, any settings saved earlier on the
page are used again; the page names them while app.env is in use.
More memory for a stack. A stack with many people, evidence or integrations may need more than the default:
sudo proofroomctl -s resize acme --app-memory 2g --db-memory 1g # what it would change, and whether it fits
sudo proofroomctl resize acme --app-memory 2g --db-memory 1g
It sets the limits (and the app’s heap, three quarters of the app’s limit) and restarts the stack’s containers, which
takes about a minute. It refuses, changing nothing, when the extra memory is more than the server has free (its
MemAvailable, less 256 MB kept for the system), and puts the previous limits back if the stack does not come up
healthy on the new ones. The app needs at least 512m and the database 256m.
Looking at stacks:
sudo proofroomctl list # every stack: version, domain, port, health, licence state, last backup
sudo proofroomctl status acme # one stack in detail, and the server's patch report (section 10)
sudo proofroomctl logs acme --since 1h
6. Updates
A stack updates in one of two ways, and both do the same thing: an admin presses Update now in the app, or you
run sudo proofroomctl update on the server. Either way it can go straight to the newest release; there is no need to
install the releases in between.
Before you update, take a backup (sudo proofroomctl backup acme) and copy it and the stack’s app.env off the server
(§7). The update takes its own backup first and restores it if the update fails, but that copy is on the same server.
What an update does
- Checks there is room: it stops, changing nothing, only when the disk Docker keeps its images on is clearly short of the new image’s download (its size in the registry) and 512 MB. When removing old Proofroom images no stack uses would make the room, it removes them first; never the image the stack runs or the one before it. When the size can’t be read, it says so and goes on.
- Takes a verified backup.
- Switches the stack to the new image and the new release’s
compose.yml. - Waits up to five minutes for it to be healthy (database changes run at start; the release notes say which releases have them).
- If it doesn’t come up healthy, puts the previous image back and restores the backup it just took, so the stack is as it was. It stops with an error and the app’s last log lines.
- When it is healthy, installs the release’s own proofroomctl and its systemd units (the patch report, the update
check and Update now), so changes to them arrive with the update. If that step fails, the update still stands and
one line says to run
sudo proofroomctl init --refresh. - When it is healthy, removes the Proofroom images no stack on the server needs. Each stack keeps the image it runs
and the one before it, for a rollback. Only Proofroom’s own images go (
ghcr.io/proofroom-team/proofroom, the old nameghcr.io/hriday/proofroomand your registry, if you set one), one by one by tag or ID; never the database’s or anything else on the server, and neverdocker image prune. A failed update removes nothing.
proofroomctl never replaces itself with an older version, and never after a rollback. If it can’t replace its own file (for example, a read-only file or a link), it says so and prints the command to do it by hand:
sudo docker run --rm --entrypoint cat ghcr.io/proofroom-team/proofroom:<version> /app/release/proofroomctl | sudo tee /usr/local/sbin/proofroomctl.new > /dev/null \
&& sudo chmod 755 /usr/local/sbin/proofroomctl.new && sudo mv /usr/local/sbin/proofroomctl.new /usr/local/sbin/proofroomctl
Update now (Admin → Updates)
Admin → Updates shows the version the stack runs, the newest release with its release notes, when the server last checked, and Update now. Admins also see a notice on the dashboard when a newer version is out.
- The check: every 4 hours (and a few minutes after each boot) root fetches one small file,
https://proofroom.com/releases/latest.json, into/srv/proofroom/host/update.json. The request has no query string and sends nothing about the server or its stacks. To check now:sudo systemctl start proofroom-update-check.service. - Turning it off: Admin → Updates → Check for updates, per stack. The server makes no request when every stack on it
has the check off.
sudo proofroomctl init --update-url ""turns it off for the whole server. - Pressing Update now: the app writes one empty file in the stack’s own request folder
(
/srv/proofroom/stacks/<name>/update-request/update). Root removes it, fetcheslatest.jsonitself, checks the release is newer than the stack runs, and runs the same update as by command. The app never chooses what root runs. - Following it: the page shows each phase through the app’s restart, then the outcome: “Updated from X to Y” with the backup taken first, or that the update failed and the stack went back to the previous version with its data. The audit log records who asked and each outcome.
- When there’s no button: the check is off, the last check failed or is over 24 hours old, there is no newer release, or an update is already waiting or running. Root also refuses a request made under a minute after the last one, or while another proofroomctl command holds the server’s lock. Nothing changes, and the page says why.
- If an update is interrupted (a reboot, the 45-minute limit, an error), the status ends “interrupted” and says what
changed. If it stopped while the new version was starting, it was not rolled back: the status names the image the
stack is set to and the command to restore the backup taken just before (
sudo proofroomctl restore <name> <backup>). - One stack, one folder: each stack’s request folder is mounted into that stack only, so one stack can never update another. Its status folder is read-only to the app.
- Logs:
journalctl -u proofroom-update@<name>andjournalctl -u proofroom-update-check.
By command
sudo proofroomctl update acme --version 1.8.0 # a given release (the newest is in the release notes)
sudo proofroomctl update acme # the release this proofroomctl came with
Without --version it never moves a stack back to an older release than it runs. After an update by button the server’s
proofroomctl is the new release’s, so that is usually the newest.
Old images and disk space
Before 1.14.3 an update kept every image it pulled, so a server updated often can fill its disk. To see what is there and free it:
sudo proofroomctl images # every Proofroom image, its size and whether a stack keeps it
sudo proofroomctl -s images --clean # what --clean would remove, changing nothing
sudo proofroomctl images --clean # remove the ones no stack needs (each keeps the one it runs and the one before)
An image Docker will not remove (a stopped container still uses it) is left, with a note. If a stack names an image that is not on the server, nothing is removed until that is sorted out (the note says which). Updates do the same clean-up on their own from 1.14.3 on.
Coming from an older version?
Every stack checking in runs 1.7.1 or later, which updates as above. If you restore or run something older: The release notes have the steps under each release’s “Upgrading”.
7. Backups
-
Every night at 01:30 UTC each stack’s app takes a verified backup (database, evidence files and a manifest) into
/srv/proofroom/stacks/<name>/backups/; the newest 14 are kept.sudo proofroomctl backup <name>takes one now;updatealways takes one first. -
Copy them off the server. They are on the same disk as the stack. The simplest way: in each stack, Admin → Backups → Off-server copy (below). Or copy them yourself, for example nightly from another machine:
rsync -a --rsync-path="sudo rsync" <you>@<server>:/srv/proofroom/stacks/ ~/proofroom-backups/ # backups/ plus each stack's .env and app.env sudo rclone copy /srv/proofroom/stacks/acme/backups remote:proofroom/acme # or to object storage, from the server -
Off-server copy (Admin → Backups). Give the stack an S3-compatible bucket you own (AWS S3, Cloudflare R2, Backblaze B2, Wasabi, MinIO and others): endpoint (
https://, without the bucket), bucket, region (autoorus-east-1where the service has none), access key, secret and prefix. After each nightly backup the app encrypts that backup’s files (AES-256-GCM, with a key derived from the stack’sMASTER_KEY) and uploads them, with the stack’s licence file, also encrypted, andcopy.jsonlast:<bucket>/<prefix>/<stack's domain>/<backup name>/db.dump.enc evidence.tar.gz.enc manifest.json.enc licence.proofroom-licence.json.enc copy.json # written last: a copy without it is incompletecopy.jsonis not encrypted: it names the backup, the key’s fingerprint, and each encrypted file’s size and SHA-256, so a download can be checked before anything is decrypted. Where the service supports it, the app also asks it to encrypt at rest (SSE-S3, a setting on the page).The domain in the path lets several stacks share one bucket and prefix. The copy goes straight from the server to your bucket; Proofroom, the company, never receives it. The secret is encrypted with the stack’s master key and never shown again. Test connection writes
proofroom-connection-test.txtunder<prefix>/<domain>/and removes it (a key that may only write passes; delete the file yourself). Upload the newest backup now and Back up now do not wait for the night. Each upload is in the audit log (backups.uploaded,backups.upload_failed), and a failure shows on the page and the dashboard in plain words until an upload succeeds. A failed upload never touches the backup on the server.Set the bucket up so a stolen key or a mistake cannot remove your backups:
- a private bucket (the files are encrypted before they leave the server; encryption at rest is a second layer);
- a key that may only write (
s3:PutObject, pluss3:AbortMultipartUpload) to<prefix>/*; - versioning or object lock, so an overwrite or delete keeps the old copy;
- a lifecycle rule for retention, for example delete objects under
<prefix>/after 30 days, and abort incomplete multipart uploads after 7 days (AbortIncompleteMultipartUpload), so an upload cut off half-way does not leave parts you pay for. Proofroom never deletes backups in the bucket;BACKUP_KEEPapplies only to the server’s own folder.
app.env(withMASTER_KEY) is not uploaded, and the copy cannot be decrypted without it: keep it off the server separately (next point). Restoring from the copy: section 8, “Restore from the off-server copy”. -
The key. Evidence in a backup is encrypted with the stack’s
MASTER_KEY, inapp.env. Without it a backup’s files cannot be read. Back up each stack’sapp.envseparately and securely (a password manager or an encrypted store), not only next to the backups. -
Test a restore now and then (section 8,
--into), then remove the test stack.
8. Restoring and moving a client
Restore a stack in place (asks you to type the stack’s name; --yes skips that):
sudo proofroomctl restore acme 20261021T013000Z # a backup in the stack's backups/, or a path to a backup directory
It first takes a safety backup of the current data (if that fails, nothing is changed), then stops the app, replaces the database and the evidence with the backup’s, and starts the app. Safety and pre-update backups never delete older ones, so restoring the oldest backup is safe; the nightly job keeps the newest 14.
Start a new stack from a backup, on the same server (to test a restore, or to move a client to a new domain):
sudo proofroomctl restore acme 20261021T013000Z --into acme-test --domain test.acme.example --licence ~/<licence file for that domain>
The new stack gets the source stack’s MASTER_KEY and version. With a firm licence file, the new site is registered
on the firm licence. On a site with a firm licence, restoring (in place or --into) or importing data from another
site on the same firm licence counts as one of the firm’s 3 releases, reactivations and imports a year, counted by
the licence server (so it needs the server, and is refused, with nothing changed, when none are left); so does a
backup taken before 1.10, which does not say which site it came from. Restoring a site’s own backup, or importing a
client’s export from their own Proofroom, is free. A counted restore that fails gives its count back.
To move a client to another server: on the new server, create the stack with --master-key-from <copy of the old stack's app.env>, copy a backup directory over, then sudo proofroomctl restore <name> /path/to/<backup directory>.
Then remove the old stack. A site on a firm licence keeps its registration when it moves: give create a licence file
made from the old stack, { "licence": <its licence/licence.json>, "installSecret": "<LICENCE_INSTALL_SECRET from its app.env>", "licenceServer": "https://licence.proofroom.com" }, not the firm file (which would register a new site, and
the domain is still taken by the old one).
Restore from the off-server copy
When the server is lost, or to check that the copy in the bucket works, restore from it on a server prepared with
init (a new one, or the same). You need the stack’s app.env that you kept off the server (it holds MASTER_KEY),
and its domain’s DNS record pointing at the server. The licence is in the copy, encrypted with the backups; there is
no need to ask for a new one.
-
Download the backup. Each backup is a folder
<prefix>/<domain>/<backup name>/in the bucket; take the newest whosecopy.jsonis there (a folder without it was cut off). With the AWS CLI, or rclone, for any S3-compatible service:aws s3 ls s3://acme-backups/proofroom/compliance.acme.example/ --endpoint-url https://<endpoint> aws s3 cp --recursive s3://acme-backups/proofroom/compliance.acme.example/20261029T013000Z/ /root/copy/20261029T013000Z/ --endpoint-url https://<endpoint> # or: rclone copy remote:acme-backups/proofroom/compliance.acme.example/20261029T013000Z /root/copy/20261029T013000Z -
Check the download against
copy.json: each file’s size and SHA-256 (the next step checks them again, and the key’s fingerprint, before it decrypts anything):cd /root/copy/20261029T013000Z python3 -c 'import json; [print(v["sha256"] + " " + k) for k, v in json.load(open("copy.json"))["files"].items()]' | sha256sum -c -
Decrypt it with the stack’s
app.env. It writes/srv/proofroom/restored/<backup name>/: a backup thatrestoretakes, with the licence file beside it. Nothing is written unless the whole copy decrypts; a copy made with anotherMASTER_KEYis refused, naming both keys’ fingerprints.sudo proofroomctl backup-decrypt /root/copy/20261029T013000Z --master-key-from /root/acme-app.env -
Create the stack (on a new server) with the same key and the licence from the copy, then restore:
sudo proofroomctl create acme --domain compliance.acme.example --licence /srv/proofroom/restored/20261029T013000Z/licence.proofroom-licence.json --master-key-from /root/acme-app.env sudo proofroomctl restore acme /srv/proofroom/restored/20261029T013000Z --yesOn the server the stack already runs on, skip
create:sudo proofroomctl restore acme /root/copy/20261029T013000Zdecrypts the copy with the stack’s own key and restores it, in one step. Put-sbefore either command to see what it would do first. -
Sign in and check the data. Admin → Backups comes back with the backup (its secret is sealed with the same key): press Upload the newest backup now to check the copy works from here. Remove
/srv/proofroom/restored/<backup name>and/root/copywhen you are done.
If the old server’s site is still registered on a firm licence, the licence file in the copy keeps that registration: release the old site only after the new one runs.
Moving an audit to an audit firm’s Proofroom
An audit firm may work on a client’s audit in its own Proofroom. The client’s admin makes a password-protected export (Admin → Export) and sends the file and the password separately. On the firm’s server:
sudo proofroomctl create acme-2026 --domain acme-2026.audits.firm.example --licence ~/<licence file> # no --admin-email: the site stays empty
sudo -i # a root shell for the import (the stack's folder is root's); exit when done
cd /srv/proofroom/stacks/acme-2026
cp /home/<you>/acme-export.zip /home/<you>/pw.txt backups/ && chown 1000:1000 backups/acme-export.zip backups/pw.txt && chmod 0600 backups/pw.txt
docker compose -p proofroom-acme-2026 stop app
docker compose -p proofroom-acme-2026 run --rm app node_modules/.bin/tsx scripts/import.ts --archive /backups/acme-export.zip --password-file /backups/pw.txt
docker compose -p proofroom-acme-2026 up -d app
rm backups/acme-export.zip backups/pw.txt
The import prints setup links for the imported admins; invite everyone else again from Admin → Users. Integrations come back switched off and without secrets: leave them off, so the copy never calls the client’s systems. Passwords, authenticators, sessions, API keys, integration secrets, the email settings and the TypeSafe API key don’t travel; every record, evidence file and the audit log do. Work done in the firm’s copy doesn’t flow back to the client’s. On a firm licence, importing a client’s export from their own Proofroom is free; only an import of a site that was on your own firm licence counts toward the firm’s limit of three releases, reactivations or imports a year.
9. Licences
Each stack checks in with the licence server once a day; renewals, extra users and grace extensions arrive that way, with nothing to do on the server.
sudo proofroomctl licence acme # the stack's licence state, licensed and active users
sudo proofroomctl licence acme ~/<file> # install a licence file by hand (servers without internet access)
sudo proofroomctl usage # licensed and active users for every stack; --csv for a spreadsheet
Admins buy, add users and renew on the stack’s Admin → Licence, paid through our payment provider; the
stack picks up the paid licence within seconds of payment, and nothing is needed on the server. A stack without
access to the licence server cannot buy itself: the billing contact gets the licence file by email after any payment,
to install with sudo proofroomctl licence <stack> <file>.
On a site whose licence a vendor (reseller) manages, Admin → Licence names the vendor instead and offers no buying:
the vendor changes the site’s users and renews from its own vendor area, and the change arrives with the daily
check-in, or at once when an admin presses Check now. Up to 5 of the vendor’s own staff accounts are free on the
site, so the vendor can help with set-up; they are marked Vendor on Admin → Users. A vendor hosting its clients
runs create with each client’s install code, one stack per client.
proofroomctl create (and licence) writes licence/profile.json beside licence.json when the licence file came
from a trial with profile answers. It holds hints for the company profile only; deleting it changes nothing about the
licence.
Up to 3 auditor accounts per open audit are not counted; an auditor on no open audit counts (activeUsers is the counted number; totalUsers and
freeAuditors are in sudo proofroomctl licence <name>). On a firm licence, licence also shows firm=true, the seats
used across the firm’s sites (poolUsed) and whether the site was released. To free a firm site’s seats, an admin
presses Release licence on its Admin → Licence: the site becomes read-only (sign-in, viewing and exports still
work).
10. Patch report
Every stack’s server needs its security updates installed, and an auditor asks for proof (PCI DSS 6.3.3: critical
patches within a month of release). init installs a daily patch report that records it. It only reads: Proofroom
never installs, upgrades or reboots anything on the server, and never changes its settings. It shows each problem
with the command that fixes it, for you to run.
- What it records: the operating system and kernels, pending updates (and which are security updates), every
security update from the day it was published to the day it was installed, whether automatic updates are on, and
whether a reboot is pending. It is written to
/srv/proofroom/host/status.json, with 90 days of history in/srv/proofroom/host/history/. Nothing is sent to us: each stack’s app reads the folder read-only at/host. - When: the
proofroom-host-report.timerruns/usr/local/lib/proofroom/host-report.shonce a day, at a time between 01:00 and 05:00 that stays the same on each server (a run missed while the server was off happens at the next boot). Run it now:sudo systemctl start proofroom-host-report; its log:systemctl status proofroom-host-report. - Check now: an admin can also run it from Admin → Server health. The button writes one empty file,
/srv/proofroom/host-request/request, and nothing else;proofroom-host-report.pathsees it and starts the same service, which removes the file and writes a new report, usually within a minute. It runs at most once every 5 minutes: a press sooner says when the last report ran. The folder is the only one the app can write to (mounted at/host-request; root’s, group 1000, mode 770). The script removes whatever is at that name without following it (a file, a link, an empty folder); only a folder with something in it is left, and the button stops working until you remove it by hand. If the path unit gave up,sudo proofroomctl init --refreshorupdateresets and restarts it. Without the folder (a server not yet updated), the page shows the command instead of the button. - After patching: press Check now on Server health, or run
sudo systemctl start proofroom-host-report.service, so the page and the next evidence show the updates installed. - First report: when
initorupdateinstalls or changes the timer, it also starts the first report in the background, so Server health has data within minutes. If that start fails, one line says so and the timer runs it that night. - Updates:
updateinstalls the release’s copy of the script when it changed, and the units (the service, the timer and the path unit) when missing or changed, as the new release’s proofroomctl writes them (section 6). A server set up before the patch report, or before Check now, gets them withsudo proofroomctl init --refresh(or on its nextupdate). A problem with the patch report prints one line and never stopsinitorupdate. - Updating from 1.2.x: the first update runs the old proofroomctl, which does not know the patch report. Afterwards,
run
sudo proofroomctl init --refreshonce (or update again) to install the daily report. - Looking at it:
sudo proofroomctl status(orsudo proofroomctl status <name>, with the stack) prints a summary and each warning:
| Warning | Fix |
|---|---|
| Security updates open more than 30 days after publication | sudo apt-get upgrade |
| Automatic updates are off | sudo apt-get install unattended-upgrades && sudo dpkg-reconfigure -plow unattended-upgrades |
| A reboot needed for more than 7 days | reboot at a quiet time (the stacks start again on their own), then sudo proofroomctl status |
| The report has not run for more than 48 hours | systemctl status proofroom-host-report |
| “none yet” | the first report is still running, or the timer was never installed: sudo proofroomctl init --refresh, then sudo systemctl start proofroom-host-report |
11. Troubleshooting
| Symptom | Look at |
|---|---|
create says the stack did not become healthy |
sudo proofroomctl logs <name>; the stack is left in place. Remove it with sudo proofroomctl remove <name> --confirm <name> --no-export and try again |
| “the licence was refused” | the message says why: another domain (the licence names exactly one), or a changed file (download it again) |
| “Caddy did not accept the change” | the Caddyfile or another site in it does not validate: caddy validate --config /etc/caddy/Caddyfile says why. (DNS is not checked then; a domain whose DNS is not ready yet gets its certificate once it is) |
| The site shows a certificate error | DNS for the domain does not point at this server yet, or ports 80/443 are blocked; Caddy retries on its own: journalctl -u caddy |
| “another proofroomctl command is running” | one changing command runs at a time; wait for the other to finish |
| A restore or rollback says its restore failed | the data is as it was before (the database restore is all-or-nothing). Read the message above it: a “master key fingerprint mismatch” means the stack’s MASTER_KEY is not the one the backup was taken with (section 8) |
| A port is in use | proofroomctl skips ports in use when it creates a stack; another program that later takes a stack’s port stops that stack starting |
| The app is read-only | the licence expired or is not valid, or a firm site was released: sudo proofroomctl licence <name>, and contact us |
| “a firm licence needs the licence server” | the server was prepared with init --licence-server ""; firm sites need to reach the licence server (init --licence-server https://licence.proofroom.com) |
| “Another site on the firm licence already uses this domain” | release that site first (its Admin → Licence), or ask us if its server is gone |
| Admin → Updates says “not set up on this server yet” | the stack has no update folders, or root has no units: sudo proofroomctl init --refresh |
| Update now waits and nothing happens | systemctl status proofroom-update@<name>.path; if it gave up, sudo proofroomctl init --refresh resets it |
12. Removing a client
sudo proofroomctl remove acme --confirm acme
It takes a final full export of the client’s data (final-export.zip) and copies the stack’s .env, app.env and
backups into /srv/proofroom/removed/acme-<time>/, then deletes the stack’s containers, volumes and Caddy site. Give
the client their export; delete that directory when you no longer need it. Without a working app the export cannot be
taken and remove stops; --no-export removes the stack anyway. A site on a firm licence keeps counting toward the
firm’s seats until it is released: press Release licence on its Admin → Licence before removing it.