Archive (1.0): Backup and restore

Archive. This is the 1.0 guide for an installation from a clone of the repository with Docker Compose (a "checkout install"). It describes 1.0 under the product's former name, form-schema-app; the product is now Fieldset. It does not apply to an instance installed with install.sh and managed with fieldset (2.0): do not run its git, docker compose, migration, restore or .env steps on such an instance; they bypass fieldset's backups, maintenance gate and rollback. For 2.0 see the current guide. To move a checkout install to 2.0, use fieldset adopt (install guide).

This guide covers what to back up, the backup script, the restore drill and a full restore. It assumes the Docker Compose installation from the install guide.

Contents:

• What to back up
• The APP_KEY
• Back up with scripts/backup.sh
• Test your backups: scripts/restore-drill.sh
• Full restore

What to back up

What	Where	Why
PostgreSQL database	volume pg-data, service postgres	Everything except files: organizations, users, forms and versions, submissions and their values, workflows and runs, the mail log, settings, the audit trail.
Storage volume	volume app-storage, mounted at /var/www/html/storage/app	private/renditions/ (the PDFs), private/attachments/ (photos, signatures, files), private/branding/ (logos), private/exports/ (report exports, deleted after 7 days), private/data-exports/ (personal data and audit log exports, deleted after 7 days).
.env, especially APP_KEY	the project directory	Without the same APP_KEY, encrypted data in the database cannot be read (details). Keep it separate from the data backups.
The release you run	git describe --tags in the project directory	Restore with the same release, then upgrade if needed.

Not needed: Redis/Valkey holds only queues, caches and locks, and Gotenberg keeps no data.

Back up the database before the storage volume. Files are written before the database rows that point to them, so every row in the dump then has its file in the archive. Files that appear between the two steps are harmless extras. scripts/backup.sh does it in this order.

The APP_KEY

APP_KEY encrypts, among other things:

• the SMTP username and password of organizations with their own mail server;
• the SSO configuration and client secret of every identity provider;
• webhook signing secrets;
• users' two-factor secrets and recovery codes;
• queued and failed emails in the mail log (so they can be retried);
• sessions, cookies and signed links (invitations, download links).

If the key is lost, the database restores fine but these values cannot be decrypted: organizations must re-enter SMTP and SSO settings, webhook secrets must be replaced (and receivers updated), users with two-factor authentication cannot sign in, failed mails cannot be retried, and everyone is signed out. There is no way to recover the values without the key.

Store APP_KEY in a password manager or secret store, separately from the backups. If you ever change the key, put the old one in APP_PREVIOUS_KEYS (comma-separated) so existing data stays readable; the application has no command that re-encrypts old values.

Back up with scripts/backup.sh

The script runs on the server, from the project directory or anywhere else, and only reads production data. It writes one directory per run:

<dest>/20261018T020000Z/
  database.dump     pg_dump custom format (from inside the postgres container)
  storage.tar.gz    the storage volume
  env               a copy of .env, only with --include-env
  manifest.txt      time, PostgreSQL version, migration count, row counts
  SHA256SUMS        checksums of the files above

scripts/backup.sh --dest /srv/backups/form-schema-app

Option	Meaning
--dest DIR	Required (or set BACKUP_DEST). Must be outside the repository: the backup contains personal data and must never be committed.
--keep-days N	Delete complete backups older than N days after a successful run (default 14; 0 keeps everything).
--min-keep N	Always keep the newest N backups, however old (default 3).
--no-storage	Database only.
--include-env	Also copy .env (with APP_KEY and all secrets). Only if the backup destination is as protected as the server.

Details:

• A run writes into <timestamp>.partial and renames it when everything succeeded, so an interrupted run never looks like a complete backup. Pruning only touches directories named like a complete or partial backup, nothing else in <dest>.
• The dump is checked with pg_restore --list before the run counts as complete.
• On a running system tar may report files that changed while being read; the script prints a warning and continues.
• The storage archive is read through the app image (docker compose exec app, or a one-off container when app is stopped). The postgres service must be running.
• Files and directories are created with mode 600/700.

Schedule it, for example nightly with cron (as a user that may run docker compose):

15 2 * * * /srv/form-schema-app/scripts/backup.sh --dest /srv/backups/form-schema-app >> /var/log/form-schema-app-backup.log 2>&1

Copy backups off the server. A backup on the same disk does not survive a lost server. Copy the backup directory to another location with your usual tool (restic, borg, rsync to another host, object storage) and encrypt it there if the destination is not under your control.

Test your backups: scripts/restore-drill.sh

A backup only counts once a restore has worked. The drill restores a backup into a new scratch database and checks it, without touching production data:

scripts/restore-drill.sh --backup /srv/backups/form-schema-app/20261018T020000Z \
  --scratch-db restore_drill_20261018

It checks:

1. the checksums in SHA256SUMS;
2. that pg_restore loads the dump into the scratch database without errors;
3. row counts of the main tables, and at least one organization;
4. that every attachment and PDF in the database has its file in storage.tar.gz, and that the newest files (50 per table, --sample N) match their recorded SHA-256 hash;
5. in a one-off container of the app image against the scratch database: whether migrations are pending (the backup is older than the code) or unknown (the backup is newer than the code), and whether the APP_KEY in .env decrypts the encrypted columns.

The output is a list of PASS, FAIL and NOTE lines, and the exit code is non-zero if a check failed.

Safety rules built into the script:

• --backup and --scratch-db are required. The name must start with restore_drill_, must not exist yet, and can never be the production database.
• It never drops a database it did not create in the same run. At the end it drops the scratch database and deletes the extracted files, unless you pass --keep.
• Files are read from the archive into a temporary directory; nothing is written to the storage volume.
• The app check refuses to run unless it is connected to a restore_drill_ database.

The drill needs the postgres service running and, for the app check, the app image and .env (--skip-app-check skips that part). It uses resources of the server it runs on; for large databases run it on a second machine with a checkout of the same release, the .env from your secret store and docker compose up -d postgres. Run a drill at least monthly and after every upgrade.

Full restore

Use these steps on a new server, or on the old one after data loss. They overwrite data, so read them first and make sure you are restoring the right backup.

1. Prepare the server as in the install guide, with the release the backup was made with (last_migration in manifest.txt helps to find it) and the saved .env, including the original APP_KEY. Build the image (docker compose build) but do not start the stack yet.

2. Check the backup:

   cd /srv/backups/form-schema-app/20261018T020000Z && sha256sum -c SHA256SUMS && cd -

3. Start only the database and make sure it is empty. A new volume is empty; on an existing server, stop the application first and recreate the database (this deletes its current content):

   docker compose up -d --wait postgres
   # Existing server only:
   docker compose stop app queue scheduler
   docker compose exec postgres sh -c 'dropdb -U "$POSTGRES_USER" "$POSTGRES_DB" && createdb -U "$POSTGRES_USER" "$POSTGRES_DB"'

4. Restore the database:

   docker compose exec -T postgres sh -c 'pg_restore --username="$POSTGRES_USER" --dbname="$POSTGRES_DB" --no-owner --no-privileges --exit-on-error' \
     < /srv/backups/form-schema-app/20261018T020000Z/database.dump

5. Restore the storage volume through a one-off container of the app image (it creates the volume if needed; files belong to the web server user):

   docker compose run --rm --no-deps -T --entrypoint tar app -C /var/www/html/storage/app -xzf - \
     < /srv/backups/form-schema-app/20261018T020000Z/storage.tar.gz

6. Start everything. The app container runs any pending migrations, which upgrades an older backup to the release you installed:

   docker compose up -d --wait

7. Check: sign in at /admin, open a submission and download its PDF, send a test email, and look at "E-Mail-Protokoll" ("Mail log") for mails left "In Warteschlange" ("Queued"). Jobs that were waiting in Redis when the backup was taken are not part of it: missing PDFs and stalled workflow steps are picked up again by the scheduler within minutes, but such mails may have to be sent again. Run docker compose exec app php artisan audit:verify: the audit chains now end at the time of the backup, so record the new chain heads (Protect the audit log).

Anything created after the backup was taken is not in it. Field App devices keep the work that is still in their outbox and send it at their next sync; ask field workers to check "Einreichungen" → "Auf diesem Gerät" ("On this device") for items marked "Entscheidung nötig" ("Needs your decision").