Database backup
Bulk Signer can back up its operational store — the job history, the operational events, the recorded approvals and the approval rules frozen onto each job — to a directory, an S3 bucket (or an S3-compatible service) or an Azure Blob Storage container, on a schedule or on demand. It is off by default, and SQLite only.
Each backup is a plain SQLite database file that any SQLite tool can open, so a restore never needs Bulk Signer itself. There is no restore button and no restore endpoint: restoring is an operator procedure, done with the service stopped, and it is written out below.
| What is backed up | The whole operational store, in one file. |
| What is not | input/, processing/, output/, error/, logs/, your certificates and your appsettings.*.json. Snapshot those separately (Retention). |
| Which database | Database:Provider = Sqlite only. Under SqlServer, Backup:Enabled = true stops the service at startup — see below. |
| Where it goes | Backup:Destination = Disk, S3 or AzureBlob. One destination per deployment. |
| What a backup is | bulksigner-<yyyyMMdd-HHmmss>Z.db (the run's start time in UTC) — an ordinary SQLite file, not encrypted. |
| When it runs | Every Backup:IntervalHours, plus Back up now on the dashboard and POST /api/backup. With no interval, backups are manual only. |
| How many are kept | Backup:RetainCount, default 14. 0 keeps every backup, for a bucket or container whose own lifecycle rules prune it. |
| What is recorded | One history entry per finished run — Succeeded, Failed or Canceled — with the backup file's SHA-256. |
Every key is in Configuration; the /backup page is described in
Dashboard.
Under SQL Server, use your database backups
With Database:Provider = SqlServer, this feature does not apply, and Backup:Enabled = true stops the
service at startup with a message naming both keys, rather than doing nothing in silence. Back the
database up with your DBMS's own tools instead:
| Store | Use |
|---|---|
| SQL Server (on-premises or on a VM) | BACKUP DATABASE … TO DISK = …, on your own schedule and retention. |
| Azure SQL Database | Its automated backups, plus a point-in-time restore point taken before each Bulk Signer upgrade. |
The service's database login holds only db_datareader, db_datawriter and db_ddladmin on one
database — not the server-level rights a backup needs. The /backup page still exists on such a
deployment and says whose job backup is.
Turning it on
Three things decide it: the switch, the destination, and whether there is a schedule.
To a directory — Disk
"Backup": {
"Enabled": true,
"Destination": "Disk",
"IntervalHours": 24,
"RetainCount": 14,
"Disk": { "Path": "/backup/bulksigner" } // a volume that is not the database's
}
Backup:Disk:Path has no default, and the service refuses to start without it: every plausible
default would put the backup on the same disk as the database. The account the service runs as needs
write access to it — bulksigner for the Linux unit, NT SERVICE\LacunaBulkSigner on Windows, or
UID 1654 in the container. The directory is created if it is missing. A UNC path or a mounted network
volume is fine.
Three paths are refused at startup:
- Inside a watched input folder. A
.dbfile written there would be picked up, signed and then deleted from the folder like any other input — the pipeline would consume the backup. - Inside
processing/,output/,error/ordb/. The service creates, sweeps and removes those trees. - An
azurefiles://location. Mount the share and give the mount's path, or useAzureBlob.
To an S3 bucket — S3
"Backup": {
"Enabled": true,
"Destination": "S3",
"IntervalHours": 24,
"RetainCount": 30,
"S3": {
"BucketName": "contoso-bulksigner-backups",
"Region": "sa-east-1",
"Prefix": "bulksigner/",
"Credential": "AccessKey", // or InstanceRole
"AccessKeyId": "AKIA…"
}
}
with Backup__S3__SecretAccessKey in the environment. The credential mode is never defaulted: write
AccessKey or InstanceRole. Under InstanceRole, leaving an access key configured is refused.
The IAM policy needs four actions. s3:ListBucket applies to the bucket, not to its keys, and is
the one most often left out; s3:GetObject is needed because the service checks whether a name is
already taken before writing.
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": ["s3:PutObject", "s3:GetObject", "s3:DeleteObject"],
"Resource": "arn:aws:s3:::contoso-bulksigner-backups/bulksigner/*"
},
{
"Effect": "Allow",
"Action": "s3:ListBucket",
"Resource": "arn:aws:s3:::contoso-bulksigner-backups"
}
]
}
For an S3-compatible service (MinIO, Ceph, Wasabi, Backblaze B2), set ServiceUrl to its API
endpoint — not its web console — and, for almost all of them, ForcePathStyle: true. Region then
becomes optional. The same policy actions apply.
To an Azure Blob container — AzureBlob
"Backup": {
"Enabled": true,
"Destination": "AzureBlob",
"IntervalHours": 24,
"RetainCount": 30,
"AzureBlob": {
"ContainerUrl": "https://contoso.blob.core.windows.net/bulksigner-backups",
"Prefix": "prod/",
"Credential": "ManagedIdentity" // or ServicePrincipal / AccountKey
}
}
ContainerUrl is the container's URL as the portal's Copy URL gives it: https://, exactly one
container, and no query string — a shared-access signature is not accepted. Put any path inside the
container in Prefix.
- The role is
Storage Blob Data Contributor. A backup is written, listed and pruned, so the reader role that a certificate blob needs is not enough: an identity with onlyStorage Blob Data Readerfails on the first backup. ManagedIdentityis the system-assigned identity, and holds nothing on the host.AccountKeygrants the whole storage account; prefer the other two.- The container is not created for you. Create it, with the access policy and lifecycle you want, and then grant the role.
Secrets
Backup__S3__SecretAccessKey, Backup__AzureBlob__AppSecret and Backup__AzureBlob__AccountKey belong
in environment variables (or a non-committed appsettings.Production.json). They guard complete copies
of the store — see Security.
What the startup banner says
The ready summary carries one backup row, which never shows a credential:
backup = Disk → /backup/bulksigner (every 24h, keeping 14)
backup = S3 → backups/bulksigner/ at minio.internal:9000 (manual only, keeping every artifact)
backup = disabled
backup = not applicable (Database:Provider = SqlServer — your DBMS regime's job)
How backups run
- A consistent copy. The copy uses SQLite's online backup, so a backup taken while jobs are running is a snapshot of one committed state. A missing database file is reported as a failure, never backed up as an empty database.
- Never overwritten. A backup name that already exists at the destination fails the run. Give each
deployment its own directory, bucket, container or at least its own
Prefix. - An interval, not a time of day.
Backup:IntervalHours(1–8760) counts from the last successful backup, read from the history — so a service that restarts nightly still backs up once per interval, and a failed run is retried one interval after the last success, not after the failure. A deployment that has never backed up is due at once. Missed intervals are skipped, never queued. A backup may start up to five minutes after it is due. - Retention by count. After each successful run, all but the newest
Backup:RetainCountbackups are deleted. Only files following the naming convention above are ever deleted. A deletion that fails does not fail the run — it is logged as a warning and counted (see Monitoring). - One run at a time. A second Back up now while one is running is refused with
backup.already-running. - Cancel is a request. It is honoured between steps and throughout the upload and the clean-up, but
not during the local copy, which takes seconds on a typical store. The dashboard shows cancel
requested until the run stops; a host shutting down cancels the run the same way. Either is recorded
as
Canceled, notFailed. - Only finished runs are recorded. A process killed mid-run leaves no history entry; the log has what it did. Nothing deletes history entries.
Taking a backup now
On the dashboard, the /backup page has Back up now while nothing is running and Cancel run
while something is. Over REST, with the operator's API key:
| Method | Path | Purpose |
|---|---|---|
GET | /api/backup | Configuration, the run in progress if any, the last successful run, and the 50 newest finished runs. |
POST | /api/backup | Start a run. 202 as soon as it is accepted — poll GET for the outcome. |
POST | /api/backup/cancel | Ask the run in progress to stop. |
curl -X POST https://bulksigner.example.com/api/backup -H "X-API-Key: $BULK_SIGNER_API_KEY"
curl -s https://bulksigner.example.com/api/backup -H "X-API-Key: $BULK_SIGNER_API_KEY"
The refusals are backup.disabled (the switch is off, or the provider is SqlServer),
backup.already-running and backup.not-running, all 409. The full response shape is in
REST API.
Restoring a backup
Everything the store recorded after the backup was taken is gone: jobs, history, operational events, the recorded approvals and the frozen approval rules, and the backup history itself. Keep the file you replace — it is the only copy of everything recorded since.
1. Find the run and its SHA-256. On the /backup page, or:
curl -s https://bulksigner.example.com/api/backup -H "X-API-Key: $BULK_SIGNER_API_KEY" \
| jq -r '.history[] | select(.status=="Succeeded") | "\(.completedAtUtc) \(.artifactName) \(.sha256)"'
2. Stop the service. A SQLite file must not be replaced under a running service.
sudo systemctl stop bulksigner # Linux
Stop-Service LacunaBulkSigner # Windows (PowerShell)
docker compose stop bulksigner # Docker
3. Fetch the backup.
# Disk — it is already a file
cp /backup/bulksigner/bulksigner-20260812-021530Z.db /tmp/restore.db
# S3
aws s3 cp s3://contoso-bulksigner-backups/bulksigner/bulksigner-20260812-021530Z.db /tmp/restore.db
# S3-compatible (MinIO)
aws --endpoint-url https://minio.internal:9000 \
s3 cp s3://bulksigner-backups/bulksigner-20260812-021530Z.db /tmp/restore.db
# Azure Blob Storage
az storage blob download --account-name contoso --container-name bulksigner-backups \
--name prod/bulksigner-20260812-021530Z.db --file /tmp/restore.db --auth-mode login
4. Check the SHA-256 against the history entry — before replacing anything.
sha256sum /tmp/restore.db
(Get-FileHash .\restore.db -Algorithm SHA256).Hash.ToLower()
A mismatch means this is not the file the history entry describes — a truncated download, the wrong file — and it must not be restored. Then check that it opens:
sqlite3 /tmp/restore.db "pragma integrity_check; select count(*) from Jobs;"
5. Put it in place, keeping the file you replace:
sudo -u bulksigner mv /var/lib/bulksigner/db/bulksigner.db \
/var/lib/bulksigner/db/bulksigner.db.replaced-$(date -u +%Y%m%dT%H%M%SZ)
sudo install -o bulksigner -g bulksigner -m 0640 /tmp/restore.db /var/lib/bulksigner/db/bulksigner.db
Delete any bulksigner.db-wal and bulksigner.db-shm files left beside the old database — they belong
to the file you moved aside. On Windows the database is
C:\ProgramData\Lacuna\BulkSigner\data\db\bulksigner.db; in the Docker tree it is
data/db/bulksigner.db, owned by UID 1654.
6. Start the service and check readiness. Migrations run at startup, so a backup from an older version is upgraded on this start — one-way, like every upgrade.
sudo systemctl start bulksigner
curl -s https://bulksigner.example.com/api/ready
7. Reconcile processing/. Startup recovery works from the job records, and the restored store has a
different set of them: a leftover directory for a job the restored store never heard of is cleaned by
nothing, so inspect and clear it by hand. A job that was awaiting approval when the backup was taken
comes back awaiting approval, without the decisions made since.
Practise it. Copy a backup to a scratch host, check its digest, open it and read a job's history out of it. The first restore should not be the one that matters.
What a backup contains
A backup is a complete copy of the operational store, and so the largest concentration of personal data Bulk Signer writes to one file: every recorded approval (who, when, from which IP address and browser), every approver's name, email and CPF, the CNAB240 payment lines of files still in flight (beneficiary names, CPF/CNPJ and accounts), and every job's file name and history. The masking on the approval page is a display rule; nothing is masked in the store or in the backup.
- The destination's access control is part of your security boundary. Use a bucket, container or directory nobody else uses, and scope the role or ACL to it.
- Its credentials are the most sensitive in the product — they open every backup at once.
- Turn on encryption at rest at the destination: SSE-S3 or SSE-KMS on the bucket, Azure Storage encryption (with a customer-managed key if your policy asks) on the container. The backup file itself is not encrypted, so that nobody ever holds a backup they cannot open.
Monitoring
Five Prometheus instruments at /api/metrics:
| Metric | Kind | What it tracks |
|---|---|---|
bulksigner_backup_runs_total{result} | Counter | Finished runs: Succeeded, Failed, Canceled. |
bulksigner_backup_duration_seconds | Histogram | Duration of successful runs. |
bulksigner_backup_last_size_bytes | Gauge | Size of the latest backup this instance stored. |
bulksigner_backup_last_success_timestamp_seconds | Gauge | Unix time of this process's last successful backup; 0 until it has one. |
bulksigner_backup_prune_failures_total | Counter | Runs whose backup was stored but whose clean-up of older backups failed. |
The alert to build is on the timestamp, which keeps growing older even if the service itself has stopped:
# No successful backup in 48 hours (pair with for: 30m so a restart does not page)
bulksigner_backup_last_success_timestamp_seconds > 0
and time() - bulksigner_backup_last_success_timestamp_seconds > 172800
# The destination has started refusing writes
increase(bulksigner_backup_runs_total{result="Failed"}[6h]) > 0
# Backups work but old ones are not being deleted
increase(bulksigner_backup_prune_failures_total[24h]) > 0
The gauge is per process and starts at 0 after a restart; the durable answer is lastSuccessAtUtc on
GET /api/backup and the history on the /backup page. On a SqlServer deployment all five stay at
zero.
Troubleshooting
Start at /backup. When you open or refresh it, the page checks the destination and names what is
wrong — a directory it cannot write to, a bucket that answered 403, a missing container, a credential
it could not obtain. That check does not prove a write on S3 or Azure Blob, and a destination
problem never stops the service starting: after configuring, press Back up now — the first run is
the real test.
| Symptom | Cause | Fix |
|---|---|---|
Startup refused: Backup:Enabled = true is not supported when Database:Provider = SqlServer | The provider was changed and the backup switch left on | Set Backup:Enabled = false and use your database's backups. |
Startup refused: Backup:Disk:Path is required | There is no default | Name a directory on a volume other than the database's. |
Startup refused: Backup:Disk:Path … is inside the watched input folder … | The pipeline would sign and then delete the backup | Choose a directory outside every watched folder. |
Startup refused: Backup:Disk:Path … is inside the pipeline's processing/output/error folder or the store's own directory | The service sweeps those trees | Choose a directory outside Storage:Root. |
Startup refused on Backup:S3:Region, Backup:S3:Credential, or a key set under InstanceRole | The region and credential are never defaulted; an unused key is refused | Write the region (or set ServiceUrl), write AccessKey or InstanceRole, and remove keys the mode does not use. |
Startup refused: Backup:AzureBlob:ContainerUrl … must not carry a query string or must name exactly one container | A SAS URL or a blob URL was pasted | Give the bare container URL; put a path in Prefix. |
Run Failed naming s3:PutObject, s3:GetObject and s3:DeleteObject | An IAM action is missing — usually s3:ListBucket on the bucket | Apply the policy above. |
Run Failed on Azure, with an identity that holds Storage Blob Data Reader | Reader can list, not write | Grant Storage Blob Data Contributor. |
Run Failed: The backup container '…' does not exist | Containers are not created automatically | Create it, then grant the role. |
Run Failed: The backup directory '…' refused a write | The service account lacks write access | Grant write to the account the service runs as. |
Run Failed: The operational store '…' does not exist | ConnectionStrings:Default names a missing file | Fix the connection string. |
Run Failed: An artifact named '…' is already at '…' | Two deployments share one destination, or the clock went back | Give each deployment its own destination or Prefix. |
| A scheduled backup did not run, and there is no history entry | The process stopped mid-run; only finished runs are recorded | Read the log for that run. |
409 backup.already-running on every attempt | A large store over a slow link is still uploading | Watch phase on GET /api/backup; cancel it if the destination is down. |
| A cancel was accepted and the run kept going | Cancel is not honoured during the local copy | Wait for the Canceled entry. |
More backups at the destination than RetainCount | A deletion failed | Check bulksigner_backup_prune_failures_total and the log; grant the delete permission (s3:DeleteObject, or Contributor). |
A large .db file under data/backup-staging/ | A process was killed mid-run | Delete it — it is a complete copy of the store. |