Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Scheduled jobs

What you will learn

  • What the daemon does and how to run it.
  • How to schedule scrubs and snapshots, and how replication is set up.
  • How to read a job's history and handle a failure.

The daemon

Jobs run unattended in the daemon, a background program on the hub: zfsctl daemon, started by the rc script you installed in chapter 4 (service zfsctl start). The title of the Jobs section tells you whether it is running (daemon running (pid …)); the screen reads a heartbeat file the daemon rewrites every second.

┌zfsctl────────────────┐┌Jobs — daemon running (pid 74775)───────────────────────────────────────────────────────────────────────┐
│Dashboard             ││Job                    Host     Target         Schedule             Next run        Last                │
│Hosts                 ││scrub-local-tpool      local    tpool          Weekly, Sunday 03:00 Sun 10-25 03:00 ok 2d ago           │
│Pools                 ││scrub-local-zroot      local    zroot          Weekly, Sunday 03:00 Sun 10-25 03:00 ok 2d ago           │
│Datasets              ││                                                                                                        │
│Snapshots             ││                                                                                                        │
│Disks                 ││                                                                                                        │
│Shares                ││                                                                                                        │
│Boot Envs             ││                                                                                                        │
│Jobs                  ││                                                                                                        │
│Cloud                 ││                                                                                                        │
│Alerts                ││                                                                                                        │
│Settings              ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      │└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
│                      │┌Job─────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                      ││nothing selected                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
│                      ││                                                                                                        │
└──────────────────────┘└────────────────────────────────────────────────────────────────────────────────────────────────────────┘
↑↓ Move  n New scrub job  N New snapshot job  R New replication job  B New backup job  C New cloud mirror  ? Help  q Quit

The Jobs section is global: it lists the jobs of every host. For the selected job, the panel shows its recent runs, its schedule in words and in cron form, and its next three run times. The Target column shows where data goes, and Last the result of the last run. Your local time zone is used everywhere.

Schedules

A schedule is five fields, minute hour day-of-month month day-of-week, with *, numbers, lists (1,15), ranges (1-5) and steps (*/15). Day of week is 0–6 with 0 Sunday. 0 3 * * 0 is "every Sunday at 03:00". In the forms F2 offers presets (Hourly, Daily, Weekly, Monthly…), and the form shows the next three run times as you type, so you can see whether you meant what you wrote. If both day fields are restricted a day matches when either does.

Catch up (y by default) means: if the hub was off at the scheduled time, run the job once when the daemon comes back. A catch-up never reaches back before the job was created.

A scrub job

Press n in Jobs, or J on a pool in Pools:

┌zfsctl────────────────┐┌Jobs — daemon running (pid 74775)───────────────────────────────────────────────────┐
│Dashboard             ││Job               Host     Target         Schedule        Next run        Last      │
│Hosts                 ││scrub-local-tpool local    tpool          Weekly, Sunday  Sun 10-25 03:00 ok 2d ago │
│Pools                 ││scrub-local-zroot local    zroot          Weekly, Sunday  Sun 10-25 03:00 ok 2d ago │
│Datasets              ││                                                                                    │
│Snapshots             ││                                                                                    │
│Disks                 ││                                                                                    │
│Shares                ││                                                                                    │
│Boot Envs             ││                                                                                    │
│Jobs                  ││                                                                                    │
│Cloud                ┌New scrub job───────────────────────────────────────────────────┐                     │
│Alerts               │> Host: local                                                   │                     │
│Settings             │  Pool:                                                         │                     │
│                     │  Schedule: 0 3 * * 0                                           │                     │
│                     │  Catch up (y/n): y                                             │─────────────────────┘
│                     │                                                                │─────────────────────┐
│                     │Weekly, Sunday 03:00                                            │                     │
│                     │Next: Sun 2026-10-25 03:00, Sun 2026-11-01 03:00, Sun 2026-11-08│                     │
│                     │03:00                                                           │                     │
│                     └────────────────────────────────────────────────────────────────┘                     │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
└──────────────────────┘└────────────────────────────────────────────────────────────────────────────────────┘
Enter Next  Tab Next field  Shift+Tab Prev field  F2 Pick  Esc Cancel

Choose the host (F2 lists your configured hosts), the pool (F2 fetches the pool names of that host live), the schedule and catch up. A monthly scrub of every pool is a good minimum.

A snapshot job

Press N in Jobs, or J on a dataset in Datasets:

┌zfsctl────────────────┐┌Jobs — daemon running (pid 74775)───────────────────────────────────────────────────┐
│Dashboard             ││Job               Host     Target         Schedule        Next run        Last      │
│Hosts                 ││scrub-local-tpool local    tpool          Weekly, Sunday  Sun 10-25 03:00 ok 2d ago │
│Pools                 ││scrub-local-zroot local    zroot          Weekly, Sunday  Sun 10-25 03:00 ok 2d ago │
│Datasets              ││                                                                                    │
│Snapshots             ││                                                                                    │
│Disks                ┌New snapshot job────────────────────────────────────────────────┐                     │
│Shares               │Only snapshots named <prefix>-<timestamp> made by this job are  │                     │
│Boot Envs            │ever pruned.                                                    │                     │
│Jobs                 │                                                                │                     │
│Cloud                │> Host: local                                                   │                     │
│Alerts               │  Dataset:                                                      │                     │
│Settings             │  Recursive (y/n): n                                            │                     │
│                     │  Prefix: auto                                                  │                     │
│                     │  Keep (newest N): 24                                           │─────────────────────┘
│                     │  Schedule: 0 * * * *                                           │─────────────────────┐
│                     │  Catch up (y/n): y                                             │                     │
│                     │                                                                │                     │
│                     │Hourly                                                          │                     │
│                     │Next: Tue 2026-10-20 08:00, Tue 2026-10-20 09:00, Tue 2026-10-20│                     │
│                     │10:00                                                           │                     │
│                     │                                                                │                     │
│                     └────────────────────────────────────────────────────────────────┘                     │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
│                      ││                                                                                    │
└──────────────────────┘└────────────────────────────────────────────────────────────────────────────────────┘
Enter Next  Tab Next field  Shift+Tab Prev field  F2 Pick  Esc Cancel

The job takes a snapshot of the dataset named <prefix>-<UTC timestamp> (for example hourly-20261010T170000Z) and then keeps only the newest Keep of the snapshots it made. Recursive includes the child datasets, taken together. The job only ever deletes names that match exactly its own <prefix>-<timestamp>: your manual snapshots, other jobs' snapshots, and look-alikes like hourly-keep-forever are never touched. A snapshot with a hold (chapter 10) is kept and the run says kept N (held); this is not an error. If a snapshot cannot be destroyed for another reason (a clone), the run is recorded as failed and says why.

A replication job

Press R in Jobs:

┌zfsctl────────────────┐┌Jobs — daemon running (pid 74775)───────────────────────────────────────────────────┐
│Dashboard             ││Job               Host     Target         Schedule        Next run        Last      │
│Hosts                 ││scrub-local-tpool local    tpool          Weekly, Sunday  Sun 10-25 03:00 ok 2d ago │
│Pools                ┌New replication job─────────────────────────────────────────────┐0-25 03:00 ok 2d ago │
│Datasets             │Sends the newest <prefix>-<timestamp> snapshot, incrementally.  │                     │
│Snapshots            │The destination is received unmounted and rolled back only if   │                     │
│Disks                │Force is y. Keep on target destroys this job's older snapshots  │                     │
│Shares               │on a host destination after each run. Raw y sends an encrypted  │                     │
│Boot Envs            │dataset as it is (it stays encrypted and locked on the          │                     │
│Jobs                 │destination, which cannot read it without the key); an encrypted│                     │
│Cloud                │source needs it.                                                │                     │
│Alerts               │                                                                │                     │
│Settings             │> Source host: local                                            │                     │
│                     │  Source dataset:                                               │                     │
│                     │  Prefix (snapshots to send): auto                              │                     │
│                     │  Destination host (or file):                                   │─────────────────────┘
│                     │  Destination dataset (or directory):                           │─────────────────────┐
│                     │  Force: roll back the destination (y/n): n                     │                     │
│                     │  Schedule: 30 * * * *                                          │                     │
│                     │  Catch up (y/n): y                                             │                     │
│                     │  Keep on target (blank = keep everything):                     │                     │
│                     │  Raw: keep it encrypted (y/n): n                               │                     │
│                     │                                                                │                     │
│                     │custom schedule                                                 │                     │
│                     │Next: Tue 2026-10-20 07:30, Tue 2026-10-20 08:30, Tue 2026-10-20│                     │
│                     │09:30                                                           │                     │
│                     │                                                                │                     │
│                     │                                                                │                     │
│                     └────────────────────────────────────────────────────────────────┘                     │
│                      ││                                                                                    │
└──────────────────────┘└────────────────────────────────────────────────────────────────────────────────────┘
Enter Next  Tab Next field  Shift+Tab Prev field  F2 Pick  Esc Cancel

It copies the newest snapshot with a given prefix (typically the ones a snapshot job takes) from a dataset on one host to a dataset on another, incrementally after the first full copy, through the same ssh and sudo path the TUI uses, so nothing new is needed in sudoers. The destination can instead be a file: a directory on the hub that receives the stream as files. A stream file is not a filesystem: nothing can be read out of it until it has been received in full with zfs receive, and an incremental file can only be received on top of the snapshot it was made from, so a series of files is a chain that needs every link (zfs-receive(8)). zfsctl cannot check that a stored stream still receives. That is why the safety check counts it as a copy for the Basic level only (chapter 6). Treat it as a convenient, weaker copy, and now and then receive one into a scratch dataset to be sure.

  • What the incremental chain needs. Each run sends the newest snapshot with the prefix, incrementally from the newest base that the source and the destination both still have (matched by GUID, not by name), and ZFS requires the destination's most recent snapshot to be that base. To keep the chain alive when the source's snapshot job deletes the shared snapshot before the next run, zfsctl keeps a ZFS bookmark of it on the source: after every successful run (and when the destination is already up to date) it bookmarks the snapshot both sides now have, named zfsctl-<job id>-<snapshot>, and keeps the newest two. A bookmark takes no space and survives its snapshot, and an incremental send may start from one (zfs-send(8)); the run's message then says incremental from … kept as a bookmark. The destination must still hold the snapshot the bookmark was made from, which Keep on target never destroys. If the base is gone on the destination too, or the bookmark was deleted, no base is left: ZFS refuses a full send into the existing destination (zfsctl never adds -F to a full stream) and the run fails until you resend by hand or remove the destination's copy. A bookmark that cannot be made is mentioned in the run's message and never fails a run that has done its work; zfsctl only ever touches bookmarks with its own zfsctl-<job id>- prefix.
  • Nothing is destroyed on the source. The destination is received unmounted, and is only touched if you set Force to y; without it, a destination that has been written to fails the run with ZFS's own message.
  • Interrupted transfers resume (to another host). After a network drop, a cancel or a restart, the next run continues where it stopped. A transfer to files cannot resume; it starts again.
  • Progress and cancel. A running replication shows sent 12.3 GiB of 40.0 GiB (30%) 85.0 MiB/s eta 5m 00s in the panel; c cancels it within a second or two. A cancelled run is not a failure and sends no alert.
  • Keep on target (optional, hosts only). The last field of the form, dest_keep in jobs.toml: keep only the newest N of this job's own snapshots on the destination and destroy the older ones after each run. Blank keeps everything, which is the default. When you save, zfsctl looks at the destination and, if the setting would destroy snapshots right now, shows how many, the oldest and the newest of them, and how many stay; it needs a second keystroke. Only snapshots named exactly <prefix>-<timestamp> are ever destroyed, never ones you made by hand, and a snapshot with a hold or a clone is left alone. A destination that is a file is never pruned.
  • Raw (y/n, the last field, default n). Send an encrypted dataset as it is, keeping it encrypted and locked on the destination; see the safety note below.
  • Two enabled jobs cannot write to the same destination, and Space refuses to enable one that would clash.

Safety: Force adds zfs receive -F: it rolls the destination back to the last snapshot both sides share, destroying destination snapshots and changes newer than that. Leave it at n unless you know the destination has stray changes you want to discard.

Safety: Keep on target permanently destroys snapshots on the destination, including ones the source no longer has. Choose N large enough to go back as far as you may need, and read the confirmation before you accept it.

Safety: a replication of an encrypted source needs Raw set to y (the last field of the form, raw = true in jobs.toml), to a host or to stream files. A plain send of an encrypted dataset delivers the data decrypted, so without Raw the form refuses to save it (when it knows the dataset is encrypted) and a run fails with ... is encrypted: sending it without Raw would deliver it decrypted. Turn Raw on .... With Raw the job runs zfs send -w: the data stays encrypted all the way, and the copy on the destination is locked and stays locked; it cannot be read there without the source's passphrase, which is the point. Opening the replica later is L on the destination with that passphrase. If you change the passphrase on the source (C) while the source is its own encryption root, the replica follows at the next run and from then on opens with the new one (checked on FreeBSD; check it on your own setup before relying on it). Raw on a plain dataset also works, but zfs send -w of a plain dataset is the same as -Lec (compressed, large blocks), a different stream type than a plain send. Raw and plain streams do not chain, so changing Raw on a job that has already run asks for a confirmation: a host destination may refuse the next incremental (the run then fails with ZFS's message and the destination has to be received again from a full send), and a stream-file job simply starts a new full stream. A raw job also refuses to resume an interrupted transfer that a plain send left behind when the source is encrypted (zfs receive -A <destination> discards it); the Jobs panel marks raw jobs with raw. The cloud mirror still refuses an encrypted source; restic backups work (restic encrypts with its own key).

Safety: replication is not a backup of mistakes: a deleted file is deleted on the copy at the next run. Keep several snapshots on the destination (a snapshot job there) if you want to go back.

Managing jobs

KeyWhat it does
Up / Downselect a job
n / N / R / B / Cnew scrub / snapshot / replication / backup / cloud mirror job
Enteredit the selected job (its history and enabled state are kept)
Spaceenable or disable it
ccancel a running replication or backup
ddelete it (yes/no); its past runs stay in the history for a while

Jobs live in /var/db/zfsctl/jobs.toml and run history in history.jsonl; the daemon notices an edit of jobs.toml within a second, and keeps the previous schedule if the new file does not parse.

When a job fails

A failed job turns the host's State in Hosts yellow (job failed), counts in the Jobs column, and, when alerts are on, sends you a message (chapter 16). Select the job: the panel lists its recent runs with the error text. A fixed job clears the state at its next successful run.

Try it

Make a snapshot job for the practice dataset with J in Datasets: prefix practice, keep 3, schedule * * * * * (every minute, only for this test). After four minutes you should see exactly three snapshots in Snapshots. Delete the job with d afterwards.

Where next

  • Backups: off-site copies with restic.
  • Alerts: be told when a job fails.