Troubleshooting
What you will learn
- What the messages mean, and the first thing to try for each.
A host shows red or yellow in Hosts
Read the State column (chapter 7 lists every text). The usual ones:
| State | First thing to try |
|---|---|
unreachable | from the hub, as the hub's user, run ssh user@host true; check the network and the host's power |
ssh login refused | the key is not authorised; see chapter 5 |
host key not accepted | run ssh user@host once and accept the key, or fix a changed key deliberately |
cannot inspect: … | run zfsctl check-host <name>; usually sudoers |
<pool> DEGRADED | a disk has failed; see "Replace a failed disk" (chapter 17) |
<pool> <n>% full | free space, or add storage |
job failed | open Jobs and read the last run |
"a password is required"
The feature runs a program that the host's sudoers entry does not allow. Run zfsctl check-host <name>; it names the missing line. Replace the entry with the output of zfsctl print-sudoers <name>.
The daemon
The Jobs title says daemon NOT running: jobs will not fire: start it with service zfsctl start, and check sysrc zfsctl_enable. The log is /var/log/zfsctl.log; it is stamped in the hub's local time.
A job did not run
- Is it enabled (
Space)? Is the daemon running? - Look at Next run in the panel: is the time what you meant (local time of the hub)?
- If the hub was off: with catch up
yit runs once when the daemon starts. - A replication whose previous transfer is still running skips the slot.
Replication or backup fails
The panel shows the tool's own message. Common ones:
destination … has been modified since most recent snapshot: something wrote to the destination; remove the change or set Force.- A resume token that cannot be continued: on the destination run
zfs receive -A <dataset>. - restic:
repository does not exist, orwrong password: check the repository and the password file path on that host; runrestic snapshotsthere as root. - rclone: the remote must be in root's rclone config on that host.
skipped: no snapshots with prefix 'x' on tank/data: the backup's Source is a prefix, and the job only counts snapshots named exactlyx-<UTC timestamp>(such asx-20261011T151500Z). A snapshot called justx, orx-old, is invisible to it. Set Source tonew(the job takes its own), to@name(that one snapshot), or to a prefix a snapshot job really makes.F2on Source lists what exists.the snapshot tank/data@x no longer exists: the Source is pinned to@xand someone destroyed it. The run fails on purpose, so you notice. Pick another Source.skipped: up to date (…): normal. The repository already holds a backup of that snapshot from this job.cannot snapshot tank/data: …: anewSource could not take its snapshot (out of space, or a snapshot limit on the dataset).could not remove the old snapshot …: the backup worked, but a clone (or another destroy failure) stops the job from pruning one of its ownzit-…snapshots; it tries again next run. A snapshot with a hold is not an error: the run sayskept the old snapshot <name> (held)and the reason instead. If zfsctl cannot check for holds, the note sayscould not remove the old snapshot <name> (could not check holds: …).could not check holds: …: zfsctl could not tell whether a snapshot is held, so it refused to destroy it (nothing was destroyed). A snapshot job or replication run with this message is Failed, not green. Check that the host answers and thatzfs get -H -p -t snapshot -o name,value userrefs <dataset>works there as the same user; chapter 10 explains holds.cannot destroy snapshot …: dataset is busy(when you runzfs destroyby hand, outside zfsctl): ZFS's own refusal for a snapshot that has a hold or a clone.zfs holds <snapshot>lists the holds. zfsctl checks first and saysprotected: …instead; the message above is what ZFS itself says.browse is not enabled for this host:fin Snapshots lists a snapshot's folders with a pinnedlsthat the host's sudoers does not allow yet. Runzfsctl print-sudoers HOST --with browse, read the file, install it as chapter 5 describes (the hub itself needs nothing). A host set up before the escaped listings needs the browse entry installed again: runprint-sudoers HOST --with browseand install the new file.this folder cannot be browsed hereis a different thing: the host shows no.zfs/snapshotfolder there, and zfsctl does not clone to get around it.unusual name, not openedmeans a folder's name has a control character (a newline, an escape), or a byte that is not UTF-8: it is shown cleaned and never opened.restore is not enabled for this host:Rin the snapshot browser copies a file with a pinned rootcpthat the host's sudoers does not allow yet. Runzfsctl print-sudoers HOST --with restore, read the file (it says what the entry allows), install it as chapter 5 describes, and try again. The hub itself needs nothing.that name already exists in the folder: nothing was copied: the restore never replaces a file. Choose another name in the form.an entry whose name differs only by case already existsis the same refusal for a look-alike name (a dataset that ignores case would treat it as the same).not copied: the file is not in the destination after the copy ...means the copy ran but nothing showed up at the new name; check the destination folder yourself, because zfsctl cannot tell what the filesystem did.a folder on the way to the destination is a symlink or is gone: the live folder the copy would go into is no longer a real folder. Nothing was copied. Check it on the host.the copy ran (cp exited 0) but its result could not be checkedandthe copy reported a failure ... a file of that name exists: the copy may have happened. Look at the destination folder yourself before using the file; zfsctl never deletes or retries.the destination folder is now inside the child dataset ...: a child dataset is mounted over that folder, so the copy would land in it instead of in the dataset the snapshot belongs to. Restore from the child's own snapshot.restore from a dataset given to a zone must be done inside the zone: such a dataset can be browsed but not restored from the host; run the restore inside the zone.sftp: no space left on device: the disk holding the repository is full. Free space there (for example the package cache withpkg clean -a, as root) or move the repository;dfon that host shows the filesystem.Note:lines underF5in the backup form are about the password file's location and permissions, not about the backup (chapter 15, "Protect the password file").
No mail arrives
- Run
zfsctl notify-testas root on the hub. It prints one line per channel. mail: failed: exited with status 78or similar: the mail command itself failed; run the same command by hand and read its error. Check~/.msmtprcof the user the daemon runs as (root).mail: okbut nothing arrives: look in spam; check the sender address (Mail from) is accepted by your provider.- Nothing configured: the Alerts screen says
Monitor off — set an email address to turn alerts on.
NFS or SMB does not work
NFS server not available on this host: startnfsd(service nfsd start).- A client cannot mount: check the network in the share matches the client's address, and the firewall.
- SMB: Samba must be installed and running; the valid user must exist as a Unix user; check
smbdis running.
The screen looks wrong
Make the terminal wider or taller; colours need a terminal that supports them (theme = "mono" is always safe). Press r to refresh.
Where to look
/var/log/zfsctl.log(daemon),/var/db/zfsctl/(state: jobs, history, events),config.toml.zfsctl check-host,zfsctl notify-test,zfsctl report.- When you report a bug, include the version (
Afor About) and the message.
Try it
Stop the daemon (service zfsctl stop), look at the Jobs title, and start it again.