How to Build a Cron Job Dead Man's Switch That Alerts You

A cron job dead man's switch is a heartbeat check that expects to hear from a scheduled job on a fixed schedule and alerts you the moment that ping stops arriving. Instead of checking whether the job succeeded from the outside, you watch for silence: no ping inside the grace window means something is wrong, whether the job crashed, the server rebooted, or cron itself never fired.

This matters because cron failures are invisible by default. A job that exits 0 says nothing when it exits 1, and a job that never even starts — a bad crontab edit, a stopped cron daemon, a deleted script — says nothing at all. You find out three weeks later when a report is missing, a backup didn't run, or a cleanup job silently died and a table has been growing ever since.

What Is a Dead Man's Switch, Exactly?

The term comes from railway and industrial equipment: a switch that triggers an alarm unless a human actively holds it down, so an operator who collapses at the controls still stops the train. Applied to software, the "human" is your cron job, and "holding the switch" means sending a heartbeat ping every time it runs. Stop pinging, and the switch releases — the alert fires.

Dead Man's Switch vs. Uptime Monitoring

Uptime monitoring pings something you control and waits for a response; a dead man's switch waits for something you control to ping it. They are inverses, and most real systems need both — see how to get notified when your website goes down for the outbound-check side of the same pattern.

Why Do Cron Jobs Fail Silently?

Four causes account for almost every silent cron failure: the crontab entry has a typo and never registers, the cron daemon itself is down after a reboot, the script exits non-zero but nothing reads the exit code, or the script hangs indefinitely and never reaches its own success line. A dead man's switch catches all four the same way, because it doesn't care why the ping didn't arrive — only that it didn't.

• Bad crontab syntax — a misplaced field silently drops the whole line; cron does not warn you by default.
• Daemon not running — common after a server migration or a kernel update that didn't restart services.
• Non-zero exit swallowed — chained commands and redirected output are easy to get wrong once and never notice again.
• Hung process — a script waiting on a lock or a stalled network call never reaches the line that would have reported success.

How Do You Build a Heartbeat Check for a Cron Job?

The mechanism is one line added to the end of the job, plus one monitor watching for it: append a request to the job so it reports it ran every time it completes, and set the monitor's expected interval to match the job's schedule plus a grace window.

1. Create a heartbeat monitor with an expected interval that matches the job's cron schedule.
2. Add one line to the end of the script that pings the monitor's token URL, guarded so it only fires on success.
3. Set a grace period wider than the job's normal runtime variance, not equal to it.
4. Point the monitor's alert channel at somewhere you'll actually see it, not an inbox nobody checks daily.
5. Test it once by commenting the heartbeat line out and confirming the alert fires.

#!/bin/bash
# nightly-backup.sh
pg_dump mydb | gzip > /backups/mydb-$(date +%F).sql.gz \
  && curl -fsS -m 10 https://pingwire.dev/hb/YOUR-TOKEN

The && is doing real work here: if pg_dump or the compression step fails, the heartbeat request never fires, and the monitor treats that exactly like the job not running at all — which is correct, because a backup that didn't finish is a backup that doesn't exist. You can also report a failure explicitly with /hb/YOUR-TOKEN/fail instead of just letting the beat go silent; full request format in the developer docs.

How Long Should the Grace Period Be?

Set the grace period to whichever is larger: three times the job's normal runtime variance, or ten minutes. Not the job's average runtime, and not the raw cron interval. A grace period equal to the average runtime fires false alarms on every slow run; a grace period equal to the full cron interval means a job that failed on run one won't alert until run two is also overdue, doubling your real detection time.

Job interval	Typical runtime	Recommended grace period
Every 5 minutes	a few seconds	10 minutes
Hourly	1–3 minutes	15 minutes
Nightly	10–30 minutes	90 minutes
Weekly	under an hour	4 hours

How Do You Verify the Switch Actually Works?

Test the failure path once, deliberately, before you trust it: comment out the heartbeat line, wait for the grace period to pass, and confirm the alert arrives on the channel you expect. This single dry run catches the two most common setup mistakes — a monitor interval that doesn't match the real cron schedule, and an alert channel nobody has muted. A dead man's switch that has never actually fired is a guess, not a safeguard, and the only way to know the grace period you picked is realistic is to watch one real alert land.

Do the same check after any change to the job itself. A script that grows a new slow step — a bigger database dump, an added upload, a retry loop — can push its runtime past a grace period that was correct when you set it, and the monitor will start firing on runs that actually succeeded, just late. Re-time the job and widen the grace period rather than silencing the alert; a switch you've learned to ignore is worse than no switch at all, because it still feels like coverage.

What Should You Do When the Alert Fires?

Check three things in order: is the cron daemon running at all, does the crontab still contain the line, and does running the script by hand reproduce a failure. In that order, because a stopped daemon and a dropped crontab line both look identical to a hung script from the monitor's side, and ruling out the cheap causes first saves you from debugging application code for a problem that was actually a stopped service.

Piecing this together with cron logs, a shell wrapper, and an inbox you half-watch works, right up until the job runs on a server you don't check daily and the failure sits for a week. That is the point where a dedicated heartbeat monitor earns its keep: one line in the script, a push alert the moment a beat is missed, and a history of every run so a "sometimes it's slow" pattern is visible before it turns into a missed beat.

Isn't This Overkill for a Small Script?

If the job's failure is genuinely harmless — a log rotation you would notice within a day anyway — skip it. But most cron jobs people add a dead man's switch to are backups, billing syncs, and data pipelines feeding something else, where silent failure compounds: a missed nightly backup means the next incident has no fallback, and a missed sync means downstream numbers are wrong for however long nobody notices. The setup cost is one line in the script and a couple of minutes creating the monitor; the failure cost is usually measured in hours of cleanup, or data that can't be recovered at all.

Create a heartbeat monitor and paste the one-line ping into your job — it takes about two minutes, and the free plan covers three monitors on a five-minute interval, enough to protect the jobs that actually matter. Sign up and add the first one to your existing backup script.