backend / linux bash / 13_cron_and_systemd_timers.md

Cron and Systemd Timers

6 interview angles 7 min read source

Cron and Systemd Timers

The two ways to run scheduled jobs on Linux. Cron is older, simpler, universal. Systemd timers are newer, integrated with logging/dependencies, harder to set up. For interview questions: cron syntax and “why isn’t my cron job running” debugging.

Cron — the basics

crontab -e                    # edit YOUR crontab
crontab -l                    # list YOUR crontab
crontab -r                    # remove YOUR crontab (careful!)
sudo crontab -e -u alice      # edit alice's crontab (need root)

System-wide crons live in:

  • /etc/crontab — system crontab (has a USER field)
  • /etc/cron.d/* — drop-in files (also have USER field)
  • /etc/cron.{hourly,daily,weekly,monthly}/ — scripts run at those intervals

User crontabs (via crontab -e) are in /var/spool/cron/crontabs/USER (don’t edit directly).

Cron syntax

# m  h  dom mon dow  command
  0  3  *   *   *    /usr/local/bin/backup.sh

Five time fields:

Field Range Meaning
minute 0–59 minute of hour
hour 0–23 hour of day (24h)
day of month 1–31 day of month
month 1–12 (or jan-dec) month
day of week 0–7 (0 and 7 are Sunday; or sun-sat) day of week

Each field can be:

Form Means
* any value
5 exactly that value
1,15,30 list of values
0-30 range
*/5 every 5 (step)
0-30/5 every 5 within 0-30

Common patterns

*/5 * * * *       # every 5 minutes
0 * * * *         # top of every hour
0 3 * * *         # 03:00 daily
0 3 * * 0         # 03:00 every Sunday
0 0 1 * *         # midnight on the 1st of each month
30 2 * * 1-5      # 02:30 weekdays only
0 9-17 * * 1-5    # top of every hour, 9am-5pm, weekdays
@reboot           # at boot (special)
@daily            # 00:00 every day (= 0 0 * * *)
@hourly           # = 0 * * * *
@weekly @monthly @yearly

@reboot runs at system startup; useful for “start my service if not using systemd.”

A real example

# m  h  dom mon dow  command
  0  3  *   *   *    /usr/local/bin/backup.sh > /var/log/backup.log 2>&1
*/15 * *   *   *    /usr/bin/curl -fsS https://hc-ping.com/abc123 > /dev/null
  0  4  *   *   *    cd /opt/myapp && /usr/bin/python3 cleanup.py

Why is my cron job not running?

The 5 most common reasons:

1. PATH is minimal in cron

PATH=/usr/bin:/bin

Your interactive shell has /usr/local/bin, ~/.local/bin, virtualenvs. Cron does not. So python may resolve to /usr/bin/python3.6 instead of your venv’s Python.

Fix: use absolute paths.

0 * * * * /opt/myapp/.venv/bin/python /opt/myapp/job.py

Or set PATH at the top of crontab:

PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin
0 * * * * job.sh

2. Working directory is $HOME

If your script uses relative paths, they’re relative to your home directory, not where the script lives.

Fix: cd first, or use absolute paths:

0 * * * * cd /opt/myapp && ./run.sh

3. Output disappears

Cron mails stdout/stderr to the user. If mail isn’t configured (most modern systems don’t), the output vanishes. You don’t see errors.

Fix: redirect explicitly:

0 * * * * /opt/myapp/job.sh >> /var/log/myapp/job.log 2>&1

For “no output unless something fails” semantics:

0 * * * * /opt/myapp/job.sh > /dev/null

(stderr still mails on failure, which gets dropped — set MAILTO=alice@example.com to actually receive it).

4. % is special in crontab

Unescaped % is treated as a newline in cron commands. The classic bug:

0 0 * * * date +%Y-%m-%d > /tmp/today.log     # broken

Fix: escape with \%:

0 0 * * * date +\%Y-\%m-\%d > /tmp/today.log

Or move the date logic into a script.

5. Environment variables aren’t loaded

~/.bashrc and ~/.profile don’t run for cron jobs. So $JAVA_HOME, $VIRTUAL_ENV, custom $PATH aren’t there.

Fix: source them explicitly:

0 * * * * . $HOME/.profile && /opt/myapp/job.sh

Or set them in the crontab:

JAVA_HOME=/opt/java
PATH=/opt/java/bin:/usr/bin:/bin

Debugging cron

# Did it run? check the cron log
sudo tail -f /var/log/cron       # RHEL
sudo tail -f /var/log/syslog | grep CRON   # Debian/Ubuntu
sudo journalctl -u cron          # systemd-cron

# Test the command in a stripped-down environment
env -i HOME="$HOME" PATH=/usr/bin:/bin sh -c '/opt/myapp/job.sh'

Locking — preventing overlapping runs

If a job runs every 5 minutes but sometimes takes 10, you’ll have overlapping invocations. Use flock:

*/5 * * * * /usr/bin/flock -n /tmp/myjob.lock /opt/myapp/job.sh

flock -n exits immediately if the lock is already held — second invocation does nothing.

healthchecks — knowing when cron failed

Cron failures are silent unless you set MAILTO. The cleaner pattern is “ping a heartbeat URL when the job succeeds”:

0 * * * * /opt/myapp/job.sh && curl -fsS https://hc-ping.com/abc123

Services like healthchecks.io / cronitor / Better Uptime alert if the heartbeat doesn’t arrive on schedule. Catches “cron stopped running entirely” — which plain logging doesn’t.

Systemd timers

Newer alternative to cron. Two unit files: .service (what to run) and .timer (when to run).

/etc/systemd/system/backup.service:

[Unit]
Description=Daily backup

[Service]
Type=oneshot
User=alice
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/backup.sh

/etc/systemd/system/backup.timer:

[Unit]
Description=Run backup daily

[Timer]
OnCalendar=daily               # or specific: "*-*-* 03:00:00"
Persistent=true                # if missed (e.g. machine off), run on next boot
RandomizedDelaySec=600         # spread load: random 0-600s offset

[Install]
WantedBy=timers.target

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now backup.timer
sudo systemctl list-timers     # see all active timers
sudo systemctl status backup.timer
sudo journalctl -u backup.service   # past run logs

OnCalendar syntax

OnCalendar=daily                    # 00:00 daily
OnCalendar=hourly                   # top of every hour
OnCalendar=weekly                   # Mon 00:00
OnCalendar=*-*-* 03:00:00           # 03:00 daily
OnCalendar=*-*-* 03:30:00           # 03:30 daily
OnCalendar=Mon..Fri *-*-* 09:00:00  # weekdays 09:00
OnCalendar=*-*-01 00:00:00          # 1st of every month
OnCalendar=*-01,07-01 00:00:00      # Jan 1 and July 1

Different from cron syntax — more verbose but unambiguous.

Cron vs systemd timers

Cron Systemd Timers
Setup one-line in crontab two unit files
Discovery crontab -l systemctl list-timers
Logs mailed (or wherever you redirect) journalctl -u service
Missed runs gone forever Persistent=true reruns
Dependency on other services none Requires=, After=
Random delay (jitter) manual RandomizedDelaySec=
Per-user vs system-wide per-user via crontab -e system-wide (or per-user via systemctl --user)
Universality every Linux system systemd systems only

For containers and cloud VMs (where systemd is universal), timers are increasingly the right choice. For “drop a one-liner on this server,” cron is faster.

When to skip both

For modern cloud environments:

  • Kubernetes CronJob — runs a pod on a schedule. Real isolation, no shared environment surprises.
  • AWS EventBridge + Lambda — for serverless scheduled tasks.
  • Celery beat / APScheduler — in-app schedulers if you already have a worker process.

Cron is for legacy or small/simple deployments. K8s CronJob is the modern equivalent in clusters.

Common pitfalls

  • % in crontab — escape as \%.
  • Output going nowhere — redirect explicitly.
  • PATH doesn’t include venv — use absolute path to python from the venv’s bin dir.
  • Overlapping runs — wrap in flock.
  • Missed runs after server downtime — cron silently misses; systemd Persistent=true reruns.
  • Server clock wrong — cron honors local time; setting timezone after creating the crontab may surprise you.

Common interview confusions

  • “Cron jobs run with my normal environment.” — no. Bare PATH, no .bashrc, no virtualenv. You must set up the environment in the crontab or script.
  • @reboot runs every minute after boot.” — runs once when cron daemon starts (i.e., at boot).
  • “Systemd timers are more accurate than cron.” — they’re more featureful, similarly accurate. Both fire at the configured time within seconds.

Interview angle

  • “Cron syntax — what does */5 * * * * mean?” — every 5 minutes, every hour, every day, every month, every weekday. */N is “every N units.”
  • “Why might a cron job work when run manually but fail when scheduled?” — different environment: bare PATH, different working directory, no shell startup files sourced, output going nowhere (you don’t see the error). Fix with absolute paths and explicit redirects.
  • “How do you redirect output of a cron job?”>> /path/to/log 2>&1 at the end of the command. Otherwise stdout/stderr are mailed (and lost if mail isn’t configured).
  • “How do you prevent overlapping runs?”flock -n /tmp/job.lock command. Second invocation finds the lock held and exits immediately.
  • “How do you know a cron job stopped running?” — alone, you don’t. Use a heartbeat service (healthchecks.io / cronitor) — the job pings on success, the service alerts if no ping arrives on schedule.
  • “Cron vs systemd timers — when each?” — cron for one-line drops on universal Linux. Timers for production systemd setups, dependency tracking, missed-run replay (Persistent=true), unified journalctl logging.