Adding a service¶

Once Caddy is fronting the traffic, adding a new service on the instance is: a binary or script + a systemd unit + a Caddyfile block. This chapter shows the pattern.

The pattern¶

Client
  │
  ▼ HTTPS
Caddy (:443)
  │  reverse_proxy localhost:8000
  ▼
Your service (localhost:8000)
  │  systemd-managed
  ▼
Data (SQLite, files, whatever)

Caddy handles TLS + hostname routing. Your service listens on localhost (unreachable from outside). systemd handles start/stop/restart and logging.

Example: a Go binary¶

Say you have a Go web app that listens on localhost:8000. Deploy it to /opt/myapp/myapp and give it a systemd unit at /etc/systemd/system/myapp.service:

[Unit]
Description=MyApp — home lab web service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/myapp
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal

# Modest hardening
NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/myapp/data /var/log/myapp
CapabilityBoundingSet=

# Environment (or use EnvironmentFile= for secrets)
Environment=PORT=8000
Environment=ENV=production

[Install]
WantedBy=multi-user.target

Create the service user (one-time):

sudo useradd --system --home /opt/myapp --shell /usr/sbin/nologin myapp
sudo mkdir -p /opt/myapp/data
sudo chown -R myapp:myapp /opt/myapp

Enable and start:

sudo systemctl daemon-reload
sudo systemctl enable --now myapp
sudo systemctl status myapp

Watch logs:

sudo journalctl -u myapp -f

Front it with Caddy¶

Add to /etc/caddy/Caddyfile:

myapp.example.com {
    reverse_proxy localhost:8000
}

Reload:

sudo systemctl reload caddy

Wait for the cert to be issued (journalctl -u caddy -f), then:

curl -sI https://myapp.example.com

Done.

Example: a Python service¶

Same pattern, different unit. Given a FastAPI app at /opt/myapp/app.py:

[Unit]
Description=MyApp — FastAPI service
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=myapp
Group=myapp
WorkingDirectory=/opt/myapp
ExecStart=/opt/myapp/venv/bin/uvicorn app:app --host 127.0.0.1 --port 8000
Restart=on-failure
RestartSec=5
StandardOutput=journal
StandardError=journal
EnvironmentFile=/opt/myapp/.env

NoNewPrivileges=true
PrivateTmp=true
ProtectSystem=strict
ProtectHome=true
ReadWritePaths=/opt/myapp/data

[Install]
WantedBy=multi-user.target

Set up the venv (as myapp user):

sudo -u myapp python3 -m venv /opt/myapp/venv
sudo -u myapp /opt/myapp/venv/bin/pip install fastapi uvicorn

Environment files for secrets¶

Never put secrets in the systemd unit itself (they'd be world-readable via systemctl cat). Use an EnvironmentFile:

sudo tee /opt/myapp/.env >/dev/null <<'EOF'
DATABASE_URL=sqlite:///data/app.db
API_KEY=super-secret-key
SESSION_SECRET=another-secret
EOF

sudo chown root:myapp /opt/myapp/.env
sudo chmod 640 /opt/myapp/.env

The systemd unit references it with EnvironmentFile=/opt/myapp/.env. The service reads them as environment variables.

Deploy script template¶

Save on your local machine as deploy.sh:

#!/usr/bin/env bash
set -euo pipefail

REMOTE=${REMOTE:-arm-1}     # SSH host
REMOTE_PATH=${REMOTE_PATH:-/opt/myapp}
SERVICE=${SERVICE:-myapp}

echo "=== Building ==="
# your build step, e.g., `go build`, `npm run build`, etc.
go build -o dist/myapp ./cmd/myapp

echo "=== Rsync to $REMOTE ==="
rsync -av --delete dist/ "$REMOTE:/tmp/${SERVICE}-deploy/"

echo "=== Installing ==="
ssh "$REMOTE" bash -s <<REMOTE_SCRIPT
set -euo pipefail
sudo rsync -av --delete /tmp/${SERVICE}-deploy/ ${REMOTE_PATH}/
sudo chown -R ${SERVICE}:${SERVICE} ${REMOTE_PATH}
sudo systemctl restart ${SERVICE}
sleep 2
sudo systemctl status ${SERVICE} --no-pager | head -10
REMOTE_SCRIPT

echo "=== Health check ==="
curl -sSI "https://myapp.example.com/health" | head -1

Run with ./deploy.sh. This is roughly the shape of the deploy scripts used in real lab setups.

Reading logs¶

systemd's journal is where all your service logs go:

sudo journalctl -u myapp             # everything, oldest first
sudo journalctl -u myapp -n 50       # last 50 lines
sudo journalctl -u myapp -f          # tail
sudo journalctl -u myapp --since "1 hour ago"
sudo journalctl -u myapp -p err      # only errors

For persistent logs across reboots (default is memory-only on some Ubuntu images):

sudo mkdir -p /var/log/journal
sudo systemd-tmpfiles --create --prefix /var/log/journal
sudo systemctl restart systemd-journald

Now journalctl shows logs from previous boots (journalctl --list-boots).

Rotating logs¶

systemd-journald handles rotation automatically based on /etc/systemd/journald.conf. Reasonable defaults for a lab:

[Journal]
SystemMaxUse=500M
SystemMaxFileSize=50M
MaxRetentionSec=1month

After editing: sudo systemctl restart systemd-journald.

Watchdog: restart if unhealthy¶

If your service exposes a /health endpoint, wire it up:

[Service]
Type=notify
NotifyAccess=main
WatchdogSec=30
Restart=on-watchdog

Your service must call sd_notify(0, "WATCHDOG=1") every ~15 seconds (half of WatchdogSec). Most languages have systemd notify libraries.

For simpler cases, just Restart=on-failure with RestartSec=5 is enough — systemd will restart on process crashes; anything more sophisticated needs Restart=on-watchdog or an external monitor.

Next¶

Head to Staying always-free for the checklist that keeps this whole setup at $0/month.