Skip to content

operations.md

Latest commit

 

History

History
234 lines (177 loc) · 12 KB

operations.md

File metadata and controls

234 lines (177 loc) · 12 KB

Operations & Maintenance

Everything you do to the stack runs through pithead. Run ./pithead help at any time to see the full list.

Command reference

Command Description
./pithead setup First-time setup (interactive): dependency check, config, Tor provisioning, kernel optimization, and start. --skip-optimize skips kernel/GRUB tuning; --skip-deps skips the dependency check/install.
./pithead apply Preview and apply config.json changes — warns before disruptive ones and recreates only what changed. -y / --yes skips the prompt.
./pithead up Start the stack.
./pithead down Stop the stack.
./pithead restart Restart the stack.
./pithead upgrade Re-render the generated config, then pull (release bundle) or rebuild (source checkout) the images and restart. Run after downloading a newer bundle or a git pull.
./pithead logs [service] Follow logs for all containers, or a single service (e.g. logs p2pool).
./pithead status Show container status and health-check every expected service — warns about anything down/unhealthy and exits non-zero if so (handy for cron/monitoring). Profile-aware, and treats a stopped p2pool/xmrig-proxy as intentional during a node-down failover or while the miner is held until the chains sync.
./pithead doctor Read-only diagnostics: deps, Docker, AVX2, HugePages, RAM/disk, .env/onion state, and container status — a paste-able health report.
./pithead backup Save config.json, .env, Caddyfile, the Tor onion keys, and the dashboard's database (your hashrate history & settings) to a timestamped tar.gz under backups/ (checks free space first; stops a running stack for a clean copy, then restarts it). --with-chains also includes the blockchain data; -y / --yes skips the prompts (low free space and stopping the stack).
./pithead restore <archive> Restore those files from a backup archive (asks before overwriting; fixes Tor key ownership). -y / --yes skips the prompt.
./pithead reset-dashboard DESTRUCTIVE — wipes and recreates the dashboard and P2Pool data. -y / --yes skips the prompt.
./pithead help Show all commands.

Service names for logs match the containers: monerod, p2pool, tari, xmrig-proxy, tor, dashboard, docker-proxy, docker-control, caddy.

Day-to-day

Check status and watch logs:

./pithead status
./pithead logs                 # everything
./pithead logs p2pool          # one service

status prints the usual compose table, then a per-service health check — a green ✓ for each running (and healthy) service, and a ⚠/✗ for anything unhealthy, restarting, stopped, or missing. It exits non-zero when something needs attention, so you can wire it into a cron/monitoring check. A stopped p2pool/xmrig-proxy is reported as intentional, not an error: the dashboard stops it either to fail workers over a node-down outage or while the miner is held until the required chains finish their initial sync — check the dashboard to see which.

Start / stop / restart:

./pithead up
./pithead down
./pithead restart

Change a setting: edit config.json, then ./pithead apply. See Configuration › Changing settings later.

Reboot resilience: every service runs with restart: unless-stopped, so the whole stack comes back on its own after a reboot or power loss — provided the Docker daemon itself starts at boot. On Ubuntu's packaged Docker that's the default, but a custom/rootless install (or setup --skip-deps) may leave it disabled. ./pithead doctor checks this and warns if Docker isn't boot-enabled; the fix is sudo systemctl enable --now docker.

Updating the stack

How you update depends on how you installed (see Getting Started).

Release bundle (the default): from your install directory, re-download the latest bundle over it, then upgrade — upgrade pulls the new published images:

curl -fsSL https://github.com/p2pool-starter-stack/pithead/releases/latest/download/pithead.tar.gz | tar xz --strip-components=1
./pithead upgrade

Source checkout: pull the latest code, then upgrade — upgrade rebuilds the images locally:

git pull
./pithead upgrade

Either way, upgrade re-renders the generated config (.env, the Caddyfile, and the Tari config) for the new release before pulling/rebuilding — so a release that changes a config template or adds an .env var takes effect, not just the new image. Your data directories and config.json are untouched, so your blockchain sync and settings are preserved across upgrades.

Switching a source checkout to release images

If you cloned the repo (so the dashboard badge reads dev · branch @ commit) and would rather run the published, tested images — a clean version badge, a working update-checker, and no local build — convert the install in place. Your config.json, .env, Tor onion keys, and data directories are all preserved:

./pithead backup -y          # safety snapshot: config.json, .env, onion keys, dashboard db (chains excluded)
# overlay the published release files — leaves config.json, .env, .git, and your data dirs untouched:
curl -fsSL https://github.com/p2pool-starter-stack/pithead/releases/latest/download/pithead.tar.gz | tar xz --strip-components=1
rm -f build/*/Dockerfile     # remove the image Dockerfiles → pithead switches from build to pull
./pithead upgrade            # re-render config, pull :vX.Y.Z, recreate

pithead chooses build-vs-pull by whether the image Dockerfiles are present (build/<svc>/Dockerfile) — deleting them is what flips it from building :dev locally to pulling the published :vX.Y.Z. To go back to building from source, git checkout vX.Y.Z (or git pull) restores the Dockerfiles, then ./pithead upgrade rebuilds locally again. The new validation in upgrade will refuse to start on a non-primary Monero payout address, so confirm yours is a 4…/95-char address first (see Configuration).

Moving the install? Data directories are stored as absolute paths in .env, so relocating or copying the stack to a different path (or running a second checkout) points it at a different, empty data/ — a full re-sync, and the dashboard history is orphaned. If you move the install, move its data/ with it, or set absolute data_dir paths in config.json and run ./pithead apply. pithead up and pithead doctor now warn when a data directory named in .env is missing.

Backups

Your important state lives in the data directories (by default under ./data/, or wherever you pointed each *.data_dir — see Configuration › Data directories):

  • config.json — your settings (keep a copy somewhere safe; it's chmod 600).
  • data/tor/ — your onion service keys. Back these up if you want to keep the same onion addresses across a rebuild.
  • data/monero/, data/tari/ — the blockchains. Large, but backing them up saves a re-sync; they can also be re-downloaded from the network if lost.
  • data/dashboard/ — the dashboard's database: your hashrate history and settings. Small but irreplaceable (it doesn't re-sync), so it's part of the default backup.

Stop the stack (./pithead down) before copying data directories so files are in a consistent state.

backup / restore

Rather than copying files by hand, let pithead do it for you:

./pithead backup

That's it. This saves the things you can't get back — your config.json, your secrets (.env), your Caddyfile (if you have one), the Tor onion address keys, and the dashboard's database (your hashrate history and settings) — into a small, timestamped file under backups/. It's quick and small because your blockchains are not included (they just re-sync). The archive is locked down to chmod 600, and pithead prints its path when it's done. Before it writes anything, it checks there's room — if free space looks tight, it asks before going ahead so a backup can't quietly fill your disk.

If the stack is running, backup offers to briefly stop it for a clean copy and start it again when it's done. Pass -y / --yes to skip the prompts (the low-space warning and the stop-the-stack question) and just back up.

Optional extras (you usually don't need these):

./pithead backup --with-chains   # also include the blockchain data — much bigger and slower

To recover — on a new machine, or after a wipe — copy the archive back and run:

./pithead down                       # stop the stack first so files restore cleanly
./pithead restore backups/pithead-backup-YYYYmmdd-HHMMSS.tar.gz
./pithead up

restore always asks before it overwrites anything, so you can change your mind (pass -y / --yes to skip that prompt). It puts the files back where they belong and sorts out the Tor key ownership for you, so your onion address comes back exactly as it was — and your hashrate history and dashboard settings come back too.

Note: After a restore, the dashboard's HTTPS certificate is regenerated, so your browser may show its "not trusted" warning once — that's expected; accept it as you did on first setup.

Troubleshooting

The dashboard is stuck on Sync Mode. This usually just means a chain is still downloading. Confirm steady progress:

./pithead logs monerod
./pithead logs tari

If a node looks genuinely stalled (no new blocks over a long period), restart it with ./pithead restart. To avoid the wait entirely, point the stack at an existing synced blockchain or a remote node — see Configuration › Reusing an existing node.

Tari shows high memory use. Usually nothing to worry about — most of it is reclaimable disk cache, not a leak. Tari has an auto-sized memory limit (tari.mem_limit) that keeps a genuine runaway from affecting the rest of the stack. Only change it if Tari restarts repeatedly (give it more) or you want to free RAM for other apps (give it less), then run ./pithead apply.

Browser warns "your connection is not private." Expected with dashboard.secure: true — Caddy uses a self-signed certificate. Accept the warning once. To use plain HTTP, set dashboard.secure: false and run ./pithead apply.

Workers don't show up in the dashboard. Check that each rig points at YOUR_STACK_IP:3333 and that port 3333 is reachable from the worker (firewall on the stack host?). See Connecting Miners.

Hashrate reads zero or the chart is blank. Give it a minute after a worker connects for stats to populate. Confirm the worker is actually hashing (./pithead logs xmrig-proxy).

P2Pool can't connect to a remote node. The node must be set up for mining — ZMQ publishing enabled (zmq-pub) and its RPC reachable by P2Pool. Public "open node" endpoints don't qualify — use a node you run and control. See Configuration › Connecting to a remote Monero node.

HugePages shows as disabled / low. Persistent HugePages require a GRUB change and a reboot. Re-run ./pithead setup (without --skip-optimize) and reboot when prompted.

Something looks broken in the dashboard data and you want a clean slate. ./pithead reset-dashboard wipes and recreates the dashboard and P2Pool data. This is destructive — you'll lose P2Pool sidechain state and dashboard history (your blockchains and wallets are unaffected). Pass -y / --yes to skip the confirmation prompt.

For developers

  • Run the test suites locally (mirrors CI): make test — or individually make test-dashboard, make test-stack, make test-compose, make lint.
  • Dashboard development — see build/dashboard/README.md for the package layout, local dev setup, and the hermetic test suite.