Architecture¶

flowchart TB
    Browser["Browser (HTMX + Alpine)"] -->|HTTPS| Caddy
    Caddy --> FastAPI["FastAPI (web)"]

    subgraph Core["Docker Compose (supported)"]
        FastAPI --> DB[(PostgreSQL)]
        FastAPI --> Scheduler["APScheduler"]
        FastAPI --> Celery["Celery worker(s)"]
        FastAPI --> CeleryNmap["celery-worker-nmap (profile nmap)"]
    end

    Scheduler -->|backup cron| Celery
    Scheduler -->|patch / check enqueue| Celery
    Scheduler -->|nmap schedules / stale cleanup| Celery
    Scheduler -->|host facts · herder backup| Celery
    Celery -->|reads/writes| DB
    Celery -->|SSH · apt · docker · rsync · self-backup| PiFleet["Remote fleet"]
    CeleryNmap -->|nmap -oX / vuln pack| LAN["Configured LAN CIDR(s)"]
    CeleryNmap -->|reads/writes| DB
    FastAPI -->|console · files| PiFleet
    FastAPI -->|DB reads for UI| DB
    FastAPI -.->|Job.details progress| Celery
    FastAPI -.->|enqueue exclusive_job| Celery
    FastAPI -.->|enqueue -Q nmap| CeleryNmap

Job execution paths¶

Work	Runs on	Concurrency rule
Backups	Celery	Parallel across hosts; one backup per host (Redis mutex)
Backup copy (`backup_replicate`)	Celery default queue	Google Drive, OneDrive, or a LAN share. Backup copy is the herder-folder hop. Direct copy is Copy now or the schedule for a host that sends files straight out. A folder row left running does not hold that direct hop. A second folder copy still waits on the folder row. Backup now on the host stays job type `backup`. A self-backup archive is Self-backup copy and does not take the slot. Up to 7 days. Acked on receive. Not the per-host backup lock. Web only enqueues
Move (`service_migrate`), fail-path undo (`service_migrate_undo`), and dest-up recover (`service_migrate_dest_recover`)	Celery	Dual-host backup mutex; recycle web is safe; recycle worker fails a running Move, undo, or recover. Undo only after cutover / rebind / validate failed. Dest-up recover is only a worker death during `dest_up`: inspect, `compose stop` dest, start source, no DNS change
OS/container patch, update checks, stack jobs, template jobs, `host_reboot`	Celery default queue (`exclusive_job`)	One active job of that type per host. Stack writes share one lane. Reboot is also refused while OS patch, container patch, or backup is active. No backup mutex. Host-down stays pending. Worker recycle fails a running job
Bulk fleet actions	Web → same enqueue paths	Feature-flag skip + exclusive rules
LAN nmap scans / vuln pack update	celery-worker-nmap (`-Q nmap`, concurrency 1)	Opt-in profile; host network; `PIHERDER_NMAP_WORKER=1` only here
Stale Jobs/Audit/nmap-run purge	Celery (default queue)	Opt-in Settings schedule
`host_facts`	Celery default queue (`exclusive_job`)	One active snapshot per host, about every 15 minutes. SSH down stays pending. A web recycle does not fail it. Each run writes a Jobs row and an Audit row. The default Jobs and Audit lists leave a successful snapshot out. A failed snapshot stays visible
`retention`, `herder_backup`	Celery default queue (`housekeeping_job`)	No host exclusive slot. One self-backup at a time. A web recycle does not fail them. Worker cap is 2 hours

Nmap privilege boundary: web + main celery set PIHERDER_NMAP_WORKER=0 in compose; tasks call worker_guard and refuse if marker is off or nmap is missing. Never put queue nmap on the main worker. See env reference · .env.example.

Key modules (pointers)¶

Concern	Location
Roles / middleware	`app/security/auth.py` — expired session: HTML 303 `/auth/login`, HTMX `HX-Redirect`, `/api/v1` JSON 401
Password policy	`app/services/password_policy.py` · Settings Security
Account / 2FA step-up policy	`app/services/account_stepup.py` · Settings Security
Web SSH console	`app/services/ssh_console.py` · `app/routers/server_console.py` · Settings Console (timeouts) / Security (factors). Mux-1: `Server.console_mux_enabled` + probe tmux/screen. Mux-2: SSH access lists and kills leftover `ph-u*` for that host (`POST /servers/{id}/ssh/mux-sessions`)
Jobs / progress / exclusive types	`app/services/jobs/` (`service.py`; package preserves `patch.object` surface). Move: `app/services/jobs_migrate.py`. Jr-1 handoff: `app/services/jobs_exclusive.py`. Backups, Move, and exclusive_job on Celery (`app/tasks.py`)
Reports layout (N3a) + Move card (N3b) + card catalog (N3c first slice)	`app/services/report_layout.py` · cookie `ph_reports_layout` · `POST /reports/layout` · the Cards row is the same six ids. Move stats from `ops_reports.collect_move_history` (`service_migrate` Jobs)
Service migrate pipeline	`app/services/service_migrate/` · Celery `app.tasks.service_migrate` · undo `undo.py` + `app.tasks.service_migrate_undo` · dest-up recover `dest_up_recover.py` + `app.tasks.service_migrate_dest_recover`
CSP	`app/security/headers.py` — per-request script nonce; `script-src-attr 'none'` (Slice 2); demo Report-Only unless `PIHERDER_CSP_ENFORCE`
Docker unused cleanup HTML	`app/services/docker_unused_html.py`
Per-server backup lock	`app/services/server_job_lock.py`
Scheduler	`app/services/scheduler.py`. About reads the latest matching job and the next fire from `app/services/scheduler_visibility.py` and opens them in a modal grouped by type. A restart does not clear the last run
Backup	`app/services/backup.py` (+ progress, profiles)
Drive copy	`app/services/backup_replicate.py` · `app/routers/backup_copies.py` · `app/tasks.py` `replicate_backup` · Alembic 047 · Settings → PiHerder backup. Google web OAuth client, refresh token in Fernet. Not in the 1.7.0 image
Docker inventory	`app/services/docker_inventory.py`
Host OS / hardware / CPU / RAM / disk snapshot	`app/services/host_facts.py` · Alembic 044 + 045 · System Info modal (DB first; icon refresh) · scheduler ~15 min · same columns as `/api/v1` + HACS
Templates (domain)	`app/services/service_templates/` — `deploy`, `host_sync` (adopt/migrate), `harden`, `schema`, `from_host`, …
Compose project files (pure)	`app/services/compose_project_files.py` — file kinds, sidecar discovery, desired→live merge (no SSH)
Compose editor workspace	`app/services/compose_editor.py` — inventory/fallback path, live files, template sidecars, drafts
Docker versions / live files	`app/services/docker_versions.py` — SFTP multi-file I/O + discovery via `compose_project_files`
Integrations (domain)	`app/services/integrations/`
Integrations (HTTP)	`integrations.py` + `integrations_common` / `_kuma` / `_grafana` / `_pihole` / `_npm` / `_nmap`
LAN nmap (scan/parse/schedules/vuln)	`app/services/nmap/` (`worker_guard`, `scan`, `device_ops`, `fabric_projection`, …) · router `integrations_nmap.py` · image `Dockerfile.nmap`
Stale data cleanup	`app/services/stale_data_cleanup.py` · Settings General
Templates (HTTP)	`templates_common` + `templates_svc` (catalog) + `templates_deploy`. On the fleet reads `StackDeployment` for that template id or slug.
Auth (HTTP)	`auth.py` + `auth_users.py` (admin users)
Network maps	`app/services/dns_fabric/` (`core`, `mesh_physical`, `mesh_logical`, `ports`, `stack_panel`) · `app/routers/dns.py`
Published port chips	`app/services/dns_fabric/ports.py` — host→container parse for stack panel
Operator pins + host jump	`app/services/nav_shortcuts.py` · `app/routers/favourites.py` · model `UserFavourite` · partials `pin_button` / `host_switcher` / `host_feature_nav`
Human-readable cron	`app/services/cron_human.py` — Jinja `cron_human` filter + `cron_presets` global
Certificates / deploy targets	`app/services/certificates.py` · `app/routers/certificates.py` — vault, stage+sudo, verify, wizard
Ops-hero pulse helpers	`app/services/ops_pulse.py`
Instance name, accent, Catalog nav	`app/services/instance_brand.py` · Settings → General → Instance · `POST /herder-backups/instance`. Demo forces official chrome.
Push	`app/services/push.py`
API tokens	`app/services/api_tokens.py`, `app/routers/api_v1.py`. Hosted MCP is `app/routers/mcp.py` + `app/services/mcp_hosted.py` (`POST /mcp`). A pasted Bearer token still works. Browser sign-in is `app/services/mcp_oauth.py` (Alembic 048) and only applies to that hosted URL. The stdio client piherder-mcp calls `/api/v1` with `PIHERDER_TOKEN` and does not run the OAuth flow. Adapter 0.4.4 lists `list_alerts`, `prune_build_cache`, `replace_stale_mac`, `fleet_summary`, `prune_docker`, `start_move`, `read_discovery`, `start_discovery`, and the LAN Discovery device tools, and sends `confirm=true` on purge. `fleet_summary` returns 404 on the 1.13.0 image. Adapter 0.4.3 does not list those four. Adapter 0.4.2 does not list `prune_docker`. Adapter 0.4.1 omits `confirm=true` on purge. Adapter 0.4.0 does not list the device tools. `uvx` installs 0.4.4. Home Assistant on this train is piherder-ha 0.5.4 (poll-only Slice 3; 0.5.4 serves the card from `/piherder/` as a Lovelace resource). Tagged v1.8.0 paired with plugin 0.4.4. v1.7.0 paired with 0.3.0. Neither repo is inside this image. Operator pages: Agents (MCP) · Home Assistant
Herder backup	`app/services/herder_backup.py`
Metrics	`app/services/metrics.py`
Bulk server actions	`app/routers/servers.py` (`POST /servers/bulk`)
Server SSH / patch sub-routers	`server_ssh.py`, `server_patch.py`, `server_common.py` (mounted under `/servers`)
Trusted client IP	`app/services/request_ip.py` — honour XFF/CF only from `PIHERDER_TRUSTED_PROXY_CIDRS`
SSH host-key pin (TOFU)	`app/services/ssh.py` (`HostKeyPinPolicy`) · columns `ssh_hostkey_*` · reset in `server_ssh.py`
Weak `SECRET_KEY`	`app/main.py` — refuse boot unless `PIHERDER_ALLOW_INSECURE` / `DEMO_MODE`
Docker UI	`server_docker.py` + `server_docker_compose.py` (thin; editor load in `compose_editor`)
Theme / map / ops CSS	`themes.css`, `fabric.css` (mesh), `fabric-stack.css`, `dns-hub.css`, `ops.css`, `ops-auth.css`, `ops-pages.css` (`ph-dense-*` lists)
Map / stack client	`fabric-mesh.js` (map open/closed + pan/zoom; `#map` / `preferMapOnLoad`) · `fabric-stack-panel.js` (stack drawer + one pointer reorder path)
App timezone display	`app/services/app_settings.describe_timezone` · Settings General card
Large templates	Prefer `partials/` — e.g. `server_detail_*_modals`, `docker_modals`, `settings_{tab}`, pin/host switcher partials

Frontend stack¶

Server-rendered Jinja2 + HTMX fragments + Alpine for small widgets
Compiled Tailwind (app/static/css/tailwind.css via scripts/build-tailwind.sh) + vendored HTMX / Alpine (no runtime CDN, no Play, no unsafe-eval)
Progressive enhancement vanilla JS for Network maps, job hold, push, compose editor
Shared ops-hero grid contract (ops.css): full main content width; desktop title left · viz right (≥768px); mobile viz under title
One list markup per surface — dense rows (ph-dense-*, nmap sched cards) reflow with CSS; avoid dual mobile-table + desktop-table DOM
Shell nav: single nav_items / secondary_items source → desktop bar + mobile drawer (chrome only)
Mobile orientation reflow in base.html; service rows stack actions on narrow viewports
Auth pages (login/register + force-password / 2FA) use shared auth-stage chrome
Empty DB → first register is admin; no default password user; then registration closes
Session cookies set Secure when PIHERDER_PUBLIC_URL is https:// (or COOKIE_SECURE=true)

Design principles¶

Privileged actions audited (incl. client IP)
Secrets encrypted at rest; decrypt only in memory for jobs
Offline/air-gapped ready once built (vendored assets)
External/dangerous actions opt-in: preview → confirm → audit
One exclusive OS/container job type per host (no silent double SSH)
Thin routers where practical; domain logic in app/services/
Compose-first deployment; DB-first operational settings
Integrations optional — core fleet never depends on Kuma/Grafana/NPM/Pi-hole