livef12rocks/ARCHITECTURE.md

Architecture — live.f12.rocks

Overview

Organizer phone                 Docker host
──────────────                  ───────────
Share / pick
  → JPEG 2000px
  → POST /api/jobs  ──►  incoming/  ──watch──►  worker
        token                              │
                                           └── rembg + gmic pipeline
                                                     │
meta/{stem}.json ◄───────────────────────────────────┘
       │
       ├── web (HTML)  ──►  guest browsers + remix
       └── GET /api/jobs/{id}  ──►  app polls, local notification,
                                    save variants + intermediates

Two containers share one image (Dockerfile) and one bind-mounted data directory (DATA_HOST_DIR → /data). The organizer app is in android/. Syncthing is an optional fallback writer for incoming/, not required.

Python modules

Module	Role
app/worker.py	Poll incoming/, copy to variants/, run process_job()
app/pipeline.py	gmic/rembg orchestration, meta/manifest I/O, variant compose
app/remix.py	User-selected remix variant (one-shot compose + manifest append)
app/main.py	FastAPI HTML routes, Jinja templates, static files
app/api.py	JSON API: health, list, detail, authenticated upload
app/jobs.py	Shared job payloads, labels, inbox filenames
app/config.py	Environment → constants (paths, timeouts, UPLOAD_TOKEN)
app/rembg_cli.py	Subprocess entry for rembg (avoids heavy rembg[cli] extra)
app/io_utils.py	Atomic JSON read/write
app/env_utils.py	Shared bool env parsing
app/logging_config.py	Entry-point logging setup
app/images.py	On-demand WebP srcset cache

Data model

One job = one sanitized stem (unique upload name). Flat files:

• variants/{stem}_original.jpg — preprocessed copy
• variants/{stem}_rembg.png — background removal
• variants/{stem}_vN.png — final composed images
• intermediates/{stem}_vN_*.png — step images (kept for debugging)
• meta/{stem}.json — status + stage + progress + manifest
• processed.json — worker bookkeeping (incoming path → handled)

Status values: queued → processing (stages preprocess / rembg / variant) → done | error.

API

Method	Path	Auth	Notes
GET	/api/health	no	{ok, upload}
GET	/api/jobs	no	list + active flag
GET	/api/jobs/{id}	no	full payload + file URLs
POST	/api/jobs	Bearer / X-Upload-Token	multipart file, 202 + job summary
GET	/jobs/{id}/files/{name}	no	original file or ?w= WebP

Empty UPLOAD_TOKEN → POST returns 503. Wrong token → 401.

Pipeline origin

The gmic filter/blend chain is ported from an external reference script make_random.py (not in this repo). config.POST_FILTERS and asset lists under assets/ mirror that script's behaviour.

Background removal

Production uses u2net + alpha matting via app/rembg_cli.py (see config.REMBG_MODEL).

Known limitations (intentional)

• No web login — anyone with the link sees all jobs (event tool).
• Upload token — organizer app only; not an account system.
• Sequential worker — one photo at a time; burst uploads queue.
• Meta JSON — worker and web share meta/*.json via locked read-modify-write (fcntl.flock); variants are merged by id so concurrent remix during processing does not clobber entries.
• Incoming is append-only — worker never deletes from incoming/.

Dev / test

python3.11 -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
ruff check app/ tests/
pytest -q

Android: see android/README.md. Full stack (gmic/rembg): docker compose up --build with DATA_HOST_DIR=./data.

Python 3.11 matches the Docker image; host 3.14+ cannot install pinned onnxruntime — use Docker or a 3.11 venv for parity.