Files

T

CI / lint-and-test (pull_request) Failing after 9s

Details

fix: lock meta RMW and harden concurrent remix paths

Prevent worker/web clobbering of meta variants via flock and merge-by-id,
make filter timeouts thread-local, harden job-id/stem sanitization, migrate
TemplateResponse API, and remove the compare-bg experiment.

Co-authored-by: Cursor <cursoragent@cursor.com>

2026-07-18 17:44:15 +02:00

3.0 KiB

Raw Blame History

Architecture — live.f12.rocks

Overview

Phone/Syncthing                Docker host
───────────────                ───────────
incoming/  ──watch──►  worker  ──►  variants/ + intermediates/
                              │         │
                              └── rembg + gmic pipeline
                                        │
meta/{stem}.json ◄──────────────────────┘
       │
       ▼
     web (FastAPI)  ──►  browser UI + remix POST

Two containers share one image (Dockerfile) and one bind-mounted data directory (DATA_HOST_DIR → /data). Syncthing runs on the host, not in compose.

Python modules

Module	Role
`app/worker.py`	Poll `incoming/`, copy to `variants/`, run `process_job()`
`app/pipeline.py`	gmic/rembg orchestration, meta/manifest I/O, variant compose
`app/remix.py`	User-selected remix variant (one-shot compose + manifest append)
`app/main.py`	FastAPI routes, Jinja templates, static files
`app/config.py`	Environment → constants (paths, timeouts, tuning)
`app/rembg_cli.py`	Subprocess entry for rembg (avoids heavy `rembg[cli]` extra)
`app/io_utils.py`	Atomic JSON read/write
`app/env_utils.py`	Shared bool env parsing
`app/logging_config.py`	Entry-point logging setup

Data model

One job = one sanitized stem (from incoming filename). Flat files:

variants/{stem}_original.jpg — preprocessed copy
variants/{stem}_rembg.png — background removal
variants/{stem}_vN.png — final composed images
intermediates/{stem}_vN_*.png — step images (kept for debugging)
meta/{stem}.json — status + manifest (variants list, filter metadata)
processed.json — worker bookkeeping (incoming path → handled)

Pipeline origin

The gmic filter/blend chain is ported from an external reference script make_random.py (not in this repo). config.POST_FILTERS and asset lists under assets/ mirror that script's behaviour.

Background removal

Production uses u2net + alpha matting via app/rembg_cli.py (see config.REMBG_MODEL).

Known limitations (intentional)

No web auth — anyone with the link sees all jobs (event tool).
Sequential worker — one photo at a time; burst uploads queue.
Meta JSON — worker and web share meta/*.json via locked read-modify-write (fcntl.flock); variants are merged by id so concurrent remix during processing does not clobber entries.
Incoming is append-only — worker never deletes from incoming/.

Dev / test

python3.11 -m venv .venv && source .venv/bin/activate
pip install -r requirements-dev.txt
ruff check app/ tests/
pytest -q

Full stack (gmic/rembg): docker compose up --build with DATA_HOST_DIR=./data.

Python 3.11 matches the Docker image; host 3.14+ cannot install pinned onnxruntime — use Docker or a 3.11 venv for parity.

3.0 KiB Raw Blame History