Frank Schwenk
fd024d31e9
CI / lint-and-test (pull_request) Successful in 2m3s
Dev setup (venv, ruff, pytest), config table entries, architecture overview, and cleanup completion report. Co-authored-by: Cursor <cursoragent@cursor.com>
80 lines
3.0 KiB
Markdown
80 lines
3.0 KiB
Markdown
# Architecture — live.f12.rocks
|
|
|
|
## Overview
|
|
|
|
```
|
|
Phone/Syncthing Docker host
|
|
─────────────── ───────────
|
|
incoming/ ──watch──► worker ──► variants/ + intermediates/
|
|
│ │
|
|
└── rembg + gmic pipeline
|
|
│
|
|
meta/{stem}.json ◄──────────────────────┘
|
|
│
|
|
▼
|
|
web (FastAPI) ──► browser UI + remix POST
|
|
```
|
|
|
|
Two containers share one image (`Dockerfile`) and one bind-mounted data
|
|
directory (`DATA_HOST_DIR` → `/data`). Syncthing runs on the host, not in
|
|
compose.
|
|
|
|
## Python modules
|
|
|
|
| Module | Role |
|
|
|--------|------|
|
|
| `app/worker.py` | Poll `incoming/`, copy to `variants/`, run `process_job()` |
|
|
| `app/pipeline.py` | gmic/rembg orchestration, meta/manifest I/O, variant compose |
|
|
| `app/remix.py` | User-selected remix variant (one-shot compose + manifest append) |
|
|
| `app/main.py` | FastAPI routes, Jinja templates, static files |
|
|
| `app/config.py` | Environment → constants (paths, timeouts, tuning) |
|
|
| `app/rembg_cli.py` | Subprocess entry for rembg (avoids heavy `rembg[cli]` extra) |
|
|
| `app/io_utils.py` | Atomic JSON read/write |
|
|
| `app/env_utils.py` | Shared bool env parsing |
|
|
| `app/logging_config.py` | Entry-point logging setup |
|
|
|
|
## Data model
|
|
|
|
One **job** = one sanitized stem (from incoming filename). Flat files:
|
|
|
|
- `variants/{stem}_original.jpg` — preprocessed copy
|
|
- `variants/{stem}_rembg.png` — background removal
|
|
- `variants/{stem}_vN.png` — final composed images
|
|
- `intermediates/{stem}_vN_*.png` — step images (kept for debugging)
|
|
- `meta/{stem}.json` — status + manifest (variants list, filter metadata)
|
|
- `processed.json` — worker bookkeeping (incoming path → handled)
|
|
|
|
## Pipeline origin
|
|
|
|
The gmic filter/blend chain is ported from an external reference script
|
|
`make_random.py` (not in this repo). `config.POST_FILTERS` and asset lists
|
|
under `assets/` mirror that script's behaviour.
|
|
|
|
## Background removal choice
|
|
|
|
`compare-bg/` is a **standalone benchmark** (rembg models vs withoutbg).
|
|
It is not part of the runtime stack. Production uses `u2net` + alpha
|
|
matting via `app/rembg_cli.py` (see `config.REMBG_MODEL`).
|
|
|
|
## Known limitations (intentional)
|
|
|
|
- **No web auth** — anyone with the link sees all jobs (event tool).
|
|
- **Sequential worker** — one photo at a time; burst uploads queue.
|
|
- **Meta JSON** — worker and web both read-modify-write `meta/*.json`
|
|
without file locking; concurrent remix during active processing can race.
|
|
- **Incoming is append-only** — worker never deletes from `incoming/`.
|
|
|
|
## Dev / test
|
|
|
|
```bash
|
|
python3.11 -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements-dev.txt
|
|
ruff check app/ tests/
|
|
pytest -q
|
|
```
|
|
|
|
Full stack (gmic/rembg): `docker compose up --build` with `DATA_HOST_DIR=./data`.
|
|
|
|
Python **3.11** matches the Docker image; host 3.14+ cannot install pinned
|
|
`onnxruntime` — use Docker or a 3.11 venv for parity.
|