Frank Schwenk
fd024d31e9
CI / lint-and-test (pull_request) Successful in 2m3s
Dev setup (venv, ruff, pytest), config table entries, architecture overview, and cleanup completion report. Co-authored-by: Cursor <cursoragent@cursor.com>
107 lines
4.3 KiB
Markdown
107 lines
4.3 KiB
Markdown
# live.f12.rocks
|
|
|
|
Event photo pep pipeline: guests drop a photo into a Syncthing folder, a
|
|
worker runs it through `rembg` (background removal) + `gmic` (random
|
|
filter/blend compositing, ported from `make_random.py`), and a small web
|
|
UI shows the results with a "remix" option to try different filters on
|
|
the same photo. Finished jobs sync back via the same share.
|
|
|
|
No web auth. This is an event tool, not a DAM.
|
|
|
|
## Services (`compose.yml`)
|
|
|
|
| Service | What | Port |
|
|
|----------|--------------------------------------------------|--------------------------|
|
|
| `worker` | Watches `incoming/`, runs the compose pipeline | none published |
|
|
| `web` | FastAPI + Jinja UI, browse jobs, remix | via Traefik only |
|
|
|
|
Both share one host directory (`DATA_HOST_DIR`) mounted at `/data`.
|
|
Syncthing runs on the host — not in this compose.
|
|
|
|
## Data layout (Syncthing share)
|
|
|
|
Prod path: `/home/frank/sync.schwenk.online/data/livef12`
|
|
|
|
```
|
|
incoming/ # phone drop — worker only ever reads/copies from here
|
|
variants/
|
|
{stem}_original.jpg # preprocessed private copy
|
|
{stem}_rembg.png
|
|
{stem}_v1.png # final composed images
|
|
intermediates/
|
|
{stem}_v1_bg_filtered.png
|
|
{stem}_v1_post_0.png # …
|
|
meta/
|
|
{stem}.json # status + manifest for the web UI
|
|
processed.json # worker bookkeeping: which incoming files were handled
|
|
```
|
|
|
|
Phone tip: `.stignore` can drop `meta/` and `processed.json` if you only
|
|
want images on the device.
|
|
|
|
`assets/` (filter lists + trimmed `filters.json`) lives in the repo and is
|
|
bind-mounted read-only into `worker`/`web` at `/data/assets`.
|
|
|
|
## Running it
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# local: set DATA_HOST_DIR=./data
|
|
# prod: leave unset (defaults to the Syncthing share path)
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Drop a photo into `incoming/` (via Syncthing or locally). After a few
|
|
seconds (poll interval + processing time) it shows up on the web UI; flat
|
|
files land in `variants/` and `intermediates/` for the phone.
|
|
|
|
## Development
|
|
|
|
Python **3.11** (same as `Dockerfile`). On a host with newer Python, use
|
|
Docker or a 3.11 venv — pinned `onnxruntime` does not publish wheels for 3.14+.
|
|
|
|
```bash
|
|
python3.11 -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements-dev.txt
|
|
|
|
ruff check app/ tests/ # lint
|
|
ruff format app/ tests/ # format
|
|
pytest -q # unit tests (no gmic/rembg)
|
|
```
|
|
|
|
CI (`.gitea/workflows/ci.yml`): ruff + pytest on push/PR.
|
|
|
|
Architecture notes: `ARCHITECTURE.md`. Cleanup log: `CLEANUP_PLAN.md`.
|
|
|
|
## Configuration (`.env`, see `.env.example`)
|
|
|
|
| Var | Default | Notes |
|
|
|-----|---------|-------|
|
|
| `DATA_HOST_DIR` | `/home/frank/sync.schwenk.online/data/livef12` | Syncthing share root; use `./data` locally |
|
|
| `OUTPUT_COUNT` | `3` | Variants generated per uploaded photo |
|
|
| `BLEND_OPACITY` | `30%` | Default blend opacity for auto-generated variants |
|
|
| `FILTER_TIMEOUT` | `120` | Seconds before a single gmic call is killed |
|
|
| `MAX_FILTER_ATTEMPTS` | `8` | Random-filter retry budget per bg/fg pick |
|
|
| `MAX_EDGE_PX` | `2000` | Longest edge after ImageMagick preprocess |
|
|
| `PREPROCESS_TIMEOUT` | `120` | Seconds for preprocess step |
|
|
| `NICE_LEVEL` | `18` | `nice -n` level for gmic/rembg subprocesses |
|
|
|
|
## Deploy notes / caveats
|
|
|
|
- **Firewall:** Traefik must already route `live.f12.rocks` — this repo
|
|
only adds the router labels; it assumes the external `traefik` docker
|
|
network exists. No SFTP port needed.
|
|
- **rembg model download:** first background-removal call downloads the
|
|
`u2net` ONNX model (~176 MB) from GitHub. This needs outbound internet
|
|
on first run and can take a minute or two depending on the link; the
|
|
model is cached in the `rembg_cache` named volume afterwards, so
|
|
restarts don't re-download it.
|
|
- **Incoming is append-only:** the worker only ever copies out of
|
|
`incoming/` and never deletes or moves anything there — plan disk space
|
|
accordingly, or clean up `incoming/` manually between events.
|
|
- **Sequential processing:** the worker handles one photo at a time
|
|
(`cpus: "1.0"`, no concurrency) — fine for an event pace, but a burst of
|
|
uploads will just queue up and get processed in order.
|
|
- Deploy itself (`docker compose up -d` on the server) is on you — this
|
|
repo doesn't run it automatically.
|