feat: initial live.f12.rocks SFTP → gmic/rembg → web pipeline

Event pep stack with SFTPGo inbox, sequential worker, FastAPI gallery/remix, and Traefik-ready compose.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Frank Schwenk
2026-07-16 21:32:10 +02:00
commit 90192cd284
31 changed files with 7739 additions and 0 deletions
+95
View File
@@ -0,0 +1,95 @@
# live.f12.rocks
Event photo pep pipeline: guests upload a photo via SFTP, a worker runs it
through `rembg` (background removal) + `gmic` (random filter/blend
compositing, ported from `make_random.py`), and a small web UI shows the
results with a "remix" option to try different filters on the same photo.
No web auth. This is an event tool, not a DAM.
## Services (`compose.yml`)
| Service | What | Port |
|----------|--------------------------------------------------|--------------------------|
| `sftpgo` | SFTP drop point, `drakkan/sftpgo` | `12121` (host) -> `2022` |
| `worker` | Watches inbox, runs the compose pipeline | none published |
| `web` | FastAPI + Jinja UI, browse jobs, remix | via Traefik only |
All three share one host directory (`DATA_HOST_DIR`, default `./data`),
mounted at different paths — see the comment at the top of `compose.yml`.
## Data layout (`./data`)
```
inbox/ # SFTP drop — worker only ever reads/copies from here
jobs/<job_id>/
original.<ext> # private copy of the uploaded photo
rembg.png # background removed (computed once, reused)
intermediates/ # every intermediate step, kept for inspection
variants/ # final composed images
manifest.json # filter names/commands/blends per variant
status.json # pending | processing | done | error
processed.json # worker bookkeeping: which inbox files were handled
```
`assets/` (filter lists + trimmed `filters.json`) lives in the repo and is
bind-mounted read-only into `worker`/`web` at `/data/assets`.
## Running it
```bash
cp .env.example .env
# edit .env: set a real SFTP_PASSWORD, and DATA_HOST_DIR on the server
docker compose up -d --build
```
Upload a photo:
```bash
sftp -P 12121 livef12@<host>
put photo.jpg
```
After a few seconds (poll interval + processing time) it shows up on the
web UI as a new job.
## Configuration (`.env`, see `.env.example`)
| Var | Default | Notes |
|-----|---------|-------|
| `SFTP_USER` / `SFTP_PASSWORD` | `livef12` / `changeme` | SFTP login, home dir is locked to the shared inbox |
| `DATA_HOST_DIR` | `./data` | On the server: `/home/frank/live.f12.rocks/data` |
| `SFTP_HOST_PORT` | `12121` | Host-side SFTP port (121212 is invalid TCP) |
| `OUTPUT_COUNT` | `3` | Variants generated per uploaded photo |
| `BLEND_OPACITY` | `30%` | Default blend opacity for auto-generated variants |
| `FILTER_TIMEOUT` | `120` | Seconds before a single gmic call is killed |
| `MAX_FILTER_ATTEMPTS` | `8` | Random-filter retry budget per bg/fg pick |
| `NICE_LEVEL` | `18` | `nice -n` level for gmic/rembg subprocesses |
## Deploy notes / caveats
- **Port note:** Spec originally said `121212`, which exceeds TCP max
(65535). Production default is **`12121`**.
- **Firewall:** open host port `12121` (SFTP) and make sure Traefik
already routes `live.f12.rocks` — this repo only adds the router labels,
it assumes the external `traefik` docker network exists.
- **SFTPGo first run:** the `sftpgo` service auto-creates the SFTP user
from `SFTP_USER`/`SFTP_PASSWORD` on every start via
`sftpgo/entrypoint.sh` (uses `jq`, bundled in the official image, to
build a `loaddata` JSON safely — no manual admin setup needed). Host SSH
keys persist in the `sftpgo_state` named volume, not in `./data`.
The SFTPGo web admin exists internally on port 8080 but is deliberately
**not** published or routed — there's no need for it here.
- **rembg model download:** first background-removal call downloads the
`u2net` ONNX model (~176 MB) from GitHub. This needs outbound internet
on first run and can take a minute or two depending on the link; the
model is cached in the `rembg_cache` named volume afterwards, so
restarts don't re-download it.
- **Inbox is append-only:** the worker only ever copies out of `inbox/`
and never deletes or moves anything there — plan disk space
accordingly, or clean up `inbox/` manually between events.
- **Sequential processing:** the worker handles one photo at a time
(`cpus: "1.0"`, no concurrency) — fine for an event pace, but a burst of
uploads will just queue up and get processed in order.
- Deploy itself (`docker compose up -d` on the server) is on you — this
repo doesn't run it automatically.