feat: initial live.f12.rocks SFTP → gmic/rembg → web pipeline
Event pep stack with SFTPGo inbox, sequential worker, FastAPI gallery/remix, and Traefik-ready compose. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,95 @@
|
||||
# live.f12.rocks
|
||||
|
||||
Event photo pep pipeline: guests upload a photo via SFTP, a worker runs it
|
||||
through `rembg` (background removal) + `gmic` (random filter/blend
|
||||
compositing, ported from `make_random.py`), and a small web UI shows the
|
||||
results with a "remix" option to try different filters on the same photo.
|
||||
|
||||
No web auth. This is an event tool, not a DAM.
|
||||
|
||||
## Services (`compose.yml`)
|
||||
|
||||
| Service | What | Port |
|
||||
|----------|--------------------------------------------------|--------------------------|
|
||||
| `sftpgo` | SFTP drop point, `drakkan/sftpgo` | `12121` (host) -> `2022` |
|
||||
| `worker` | Watches inbox, runs the compose pipeline | none published |
|
||||
| `web` | FastAPI + Jinja UI, browse jobs, remix | via Traefik only |
|
||||
|
||||
All three share one host directory (`DATA_HOST_DIR`, default `./data`),
|
||||
mounted at different paths — see the comment at the top of `compose.yml`.
|
||||
|
||||
## Data layout (`./data`)
|
||||
|
||||
```
|
||||
inbox/ # SFTP drop — worker only ever reads/copies from here
|
||||
jobs/<job_id>/
|
||||
original.<ext> # private copy of the uploaded photo
|
||||
rembg.png # background removed (computed once, reused)
|
||||
intermediates/ # every intermediate step, kept for inspection
|
||||
variants/ # final composed images
|
||||
manifest.json # filter names/commands/blends per variant
|
||||
status.json # pending | processing | done | error
|
||||
processed.json # worker bookkeeping: which inbox files were handled
|
||||
```
|
||||
|
||||
`assets/` (filter lists + trimmed `filters.json`) lives in the repo and is
|
||||
bind-mounted read-only into `worker`/`web` at `/data/assets`.
|
||||
|
||||
## Running it
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# edit .env: set a real SFTP_PASSWORD, and DATA_HOST_DIR on the server
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
Upload a photo:
|
||||
|
||||
```bash
|
||||
sftp -P 12121 livef12@<host>
|
||||
put photo.jpg
|
||||
```
|
||||
|
||||
After a few seconds (poll interval + processing time) it shows up on the
|
||||
web UI as a new job.
|
||||
|
||||
## Configuration (`.env`, see `.env.example`)
|
||||
|
||||
| Var | Default | Notes |
|
||||
|-----|---------|-------|
|
||||
| `SFTP_USER` / `SFTP_PASSWORD` | `livef12` / `changeme` | SFTP login, home dir is locked to the shared inbox |
|
||||
| `DATA_HOST_DIR` | `./data` | On the server: `/home/frank/live.f12.rocks/data` |
|
||||
| `SFTP_HOST_PORT` | `12121` | Host-side SFTP port (121212 is invalid TCP) |
|
||||
| `OUTPUT_COUNT` | `3` | Variants generated per uploaded photo |
|
||||
| `BLEND_OPACITY` | `30%` | Default blend opacity for auto-generated variants |
|
||||
| `FILTER_TIMEOUT` | `120` | Seconds before a single gmic call is killed |
|
||||
| `MAX_FILTER_ATTEMPTS` | `8` | Random-filter retry budget per bg/fg pick |
|
||||
| `NICE_LEVEL` | `18` | `nice -n` level for gmic/rembg subprocesses |
|
||||
|
||||
## Deploy notes / caveats
|
||||
|
||||
- **Port note:** Spec originally said `121212`, which exceeds TCP max
|
||||
(65535). Production default is **`12121`**.
|
||||
- **Firewall:** open host port `12121` (SFTP) and make sure Traefik
|
||||
already routes `live.f12.rocks` — this repo only adds the router labels,
|
||||
it assumes the external `traefik` docker network exists.
|
||||
- **SFTPGo first run:** the `sftpgo` service auto-creates the SFTP user
|
||||
from `SFTP_USER`/`SFTP_PASSWORD` on every start via
|
||||
`sftpgo/entrypoint.sh` (uses `jq`, bundled in the official image, to
|
||||
build a `loaddata` JSON safely — no manual admin setup needed). Host SSH
|
||||
keys persist in the `sftpgo_state` named volume, not in `./data`.
|
||||
The SFTPGo web admin exists internally on port 8080 but is deliberately
|
||||
**not** published or routed — there's no need for it here.
|
||||
- **rembg model download:** first background-removal call downloads the
|
||||
`u2net` ONNX model (~176 MB) from GitHub. This needs outbound internet
|
||||
on first run and can take a minute or two depending on the link; the
|
||||
model is cached in the `rembg_cache` named volume afterwards, so
|
||||
restarts don't re-download it.
|
||||
- **Inbox is append-only:** the worker only ever copies out of `inbox/`
|
||||
and never deletes or moves anything there — plan disk space
|
||||
accordingly, or clean up `inbox/` manually between events.
|
||||
- **Sequential processing:** the worker handles one photo at a time
|
||||
(`cpus: "1.0"`, no concurrency) — fine for an event pace, but a burst of
|
||||
uploads will just queue up and get processed in order.
|
||||
- Deploy itself (`docker compose up -d` on the server) is on you — this
|
||||
repo doesn't run it automatically.
|
||||
Reference in New Issue
Block a user