Frank Schwenk
c03f17488c
CI / lint-and-test (push) Failing after 1m18s
Replace Syncthing as the product upload path with a share-target app and POST /api/jobs. Web UI live-polls job progress. Co-authored-by: Cursor <cursoragent@cursor.com>
116 lines
4.4 KiB
Markdown
116 lines
4.4 KiB
Markdown
# live.f12.rocks
|
|
|
|
Event photo pep pipeline: the organizer shares a photo into the Android
|
|
app, a worker runs it through `rembg` (background removal) + `gmic`
|
|
(random filter/blend compositing, ported from `make_random.py`), and a
|
|
small web UI shows the results with a "remix" option. Finished jobs
|
|
land on the phone (gallery + local notification) and on the public page.
|
|
|
|
No web login. Upload uses a shared token in the organizer app. This is
|
|
an event tool, not a DAM.
|
|
|
|
## Services (`compose.yml`)
|
|
|
|
| Service | What | Port |
|
|
|----------|--------------------------------------------------|--------------------------|
|
|
| `worker` | Watches `incoming/`, runs the compose pipeline | none published |
|
|
| `web` | FastAPI HTML + JSON API, remix | via Traefik only |
|
|
|
|
Both share one host directory (`DATA_HOST_DIR`) mounted at `/data`.
|
|
The organizer app lives in `android/`. Syncthing can still write
|
|
`incoming/` as a fallback — it is not in this compose.
|
|
|
|
## Data layout
|
|
|
|
Prod path: `/home/frank/sync.schwenk.online/data/livef12`
|
|
|
|
```
|
|
incoming/ # HTTP upload (and optional Syncthing drop)
|
|
variants/
|
|
{stem}_original.jpg # preprocessed private copy
|
|
{stem}_rembg.png
|
|
{stem}_v1.png # final composed images
|
|
intermediates/
|
|
{stem}_v1_bg_filtered.png
|
|
{stem}_v1_post_0.png # …
|
|
meta/
|
|
{stem}.json # status + stage + manifest
|
|
processed.json # worker bookkeeping
|
|
```
|
|
|
|
`assets/` (filter lists + trimmed `filters.json`) lives in the repo and is
|
|
bind-mounted read-only into `worker`/`web` at `/data/assets`.
|
|
|
|
## API
|
|
|
|
| Method | Path | Auth |
|
|
|--------|------|------|
|
|
| `GET /api/health` | `{ok, upload}` | no |
|
|
| `GET /api/jobs` | job list | no |
|
|
| `GET /api/jobs/{id}` | status, variants, intermediates | no |
|
|
| `POST /api/jobs` | multipart `file` | `Authorization: Bearer <UPLOAD_TOKEN>` |
|
|
|
|
Empty `UPLOAD_TOKEN` disables uploads (fail closed). File download stays
|
|
on `GET /jobs/{id}/files/{name}`.
|
|
|
|
## Running it
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
# set UPLOAD_TOKEN (python3 -c "import secrets; print(secrets.token_urlsafe(32))")
|
|
# local: set DATA_HOST_DIR=./data
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Share a photo into the Android app, or drop one into `incoming/` locally.
|
|
After a few seconds it shows up on the web UI.
|
|
|
|
Android build: [`android/README.md`](android/README.md).
|
|
|
|
## Development
|
|
|
|
Python **3.11** (same as `Dockerfile`). On a host with newer Python, use
|
|
Docker or a 3.11 venv — pinned `onnxruntime` does not publish wheels for 3.14+.
|
|
|
|
```bash
|
|
python3.11 -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements-dev.txt
|
|
|
|
ruff check app/ tests/ # lint
|
|
ruff format app/ tests/ # format
|
|
pytest -q # unit tests (no gmic/rembg)
|
|
```
|
|
|
|
CI (`.gitea/workflows/ci.yml`): ruff + pytest on push/PR.
|
|
|
|
Architecture notes: `ARCHITECTURE.md`.
|
|
|
|
## Configuration (`.env`, see `.env.example`)
|
|
|
|
| Var | Default | Notes |
|
|
|-----|---------|-------|
|
|
| `DATA_HOST_DIR` | `/home/frank/sync.schwenk.online/data/livef12` | data root; use `./data` locally |
|
|
| `UPLOAD_TOKEN` | empty | required for `POST /api/jobs`; empty disables upload |
|
|
| `OUTPUT_COUNT` | `3` | Variants generated per uploaded photo |
|
|
| `BLEND_OPACITY` | `30%` | Default blend opacity for auto-generated variants |
|
|
| `FILTER_TIMEOUT` | `120` | Seconds before a single gmic call is killed |
|
|
| `MAX_FILTER_ATTEMPTS` | `8` | Random-filter retry budget per bg/fg pick |
|
|
| `MAX_EDGE_PX` | `2000` | Longest edge after ImageMagick preprocess |
|
|
| `PREPROCESS_TIMEOUT` | `120` | Seconds for preprocess step |
|
|
| `NICE_LEVEL` | `18` | `nice -n` level for gmic/rembg subprocesses |
|
|
|
|
## Deploy notes / caveats
|
|
|
|
- **Firewall:** Traefik must already route `live.f12.rocks` — this repo
|
|
only adds the router labels; it assumes the external `traefik` docker
|
|
network exists.
|
|
- **rembg model download:** first background-removal call downloads the
|
|
`u2net` ONNX model (~176 MB) from GitHub. Cached in `rembg_cache`.
|
|
- **Incoming is append-only:** the worker only ever copies out of
|
|
`incoming/` and never deletes or moves anything there.
|
|
- **Sequential processing:** the worker handles one photo at a time
|
|
(`cpus: "1.0"`, no concurrency).
|
|
- Deploy itself (`docker compose up -d` on the server) is on you — this
|
|
repo doesn't run it automatically. Remember `UPLOAD_TOKEN` in the
|
|
server `.env` or the app cannot upload.
|