froxxxy/livef12rocks
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: replace SFTP drop with Syncthing share path

Browse Source 
Remove SFTPGo; mount event data from the Syncthing folder, watch
incoming/, and name variants after the source stem.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Frank Schwenk
2026-07-18 14:06:57 +02:00
parent 83d4468b69
commit d5b44b221e
13 changed files with 167 additions and 189 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+26 -40
View File
@@ -1,9 +1,10 @@
# live.f12.rocks
Event photo pep pipeline: guests upload a photo via SFTP, a worker runs it
through `rembg` (background removal) + `gmic` (random filter/blend
compositing, ported from `make_random.py`), and a small web UI shows the
results with a "remix" option to try different filters on the same photo.
Event photo pep pipeline: guests drop a photo into a Syncthing folder, a
worker runs it through `rembg` (background removal) + `gmic` (random
filter/blend compositing, ported from `make_random.py`), and a small web
UI shows the results with a "remix" option to try different filters on
the same photo. Finished jobs sync back via the same share.
No web auth. This is an event tool, not a DAM.
@@ -11,25 +12,26 @@ No web auth. This is an event tool, not a DAM.
| Service | What | Port |
|----------|--------------------------------------------------|--------------------------|
| `sftpgo` | SFTP drop point, `drakkan/sftpgo` | `12121` (host) -> `2022` |
| `worker` | Watches inbox, runs the compose pipeline | none published |
| `worker` | Watches `incoming/`, runs the compose pipeline | none published |
| `web` | FastAPI + Jinja UI, browse jobs, remix | via Traefik only |
All three share one host directory (`DATA_HOST_DIR`, default `./data`),
mounted at different paths — see the comment at the top of `compose.yml`.
Both share one host directory (`DATA_HOST_DIR`) mounted at `/data`.
Syncthing runs on the host — not in this compose.
## Data layout (`./data`)
## Data layout (Syncthing share)
Prod path: `/home/frank/sync.schwenk.online/data/livef12`
```
inbox/ # SFTP drop — worker only ever reads/copies from here
incoming/ # phone drop — worker only ever reads/copies from here
jobs/<job_id>/
original.<ext> # private copy of the uploaded photo
rembg.png # background removed (computed once, reused)
intermediates/ # every intermediate step, kept for inspection
variants/ # final composed images
variants/ # final composed images ({stem}_v1.png, …)
manifest.json # filter names/commands/blends per variant
status.json # pending | processing | done | error
processed.json # worker bookkeeping: which inbox files were handled
processed.json # worker bookkeeping: which incoming files were handled
```
`assets/` (filter lists + trimmed `filters.json`) lives in the repo and is
@@ -39,27 +41,20 @@ bind-mounted read-only into `worker`/`web` at `/data/assets`.
```bash
cp .env.example .env
# edit .env: set a real SFTP_PASSWORD, and DATA_HOST_DIR on the server
# local: set DATA_HOST_DIR=./data
# prod: leave unset (defaults to the Syncthing share path)
docker compose up -d --build
```
Upload a photo:
```bash
sftp -P 12121 livef12@<host>
put photo.jpg
```
After a few seconds (poll interval + processing time) it shows up on the
web UI as a new job.
Drop a photo into `incoming/` (via Syncthing or locally). After a few
seconds (poll interval + processing time) it shows up on the web UI as a
new job; the full `jobs/<id>/` tree syncs back to the phone.
## Configuration (`.env`, see `.env.example`)
| Var | Default | Notes |
|-----|---------|-------|
| `SFTP_USER` / `SFTP_PASSWORD` | `livef12` / `changeme` | SFTP login, home dir is locked to the shared inbox |
| `DATA_HOST_DIR` | `./data` | On the server: `/home/frank/live.f12.rocks/data` |
| `SFTP_HOST_PORT` | `12121` | Host-side SFTP port (121212 is invalid TCP) |
| `DATA_HOST_DIR` | `/home/frank/sync.schwenk.online/data/livef12` | Syncthing share root; use `./data` locally |
| `OUTPUT_COUNT` | `3` | Variants generated per uploaded photo |
| `BLEND_OPACITY` | `30%` | Default blend opacity for auto-generated variants |
| `FILTER_TIMEOUT` | `120` | Seconds before a single gmic call is killed |
@@ -68,26 +63,17 @@ web UI as a new job.
## Deploy notes / caveats
- **Port note:** Spec originally said `121212`, which exceeds TCP max
(65535). Production default is **`12121`**.
- **Firewall:** open host port `12121` (SFTP) and make sure Traefik
already routes `live.f12.rocks` — this repo only adds the router labels,
it assumes the external `traefik` docker network exists.
- **SFTPGo first run:** the `sftpgo` service auto-creates the SFTP user
from `SFTP_USER`/`SFTP_PASSWORD` on every start via
`sftpgo/entrypoint.sh` (uses `jq`, bundled in the official image, to
build a `loaddata` JSON safely — no manual admin setup needed). Host SSH
keys persist in the `sftpgo_state` named volume, not in `./data`.
The SFTPGo web admin exists internally on port 8080 but is deliberately
**not** published or routed — there's no need for it here.
- **Firewall:** Traefik must already route `live.f12.rocks` — this repo
only adds the router labels; it assumes the external `traefik` docker
network exists. No SFTP port needed.
- **rembg model download:** first background-removal call downloads the
`u2net` ONNX model (~176 MB) from GitHub. This needs outbound internet
on first run and can take a minute or two depending on the link; the
model is cached in the `rembg_cache` named volume afterwards, so
restarts don't re-download it.
- **Inbox is append-only:** the worker only ever copies out of `inbox/`
and never deletes or moves anything there — plan disk space
accordingly, or clean up `inbox/` manually between events.
- **Incoming is append-only:** the worker only ever copies out of
`incoming/` and never deletes or moves anything there — plan disk space
accordingly, or clean up `incoming/` manually between events.
- **Sequential processing:** the worker handles one photo at a time
(`cpus: "1.0"`, no concurrency) — fine for an event pace, but a burst of
uploads will just queue up and get processed in order.