Frank Schwenk
b09f90bfcc
CI / lint-and-test (pull_request) Failing after 9s
Prevent worker/web clobbering of meta variants via flock and merge-by-id, make filter timeouts thread-local, harden job-id/stem sanitization, migrate TemplateResponse API, and remove the compare-bg experiment. Co-authored-by: Cursor <cursoragent@cursor.com>
91 lines
3.7 KiB
Markdown
91 lines
3.7 KiB
Markdown
# Cleanup Report — live.f12.rocks
|
|
|
|
Branch: `cleanup/code-quality` (local only, not pushed).
|
|
Completed: 2026-07-18.
|
|
|
|
## Summary
|
|
|
|
Incremental code-quality cleanup across 10 commits. **No intentional
|
|
behavior changes** to the photo pipeline, worker loop, or web UX.
|
|
|
|
## What changed
|
|
|
|
| Step | What | Why | Risk | Verification |
|
|
|------|------|-----|------|--------------|
|
|
| Phase 0 | `CLEANUP_PLAN.md` | Analysis + roadmap | Low | — |
|
|
| Tooling | `pyproject.toml`, `requirements-dev.txt`, `.python-version` | Ruff + pytest baseline, pin 3.11 | Low | `ruff check`, `pytest` |
|
|
| Style | Ruff format entire `app/` | Consistent style | Low | `ruff format --check` |
|
|
| Dead code | Removed unused `blend_layers()` | Less noise | Low | grep + pytest |
|
|
| `.gitignore` | Build/coverage/venv patterns | Keep repo clean | Low | — |
|
|
| `io_utils` | Atomic JSON helpers | DRY meta/processed I/O | Low | `test_io_utils`, pipeline/worker tests |
|
|
| `env_utils` / `logging_config` | Shared bool env + logging | DRY | Low | `test_env_utils`, pytest |
|
|
| `compose.yml` | YAML anchor for env | DRY, fewer drift bugs | Low | `docker compose config` (manual) |
|
|
| Config | `PREPROCESS_TIMEOUT` constant | Named magic number | Low | default still 120s |
|
|
| `read_meta()` | Public meta accessor; `list_jobs` reads once | Perf + clarity | Low | `test_pipeline`, `test_main` |
|
|
| Tests | 33 characterization tests | Safe refactor baseline | Low | `pytest -q` |
|
|
| CI | `.gitea/workflows/ci.yml` | Lint + test on push/PR | Low | workflow syntax |
|
|
| Docs | `README.md`, `ARCHITECTURE.md` | Onboarding | Low | — |
|
|
|
|
## What was NOT changed (and why)
|
|
|
|
| Item | Reason |
|
|
|------|--------|
|
|
| `pipeline.py` module split | Medium risk without broader integration tests; deferred |
|
|
| Dependency major bumps | Per plan — list only, no auto-bump |
|
|
| Deploy CI | Manual deploy on boka stays; STANDARDS says ask first |
|
|
|
|
## Follow-ups (later branch)
|
|
|
|
Meta locking, timeout thread-local, TemplateResponse migration, stem/validate
|
|
hardening, and removal of `compare-bg/` were done on a subsequent branch
|
|
(`fix/concurrency-and-cleanup`).
|
|
|
|
## Bugs noticed (historical — fixed later)
|
|
|
|
1. **Meta JSON race** — fixed with `fcntl.flock` + variant merge-by-id.
|
|
2. **`_filter_timeout_override` global** — fixed with `threading.local`.
|
|
3. **`sanitize_stem("a/b")` → `"b"`** — fixed (separators replaced before `Path.stem`).
|
|
4. **`_validate_job_id`** — rejection path restored for path-like raw IDs.
|
|
|
|
## Dependency notes (suggestions only)
|
|
|
|
| Package | Current pin | Note |
|
|
|---------|-------------|------|
|
|
| `fastapi` | `<0.117` | 0.117+ available; test before bump |
|
|
| `onnxruntime` | `<1.20` | Very old pin; rembg may allow newer in future |
|
|
| `pillow` | `<11` | Pillow 11 exists; verify rembg compatibility |
|
|
|
|
No lockfile added — soft pins kept to avoid changing Docker resolve behavior.
|
|
|
|
## How to verify locally
|
|
|
|
```bash
|
|
git checkout cleanup/code-quality
|
|
python3.11 -m venv .venv && source .venv/bin/activate
|
|
pip install -r requirements-dev.txt
|
|
ruff check app/ tests/
|
|
pytest -q
|
|
# Full stack:
|
|
cp .env.example .env # set DATA_HOST_DIR=./data
|
|
docker compose up --build
|
|
```
|
|
|
|
## Commits on branch
|
|
|
|
```
|
|
docs: add cleanup plan from phase-0 analysis
|
|
chore: add pyproject.toml with ruff and pytest config
|
|
style: apply ruff format and extend gitignore
|
|
refactor: extract atomic JSON helpers to io_utils
|
|
refactor: share env-bool parsing and logging setup
|
|
chore: deduplicate compose environment blocks
|
|
refactor: named preprocess timeout and single meta read
|
|
test: add characterization tests for core modules (+ CI workflow)
|
|
docs: update README and add ARCHITECTURE
|
|
```
|
|
|
|
## External references
|
|
|
|
- `make_random.py` — external reference script, not vendored
|
|
- Submodule/symlink: none
|