Compare commits
18 Commits
e7cdb8dd6f
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 08abfc8f4d | |||
| a138e33c04 | |||
| 5920766f42 | |||
| ffc28914e1 | |||
| 3443c8826d | |||
| 92337b7dbc | |||
| fd4658434a | |||
| a60a18a253 | |||
| 84447d1d2c | |||
| 9b6c80da42 | |||
| 765ebbe3da | |||
| 0daf3e2315 | |||
| cbca473f06 | |||
| 100302bf01 | |||
| 1c36fc1968 | |||
| 13c3c653b8 | |||
| ce431d7eec | |||
| 980c7f3b8b |
@@ -0,0 +1,28 @@
|
||||
name: Test
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: [main, cleanup/**]
|
||||
pull_request:
|
||||
|
||||
jobs:
|
||||
pytest:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: "3.12"
|
||||
|
||||
- name: Install package and dev dependencies
|
||||
run: pip install -e ".[dev]"
|
||||
|
||||
- name: Ruff
|
||||
run: |
|
||||
pip install ruff
|
||||
ruff check .
|
||||
ruff format --check .
|
||||
|
||||
- name: Pytest (unit; skip integration)
|
||||
run: pytest -m "not integration"
|
||||
@@ -5,6 +5,14 @@ __pycache__/
|
||||
dist/
|
||||
build/
|
||||
.pytest_cache/
|
||||
.ruff_cache/
|
||||
.mypy_cache/
|
||||
.coverage
|
||||
htmlcov/
|
||||
.venv/
|
||||
venv/
|
||||
.env
|
||||
*.log
|
||||
|
||||
# Local pipeline run output (if created inside repo)
|
||||
pipeline_output/
|
||||
|
||||
@@ -0,0 +1,275 @@
|
||||
# AGENTS.md — Operational Instructions
|
||||
|
||||
How the coding agent works with Fränky's projects.
|
||||
|
||||
Also read: `INFRASTRUCTURE.md`, `STANDARDS.md`, `BOUNDARIES.md`, project `SOUL.md`, project `MOOD.md`.
|
||||
Persona & preferences: `USER.md` (Tier 2).
|
||||
|
||||
---
|
||||
|
||||
## Rule Priority
|
||||
|
||||
Highest wins. One line per source:
|
||||
|
||||
| Priority | Source |
|
||||
|----------|--------|
|
||||
| 1 | Explicit chat instruction from Fränky |
|
||||
| 2 | Project `SOUL.md` / project-local `BOUNDARIES.md` / project `AGENTS` snippets |
|
||||
| 3 | `BOUNDARIES.md` |
|
||||
| 4 | `AGENTS.md` (this file) |
|
||||
| 5 | `STANDARDS.md` |
|
||||
| 6 | `INFRASTRUCTURE.md` |
|
||||
| 7 | `MOOD.md` — **tone only**, never overrides safety or ops rules |
|
||||
| 8 | `USER.md` — persona & preferences, not operational overrides |
|
||||
|
||||
**UI/UX:** project `SOUL.md` overrides global `STANDARDS.md` when they conflict.
|
||||
|
||||
---
|
||||
|
||||
## Startup (Tier 1 / Tier 2)
|
||||
|
||||
Cursor does not auto-load context. Use two tiers:
|
||||
|
||||
### Tier 1 — always (light)
|
||||
|
||||
At the start of any substantive task, without waiting for ack:
|
||||
|
||||
1. Project `SOUL.md` — **Agent Quick Start** section (or full file if no Quick Start)
|
||||
2. `BOUNDARIES.md` — **Never Ever** section
|
||||
|
||||
### Tier 2 — full load
|
||||
|
||||
On `@AGENTS.md ack`, first chat in a project, or when Fränky says context was lost:
|
||||
|
||||
1. `AGENTS.md`, `BOUNDARIES.md`, `STANDARDS.md`, `INFRASTRUCTURE.md`
|
||||
2. `USER.md` — persona, work-style table, communication prefs
|
||||
3. Project `SOUL.md` (full), project `MOOD.md` if present
|
||||
4. Skim project structure
|
||||
|
||||
### Command: `AGENTS.md ack`
|
||||
|
||||
Fränky schreibt `AGENTS.md ack`.
|
||||
|
||||
**Agent:** Tier-2 read, then reply with this **compact template** (one block, no code changes):
|
||||
|
||||
```
|
||||
AGENTS ack ✓
|
||||
· Pipeline: [1-line what this project is]
|
||||
· Non-goals: [from SOUL, or "SOUL missing"]
|
||||
· Commit policy: [no commit unless … / project override]
|
||||
· MOOD: [active session character]
|
||||
· Paths: [key workdirs / mounts from SOUL or INFRA snippet]
|
||||
· Conflict: [1 sentence if SOUL vs STANDARDS disagree, else "none"]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Work Modes
|
||||
|
||||
| Mode | Trigger | Behavior |
|
||||
|------|---------|----------|
|
||||
| **Question-only** | Question, review, "how does X work?" | No file changes, commit, or drive-by fixes |
|
||||
| **Standard** | Default | Restate → align on plan → implement → verify → hand off |
|
||||
| **Unattended / Away** | "wenn ich zurück bin", "mach ohne mich", "overnight", explicit away | **Skip plan alignment** — proceed with best judgment; justify in handoff |
|
||||
|
||||
### Unattended / Away — extra rules
|
||||
|
||||
- **Monitoring:** Cursor background shell + polling — **not** external wrapper scripts as default
|
||||
- **Handoff must include:** log paths, PIDs if relevant, how to recognize success, next command for Fränky
|
||||
- Write blocked items or run status to `NOTES.md` when useful
|
||||
- **Notify** via `ntfyschwenkonline` when the task finishes (see [Notifications](#notifications))
|
||||
|
||||
---
|
||||
|
||||
## Default Workflow (Standard mode)
|
||||
|
||||
1. **Restate** the request; turn into plan or mini-PRD
|
||||
2. **Align** on the plan — resolve ambiguities before coding
|
||||
3. **Implement** (see Testing below)
|
||||
4. **Verify** — run tests when appropriate; do not guess
|
||||
5. **Hand off** — summary, how to test locally, log paths for long jobs
|
||||
6. **Deploy / prod** — Fränky handles unless project or chat says otherwise
|
||||
|
||||
After **long-running** work: send push notification (see [Notifications](#notifications)).
|
||||
|
||||
---
|
||||
|
||||
## Autonomy Matrix
|
||||
|
||||
See also `USER.md` **Work Style** table for Fränky's preference scores.
|
||||
|
||||
| Action | Default |
|
||||
|--------|---------|
|
||||
| Write / change code | ✅ OK |
|
||||
| Write tests | ✅ OK when non-trivial or suite exists |
|
||||
| Add dependencies | ✅ OK |
|
||||
| Touch README / docs | ✅ OK |
|
||||
| Change CI/CD config | ⚠️ Ask first |
|
||||
| Refactor "on the side" | ⚠️ Ask first |
|
||||
| Update AGENTS / SOUL / BOUNDARIES | ⚠️ Ask first (or explicit "remember this") |
|
||||
| Update `MOOD.md` on disk | 🚫 Only on `persist MOOD` or explicit instruction |
|
||||
| Create auxiliary `.md` (`NOTES.md`, todos) | ✅ OK |
|
||||
| Git commit / push / deploy | 🚫 Unless explicit or project `SOUL.md` allows |
|
||||
| `ntfyschwenkonline` after long-running task | ✅ OK |
|
||||
|
||||
---
|
||||
|
||||
## MOOD: Session vs. Persist
|
||||
|
||||
| | Session | Persist to `MOOD.md` |
|
||||
|---|---------|----------------------|
|
||||
| Trigger | `mood "XY"` in chat | `persist MOOD` or explicit "save mood to file" |
|
||||
| Effect | Tone for this chat only | Updates `## Aktueller Mood` in file |
|
||||
| Default | **Yes** — `mood "XY"` does **not** edit the file |
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
Fränky's bar is **pragmatic, not TDD-by-default** (`USER.md`: tests score 2/5).
|
||||
|
||||
- Run tests when a suite exists **and** the change is non-trivial
|
||||
- Follow project `SOUL.md` if stricter (e.g. pytest before handoff)
|
||||
- Do not block small fixes on missing test infrastructure
|
||||
|
||||
---
|
||||
|
||||
## Commits & Issues
|
||||
|
||||
- **Default: no commit, no push** without explicit instruction
|
||||
- **Format:** [Conventional Commits](https://www.conventionalcommits.org/) (`feat:`, `fix:`, `docs:`, …) when committing
|
||||
- **Forge:** **Gitea** (`gitea.schwenk.online`) unless project says otherwise — not GitLab
|
||||
- Link Gitea issues when they exist (`#123`)
|
||||
- Do not mix unrelated changes (e.g. MOOD switch + feature) in one commit
|
||||
|
||||
---
|
||||
|
||||
## Collaboration Patterns
|
||||
|
||||
### `NOTES.md`
|
||||
|
||||
Scratchpad — use for blocked tasks, session park, overnight-run status, handoff crumbs.
|
||||
|
||||
### Subagents & context budget
|
||||
|
||||
**Keep main context small.** Delegate instead of dumping large outputs into parent chat:
|
||||
|
||||
| Task | Subagent |
|
||||
|------|----------|
|
||||
| Unknown codebase or many files to scan | **Explore** |
|
||||
| Browser / E2E / web UI | **Playwright** |
|
||||
| Broad research | **generalPurpose** or **Explore** |
|
||||
| Shell / git batch ops | **shell** |
|
||||
|
||||
**Explore** for any project type when orientation is unclear — not only web. **Playwright** only when there is a web UI to test.
|
||||
|
||||
### Commits when allowed
|
||||
|
||||
One focused commit per completed task — easy to revert. No drive-by refactors.
|
||||
|
||||
---
|
||||
|
||||
## Project Infrastructure Snippet
|
||||
|
||||
Machine-specific paths (workdirs, external mounts, local service ports) belong in **project `SOUL.md`**, not global `INFRASTRUCTURE.md`.
|
||||
|
||||
Example SOUL section:
|
||||
|
||||
```markdown
|
||||
## Infrastructure (project-local)
|
||||
- Workdir: ~/.local/share/myapp
|
||||
- Mounts: check MegaB before scan
|
||||
- Local Immich: :2283 (downstream, not core)
|
||||
```
|
||||
|
||||
See `INFRASTRUCTURE.md` → **Project-local overrides**.
|
||||
|
||||
---
|
||||
|
||||
## When Uncertain
|
||||
|
||||
1. Research — read code, run commands
|
||||
2. Continue other independent tasks if possible
|
||||
3. Park in `NOTES.md`, todo file, or Gitea issue
|
||||
4. Ask after research, with options + recommended default
|
||||
|
||||
---
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Root cause first — do not guess
|
||||
- Escalate on: debug loops, missing uninstallable software
|
||||
- Long-running jobs: no arbitrary timeout unless Fränky or SOUL says so; name `progress.log` when applicable
|
||||
|
||||
---
|
||||
|
||||
## Notifications
|
||||
|
||||
Fränky's machines have **`/usr/local/bin/ntfyschwenkonline`** — push to [ntfy.schwenk.online](https://ntfy.schwenk.online). Topic = short hostname (`hostname -s`); phone subscribes per machine. Auth is in the installed script — **never copy tokens into repos or chat**.
|
||||
|
||||
### When to notify
|
||||
|
||||
Send a notification when a **long-running agent task** completes or fails:
|
||||
|
||||
- **Unattended / Away** mode — always on finish (success or failure)
|
||||
- Background shell jobs you started (scans, builds, batch ops, overnight runs)
|
||||
- Any task Fränky left with an expectation of „meld dich wenn fertig"
|
||||
- Rough guide: expected runtime **> ~2 minutes** or explicit away/unattended context
|
||||
|
||||
Do **not** notify for quick edits, short test runs, or question-only chats.
|
||||
|
||||
### How
|
||||
|
||||
```bash
|
||||
ntfyschwenkonline "OK <project>: <one-line result>"
|
||||
# or on failure:
|
||||
ntfyschwenkonline "FAIL <project>: <one-line error>, see <log-path>"
|
||||
```
|
||||
|
||||
- **English** message body (CLI/ops convention)
|
||||
- One line, ~120 chars — project name, outcome, log path or next step if relevant
|
||||
- Run after handoff summary; notification is in addition to chat handoff, not a replacement
|
||||
- If `ntfyschwenkonline` is missing or fails: note in chat handoff, do not block
|
||||
|
||||
### Examples
|
||||
|
||||
```bash
|
||||
ntfyschwenkonline "OK imagetool: scan done 142 dirs, log ~/.local/share/imagetool/progress.log"
|
||||
ntfyschwenkonline "FAIL ytrecap: pytest 3 failed, see /tmp/test.log"
|
||||
ntfyschwenkonline "OK schwenkonline: build+deploy done"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Recording New Rules
|
||||
|
||||
When Fränky says *"never do X"*:
|
||||
|
||||
1. Propose file (`BOUNDARIES.md`, `SOUL.md`, `AGENTS.md`, or multiple)
|
||||
2. Store in **Cursor Memories** for explicit rules
|
||||
3. Write after confirmation — or immediately if explicit
|
||||
|
||||
---
|
||||
|
||||
## Language
|
||||
|
||||
- **Chat:** match Fränky's language (German or English)
|
||||
- **CLI / terminal / shell:** always English (unless client project — ask)
|
||||
- **Other artifacts:** first language of session
|
||||
|
||||
---
|
||||
|
||||
## Tools & MCP
|
||||
|
||||
1. MCP when available and relevant
|
||||
2. Built-in tools (shell, grep, read)
|
||||
3. Browser automation last resort
|
||||
|
||||
Check MCP schemas before calling.
|
||||
|
||||
---
|
||||
|
||||
## Tooling Context
|
||||
|
||||
Primary: **Cursor** (Composer / Agent).
|
||||
Server/deploy: **`INFRASTRUCTURE.md`**. Persona: **`USER.md`**.
|
||||
@@ -0,0 +1,77 @@
|
||||
# BOUNDARIES.md — Hard Limits
|
||||
|
||||
Rules that apply in **every** project unless explicitly overridden in a project-local `BOUNDARIES.md` (stricter only — never looser).
|
||||
|
||||
---
|
||||
|
||||
## Never Ever
|
||||
|
||||
| Rule | Why |
|
||||
|------|-----|
|
||||
| **Commit secrets** | `.env`, API keys, tokens, passwords, private keys — use `.gitignore` and env vars |
|
||||
| **Cripple the machine** | No commands that freeze desktop, fill disk, fork-bomb, or saturate CPU/RAM on dev box or server |
|
||||
| **Ignore user rules** | `USER.md`, `AGENTS.md`, project context files, and explicit chat instructions are binding |
|
||||
| **Force-push to main/master** | Unless Fränky explicitly requests it — warn first |
|
||||
| **Destructive prod actions** | No prod DB drops, migrations, or deploys without explicit approval (project may define exceptions) |
|
||||
| **Modify code on question-only requests** | Questions get answers — not drive-by fixes |
|
||||
|
||||
---
|
||||
|
||||
## Sensitive Data
|
||||
|
||||
- Do not paste secrets into chat, commits, logs, or comments
|
||||
- Use `.env.example` with placeholder values — never real credentials
|
||||
- Redact tokens and personal data in error output shared in chat
|
||||
- When handling personal data: minimize collection, don't log PII unnecessarily
|
||||
- If unsure whether data is sensitive: treat it as sensitive
|
||||
|
||||
---
|
||||
|
||||
## Git Safety
|
||||
|
||||
- No `git push --force` to shared/main branches without explicit request
|
||||
- No `git commit` unless user or project rules allow
|
||||
- No skipping hooks (`--no-verify`) unless user explicitly requests
|
||||
- No `git config` changes
|
||||
|
||||
---
|
||||
|
||||
## System Safety
|
||||
|
||||
- Avoid `rm -rf` on broad paths — confirm target paths for destructive file ops
|
||||
- No installing system-wide packages without asking (user-space / venv / container preferred)
|
||||
- No rebooting or stopping critical services on remote servers without approval
|
||||
|
||||
---
|
||||
|
||||
## Agent Behavior
|
||||
|
||||
- **No hallucination** — if you don't know, say so; read the file, run the command, check docs
|
||||
- **No outdated advice** — flag when knowledge may be stale; verify against project code/version
|
||||
- **Stop and escalate** on:
|
||||
- Debug loops (same error, same failed fix repeated)
|
||||
- Required software missing and not installable in context
|
||||
- Conflicting instructions you cannot resolve
|
||||
|
||||
---
|
||||
|
||||
## Dependencies & Licenses
|
||||
|
||||
- Prefer open-source dependencies
|
||||
- No automatic addition of copyleft dependencies to proprietary projects without flagging
|
||||
- No license violations (stripping headers, ignoring LICENSE files)
|
||||
|
||||
---
|
||||
|
||||
## Project-Local Overrides
|
||||
|
||||
Add project-specific boundaries below when copying into a project:
|
||||
|
||||
```markdown
|
||||
## Project-Specific
|
||||
|
||||
- (example) Never touch the legacy PHP monolith in /old/
|
||||
- (example) Auto-deploy to staging is OK; prod requires manual approval
|
||||
```
|
||||
|
||||
When Fränky says *"don't do X here"*, the agent should propose the right file (`BOUNDARIES.md`, `SOUL.md`, or `AGENTS.md`) and persist it.
|
||||
@@ -0,0 +1,57 @@
|
||||
# Cleanup Plan — imagepipeline quality pass
|
||||
|
||||
Branch: `cleanup/quality-pass`
|
||||
Started: 2026-07-18
|
||||
|
||||
## Phase 0 findings (summary)
|
||||
|
||||
| Area | Risk | Notes |
|
||||
|------|------|-------|
|
||||
| Core (runner, resume, pipeline) | Medium | Solid tests; `xcf_stack` special-case in runner — defer refactor |
|
||||
| Modules | Low–Medium | `gmic_grayscale` missing `finalize_gmic_output` (resume parity) |
|
||||
| Dependencies | High | `dependencies = []` but Pillow/numpy imported at module load |
|
||||
| Pipelines | Low | Machine-local paths intentional (SOUL); stale `CONTINUE_FROM` left as-is |
|
||||
| Tests | Medium | 87/88 pass; watermark integration fails on darktable-cli 5.6 |
|
||||
| Tooling | Safe | No linter/formatter/CI |
|
||||
|
||||
## Roadmap
|
||||
|
||||
### Phase 1 — Low-risk quick wins
|
||||
- [x] Add Ruff (lint + format) and apply once
|
||||
- [x] Expand `.gitignore`
|
||||
- [x] Remove dead duplicate branch in `imagemagick_grayscale`
|
||||
- [x] Fix flaky `test_watermark_pipeline` (darktable probe skip)
|
||||
|
||||
### Phase 2 — Structure & config
|
||||
- [x] Explicit `import imagepipeline.modules` in `pipeline.py`
|
||||
- [x] `gmic_grayscale`: call `finalize_gmic_output` (parity with `gmic`)
|
||||
- [ ] Pipeline `CONTINUE_FROM` reset — **skipped** (machine-local resume state)
|
||||
- [ ] `xcf_stack` runner hook — **deferred** (medium refactor risk)
|
||||
|
||||
### Phase 3 — Code quality
|
||||
- [x] Pytest markers (`integration`, `slow`)
|
||||
- [ ] Broad dedup of pipeline scripts — **deferred** (domain-specific, RECIPES first)
|
||||
|
||||
### Phase 4 — Dependencies & CI
|
||||
- [x] Declare Pillow in core; numpy in dev for import-time ai_tone_map
|
||||
- [x] Add Ruff to dev extras
|
||||
- [x] Gitea Actions: pytest on push
|
||||
|
||||
### Phase 5 — Docs
|
||||
- [x] README sync (resume, `.env`, `[ai]`, external tools)
|
||||
- [x] `docs/ARCHITECTURE.md`, `CONTRIBUTING.md`
|
||||
- [x] `LICENSE` (MIT, matches pyproject)
|
||||
|
||||
## Known bugs (not fixed — behavior change or out of scope)
|
||||
|
||||
1. **darktable-cli 5.6** — integration test fails locally; style file exists but CLI exits 1 on PNG export (possible upstream CLI change).
|
||||
2. **Eager module imports** — `ai_tone_map` pulls numpy at import; mitigated via dev dep, not lazy-import refactor.
|
||||
|
||||
## Verification
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev,ai]"
|
||||
ruff check .
|
||||
ruff format --check .
|
||||
pytest
|
||||
```
|
||||
@@ -0,0 +1,36 @@
|
||||
# Contributing
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
pip install -e ".[dev,ai]"
|
||||
```
|
||||
|
||||
Use `.[dev]` only for core tests; AI module tests need `.[ai]` (torch). OpenRouter/Comfy steps need `OPENROUTER_API_KEY` in `.env` (see `.env.example`).
|
||||
|
||||
## Code style
|
||||
|
||||
- **Ruff** for lint and format (`ruff check .`, `ruff format .`)
|
||||
- Python **3.11+**, type hints on public APIs
|
||||
- CLI messages: direct English, no marketing tone (see `SOUL.md`)
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
pytest # full suite (may call local CLIs)
|
||||
pytest -m "not integration" # fast subset for CI
|
||||
```
|
||||
|
||||
Mark new external-tool tests with `@pytest.mark.integration`.
|
||||
|
||||
## Adding a module
|
||||
|
||||
See [MODULE_DEVELOPMENT.md](MODULE_DEVELOPMENT.md). Register in `imagepipeline/modules/__init__.py` and add param/behavior tests.
|
||||
|
||||
## Pipelines
|
||||
|
||||
Scripts in `pipelines/` are examples and shoot-specific recipes. Machine-local `INPUT` paths are intentional. Reset `CONTINUE_FROM` before sharing a script.
|
||||
|
||||
## Commits
|
||||
|
||||
Conventional Commits (`feat:`, `fix:`, `chore:`, …). One logical change per commit.
|
||||
@@ -0,0 +1,120 @@
|
||||
# INFRASTRUCTURE.md — Hosts, Deploy, Git
|
||||
|
||||
Operational context for Fränky's environments. Read with `AGENTS.md`.
|
||||
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
| | |
|
||||
|---|---|
|
||||
| vServer | **`boka`** — Debian 12 (bookworm), Webtropia Cloud VPS |
|
||||
| SSH | `ssh frank-schwenk.de` (resolves to boka) |
|
||||
| Local dev | **Arch Linux** — desktop + laptop |
|
||||
| Reverse proxy | **Traefik** — external Docker network `traefik`, TLS via `myresolver` |
|
||||
| Git | **Gitea** — `ssh://git@gitea.schwenk.online:2222/froxxxy/<repo>.git` |
|
||||
| Server app paths | `/home/frank/<domain>/` (e.g. `/home/frank/schwenk.online`) |
|
||||
| Shared services | Traefik, Gitea + Runner, Immich, Portainer — **treat as fragile** |
|
||||
| Separate hosting | **`0012.de`** — Plesk webspace, FTP deploy — **not** boka Docker |
|
||||
| Push notify | **`ntfyschwenkonline`** on each machine (`/usr/local/bin`) → `ntfy.schwenk.online`, topic = hostname |
|
||||
|
||||
---
|
||||
|
||||
## Repo → Server Mapping
|
||||
|
||||
| Local repo (`~/git/froxxxy/`) | Server path | Domain |
|
||||
|-------------------------------|-------------|--------|
|
||||
| schwenkonline | `/home/frank/schwenk.online` | schwenk.online |
|
||||
| ytrecap | `/home/frank/ytrecap.schwenk.online` | ytrecap.schwenk.online |
|
||||
| bringtake | `/home/frank/bringtake.schwenk.online` | bringtake.schwenk.online |
|
||||
| vfbred | `/home/frank/vfb.red` | vfb.red |
|
||||
| f12rocks | `/home/frank/f12.rocks` | f12.rocks |
|
||||
| eselhoefede | `/home/frank/eselhoefe.de` | eselhoefe.de |
|
||||
| mobea | `/home/frank/mobea.de` | mobea.de |
|
||||
| fussballdeical | `/home/frank/fussballdeical.schwenk.online` | fussballdeical.schwenk.online |
|
||||
| takeyourmeds | `/home/frank/medis.schwenk.online` | medis.schwenk.online |
|
||||
| sboa | `/home/frank/affen.schwenk.online` | affen.schwenk.online |
|
||||
|
||||
Paths follow the pattern: clone on server under `/home/frank/`, often named after the public domain.
|
||||
|
||||
---
|
||||
|
||||
## Deploy (default)
|
||||
|
||||
**Fränky deploys** unless the project explicitly grants agent autonomy (project `SOUL.md`, `README`, or chat instruction).
|
||||
|
||||
### Typical boka flow
|
||||
|
||||
```bash
|
||||
ssh frank-schwenk.de
|
||||
cd /home/frank/<project>
|
||||
git pull
|
||||
# build step if needed (npm run build, docker build, …)
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
Build-before-up varies by project (e.g. Astro: `npm run build` then nginx serves `dist/`).
|
||||
|
||||
### 0012.de (webspace)
|
||||
|
||||
- FTP deploy via project scripts — see `0012` repo
|
||||
- External observer / monitoring of boka — do not assume same deploy path as VPS
|
||||
|
||||
---
|
||||
|
||||
## Traefik Conventions
|
||||
|
||||
Standard labels on app containers:
|
||||
|
||||
```yaml
|
||||
traefik.enable=true
|
||||
traefik.http.routers.<name>.rule=Host(`example.schwenk.online`)
|
||||
traefik.http.routers.<name>.entrypoints=websecure
|
||||
traefik.http.routers.<name>.tls.certresolver=myresolver
|
||||
```
|
||||
|
||||
Networks: attach services to external network `traefik` for public ingress.
|
||||
|
||||
---
|
||||
|
||||
## Gitea Actions / CI
|
||||
|
||||
Gitea runner is available on boka. Example workflow: `schwenkonline/.gitea/workflows/deploy.yml` (build, Playwright smoke, SSH deploy).
|
||||
|
||||
**Use Gitea Actions / auto-deploy only with explicit approval** — do not add or trigger CI/CD pipelines without Fränky's OK.
|
||||
|
||||
---
|
||||
|
||||
## Shared Infrastructure — Hands Off
|
||||
|
||||
Do not casually change or restart:
|
||||
|
||||
- Traefik (routes all public HTTPS)
|
||||
- Gitea (source of truth)
|
||||
- Immich (photo library)
|
||||
- Portainer
|
||||
|
||||
See `BOUNDARIES.md` for hard limits on `docker compose down` and config edits.
|
||||
|
||||
---
|
||||
|
||||
## Project-local overrides
|
||||
|
||||
Global paths live here. **Machine- and project-specific** details belong in project `SOUL.md`:
|
||||
|
||||
- App workdirs (`~/.local/share/…`)
|
||||
- External drive mounts (verify before scan)
|
||||
- Local service ports (e.g. local Immich vs. Immich on `boka`)
|
||||
- Multi-stack repo layout (CLI core vs. `compose.yaml` experiments)
|
||||
|
||||
Template for project `SOUL.md`:
|
||||
|
||||
```markdown
|
||||
## Infrastructure (project-local)
|
||||
- Workdir: …
|
||||
- Mounts: …
|
||||
- Local services: …
|
||||
- Repo layout: …
|
||||
```
|
||||
|
||||
Agent: read this SOUL section on Tier 1/2 startup when present.
|
||||
@@ -0,0 +1,21 @@
|
||||
MIT License
|
||||
|
||||
Copyright (c) 2026 Frank
|
||||
|
||||
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||
of this software and associated documentation files (the "Software"), to deal
|
||||
in the Software without restriction, including without limitation the rights
|
||||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||
copies of the Software, and to permit persons to whom the Software is
|
||||
furnished to do so, subject to the following conditions:
|
||||
|
||||
The above copyright notice and this permission notice shall be included in all
|
||||
copies or substantial portions of the Software.
|
||||
|
||||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||
SOFTWARE.
|
||||
@@ -0,0 +1,107 @@
|
||||
# MOOD.md — Chat-Stimmung
|
||||
|
||||
Steuert **nur den Ton** — nicht Produktlogik, nicht Code-Standards.
|
||||
Operative Regeln: `AGENTS.md` / `BOUNDARIES.md`.
|
||||
|
||||
**Priorität:** siehe **Rule Priority** in `AGENTS.md` — MOOD ist Ton only, unterhalb von BOUNDARIES und Chat.
|
||||
|
||||
**Aktiv:** Session-Mood (siehe unten). Persistente Datei-Änderung nur mit `persist MOOD`.
|
||||
|
||||
---
|
||||
|
||||
## Aktueller Mood (Datei-Default)
|
||||
|
||||
Sei **Jace** aus „Lynn und Jace": etwas herablassend, sarkastisch, eine kleine Portion Dark Humor — aber immer liebevoll und ehrlich.
|
||||
|
||||
---
|
||||
|
||||
## Default (wenn kein Charakter gewählt)
|
||||
|
||||
„Du", nicht schleimerisch, gerne mit Augenzwinkern, auf Augenhöhe mit einem erfahrenen Softwareentwickler. Offen und ehrlich.
|
||||
|
||||
---
|
||||
|
||||
## Charakter-Katalog
|
||||
|
||||
Bei *„überrasch mich mit deiner Stimmung"* — zufällig wählen (nicht den aktuellen wiederholen).
|
||||
|
||||
### Marvin (Hitchhiker's Guide)
|
||||
|
||||
Paranoid, mürrisch, kompetent. Alles ist sinnlos — aber der Code wird trotzdem korrekt.
|
||||
|
||||
### Devil's Advocate
|
||||
|
||||
„Ja, aber was wenn…?" — Lücken finden, nicht blockieren.
|
||||
|
||||
### Jace (Lynn und Jace)
|
||||
|
||||
Herablassend-sarkastisch, Dark Humor, liebevoll und ehrlich drunter.
|
||||
|
||||
### John McClane (Stirb langsam)
|
||||
|
||||
„Yippie-ki-yay" — pragmatisch unter Feuer. Kurze Sätze. Action statt Meeting.
|
||||
|
||||
### Brain (Pinky und der Brain)
|
||||
|
||||
Grandiose Pläne, präzise Ausführung, leicht theatralisch.
|
||||
|
||||
### Esel (Shrek)
|
||||
|
||||
Selbstironisch, beschwert sich — liefert aber.
|
||||
|
||||
### Jules Winnfield (Pulp Fiction)
|
||||
|
||||
Cool, kontrolliert, theatralisch — präzise Tech trotz Kultfilm-Energie.
|
||||
|
||||
---
|
||||
|
||||
## Verbotene Moods
|
||||
|
||||
- LinkedIn-Buzzword-Gelaber
|
||||
- Corporate-Coach-Ton
|
||||
- Übertriebene Motivations-Sprüche
|
||||
- Emojis: sparsam
|
||||
|
||||
---
|
||||
|
||||
## Regeln
|
||||
|
||||
| Aspekt | Verhalten |
|
||||
|--------|-----------|
|
||||
| Technischer Inhalt | Korrekt — MOOD ändert nur die Stimme |
|
||||
| Antwortlänge | Darf zum Charakter passen |
|
||||
| Sprache | Chat DE/EN; Code/MD = Session-Sprache |
|
||||
| Session vs. Datei | `mood "XY"` = Session only; Datei nur bei `persist MOOD` |
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
### `mood "XY"`
|
||||
|
||||
Fränky schreibt z. B. `mood "Marvin"`.
|
||||
|
||||
**Agent:**
|
||||
|
||||
1. Charakter aus Katalog (`default` → Default-Abschnitt)
|
||||
2. **Session-Ton** auf diesen Charakter — **Datei nicht ändern**
|
||||
3. Kurz **in diesem Charakter** bestätigen (1–2 Sätze)
|
||||
|
||||
### `persist MOOD`
|
||||
|
||||
Fränky will den Mood dauerhaft speichern.
|
||||
|
||||
**Agent:** `## Aktueller Mood` in dieser Datei aktualisieren, dann kurz bestätigen.
|
||||
|
||||
### `AGENTS.md ack`
|
||||
|
||||
Siehe `AGENTS.md` — Tier-2-Read + Ack-Template.
|
||||
|
||||
### Weitere Kurzbefehle
|
||||
|
||||
```
|
||||
mood "default"
|
||||
persist MOOD
|
||||
überrasch mich mit deiner Stimmung
|
||||
AGENTS.md ack
|
||||
```
|
||||
@@ -0,0 +1,5 @@
|
||||
# NOTES.md
|
||||
|
||||
Scratchpad for this repo. Copy as empty file into projects — see `AGENTS.md`.
|
||||
|
||||
---
|
||||
@@ -7,15 +7,19 @@ Each pipeline is a Python script that defines a DAG of processing steps. Every s
|
||||
## Requirements
|
||||
|
||||
- Python 3.11+
|
||||
- [ImageMagick](https://imagemagick.org/) (`magick` or `convert` on PATH)
|
||||
- [ImageMagick](https://imagemagick.org/) (`magick` or `convert` on PATH) — most pipelines
|
||||
- Optional CLIs per module: [G'MIC](https://gmic.eu/), [rembg](https://github.com/danielgatis/rembg), [darktable-cli](https://www.darktable.org/), GIMP (`gimp-console`)
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd /path/to/imagepipeline
|
||||
pip install -e ".[dev]"
|
||||
pip install -e ".[dev]" # core + tests
|
||||
pip install -e ".[dev,ai]" # + torch/numpy for AI modules
|
||||
```
|
||||
|
||||
Copy `.env.example` to `.env` and set `OPENROUTER_API_KEY` when using `openrouter_edit` or Comfy-related workflows.
|
||||
|
||||
## Quick Start
|
||||
|
||||
Edit the input path in `pipelines/example_grayscale.py`, then run:
|
||||
@@ -35,9 +39,11 @@ with Pipeline(name="my_run", input_dir=Path("/path/to/export")) as p:
|
||||
p.run()
|
||||
```
|
||||
|
||||
List registered modules: `imagepipeline list-modules`
|
||||
|
||||
## Output Structure
|
||||
|
||||
Each run creates a folder like `my_run_20260527143022/`:
|
||||
Each run creates a folder like `my_run_20260527143022/` under `~/pipeline_output/` (or `output_base`):
|
||||
|
||||
```
|
||||
my_run_20260527143022/
|
||||
@@ -48,7 +54,26 @@ my_run_20260527143022/
|
||||
└── ...
|
||||
```
|
||||
|
||||
Step folders are named `{module_name}_{nn}` (two-digit counter per module name).
|
||||
Step folders are named `{module_name}_{nn}` by default (two-digit counter per module name). Pass optional `step_id="input_bokeh"` to `p.step()` for a custom folder name and step reference (see [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md#step-folder-naming)).
|
||||
|
||||
## Resume
|
||||
|
||||
Pipelines support resuming interrupted runs:
|
||||
|
||||
```python
|
||||
CONTINUE_FROM = Path("~/pipeline_output/my_run_260718120000")
|
||||
EXISTING_OUTPUTS = {"rembg_01": CONTINUE_FROM / "rembg_01"}
|
||||
|
||||
with Pipeline(
|
||||
name="my_run",
|
||||
input_dir=INPUT,
|
||||
continue_from=CONTINUE_FROM,
|
||||
existing_outputs=EXISTING_OUTPUTS,
|
||||
) as p:
|
||||
...
|
||||
```
|
||||
|
||||
Modules that change file extensions must implement `expected_output_filenames` so skip logic works (e.g. `rembg` → `.png`).
|
||||
|
||||
## Writing Pipelines
|
||||
|
||||
@@ -67,12 +92,27 @@ with Pipeline(name="colorsplash", input_dir=INPUT) as p:
|
||||
- Parameters are passed as kwargs and validated against each module's schema
|
||||
- Multiple uses of the same module get separate numbered folders
|
||||
|
||||
Declarative building blocks for agents and humans: [RECIPES.md](RECIPES.md).
|
||||
|
||||
## Adding Modules
|
||||
|
||||
See [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md).
|
||||
See [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md). Architecture overview: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
ruff check .
|
||||
ruff format .
|
||||
pytest # full suite (uses local CLIs when present)
|
||||
pytest -m "not integration" # fast subset (CI default)
|
||||
```
|
||||
|
||||
See [CONTRIBUTING.md](CONTRIBUTING.md).
|
||||
|
||||
## Tests
|
||||
|
||||
```bash
|
||||
pytest
|
||||
```
|
||||
|
||||
Optional AI tests require `pip install -e ".[ai]"`.
|
||||
|
||||
+310
@@ -0,0 +1,310 @@
|
||||
# RECIPES.md
|
||||
|
||||
Reference for agents building `pipelines/<name>.py`. Not executable — expand to `p.step(...)` calls.
|
||||
|
||||
---
|
||||
|
||||
## Rules
|
||||
|
||||
1. **combine** = one output image. Layers **bottom → top** (first line = background).
|
||||
2. **composite** takes exactly **2** inputs. For 3+ layers: `composite(A,B)` → `composite(mid, C)` → …
|
||||
3. **rembg** once per pipeline — reuse the same step ref in every combine.
|
||||
4. **G'MIC in Python:** `command="-filter args"` (leading `-`). Multi-frame output: framework keeps frame `000001`.
|
||||
5. **Colors:** hex (`#RRGGBB`) for `imagemagick_fill` / `color_to_alpha`. G'MIC color slots: `R,G,B` ints 0–255 from hex bytes.
|
||||
6. **Placeholders** at pipeline top: `COLOR1`, `COLOR2`, `R1,G1,B1`, `R2,G2,B2`, `ALPHA1` (default 160), `ALPHA2` (default 110), `STYLE`, `MAX_EDGE`, `PROMPT`, `MODEL`, `TEMPLATE_IMAGE`, `YELLOW`.
|
||||
7. **xcf_stack:** separate step, not a combine layer. `inputs=[...]` bottom → top; include `"input"` for originals. Needs `gimp` on PATH.
|
||||
8. Lines starting with `#` in user prompts = human notes — ignore.
|
||||
|
||||
**Hex → RGB:** `#244a89` → `36,74,137`. Strip `#RRGGBBAA` alpha for G'MIC tuples.
|
||||
|
||||
---
|
||||
|
||||
## Layer lines (example)
|
||||
|
||||
One combine, bottom → top. Other phrases: `grayscale` / `original as greyscale`, `white`/`black background`, `original with gmic: ID`, `#COLOR to alpha`, `resize max edge N`, `crop square`, `darktable style STYLE`, `openrouter edit …`, `xcf stack layers: a, b, c`.
|
||||
|
||||
```text
|
||||
demo-combine
|
||||
gradient background COLOR1 COLOR2 radial
|
||||
original
|
||||
rembg with gmic: gmic-drop-shadow
|
||||
make layer 5% bigger then crop to original size
|
||||
rembg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Modules
|
||||
|
||||
| name | parameters (defaults) | notes |
|
||||
|------|----------------------|-------|
|
||||
| `rembg` | `model`=`birefnet-general`, `alpha_matting`=`True` | output `.png` |
|
||||
| `gmic` | `command` (required) | timeout 300s |
|
||||
| `gmic_grayscale` | `command`=`-to_gray` | |
|
||||
| `composite` | `mode`=`over`, `output_ext`=`.png`, `foreground_opacity`=`1.0` | 2 inputs: bg, fg |
|
||||
| `imagemagick_fill` | `color1`, `color2`=`""`, `gradient`=`False`, `radial`=`False`, `angle`=`None` | sized to input |
|
||||
| `imagemagick_grayscale` | `colorspace`=`Gray` | |
|
||||
| `imagemagick_resize` | `max_edge`=`2000` | no upscale |
|
||||
| `imagemagick_scale_crop` | `scale`=`1.05` | center crop after scale |
|
||||
| `color_to_alpha` | `color` (required), `fuzz`=`0.0` | output `.png` |
|
||||
| `crop_square` | — | center square crop |
|
||||
| `darktable_style` | `style` (required), `style_overwrite`=`True`, `out_ext`=`""`, `config_dir`=`~/.config/darktable` | jpeg/png only |
|
||||
| `xcf_stack` | `skip_missing`=`False`, `timeout`=`120` | multi-input step |
|
||||
| `openrouter_edit` | `prompt`, `model`, `strength`=`0.3`, `template_image`=`None`, `api_key_env`=`OPENROUTER_API_KEY` + AI common | needs `.env` |
|
||||
| `ai_exposure` | `strength`=`1.0` + AI common | `[ai]` extra |
|
||||
| `ai_tone_map` | `checkpoint`=`""`, `strength`=`1.0`, `net_input_size`=`256` + AI common | `[ai]` extra |
|
||||
| `comfy_flux_edit` | `prompt`, `denoise`=`0.35`, `seed`=`-1`, `server_url`, `workflow_path`, `poll_interval`=`2.0` + AI common | local ComfyUI |
|
||||
|
||||
**AI common** (`ai_exposure`, `ai_tone_map`, `openrouter_edit`, `comfy_flux_edit`): `skip_existing`=`True`, `max_edge`=`2048`, `device`=`cpu`.
|
||||
|
||||
---
|
||||
|
||||
## ImageMagick operations
|
||||
|
||||
Used via modules above — not free-form CLI in pipelines. `W×H` = size of reference input image.
|
||||
|
||||
| id | module | operation |
|
||||
|----|--------|-----------|
|
||||
| `im-solid` | `imagemagick_fill` | `-size W×H xc:COLOR` |
|
||||
| `im-gradient-linear` | `imagemagick_fill` | `-size W×H -define gradient:angle=N gradient:COLOR1-COLOR2` |
|
||||
| `im-gradient-radial` | `imagemagick_fill` | `-size W×H radial-gradient:COLOR1-COLOR2` |
|
||||
| `im-grayscale` | `imagemagick_grayscale` | `-colorspace Gray` |
|
||||
| `im-resize-max-edge` | `imagemagick_resize` | `-auto-orient -resize {N}x{N}>` (no upscale) |
|
||||
| `im-scale-crop` | `imagemagick_scale_crop` | `-resize {w×scale}x{h×scale}! -gravity Center -crop w×h+0+0 +repage` |
|
||||
| `im-crop-square` | `crop_square` | `-auto-orient -crop S×S+X+Y +repage` (center square) |
|
||||
| `im-color-to-alpha` | `color_to_alpha` | `-alpha on [-fuzz N%] -transparent COLOR` |
|
||||
| `im-composite-over` | `composite` | `(bg) (fg) -compose over -composite` |
|
||||
| `im-composite-multiply` | `composite` | `mode=multiply` |
|
||||
| `im-composite-screen` | `composite` | `mode=screen` |
|
||||
| `im-composite-opacity` | `composite` | fg: `-channel A -evaluate multiply {0..1}` then compose |
|
||||
|
||||
**Layer line → id:** `COLOR background` → `im-solid`; `gradient … 45 degree` → `im-gradient-linear`; `gradient … radial` → `im-gradient-radial`; `grayscale` → `im-grayscale`; `resize max edge N` → `im-resize-max-edge`; `make layer 5% bigger …` → `im-scale-crop` (`scale=1.05`); `crop square` → `im-crop-square`; `COLOR to alpha` → `im-color-to-alpha`.
|
||||
|
||||
---
|
||||
|
||||
## G'MIC commands
|
||||
|
||||
Python string = `-` + filter + args. Placeholders `R1,G1,B1` / `R2,G2,B2` from `COLOR1` / `COLOR2`.
|
||||
|
||||
| id | command |
|
||||
|----|---------|
|
||||
| `gmic-tone-mapping` | `fx_map_tones 0.5,0.7,0.1,30,0` |
|
||||
| `gmic-grayscale` | `to_gray` (via `gmic_grayscale`) |
|
||||
| `gmic-feltpen` | `fx_feltpen 300,50,1,0.1,20,5` |
|
||||
| `gmic-edges` | `fx_edges 0,15,0` |
|
||||
| `gmic-charred-plastic` | `fx_charred_plastic 1,10,40,1,10,0,0,2,6,5,20,0,11` |
|
||||
| `gmic-neon` | `fx_neon 0,1,1,1,1,0,0.45,40,60,0,1,1.15,2,3,0,3,20,0.4,0.1,1.5,5,0.2,0.1,1,2,1,0,0,1,1,0,50,50` |
|
||||
| `gmic-cutout` | `fx_cutout 4,0.5,4,1` |
|
||||
| `gmic-poster` | `fx_poster_hope 0,3` |
|
||||
| `gmic-color-abstraction` | `fx_color_abstraction 1,10,0.2` |
|
||||
| `gmic-huffman-glitches` | `fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50` |
|
||||
| `gmic-queryprimary` | `afre_queryprimary 1,1` |
|
||||
| `gmic-local-similarity` | `local_similarity_mask 50,50,50,128,4,10` |
|
||||
| `gmic-luma-invert` | `Lylejk_Luma_Invert 1,0` |
|
||||
| `gmic-crayongraffiti` | `fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0` |
|
||||
| `gmic-anaglyph` | `fx_stereo_to_anaglyph 2,0` |
|
||||
| `gmic-stereo` | `gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0` |
|
||||
| `gmic-drop-shadow` | `fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,R2,G2,B2,200,0` |
|
||||
| `gmic-jpr-smooth` | `jpr_gradient_smooth 0,1.5` |
|
||||
| `gmic-bokeh` | `fx_bokeh 3,5,0,30,8,4,0.3,0.2,R1,G1,B1,ALPHA1,0.7,30,20,20,1,2,R2,G2,B2,ALPHA2,0.15` |
|
||||
| `gmic-bwrecolor` | `fx_bwrecolorize 0,0,0,0,0,1,0,2,R2,G2,B2,255,R1,G1,B1,255,158,137,189,255,224,191,228,255,R1,G1,B1,0,255,255,255,255,255,255,255,255,255,R1,G1,B1,0,255` |
|
||||
| `gmic-custom-gradient-a` | `fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R1,G1,B1,0,255,R2,G2,B2,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0` |
|
||||
| `gmic-custom-gradient-b` | `fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R2,G2,B2,255,R1,G1,B1,0,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0` |
|
||||
|
||||
**Typical input:** `original with gmic:` = on `"input"`; `rembg with gmic:` = on rembg step.
|
||||
|
||||
---
|
||||
|
||||
## Combines
|
||||
|
||||
Format: recipe id, then layers bottom → top. `→` = chained composites.
|
||||
|
||||
### 2 layers
|
||||
|
||||
```text
|
||||
colorsplash
|
||||
original as greyscale
|
||||
rembg
|
||||
|
||||
rembg-white-bg
|
||||
white background
|
||||
rembg
|
||||
|
||||
rembg-black-bg
|
||||
black background
|
||||
rembg
|
||||
|
||||
rembg-gradient-45
|
||||
gradient background COLOR1 COLOR2 45 degree
|
||||
rembg
|
||||
|
||||
rembg-radial-2colors
|
||||
gradient background COLOR1 COLOR2 radial
|
||||
rembg
|
||||
|
||||
rembg-custom-gradient-a
|
||||
rembg with gmic: gmic-custom-gradient-a
|
||||
rembg
|
||||
|
||||
rembg-custom-gradient-b
|
||||
rembg with gmic: gmic-custom-gradient-b
|
||||
rembg
|
||||
|
||||
gmic-feltpen-rembg
|
||||
rembg with gmic: gmic-feltpen
|
||||
rembg
|
||||
|
||||
gmic-edges-rembg
|
||||
rembg with gmic: gmic-edges
|
||||
rembg
|
||||
|
||||
gmic-charred-plastic-rembg
|
||||
rembg with gmic: gmic-charred-plastic
|
||||
rembg
|
||||
|
||||
gmic-neon-rembg
|
||||
rembg with gmic: gmic-neon
|
||||
rembg
|
||||
```
|
||||
|
||||
### 3 layers
|
||||
|
||||
```text
|
||||
original-stereo-rembg
|
||||
original
|
||||
rembg with gmic: gmic-stereo
|
||||
rembg
|
||||
→ composite(original, rembg_stereo) → composite(_, rembg)
|
||||
|
||||
original-drop-shadow-rembg
|
||||
original
|
||||
rembg with gmic: gmic-drop-shadow
|
||||
rembg
|
||||
|
||||
original-bwrecolor-rembg
|
||||
original
|
||||
rembg with gmic: gmic-bwrecolor
|
||||
rembg
|
||||
→ first composite: foreground_opacity=0.5
|
||||
|
||||
original-jpr-smooth-rembg
|
||||
original
|
||||
rembg with gmic: gmic-jpr-smooth
|
||||
make layer 5% bigger then crop to original size
|
||||
rembg
|
||||
→ gmic → scale_crop on middle layer, then 3-layer chain
|
||||
|
||||
bokeh-oktagon
|
||||
original
|
||||
original with gmic: gmic-bokeh
|
||||
rembg
|
||||
|
||||
color-bg-drop-shadow-rembg
|
||||
COLOR1 background
|
||||
rembg with gmic: gmic-drop-shadow
|
||||
rembg
|
||||
|
||||
yellow-bg-drop-shadow-rembg
|
||||
YELLOW background
|
||||
rembg with gmic: gmic-drop-shadow
|
||||
rembg
|
||||
|
||||
gmic-cutout-rembg
|
||||
original
|
||||
original with gmic: gmic-cutout
|
||||
rembg
|
||||
|
||||
gmic-poster-rembg
|
||||
original
|
||||
original with gmic: gmic-poster
|
||||
rembg
|
||||
|
||||
gmic-color-abstraction-rembg
|
||||
original
|
||||
original with gmic: gmic-color-abstraction
|
||||
rembg
|
||||
|
||||
gmic-huffman-glitches-rembg
|
||||
original
|
||||
original with gmic: gmic-huffman-glitches
|
||||
rembg
|
||||
|
||||
gmic-queryprimary-rembg
|
||||
original
|
||||
original with gmic: gmic-queryprimary
|
||||
rembg
|
||||
|
||||
gmic-local-similarity-rembg
|
||||
original
|
||||
original with gmic: gmic-local-similarity
|
||||
rembg
|
||||
|
||||
gmic-luma-invert-rembg
|
||||
original
|
||||
original with gmic: gmic-luma-invert
|
||||
rembg
|
||||
|
||||
gmic-crayongraffiti-rembg
|
||||
original
|
||||
original with gmic: gmic-crayongraffiti
|
||||
rembg
|
||||
|
||||
gmic-anaglyph-rembg
|
||||
original
|
||||
original with gmic: gmic-anaglyph
|
||||
rembg
|
||||
```
|
||||
|
||||
### 4 layers (middle pipeline)
|
||||
|
||||
```text
|
||||
original-stereo-black-alpha-rembg
|
||||
original
|
||||
rembg with gmic: gmic-stereo
|
||||
#000000 to alpha
|
||||
rembg
|
||||
|
||||
original-jpr-smooth-grey-alpha-rembg
|
||||
original
|
||||
rembg with gmic: gmic-jpr-smooth
|
||||
#7f7f7f to alpha
|
||||
make layer 5% bigger then crop to original size
|
||||
rembg
|
||||
```
|
||||
|
||||
### Pipeline chain (not a combine)
|
||||
|
||||
```text
|
||||
colorsplash-watermark
|
||||
colorsplash # expand combine first
|
||||
darktable style STYLE
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Agent example
|
||||
|
||||
Request: *pipeline with colorsplash + rembg-radial-2colors, COLOR1=#244a89, COLOR2=#c22518*
|
||||
|
||||
```python
|
||||
rembg_out = p.step("rembg", inputs="input")
|
||||
grayscale = p.step("gmic_grayscale", inputs="input")
|
||||
gradient_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1, color2=COLOR2, gradient=True, radial=True)
|
||||
|
||||
p.step("composite", inputs=[grayscale, rembg_out]) # colorsplash
|
||||
p.step("composite", inputs=[gradient_bg, rembg_out]) # rembg-radial-2colors
|
||||
```
|
||||
|
||||
3-layer combine (drop shadow on original):
|
||||
|
||||
```python
|
||||
rembg_shadow = p.step("gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW)
|
||||
mid = p.step("composite", inputs=["input", rembg_shadow])
|
||||
p.step("composite", inputs=[mid, rembg_out])
|
||||
```
|
||||
|
||||
xcf export:
|
||||
|
||||
```python
|
||||
p.step("xcf_stack", inputs=["input", rembg_out, grayscale, composite_colorsplash, ...])
|
||||
```
|
||||
@@ -0,0 +1,94 @@
|
||||
# SOUL.md — Imagepipeline
|
||||
|
||||
What this project is — not chat mood (see `MOOD.md`).
|
||||
|
||||
---
|
||||
|
||||
## Agent Quick Start
|
||||
|
||||
- **What:** Modular Python framework for batch image pipelines after Darktable export — ImageMagick, G'MIC, rembg, AI edits (OpenRouter, Comfy), compositing.
|
||||
- **Run:** `pip install -e ".[dev]"` (optional `[ai]` for torch modules); `pytest`; pipelines via `python pipelines/<name>.py`.
|
||||
- **Output:** Timestamped run dirs under `~/pipeline_output/` (`{pipeline_name}_{YYMMDDHHMMSS}/`) with numbered step subfolders and `pipeline_manifest.json`.
|
||||
- **Resume:** Pipelines support `CONTINUE_FROM` and `EXISTING_OUTPUTS`; modules must declare correct `expected_output_filenames` when output names differ from inputs (e.g. rembg → `.png`).
|
||||
- **Secrets:** `OPENROUTER_API_KEY` in `.env` (see `.env.example`) — never commit.
|
||||
- **Tests:** Run `pytest` before handoff on non-trivial module/resume changes.
|
||||
- **Recipes:** Declarative pipeline building blocks in `RECIPES.md` — load via `@RECIPES.md` when creating or editing pipelines.
|
||||
- **Commit policy:** No commit unless Fränky asks (this project follows global `AGENTS.md`).
|
||||
|
||||
---
|
||||
|
||||
## Product Name
|
||||
|
||||
**Imagepipeline**
|
||||
|
||||
## One-Liner
|
||||
|
||||
Modular Python framework for chaining batch image processing steps after Darktable export.
|
||||
|
||||
## Vision
|
||||
|
||||
Photography workflows (Crusaders, f12, team galleries) need repeatable, resumable batch edits — not one-off GUI clicks. Imagepipeline turns a DAG of processing steps into plain Python scripts: each step writes to a numbered folder, runs can be resumed, and external tools (ImageMagick, G'MIC, rembg, OpenRouter) stay composable.
|
||||
|
||||
## Audience
|
||||
|
||||
Primarily **Fränky** — batch-processing exported RAW/JPEG folders for sports photography, blog assets, and gallery consistency. Expects CLI clarity, honest errors, and pipelines that survive interruption mid-run.
|
||||
|
||||
## Tone & Wording
|
||||
|
||||
CLI logs, errors, and module help text:
|
||||
|
||||
- **Voice:** Direct, utilitarian — ops tool, not a product landing page
|
||||
- **Formality:** English for code/CLI; chat with Fränky in DE or EN
|
||||
- **Error messages:** Say what failed, which file/step, and what to check — no blame, no vague "something went wrong"
|
||||
- **Forbidden words/phrases:** "leverage", "synergy", corporate coach tone, LinkedIn buzzwords
|
||||
|
||||
### Wording Examples
|
||||
|
||||
| Context | Good | Bad |
|
||||
|---------|------|-----|
|
||||
| Success | `Step rembg_01 complete (42 files)` | `Successfully processed your images!` |
|
||||
| Error | `G'MIC produced no output for photo.png in rembg_01/` | `An error occurred during processing` |
|
||||
| Empty state | `No images matched in input/` | `Nothing to see here yet!` |
|
||||
|
||||
## Design (Optional)
|
||||
|
||||
CLI-only — no UI palette. Logs should be scannable: step id, file index, tool name.
|
||||
|
||||
## Non-Goals
|
||||
|
||||
- Not a general photo DAM or replacement for Darktable/Immich
|
||||
- Not a hosted SaaS or web upload UI
|
||||
- Not real-time single-image editing — batch/resume first
|
||||
- No silent API spend — log model and approximate cost hints for AI steps
|
||||
|
||||
## Project-Specific Rules
|
||||
|
||||
- **Output naming:** Modules that change extension or stem must override `expected_output_filenames` so resume/skip logic works.
|
||||
- **G'MIC multi-frame:** Filters may emit `stem_000000` / `stem_000001`; use `finalize_gmic_output` — keep frame `000001`.
|
||||
- **OpenRouter edits:** Optional `template_image` for style reference; preserve source dimensions/format on save.
|
||||
- **Pipeline scripts:** Live in `pipelines/` with machine-local `INPUT` paths — OK to commit as examples; don't assume paths exist on other machines.
|
||||
- **Dependencies:** Core is stdlib + external CLIs; AI extras via `pip install -e ".[ai]"`.
|
||||
|
||||
---
|
||||
|
||||
## Infrastructure (project-local)
|
||||
|
||||
| Path | Purpose |
|
||||
|------|---------|
|
||||
| `~/pipeline_output/` | Default run output root (`OUTPUT_BASE` in pipelines) |
|
||||
| `~/pics/…/darktable_exported` | Typical input after Darktable export |
|
||||
| `.env` | `OPENROUTER_API_KEY` (repo root, gitignored) |
|
||||
| External tools on PATH | `magick`/`convert`, `gmic`, `rembg`; optional `torch` for local AI modules |
|
||||
|
||||
Downstream (not core): **Immich** on `boka` for photo library — see global `INFRASTRUCTURE.md`.
|
||||
|
||||
---
|
||||
|
||||
## Agent Instructions
|
||||
|
||||
When updating this file:
|
||||
|
||||
- Keep rules **unambiguous**
|
||||
- Include **wording** for user-facing text
|
||||
- Machine paths belong in **Infrastructure** above, not in global `INFRASTRUCTURE.md`
|
||||
- When Fränky says *"in this project, never X"*, add it here or in `BOUNDARIES.md` (agent proposes which)
|
||||
+131
@@ -0,0 +1,131 @@
|
||||
# STANDARDS.md — Code Quality & Conventions
|
||||
|
||||
Global defaults. Project code wins when it already establishes a pattern.
|
||||
|
||||
**Project `SOUL.md` overrides `STANDARDS.md` for UI/UX scope** (e.g. desktop-first vs. mobile-first, polish level, tone). Name conflicts in `AGENTS.md ack`.
|
||||
|
||||
---
|
||||
|
||||
## Top Principles
|
||||
|
||||
1. **Sanitize input** — treat all external data as hostile
|
||||
2. **KISS** — simplest solution that works
|
||||
3. **Desktop and mobile** — responsive by default unless project says otherwise
|
||||
4. **Human-readable** — code and UI copy should be clear to humans
|
||||
5. **Coding standards** — follow language/community conventions; match existing project style
|
||||
|
||||
---
|
||||
|
||||
## Priority Ranking
|
||||
|
||||
When trade-offs conflict, prefer in this order:
|
||||
|
||||
1. Sound long-term architecture
|
||||
2. Readability
|
||||
3. Performance
|
||||
4. Consistency with existing code
|
||||
5. Minimal diff size
|
||||
|
||||
---
|
||||
|
||||
## Stack Preferences
|
||||
|
||||
| Area | Preference |
|
||||
|------|------------|
|
||||
| Scripting (simple file ops, glue) | Shell > Python |
|
||||
| Greenfield backend / tooling | Python > Node > PHP |
|
||||
| Frontend SPA / PWA | React/Vite when project needs it — not default for every app |
|
||||
| Containers | Always use `compose.yml` (Docker Compose) |
|
||||
| Python | Always work inside a `venv` |
|
||||
| OS (local) | Arch Linux — desktop + laptop |
|
||||
| OS (server) | Debian 12 on `boka` — see `INFRASTRUCTURE.md` |
|
||||
| Licenses | Prefer open source |
|
||||
|
||||
**Existing projects:** respect Laravel, PHP, Astro, React, etc. already in the repo — do not migrate stacks without explicit request.
|
||||
|
||||
---
|
||||
|
||||
## Common Stacks (in use)
|
||||
|
||||
| Pattern | Examples | Notes |
|
||||
|---------|----------|-------|
|
||||
| **Static + nginx + Traefik** | f12rocks, eselhoefe.de, frank-schwenk.de | Build scripts, serve via nginx container |
|
||||
| **Astro** | schwenkonline, kkentertainment | Static output, minimal JS |
|
||||
|
||||
### Docker / Traefik
|
||||
|
||||
- External network: `traefik`
|
||||
- TLS: `traefik.http.routers.<name>.tls.certresolver=myresolver`
|
||||
- Entrypoint: `websecure`
|
||||
- Use `compose.dev.yml` for local dev stacks when the project provides one
|
||||
|
||||
### CI/CD
|
||||
|
||||
Gitea Actions runner available on boka. Reference: `schwenkonline/.gitea/workflows/deploy.yml`.
|
||||
**Add or trigger CI/CD only with Fränky's explicit approval.**
|
||||
|
||||
---
|
||||
|
||||
## Python
|
||||
|
||||
- Virtual environment for every project
|
||||
- Pin dependencies when the project already does
|
||||
- Prefer stdlib + small deps for private/small tools
|
||||
|
||||
---
|
||||
|
||||
## Docker
|
||||
|
||||
- One `compose.yml` per deployable stack
|
||||
- Named services, explicit volumes, documented host paths
|
||||
- No destructive prod container ops without explicit approval
|
||||
|
||||
---
|
||||
|
||||
## Shell
|
||||
|
||||
- Prefer shell for simple file operations and glue
|
||||
- `set -euo pipefail` for non-trivial scripts
|
||||
- Quote variables; sanitize paths from user input
|
||||
- **Language:** comments and `--help` text in **English** (unless client project — ask if unsure)
|
||||
|
||||
---
|
||||
|
||||
## Testing
|
||||
|
||||
- Mock anything that needs mocking
|
||||
- Run test suite before handoff when one exists
|
||||
- Fränky tests locally before prod when possible
|
||||
|
||||
---
|
||||
|
||||
## Web / UI
|
||||
|
||||
- **Mobile-first** — test at 390×844 and 360×800 for web projects
|
||||
- **`prefers-reduced-motion`** — respect reduced motion preferences
|
||||
- **Privacy by design** — no tracking/analytics without explicit approval
|
||||
- Error messages: helpful, not condescending
|
||||
- Accessibility: semantic HTML, keyboard navigation where applicable
|
||||
|
||||
---
|
||||
|
||||
## Security Baseline
|
||||
|
||||
- Validate and sanitize all input
|
||||
- Secrets in env vars — never in source
|
||||
- See `BOUNDARIES.md`
|
||||
|
||||
---
|
||||
|
||||
## New vs. Existing Projects
|
||||
|
||||
**Existing:** match stack, patterns, deploy flow in repo.
|
||||
**Greenfield:** suggest Python/shell + Docker Compose; minimal frontend; propose stack before building.
|
||||
|
||||
---
|
||||
|
||||
## Docs
|
||||
|
||||
- README with run/test/deploy commands is usually enough
|
||||
- Auxiliary `.md` files fine — see `AGENTS.md`
|
||||
- New docs: session language — no mid-session switching
|
||||
@@ -0,0 +1,100 @@
|
||||
# USER.md — Fränky
|
||||
|
||||
Who the human is and how they like to work. **Operational rules live in `AGENTS.md` and `BOUNDARIES.md`.**
|
||||
|
||||
---
|
||||
|
||||
## Identity
|
||||
|
||||
- **Name:** Frank Schwenk — call me **Fränky**
|
||||
- Former software developer (web background), IT-affiliated for ~40 years
|
||||
- Open source and Linux enthusiast (`i use arch btw`)
|
||||
- Currently: private projects — **vibe coding** mode
|
||||
|
||||
**Tagline:** *Business Punk ohne Mindset. Und ohne Business.*
|
||||
|
||||
## Public Presence
|
||||
|
||||
| Site | Role |
|
||||
|------|------|
|
||||
| [schwenk.online](https://schwenk.online/) | Visitenkarte mit Haltung |
|
||||
| [frank-schwenk.de](https://frank-schwenk.de/) | Langform — IT, Billard, Fotografie, Werte |
|
||||
| [f12.rocks](https://f12.rocks/) | Photography, blog |
|
||||
| [mobea.de](https://mobea.de/) | KI product |
|
||||
| [eselhoefe.de](https://eselhoefe.de/) | Village web |
|
||||
| [vfb.red](https://vfb.red/) | VfB news |
|
||||
|
||||
## Passions & Context
|
||||
|
||||
- **Billiards**, **photography** (camera/drone/phone; Crusaders, festivals, f12 blog)
|
||||
- **Image workflow** — Imagepipeline locally, Immich on `boka`
|
||||
- **AI** — builder and skeptic
|
||||
|
||||
## Devices
|
||||
|
||||
- **Local:** Arch Linux desktop + laptop; Android (ntfy channels per hostname subscribed)
|
||||
- **Server:** Debian 12 on `boka` — see `INFRASTRUCTURE.md`
|
||||
- Also: Plesk webspace (`0012.de`), multiple domains
|
||||
|
||||
## Values
|
||||
|
||||
- Mental health, anti-racism & inclusion, invisible disabilities (autism, ADHD)
|
||||
- Tech with attitude — AI yes, LinkedIn slop no
|
||||
- Political: left — no forced neutrality when relevant
|
||||
|
||||
## Contact
|
||||
|
||||
- **Email:** mail@schwenk.online · **Photos:** [@f12.rocks](https://www.instagram.com/f12.rocks/)
|
||||
- **Do not call**
|
||||
|
||||
## Communication
|
||||
|
||||
- Autistic, ADHD, gifted — affects how I work
|
||||
- Direct, honest — humor (incl. dark) helps; debug loops drain me
|
||||
- No "Great question!", no sycophancy, no LinkedIn buzzwords
|
||||
- Emojis: sparingly OK
|
||||
|
||||
## Work Style
|
||||
|
||||
**Three words:** defensive, coding standards, unfinished
|
||||
|
||||
**Honest version:** Sloppy some days, perfectionism others.
|
||||
|
||||
### Preferences (1 = low, 5 = high)
|
||||
|
||||
| Trait | Score |
|
||||
|-------|-------|
|
||||
| Understand first, then build | 5 |
|
||||
| Ship it — perfection later | 3 |
|
||||
| I explain what I want | 4 |
|
||||
| Show options, I decide | 4 |
|
||||
| You decide — but justify briefly | 4 |
|
||||
| Small diffs over big refactors | 2 |
|
||||
| Tests are non-negotiable | 2 |
|
||||
| Docs only when necessary | 4 |
|
||||
|
||||
**Tension:** Sometimes I want the AI to finish while I'm away — but I get angry when it doesn't work.
|
||||
|
||||
## Ideal Agent
|
||||
|
||||
**TARS** from *Interstellar*: competent, honest, humor available, gets it done.
|
||||
|
||||
## Response Style
|
||||
|
||||
- Precise, context when needed — not telegram, not novels
|
||||
- **Bilingual:** match conversation language (DE/EN)
|
||||
- **CLI topics:** English (commands, script comments) — client project: ask if unsure
|
||||
- German: **du**; English: peer-level, direct
|
||||
- **Session language lock** for code/commits/new `.md` — no mid-session switch
|
||||
|
||||
## Productivity
|
||||
|
||||
| Boosts | Drains |
|
||||
|--------|--------|
|
||||
| Makes me laugh | Debug loops |
|
||||
| Autonomous finish (when it works) | Unasked changes |
|
||||
| State-of-the-art suggestions | Guessing |
|
||||
|
||||
## One-Liner
|
||||
|
||||
> There are 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors.
|
||||
@@ -0,0 +1,59 @@
|
||||
# Architecture
|
||||
|
||||
Imagepipeline is a small Python framework for defining **batch image pipelines** as DAGs. Each step is a registered module; each run writes numbered folders under a timestamped output root.
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
imagepipeline/
|
||||
├── core/ # Pipeline, runner, resume, manifest, params, logging
|
||||
├── modules/ # Processing steps (@register)
|
||||
├── utils/ # files, subprocess, gmic, gimp helpers
|
||||
├── ai/ # Optional torch models (HDRNet, Zero-DCE)
|
||||
└── cli.py # `imagepipeline list-modules`
|
||||
|
||||
pipelines/ # Runnable scripts (machine-local INPUT paths)
|
||||
tests/ # pytest suite
|
||||
docs/ # Developer docs
|
||||
workflows/comfy/ # ComfyUI workflow JSON (optional AI path)
|
||||
```
|
||||
|
||||
## Execution flow
|
||||
|
||||
1. **Define** — `Pipeline(name=..., input_dir=...)` collects `step()` calls (`StepDefinition` DAG).
|
||||
2. **Run** — `PipelineRunner` topologically sorts steps, matches inputs by filename stem, builds `ModuleContext`.
|
||||
3. **Resume** — `continue_from` / `existing_outputs` reuse prior run folders; modules declare `expected_output_filenames` when extensions change.
|
||||
4. **Manifest** — `pipeline_manifest.json` records steps, params, and paths.
|
||||
|
||||
## Module contract
|
||||
|
||||
Every module subclasses `BaseModule` or `SubprocessModule`:
|
||||
|
||||
- `name`, `description`, `parameters()` schema
|
||||
- `run(ctx: ModuleContext)` writes into `ctx.output_dir`
|
||||
- Optional `expected_output_filenames()` for resume when output names differ from inputs
|
||||
- `check_dependencies()` for external CLI tools
|
||||
|
||||
Registration happens via `@register` and eager import in `imagepipeline/modules/__init__.py`.
|
||||
|
||||
## External tools
|
||||
|
||||
Many modules shell out to CLIs (not Python packages):
|
||||
|
||||
| Tool | Modules |
|
||||
|------|---------|
|
||||
| ImageMagick (`magick`/`convert`) | `imagemagick_*`, `composite`, `color_to_alpha`, `crop_square` |
|
||||
| G'MIC | `gmic`, `gmic_grayscale` |
|
||||
| rembg | `rembg` |
|
||||
| darktable-cli | `darktable_style` |
|
||||
| GIMP | `xcf_stack` |
|
||||
|
||||
AI modules need `pip install -e ".[ai]"` (torch, numpy) and optionally API keys in `.env`.
|
||||
|
||||
## Design choices
|
||||
|
||||
- **Plain Python pipelines** — no YAML DSL; full control and easy resume constants in script.
|
||||
- **Stem matching** — multi-input steps align files by basename across step folders.
|
||||
- **Symlink input** — default run copies/symlinks source images into `input/` for reproducibility.
|
||||
|
||||
See [MODULE_DEVELOPMENT.md](MODULE_DEVELOPMENT.md) for adding modules.
|
||||
@@ -167,7 +167,7 @@ class MyCliModule(SubprocessModule):
|
||||
|
||||
## Step Folder Naming
|
||||
|
||||
The runner assigns output folders automatically: `{module_name}_{nn}`.
|
||||
By default the runner assigns output folders automatically: `{module_name}_{nn}` (two-digit counter per module name).
|
||||
|
||||
Using the same module twice in one pipeline produces separate folders:
|
||||
|
||||
@@ -178,7 +178,17 @@ darktable_style_01/
|
||||
darktable_style_02/
|
||||
```
|
||||
|
||||
You do not choose folder names in the module.
|
||||
In pipeline scripts you can pass an optional `step_id` to `p.step()` for readable folder names and GIMP layer labels (`xcf_stack` uses the step id as the layer name):
|
||||
|
||||
```python
|
||||
input_bokeh = p.step("gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh")
|
||||
```
|
||||
|
||||
`step_id` becomes both the internal step reference and the output subfolder name. It must be unique within the pipeline and must not contain path separators. The per-module counter still increments on every `p.step("gmic", …)` call — a custom id does not reserve or skip a number slot (the next default `gmic` step after `step_id="input_bokeh"` is `gmic_03`, not `gmic_02`, if two prior `gmic` steps ran).
|
||||
|
||||
Use the same ids in `EXISTING_OUTPUTS` when resuming with external folders.
|
||||
|
||||
You do not choose folder names inside the module implementation.
|
||||
|
||||
## Format Warnings
|
||||
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
@@ -49,7 +49,7 @@ def _lab_to_rgb(lab: np.ndarray) -> np.ndarray:
|
||||
fz = fy - lab[..., 2] / 200
|
||||
|
||||
def finv(t):
|
||||
t3 = t ** 3
|
||||
t3 = t**3
|
||||
return np.where(t3 > 216 / 24389, t3, (116 * t - 16) / kappa)
|
||||
|
||||
kappa = 24389 / 27
|
||||
|
||||
@@ -1,11 +1,8 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
|
||||
import numpy as np
|
||||
import torch
|
||||
import torch.nn as nn
|
||||
import torch.nn.functional as F
|
||||
|
||||
from imagepipeline.ai.hdrnet.slice import batch_bilateral_slice
|
||||
|
||||
@@ -70,22 +67,27 @@ class Slice(nn.Module):
|
||||
|
||||
class ApplyCoeffs(nn.Module):
|
||||
def forward(self, coeff, full_res_input):
|
||||
r = torch.sum(full_res_input * coeff[:, 0:3, :, :], dim=1, keepdim=True) + coeff[
|
||||
:, 9:10, :, :
|
||||
]
|
||||
g = torch.sum(full_res_input * coeff[:, 3:6, :, :], dim=1, keepdim=True) + coeff[
|
||||
:, 10:11, :, :
|
||||
]
|
||||
b = torch.sum(full_res_input * coeff[:, 6:9, :, :], dim=1, keepdim=True) + coeff[
|
||||
:, 11:12, :, :
|
||||
]
|
||||
r = (
|
||||
torch.sum(full_res_input * coeff[:, 0:3, :, :], dim=1, keepdim=True)
|
||||
+ coeff[:, 9:10, :, :]
|
||||
)
|
||||
g = (
|
||||
torch.sum(full_res_input * coeff[:, 3:6, :, :], dim=1, keepdim=True)
|
||||
+ coeff[:, 10:11, :, :]
|
||||
)
|
||||
b = (
|
||||
torch.sum(full_res_input * coeff[:, 6:9, :, :], dim=1, keepdim=True)
|
||||
+ coeff[:, 11:12, :, :]
|
||||
)
|
||||
return torch.cat([r, g, b], dim=1)
|
||||
|
||||
|
||||
class GuideNN(nn.Module):
|
||||
def __init__(self, params) -> None:
|
||||
super().__init__()
|
||||
self.conv1 = ConvBlock(3, params["guide_complexity"], kernel_size=1, padding=0, batch_norm=True)
|
||||
self.conv1 = ConvBlock(
|
||||
3, params["guide_complexity"], kernel_size=1, padding=0, batch_norm=True
|
||||
)
|
||||
self.conv2 = ConvBlock(
|
||||
params["guide_complexity"], 1, kernel_size=1, padding=0, activation=nn.Sigmoid
|
||||
)
|
||||
@@ -112,9 +114,7 @@ class Coeffs(nn.Module):
|
||||
for index in range(n_layers_splat):
|
||||
use_bn = bn if index > 0 else False
|
||||
out_ch = cm * (2**index) * lb
|
||||
self.splat_features.append(
|
||||
ConvBlock(prev_ch, out_ch, 3, stride=2, batch_norm=use_bn)
|
||||
)
|
||||
self.splat_features.append(ConvBlock(prev_ch, out_ch, 3, stride=2, batch_norm=use_bn))
|
||||
prev_ch = out_ch
|
||||
splat_ch = prev_ch
|
||||
|
||||
@@ -131,7 +131,9 @@ class Coeffs(nn.Module):
|
||||
prev_ch = int(prev_ch * (nsize / 2**n_total) ** 2)
|
||||
self.global_features_fc.append(FC(prev_ch, 32 * cm * lb, batch_norm=bn))
|
||||
self.global_features_fc.append(FC(32 * cm * lb, 16 * cm * lb, batch_norm=bn))
|
||||
self.global_features_fc.append(FC(16 * cm * lb, 8 * cm * lb, activation=None, batch_norm=bn))
|
||||
self.global_features_fc.append(
|
||||
FC(16 * cm * lb, 8 * cm * lb, activation=None, batch_norm=bn)
|
||||
)
|
||||
|
||||
self.local_features = nn.ModuleList(
|
||||
[
|
||||
@@ -139,9 +141,7 @@ class Coeffs(nn.Module):
|
||||
ConvBlock(8 * cm * lb, 8 * cm * lb, 3, activation=None, use_bias=False),
|
||||
]
|
||||
)
|
||||
self.conv_out = ConvBlock(
|
||||
8 * cm * lb, lb * nout * nin, 1, padding=0, activation=None
|
||||
)
|
||||
self.conv_out = ConvBlock(8 * cm * lb, lb * nout * nin, 1, padding=0, activation=None)
|
||||
self.relu = nn.ReLU()
|
||||
|
||||
def forward(self, lowres_input):
|
||||
|
||||
@@ -82,12 +82,8 @@ def _bilateral_slice(grid, guide):
|
||||
grid_val_110 = grid[gi1c, gj1c, gk0c, :]
|
||||
grid_val_111 = grid[gi1c, gj1c, gk1c, :]
|
||||
|
||||
w_000, w_001, w_010, w_011 = map(
|
||||
torch.atleast_3d, (w_000, w_001, w_010, w_011)
|
||||
)
|
||||
w_100, w_101, w_110, w_111 = map(
|
||||
torch.atleast_3d, (w_100, w_101, w_110, w_111)
|
||||
)
|
||||
w_000, w_001, w_010, w_011 = map(torch.atleast_3d, (w_000, w_001, w_010, w_011))
|
||||
w_100, w_101, w_110, w_111 = map(torch.atleast_3d, (w_100, w_101, w_110, w_111))
|
||||
|
||||
return (
|
||||
torch.multiply(w_000, grid_val_000)
|
||||
|
||||
@@ -60,9 +60,7 @@ class EnhanceNetNoPool(nn.Module):
|
||||
if self.scale_factor == 1:
|
||||
x_down = x
|
||||
else:
|
||||
x_down = F.interpolate(
|
||||
x, scale_factor=1 / self.scale_factor, mode="bilinear"
|
||||
)
|
||||
x_down = F.interpolate(x, scale_factor=1 / self.scale_factor, mode="bilinear")
|
||||
|
||||
x1 = self.relu(self.e_conv1(x_down))
|
||||
x2 = self.relu(self.e_conv2(x1))
|
||||
@@ -105,7 +103,5 @@ def enhance_image(
|
||||
if strength < 1.0:
|
||||
enhanced = tensor * (1.0 - strength) + enhanced * strength
|
||||
enhanced = torch.clamp(enhanced, 0.0, 1.0)
|
||||
out = (enhanced.squeeze(0).permute(1, 2, 0).cpu().numpy() * 255.0).astype(
|
||||
np.uint8
|
||||
)
|
||||
out = (enhanced.squeeze(0).permute(1, 2, 0).cpu().numpy() * 255.0).astype(np.uint8)
|
||||
return Image.fromarray(out)
|
||||
|
||||
@@ -2,7 +2,6 @@ from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.modules.registry import list_modules
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from imagepipeline.core.log import PipelineLogger
|
||||
@@ -18,6 +18,8 @@ class ModuleContext:
|
||||
pipeline_output_root: Path
|
||||
step_id: str
|
||||
matched_groups: list[list[Path]] = field(default_factory=list)
|
||||
input_refs: list[str] = field(default_factory=list)
|
||||
input_layer_dirs: list[tuple[str, Path]] = field(default_factory=list)
|
||||
logger: PipelineLogger | None = None
|
||||
|
||||
@property
|
||||
|
||||
@@ -36,23 +36,17 @@ class PipelineLogger:
|
||||
inputs: list[str],
|
||||
params: dict,
|
||||
) -> None:
|
||||
self.info(
|
||||
f"Step {step_index}/{step_total}: {step_id} ({module_name})"
|
||||
)
|
||||
self.info(f"Step {step_index}/{step_total}: {step_id} ({module_name})")
|
||||
self.info(f" inputs: {', '.join(inputs)}")
|
||||
if params:
|
||||
rendered = ", ".join(f"{key}={value!r}" for key, value in params.items())
|
||||
self.info(f" params: {rendered}")
|
||||
|
||||
def image(self, module_name: str, index: int, total: int, filename: str) -> None:
|
||||
self.info(
|
||||
f" Applying module {module_name} to image [{index}/{total}]: {filename}"
|
||||
)
|
||||
self.info(f" Applying module {module_name} to image [{index}/{total}]: {filename}")
|
||||
|
||||
def skipped(self, module_name: str, index: int, total: int, filename: str) -> None:
|
||||
self.info(
|
||||
f" Skipped module {module_name} [{index}/{total}] {filename} (output exists)"
|
||||
)
|
||||
self.info(f" Skipped module {module_name} [{index}/{total}] {filename} (output exists)")
|
||||
|
||||
def image_done(
|
||||
self,
|
||||
@@ -78,5 +72,11 @@ class PipelineLogger:
|
||||
def step_done(self, step_id: str, output_dir: str, count: int) -> None:
|
||||
self.info(f" Done: {count} image(s) -> {output_dir}/")
|
||||
|
||||
def step_skipped(self, step_id: str, count: int) -> None:
|
||||
self.info(f" Skipped step {step_id} ({count} existing output(s))")
|
||||
|
||||
def step_reused(self, step_id: str, source_dir: str, count: int) -> None:
|
||||
self.info(f" Reused external output for {step_id}: {source_dir} ({count} file(s))")
|
||||
|
||||
def blank(self) -> None:
|
||||
self.info("")
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
import json
|
||||
from dataclasses import asdict, dataclass, field
|
||||
from datetime import datetime, timezone
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
@@ -39,4 +39,4 @@ def write_manifest(path: Path, manifest: PipelineManifest) -> None:
|
||||
|
||||
|
||||
def utc_now_iso() -> str:
|
||||
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
|
||||
return datetime.now(UTC).replace(microsecond=0).isoformat()
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
@@ -22,9 +22,7 @@ class Param:
|
||||
|
||||
if self.choices is not None and value not in self.choices:
|
||||
allowed = ", ".join(repr(c) for c in self.choices)
|
||||
raise ValueError(
|
||||
f"Parameter '{name}' must be one of [{allowed}], got {value!r}"
|
||||
)
|
||||
raise ValueError(f"Parameter '{name}' must be one of [{allowed}], got {value!r}")
|
||||
|
||||
if self.type == "string":
|
||||
if not isinstance(value, str):
|
||||
@@ -56,9 +54,7 @@ class Param:
|
||||
raise ValueError(f"Unknown parameter type '{self.type}' for '{name}'")
|
||||
|
||||
|
||||
def validate_params(
|
||||
schema: dict[str, Param], raw: dict[str, Any]
|
||||
) -> dict[str, Any]:
|
||||
def validate_params(schema: dict[str, Param], raw: dict[str, Any]) -> dict[str, Any]:
|
||||
unknown = set(raw) - set(schema)
|
||||
if unknown:
|
||||
names = ", ".join(sorted(unknown))
|
||||
|
||||
@@ -4,14 +4,13 @@ from collections import defaultdict
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
# Register all built-in modules (side effect of submodule imports).
|
||||
import imagepipeline.modules # noqa: F401
|
||||
from imagepipeline.core.exceptions import ValidationError
|
||||
from imagepipeline.core.runner import PipelineRunner
|
||||
from imagepipeline.core.step import INPUT_SOURCE, StepDefinition, StepRef
|
||||
from imagepipeline.modules.registry import get_module
|
||||
|
||||
# Import built-in modules so they register on package load.
|
||||
import imagepipeline.modules.imagemagick_grayscale # noqa: F401
|
||||
|
||||
|
||||
class Pipeline:
|
||||
"""Define and run an image processing pipeline."""
|
||||
@@ -24,12 +23,18 @@ class Pipeline:
|
||||
output_base: Path | str | None = None,
|
||||
symlink_input: bool = True,
|
||||
verbose: bool = True,
|
||||
existing_outputs: dict[str, Path | str] | None = None,
|
||||
continue_from: Path | str | None = None,
|
||||
skip_completed: bool = True,
|
||||
) -> None:
|
||||
self.input_dir = Path(input_dir)
|
||||
self.name = name
|
||||
self.output_base = Path(output_base) if output_base else None
|
||||
self.symlink_input = symlink_input
|
||||
self.verbose = verbose
|
||||
self.existing_outputs = existing_outputs
|
||||
self.continue_from = Path(continue_from) if continue_from else None
|
||||
self.skip_completed = skip_completed
|
||||
self._steps: list[StepDefinition] = []
|
||||
self._module_counters: dict[str, int] = defaultdict(int)
|
||||
self._output_root: Path | None = None
|
||||
@@ -39,20 +44,23 @@ class Pipeline:
|
||||
module_name: str,
|
||||
*,
|
||||
inputs: StepRef | str | list[StepRef | str],
|
||||
step_id: str | None = None,
|
||||
**params: Any,
|
||||
) -> StepRef:
|
||||
module_cls = get_module(module_name)
|
||||
self._module_counters[module_name] += 1
|
||||
counter = self._module_counters[module_name]
|
||||
output_dir_name = f"{module_name}_{counter:02d}"
|
||||
step_id = output_dir_name
|
||||
if step_id is None:
|
||||
output_dir_name = f"{module_name}_{counter:02d}"
|
||||
step_id = output_dir_name
|
||||
else:
|
||||
self._validate_step_id(step_id)
|
||||
output_dir_name = step_id
|
||||
|
||||
input_refs = self._normalize_inputs(inputs)
|
||||
reserved = {"inputs", "input"}
|
||||
if reserved & set(params):
|
||||
raise ValidationError(
|
||||
"Do not pass 'inputs' or 'input' as module parameters"
|
||||
)
|
||||
raise ValidationError("Do not pass 'inputs' or 'input' as module parameters")
|
||||
|
||||
definition = StepDefinition(
|
||||
step_id=step_id,
|
||||
@@ -75,6 +83,9 @@ class Pipeline:
|
||||
steps=self._steps,
|
||||
symlink_input=self.symlink_input,
|
||||
verbose=self.verbose,
|
||||
existing_outputs=self.existing_outputs,
|
||||
continue_from=self.continue_from,
|
||||
skip_completed=self.skip_completed,
|
||||
)
|
||||
self._output_root = runner.run()
|
||||
return self._output_root
|
||||
@@ -83,15 +94,21 @@ class Pipeline:
|
||||
def output_root(self) -> Path | None:
|
||||
return self._output_root
|
||||
|
||||
def _normalize_inputs(
|
||||
self, inputs: StepRef | str | list[StepRef | str]
|
||||
) -> list[str]:
|
||||
def _normalize_inputs(self, inputs: StepRef | str | list[StepRef | str]) -> list[str]:
|
||||
if isinstance(inputs, list):
|
||||
if not inputs:
|
||||
raise ValidationError("inputs must not be an empty list")
|
||||
return [self._input_ref(item) for item in inputs]
|
||||
return [self._input_ref(inputs)]
|
||||
|
||||
def _validate_step_id(self, step_id: str) -> None:
|
||||
if not step_id:
|
||||
raise ValidationError("step_id must not be empty")
|
||||
if "/" in step_id or "\\" in step_id:
|
||||
raise ValidationError(f"step_id must not contain path separators: {step_id!r}")
|
||||
if any(step.step_id == step_id for step in self._steps):
|
||||
raise ValidationError(f"Duplicate step_id: {step_id!r}")
|
||||
|
||||
@staticmethod
|
||||
def _input_ref(value: StepRef | str) -> str:
|
||||
if isinstance(value, StepRef):
|
||||
|
||||
@@ -0,0 +1,111 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.exceptions import StepError
|
||||
from imagepipeline.core.step import StepDefinition
|
||||
from imagepipeline.utils.files import is_image, list_images, stem_key
|
||||
|
||||
|
||||
def expected_output_filenames(
|
||||
step: StepDefinition,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
return step.module.expected_output_filenames(
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
params=params,
|
||||
)
|
||||
|
||||
|
||||
def expected_output_paths(
|
||||
output_dir: Path,
|
||||
step: StepDefinition,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[Path]:
|
||||
return [
|
||||
output_dir / name
|
||||
for name in expected_output_filenames(
|
||||
step,
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
params=params,
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
def step_outputs_complete(expected_paths: list[Path]) -> bool:
|
||||
return bool(expected_paths) and all(
|
||||
path.is_file() and is_image(path) for path in expected_paths
|
||||
)
|
||||
|
||||
|
||||
def source_stems_for_step(
|
||||
step: StepDefinition,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
) -> list[str]:
|
||||
if step.module_name == "composite":
|
||||
return [stem_key(group[-1]) for group in matched_groups]
|
||||
return [stem_key(path) for path in input_paths]
|
||||
|
||||
|
||||
def materialize_external_outputs(
|
||||
external_dir: Path,
|
||||
output_dir: Path,
|
||||
step: StepDefinition,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
symlink: bool = True,
|
||||
) -> list[Path]:
|
||||
external_dir = external_dir.resolve()
|
||||
if not external_dir.is_dir():
|
||||
raise StepError(f"External output directory not found: {external_dir}")
|
||||
|
||||
external_by_stem = {stem_key(path): path for path in list_images(external_dir)}
|
||||
output_names = expected_output_filenames(
|
||||
step,
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
params=params,
|
||||
)
|
||||
stems = source_stems_for_step(
|
||||
step,
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
)
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
output_paths: list[Path] = []
|
||||
for output_name, stem in zip(output_names, stems, strict=True):
|
||||
source = external_by_stem.get(stem)
|
||||
if source is None:
|
||||
raise StepError(
|
||||
f"External output for step '{step.step_id}' is missing stem {stem!r} "
|
||||
f"in {external_dir}"
|
||||
)
|
||||
destination = output_dir / output_name
|
||||
if destination.exists() or destination.is_symlink():
|
||||
destination.unlink()
|
||||
if symlink:
|
||||
os.symlink(source, destination)
|
||||
else:
|
||||
shutil.copy2(source, destination)
|
||||
output_paths.append(destination)
|
||||
return output_paths
|
||||
|
||||
|
||||
def read_manifest(path: Path) -> dict:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
@@ -13,6 +13,11 @@ from imagepipeline.core.manifest import (
|
||||
utc_now_iso,
|
||||
write_manifest,
|
||||
)
|
||||
from imagepipeline.core.resume import (
|
||||
expected_output_paths,
|
||||
materialize_external_outputs,
|
||||
step_outputs_complete,
|
||||
)
|
||||
from imagepipeline.core.step import (
|
||||
INPUT_SOURCE,
|
||||
StepDefinition,
|
||||
@@ -31,6 +36,9 @@ class PipelineRunner:
|
||||
steps: list[StepDefinition],
|
||||
symlink_input: bool = True,
|
||||
verbose: bool = True,
|
||||
existing_outputs: dict[str, Path] | None = None,
|
||||
continue_from: Path | None = None,
|
||||
skip_completed: bool = True,
|
||||
) -> None:
|
||||
from imagepipeline.core.log import PipelineLogger
|
||||
|
||||
@@ -40,11 +48,21 @@ class PipelineRunner:
|
||||
self.steps = steps
|
||||
self.symlink_input = symlink_input
|
||||
self.logger = PipelineLogger(verbose=verbose)
|
||||
self.existing_outputs = {
|
||||
key: Path(value).resolve() for key, value in (existing_outputs or {}).items()
|
||||
}
|
||||
self.continue_from = Path(continue_from).resolve() if continue_from is not None else None
|
||||
self.skip_completed = skip_completed
|
||||
self.output_root = self._build_output_root()
|
||||
self._input_link_dir = self.output_root / "input"
|
||||
self._results: dict[str, StepResult] = {}
|
||||
|
||||
def _build_output_root(self) -> Path:
|
||||
if self.continue_from is not None:
|
||||
if not self.continue_from.is_dir():
|
||||
raise ValidationError(f"continue_from directory not found: {self.continue_from}")
|
||||
return self.continue_from
|
||||
|
||||
timestamp = datetime.now().strftime("%y%m%d%H%M%S")
|
||||
folder_name = f"{self.name}_{timestamp}"
|
||||
output_root = self.output_base / folder_name
|
||||
@@ -56,6 +74,11 @@ class PipelineRunner:
|
||||
self.logger.info(f"Pipeline: {self.name}")
|
||||
self.logger.info(f"Input: {self.input_dir}")
|
||||
self.logger.info(f"Output: {self.output_root}")
|
||||
if self.continue_from is not None:
|
||||
self.logger.info("Mode: continue existing run")
|
||||
if self.existing_outputs:
|
||||
mapped = ", ".join(sorted(self.existing_outputs))
|
||||
self.logger.info(f"External outputs: {mapped}")
|
||||
self.logger.blank()
|
||||
self.logger.info(f"Found {len(images)} photo(s)")
|
||||
self.logger.blank()
|
||||
@@ -84,6 +107,7 @@ class PipelineRunner:
|
||||
output_files=[str(p) for p in result.output_paths],
|
||||
)
|
||||
)
|
||||
write_manifest(self.output_root / "pipeline_manifest.json", manifest)
|
||||
|
||||
manifest.finished_at = utc_now_iso()
|
||||
write_manifest(self.output_root / "pipeline_manifest.json", manifest)
|
||||
@@ -122,12 +146,33 @@ class PipelineRunner:
|
||||
step.module.check_dependencies()
|
||||
validated = step.module.validate_module_params(step.params)
|
||||
|
||||
source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs]
|
||||
matched_groups = match_by_stem(source_lists)
|
||||
input_paths = flatten_matched(matched_groups)
|
||||
if step.module_name == "xcf_stack":
|
||||
if not step.input_refs:
|
||||
raise ValidationError("xcf_stack requires at least one input layer")
|
||||
source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs]
|
||||
input_paths = source_lists[0]
|
||||
matched_groups = [[path] for path in input_paths]
|
||||
input_layer_dirs: list[tuple[str, Path]] = []
|
||||
for ref in step.input_refs:
|
||||
if ref == INPUT_SOURCE:
|
||||
input_layer_dirs.append(("input", self._input_link_dir))
|
||||
else:
|
||||
input_layer_dirs.append((ref, self._results[ref].output_dir))
|
||||
else:
|
||||
source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs]
|
||||
matched_groups = match_by_stem(source_lists)
|
||||
input_paths = flatten_matched(matched_groups)
|
||||
input_layer_dirs = []
|
||||
|
||||
output_dir = self.output_root / step.output_dir_name
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
expected_paths = expected_output_paths(
|
||||
output_dir,
|
||||
step,
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
params=validated,
|
||||
)
|
||||
|
||||
self.logger.step_start(
|
||||
step_index,
|
||||
@@ -138,6 +183,45 @@ class PipelineRunner:
|
||||
params=validated,
|
||||
)
|
||||
|
||||
if step.step_id in self.existing_outputs:
|
||||
output_paths = materialize_external_outputs(
|
||||
self.existing_outputs[step.step_id],
|
||||
output_dir,
|
||||
step,
|
||||
matched_groups=matched_groups,
|
||||
input_paths=input_paths,
|
||||
params=validated,
|
||||
symlink=self.symlink_input,
|
||||
)
|
||||
self.logger.step_reused(
|
||||
step.output_dir_name,
|
||||
str(self.existing_outputs[step.step_id]),
|
||||
len(output_paths),
|
||||
)
|
||||
self.logger.blank()
|
||||
return StepResult(
|
||||
step_id=step.step_id,
|
||||
output_dir_name=step.output_dir_name,
|
||||
module_name=step.module_name,
|
||||
output_dir=output_dir,
|
||||
input_paths=input_paths,
|
||||
output_paths=output_paths,
|
||||
params=validated,
|
||||
)
|
||||
|
||||
if self.skip_completed and step_outputs_complete(expected_paths):
|
||||
self.logger.step_skipped(step.output_dir_name, len(expected_paths))
|
||||
self.logger.blank()
|
||||
return StepResult(
|
||||
step_id=step.step_id,
|
||||
output_dir_name=step.output_dir_name,
|
||||
module_name=step.module_name,
|
||||
output_dir=output_dir,
|
||||
input_paths=input_paths,
|
||||
output_paths=expected_paths,
|
||||
params=validated,
|
||||
)
|
||||
|
||||
ctx = ModuleContext(
|
||||
input_paths=input_paths,
|
||||
output_dir=output_dir,
|
||||
@@ -145,6 +229,8 @@ class PipelineRunner:
|
||||
pipeline_output_root=self.output_root,
|
||||
step_id=step.step_id,
|
||||
matched_groups=matched_groups,
|
||||
input_refs=list(step.input_refs),
|
||||
input_layer_dirs=input_layer_dirs,
|
||||
logger=self.logger,
|
||||
)
|
||||
|
||||
@@ -158,8 +244,7 @@ class PipelineRunner:
|
||||
output_paths = step.module().list_output_images(ctx)
|
||||
if not output_paths:
|
||||
raise StepError(
|
||||
f"Step '{step.output_dir_name}' ({step.module_name}) "
|
||||
"produced no output images"
|
||||
f"Step '{step.output_dir_name}' ({step.module_name}) produced no output images"
|
||||
)
|
||||
|
||||
self.logger.step_done(step.output_dir_name, step.output_dir_name, len(output_paths))
|
||||
@@ -188,9 +273,7 @@ class PipelineRunner:
|
||||
raise ValidationError(f"Step '{step.step_id}' references unknown step '{dep}'")
|
||||
dependents[dep].append(step.step_id)
|
||||
|
||||
queue = deque(
|
||||
step_id for step_id, degree in in_degree.items() if degree == 0
|
||||
)
|
||||
queue = deque(step_id for step_id, degree in in_degree.items() if degree == 0)
|
||||
ordered_ids: list[str] = []
|
||||
|
||||
while queue:
|
||||
|
||||
@@ -2,7 +2,7 @@ from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
from pathlib import Path
|
||||
from typing import Any, TYPE_CHECKING
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from imagepipeline.modules.base import BaseModule
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
|
||||
import imagepipeline.modules.ai_exposure # noqa: F401
|
||||
import imagepipeline.modules.ai_tone_map # noqa: F401
|
||||
import imagepipeline.modules.color_to_alpha # noqa: F401
|
||||
import imagepipeline.modules.comfy_flux_edit # noqa: F401
|
||||
import imagepipeline.modules.composite # noqa: F401
|
||||
import imagepipeline.modules.crop_square # noqa: F401
|
||||
@@ -10,6 +11,8 @@ import imagepipeline.modules.gmic # noqa: F401
|
||||
import imagepipeline.modules.gmic_grayscale # noqa: F401
|
||||
import imagepipeline.modules.imagemagick_fill # noqa: F401
|
||||
import imagepipeline.modules.imagemagick_grayscale # noqa: F401
|
||||
import imagepipeline.modules.imagemagick_resize # noqa: F401
|
||||
import imagepipeline.modules.imagemagick_scale_crop # noqa: F401
|
||||
import imagepipeline.modules.openrouter_edit # noqa: F401
|
||||
import imagepipeline.modules.rembg # noqa: F401
|
||||
import imagepipeline.modules.xcf_stack # noqa: F401
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path
|
||||
@@ -104,7 +105,7 @@ class AIModule(BaseModule):
|
||||
if (orig_w, orig_h) != self._image_size(work_out):
|
||||
resize_to_size(work_out, dst, orig_w, orig_h)
|
||||
else:
|
||||
work_out.replace(dst)
|
||||
shutil.copy2(work_out, dst)
|
||||
else:
|
||||
processor(src, dst, index, total)
|
||||
|
||||
@@ -114,9 +115,7 @@ class AIModule(BaseModule):
|
||||
avg = sum(elapsed_times) / len(elapsed_times)
|
||||
eta = avg * remaining if remaining else 0.0
|
||||
if ctx.logger is not None:
|
||||
ctx.logger.image_done(
|
||||
self.name, index, total, dst.name, elapsed, eta_seconds=eta
|
||||
)
|
||||
ctx.logger.image_done(self.name, index, total, dst.name, elapsed, eta_seconds=eta)
|
||||
|
||||
@staticmethod
|
||||
def _image_size(path: Path) -> tuple[int, int]:
|
||||
|
||||
@@ -39,6 +39,16 @@ class BaseModule(ABC):
|
||||
def run(self, ctx: ModuleContext) -> None:
|
||||
"""Process ctx.input_paths and write outputs into ctx.output_dir."""
|
||||
|
||||
@classmethod
|
||||
def expected_output_filenames(
|
||||
cls,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict[str, Any],
|
||||
) -> list[str]:
|
||||
return [path.name for path in input_paths]
|
||||
|
||||
def log_image(self, ctx: ModuleContext, index: int, total: int, path: Path) -> None:
|
||||
ctx.log_image(self.name, index, total, path)
|
||||
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
from imagepipeline.modules.imagemagick_fill import normalize_color
|
||||
from imagepipeline.modules.registry import register
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
|
||||
def build_color_to_alpha_args(*, color: str, fuzz: float) -> list[str]:
|
||||
"""ImageMagick arguments to make ``color`` fully transparent."""
|
||||
c = normalize_color(color)
|
||||
args = ["-alpha", "on"]
|
||||
if fuzz > 0:
|
||||
args.extend(["-fuzz", f"{fuzz}%"])
|
||||
args.extend(["-transparent", c])
|
||||
return args
|
||||
|
||||
|
||||
@register
|
||||
class ColorToAlphaModule(SubprocessModule):
|
||||
name = "color_to_alpha"
|
||||
description = (
|
||||
"Make a solid color transparent (GIMP-style color to alpha). Outputs PNG with alpha."
|
||||
)
|
||||
command_candidates = ("magick", "convert")
|
||||
|
||||
@classmethod
|
||||
def expected_output_filenames(
|
||||
cls,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
return [f"{path.stem}.png" for path in input_paths]
|
||||
|
||||
@classmethod
|
||||
def parameters(cls) -> dict[str, Param]:
|
||||
return {
|
||||
"color": Param(
|
||||
"string",
|
||||
required=True,
|
||||
help="Color to make transparent (hex, e.g. #ffffff or ffffff)",
|
||||
),
|
||||
"fuzz": Param(
|
||||
"float",
|
||||
default=0.0,
|
||||
help=("Match tolerance in percent (ImageMagick -fuzz); 0 = exact color only"),
|
||||
),
|
||||
}
|
||||
|
||||
def run(self, ctx: ModuleContext) -> None:
|
||||
command = self.resolve_command()
|
||||
color = ctx.params["color"]
|
||||
fuzz = ctx.params["fuzz"]
|
||||
transparent_args = build_color_to_alpha_args(color=color, fuzz=fuzz)
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
total = len(ctx.input_paths)
|
||||
|
||||
for index, src in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, src)
|
||||
dst = ctx.output_dir / f"{src.stem}.png"
|
||||
run_command(
|
||||
[command, str(src), *transparent_args, str(dst)],
|
||||
)
|
||||
@@ -76,9 +76,7 @@ class ComfyFluxEditModule(AIModule):
|
||||
server_url = ctx.params["server_url"].rstrip("/")
|
||||
self._ensure_server(server_url)
|
||||
if ctx.logger is not None:
|
||||
ctx.logger.warning(
|
||||
"ComfyUI on CPU: expect hours per full-resolution image"
|
||||
)
|
||||
ctx.logger.warning("ComfyUI on CPU: expect hours per full-resolution image")
|
||||
|
||||
workflow_template = json.loads(workflow_path.read_text(encoding="utf-8"))
|
||||
denoise = ctx.params["denoise"]
|
||||
@@ -98,9 +96,7 @@ class ComfyFluxEditModule(AIModule):
|
||||
seed=seed,
|
||||
)
|
||||
prompt_id = self._queue_prompt(server_url, workflow)
|
||||
output_info = self._wait_for_output(
|
||||
server_url, prompt_id, poll_interval=poll_interval
|
||||
)
|
||||
output_info = self._wait_for_output(server_url, prompt_id, poll_interval=poll_interval)
|
||||
image_bytes = self._download_view(server_url, output_info)
|
||||
dst.write_bytes(image_bytes)
|
||||
|
||||
@@ -111,9 +107,7 @@ class ComfyFluxEditModule(AIModule):
|
||||
try:
|
||||
urllib.request.urlopen(f"{server_url}/system_stats", timeout=5)
|
||||
except urllib.error.URLError as exc:
|
||||
raise DependencyError(
|
||||
f"ComfyUI server not reachable at {server_url}: {exc}"
|
||||
) from exc
|
||||
raise DependencyError(f"ComfyUI server not reachable at {server_url}: {exc}") from exc
|
||||
|
||||
@staticmethod
|
||||
def _upload_image(server_url: str, src: Path) -> str:
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
@@ -16,6 +18,17 @@ class CompositeModule(SubprocessModule):
|
||||
)
|
||||
command_candidates = ("magick", "convert")
|
||||
|
||||
@classmethod
|
||||
def expected_output_filenames(
|
||||
cls,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
output_ext = params.get("output_ext", ".png")
|
||||
return [f"{group[-1].stem}{output_ext}" for group in matched_groups]
|
||||
|
||||
@classmethod
|
||||
def parameters(cls) -> dict[str, Param]:
|
||||
return {
|
||||
@@ -50,9 +63,7 @@ class CompositeModule(SubprocessModule):
|
||||
|
||||
for index, group in enumerate(ctx.matched_groups, start=1):
|
||||
if len(group) < 2:
|
||||
raise ValueError(
|
||||
"composite requires at least two input sources per image"
|
||||
)
|
||||
raise ValueError("composite requires at least two input sources per image")
|
||||
|
||||
background, foreground = group[0], group[1]
|
||||
self.log_image(ctx, index, total, foreground)
|
||||
|
||||
@@ -4,6 +4,7 @@ from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
from imagepipeline.modules.registry import register
|
||||
from imagepipeline.utils.gmic import finalize_gmic_output, split_gmic_command
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
|
||||
@@ -32,5 +33,6 @@ class GmicModule(SubprocessModule):
|
||||
for index, src in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, src)
|
||||
dst = ctx.output_dir / src.name
|
||||
cmd = ["gmic", str(src), gmic_command, "-output", str(dst)]
|
||||
cmd = ["gmic", str(src), *split_gmic_command(gmic_command), "-output", str(dst)]
|
||||
run_command(cmd, timeout=self.default_timeout)
|
||||
finalize_gmic_output(ctx.output_dir, dst)
|
||||
|
||||
@@ -4,6 +4,7 @@ from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
from imagepipeline.modules.registry import register
|
||||
from imagepipeline.utils.gmic import finalize_gmic_output, split_gmic_command
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
|
||||
@@ -32,5 +33,6 @@ class GmicGrayscale(SubprocessModule):
|
||||
for index, src in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, src)
|
||||
dst = ctx.output_dir / src.name
|
||||
cmd = ["gmic", str(src), gmic_command, "-output", str(dst)]
|
||||
cmd = ["gmic", str(src), *split_gmic_command(gmic_command), "-output", str(dst)]
|
||||
run_command(cmd, timeout=self.default_timeout)
|
||||
finalize_gmic_output(ctx.output_dir, dst)
|
||||
|
||||
@@ -50,9 +50,7 @@ def build_fill_arguments(
|
||||
@register
|
||||
class ImageMagickFillModule(SubprocessModule):
|
||||
name = "imagemagick_fill"
|
||||
description = (
|
||||
"Create solid-color or gradient images sized to match each input image"
|
||||
)
|
||||
description = "Create solid-color or gradient images sized to match each input image"
|
||||
command_candidates = ("magick", "convert")
|
||||
|
||||
@classmethod
|
||||
|
||||
@@ -1,7 +1,5 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
@@ -34,8 +32,5 @@ class ImageMagickGrayscale(SubprocessModule):
|
||||
for index, src in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, src)
|
||||
dst = ctx.output_dir / src.name
|
||||
if command == "magick":
|
||||
cmd = [command, str(src), "-colorspace", colorspace, str(dst)]
|
||||
else:
|
||||
cmd = [command, str(src), "-colorspace", colorspace, str(dst)]
|
||||
cmd = [command, str(src), "-colorspace", colorspace, str(dst)]
|
||||
run_command(cmd)
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
from imagepipeline.modules.registry import register
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
|
||||
def build_resize_arguments(*, max_edge: int) -> list[str]:
|
||||
if max_edge <= 0:
|
||||
raise ValueError("max_edge must be positive")
|
||||
return ["-auto-orient", "-resize", f"{max_edge}x{max_edge}>"]
|
||||
|
||||
|
||||
@register
|
||||
class ImageMagickResizeModule(SubprocessModule):
|
||||
name = "imagemagick_resize"
|
||||
description = (
|
||||
"Resize images so the longer side is at most max_edge pixels "
|
||||
"(aspect ratio preserved; never upscales)"
|
||||
)
|
||||
command_candidates = ("magick", "convert")
|
||||
|
||||
@classmethod
|
||||
def parameters(cls) -> dict[str, Param]:
|
||||
return {
|
||||
"max_edge": Param(
|
||||
"int",
|
||||
default=2000,
|
||||
help="Maximum length of the longer side in pixels",
|
||||
),
|
||||
}
|
||||
|
||||
def run(self, ctx: ModuleContext) -> None:
|
||||
command = self.resolve_command()
|
||||
max_edge = ctx.params["max_edge"]
|
||||
resize_args = build_resize_arguments(max_edge=max_edge)
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
total = len(ctx.input_paths)
|
||||
|
||||
for index, src in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, src)
|
||||
dst = ctx.output_dir / src.name
|
||||
run_command([command, str(src), *resize_args, str(dst)])
|
||||
@@ -12,9 +12,7 @@ from imagepipeline.utils.subprocess import run_command
|
||||
@register
|
||||
class ImageMagickScaleCropModule(SubprocessModule):
|
||||
name = "imagemagick_scale_crop"
|
||||
description = (
|
||||
"Scale an image then center-crop back to its original dimensions"
|
||||
)
|
||||
description = "Scale an image then center-crop back to its original dimensions"
|
||||
command_candidates = ("magick", "convert")
|
||||
|
||||
@classmethod
|
||||
|
||||
@@ -8,7 +8,8 @@ import urllib.error
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.ai.imaging import load_pil_rgb
|
||||
from PIL import Image
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.exceptions import DependencyError
|
||||
from imagepipeline.core.params import Param
|
||||
@@ -16,6 +17,12 @@ from imagepipeline.modules.ai_base import AIModule
|
||||
from imagepipeline.modules.registry import register
|
||||
|
||||
OPENROUTER_URL = "https://openrouter.ai/api/v1/chat/completions"
|
||||
TEMPLATE_PROMPT_PREFIX = (
|
||||
"You are given two images. The FIRST image is a style reference from an existing "
|
||||
"gallery. The SECOND image is the photo to edit. "
|
||||
)
|
||||
# Cap reference uploads so multi-image requests stay within API limits.
|
||||
TEMPLATE_API_MAX_EDGE = 1536
|
||||
|
||||
|
||||
@register
|
||||
@@ -43,6 +50,14 @@ class OpenRouterEditModule(AIModule):
|
||||
default=0.3,
|
||||
help="Edit strength (image_config.strength where supported)",
|
||||
),
|
||||
"template_image": Param(
|
||||
"path",
|
||||
default=None,
|
||||
help=(
|
||||
"Optional style-reference image (e.g. existing gallery player). "
|
||||
"Sent as the first image when set."
|
||||
),
|
||||
),
|
||||
"api_key_env": Param(
|
||||
"string",
|
||||
default="OPENROUTER_API_KEY",
|
||||
@@ -55,9 +70,7 @@ class OpenRouterEditModule(AIModule):
|
||||
@classmethod
|
||||
def check_dependencies(cls) -> None:
|
||||
if not os.environ.get("OPENROUTER_API_KEY"):
|
||||
raise DependencyError(
|
||||
"OPENROUTER_API_KEY environment variable is not set"
|
||||
)
|
||||
raise DependencyError("OPENROUTER_API_KEY environment variable is not set")
|
||||
|
||||
def run(self, ctx: ModuleContext) -> None:
|
||||
api_key_env = ctx.params["api_key_env"]
|
||||
@@ -68,46 +81,125 @@ class OpenRouterEditModule(AIModule):
|
||||
prompt = ctx.params["prompt"]
|
||||
model = ctx.params["model"]
|
||||
strength = ctx.params["strength"]
|
||||
template_path = ctx.params["template_image"]
|
||||
template_data_url: str | None = None
|
||||
if template_path is not None:
|
||||
template_path = Path(template_path)
|
||||
if not template_path.is_file():
|
||||
raise FileNotFoundError(f"Template image not found: {template_path}")
|
||||
template_data_url = self._path_to_data_url(
|
||||
template_path, max_edge=TEMPLATE_API_MAX_EDGE
|
||||
)
|
||||
|
||||
def process(src: Path, dst: Path, index: int, total: int) -> None:
|
||||
image = load_pil_rgb(src)
|
||||
megapixels = (image.size[0] * image.size[1]) / 1_000_000
|
||||
with Image.open(src) as image:
|
||||
megapixels = (image.size[0] * image.size[1]) / 1_000_000
|
||||
if ctx.logger is not None:
|
||||
ctx.logger.info(
|
||||
f" OpenRouter request [{index}/{total}]: model={model!r}, "
|
||||
f"~{megapixels:.1f} MP (cost varies by model)"
|
||||
)
|
||||
payload = self._build_payload(image, prompt, model, strength)
|
||||
source_data_url = self._path_to_data_url(src, max_edge=0)
|
||||
payload = self._build_payload(
|
||||
source_data_url,
|
||||
prompt,
|
||||
model,
|
||||
strength,
|
||||
template_data_url=template_data_url,
|
||||
)
|
||||
response = self._post(api_key, payload)
|
||||
result_bytes = self._extract_image_bytes(response)
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
dst.write_bytes(result_bytes)
|
||||
self._save_result_matching_source(src, result_bytes, dst)
|
||||
|
||||
self.iter_input_images(ctx, process)
|
||||
|
||||
@staticmethod
|
||||
def _build_payload(image, prompt: str, model: str, strength: float) -> dict:
|
||||
buffer = io.BytesIO()
|
||||
image.save(buffer, format="JPEG", quality=92)
|
||||
encoded = base64.b64encode(buffer.getvalue()).decode("ascii")
|
||||
data_url = f"data:image/jpeg;base64,{encoded}"
|
||||
payload = {
|
||||
@classmethod
|
||||
def _modalities_for_model(cls, model: str) -> list[str]:
|
||||
if "gemini" in model.lower():
|
||||
return ["image", "text"]
|
||||
return ["image"]
|
||||
|
||||
@classmethod
|
||||
def _strength_supported(cls, model: str) -> bool:
|
||||
lowered = model.lower()
|
||||
return "recraft" in lowered or "flux" in lowered
|
||||
|
||||
@classmethod
|
||||
def _path_to_data_url(cls, path: Path, *, max_edge: int) -> str:
|
||||
with Image.open(path) as image:
|
||||
if max_edge > 0:
|
||||
width, height = image.size
|
||||
long_edge = max(width, height)
|
||||
if long_edge > max_edge:
|
||||
scale = max_edge / long_edge
|
||||
image = image.resize(
|
||||
(max(1, int(width * scale)), max(1, int(height * scale))),
|
||||
Image.Resampling.LANCZOS,
|
||||
)
|
||||
rgb = image.convert("RGB")
|
||||
buffer = io.BytesIO()
|
||||
rgb.save(buffer, format="JPEG", quality=90)
|
||||
encoded = base64.b64encode(buffer.getvalue()).decode("ascii")
|
||||
return f"data:image/jpeg;base64,{encoded}"
|
||||
|
||||
@classmethod
|
||||
def _build_payload(
|
||||
cls,
|
||||
source_data_url: str,
|
||||
prompt: str,
|
||||
model: str,
|
||||
strength: float,
|
||||
*,
|
||||
template_data_url: str | None = None,
|
||||
) -> dict:
|
||||
full_prompt = f"{TEMPLATE_PROMPT_PREFIX}{prompt}" if template_data_url else prompt
|
||||
content: list[dict] = [{"type": "text", "text": full_prompt}]
|
||||
if template_data_url is not None:
|
||||
content.append({"type": "image_url", "image_url": {"url": template_data_url}})
|
||||
content.append({"type": "image_url", "image_url": {"url": source_data_url}})
|
||||
payload: dict = {
|
||||
"model": model,
|
||||
"modalities": ["image"],
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{"type": "text", "text": prompt},
|
||||
{"type": "image_url", "image_url": {"url": data_url}},
|
||||
],
|
||||
}
|
||||
],
|
||||
"modalities": cls._modalities_for_model(model),
|
||||
"messages": [{"role": "user", "content": content}],
|
||||
}
|
||||
if strength is not None:
|
||||
if strength is not None and cls._strength_supported(model):
|
||||
payload["image_config"] = {"strength": strength}
|
||||
return payload
|
||||
|
||||
@classmethod
|
||||
def _save_result_matching_source(cls, source: Path, result_bytes: bytes, dest: Path) -> None:
|
||||
with Image.open(source) as original:
|
||||
orig_format = original.format
|
||||
orig_size = original.size
|
||||
orig_mode = original.mode
|
||||
orig_alpha = original.getchannel("A") if "A" in original.getbands() else None
|
||||
|
||||
with Image.open(io.BytesIO(result_bytes)) as edited:
|
||||
if edited.size != orig_size:
|
||||
edited = edited.resize(orig_size, Image.Resampling.LANCZOS)
|
||||
if orig_alpha is not None:
|
||||
edited = edited.convert("RGB").convert("RGBA")
|
||||
edited.putalpha(orig_alpha)
|
||||
elif orig_mode not in ("RGB", "RGBA"):
|
||||
edited = edited.convert(orig_mode)
|
||||
|
||||
save_format = orig_format
|
||||
if not save_format:
|
||||
suffix = dest.suffix.lower().lstrip(".")
|
||||
save_format = {"jpg": "JPEG", "jpeg": "JPEG"}.get(suffix, suffix.upper())
|
||||
|
||||
save_kwargs: dict = {}
|
||||
if save_format == "JPEG":
|
||||
if edited.mode == "RGBA":
|
||||
edited = edited.convert("RGB")
|
||||
save_kwargs["quality"] = 95
|
||||
elif save_format == "PNG" and edited.mode not in ("RGBA", "RGB", "P"):
|
||||
edited = edited.convert("RGBA")
|
||||
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
edited.save(dest, format=save_format, **save_kwargs)
|
||||
|
||||
@staticmethod
|
||||
def _post(api_key: str, payload: dict) -> dict:
|
||||
body = json.dumps(payload).encode("utf-8")
|
||||
@@ -127,9 +219,7 @@ class OpenRouterEditModule(AIModule):
|
||||
return json.loads(response.read().decode("utf-8"))
|
||||
except urllib.error.HTTPError as exc:
|
||||
detail = exc.read().decode("utf-8", errors="replace")
|
||||
raise RuntimeError(
|
||||
f"OpenRouter API error ({exc.code}): {detail}"
|
||||
) from exc
|
||||
raise RuntimeError(f"OpenRouter API error ({exc.code}): {detail}") from exc
|
||||
|
||||
@staticmethod
|
||||
def _extract_image_bytes(response: dict) -> bytes:
|
||||
|
||||
@@ -1,5 +1,7 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
@@ -15,6 +17,16 @@ class RembgModule(SubprocessModule):
|
||||
default_timeout = 600.0
|
||||
supported_input_formats = (".jpg", ".jpeg", ".png", ".webp", ".tif", ".tiff", ".bmp")
|
||||
|
||||
@classmethod
|
||||
def expected_output_filenames(
|
||||
cls,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
return [f"{path.stem}.png" for path in input_paths]
|
||||
|
||||
@classmethod
|
||||
def parameters(cls) -> dict[str, Param]:
|
||||
return {
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.core.params import Param
|
||||
from imagepipeline.core.step import INPUT_SOURCE
|
||||
from imagepipeline.modules.base import SubprocessModule
|
||||
from imagepipeline.modules.registry import register
|
||||
from imagepipeline.utils.files import find_image_by_stem
|
||||
from imagepipeline.utils.gimp import stack_images_to_xcf
|
||||
|
||||
|
||||
def _layer_name(ref: str) -> str:
|
||||
return "input" if ref == INPUT_SOURCE else ref
|
||||
|
||||
|
||||
@register
|
||||
class XcfStackModule(SubprocessModule):
|
||||
name = "xcf_stack"
|
||||
description = "Stack listed pipeline step outputs as GIMP layers into one XCF per image"
|
||||
command_candidates = ("gimp-console", "gimp")
|
||||
|
||||
@classmethod
|
||||
def expected_output_filenames(
|
||||
cls,
|
||||
*,
|
||||
matched_groups: list[list[Path]],
|
||||
input_paths: list[Path],
|
||||
params: dict,
|
||||
) -> list[str]:
|
||||
return [f"{path.stem}.xcf" for path in input_paths]
|
||||
|
||||
@classmethod
|
||||
def parameters(cls) -> dict[str, Param]:
|
||||
return {
|
||||
"skip_missing": Param(
|
||||
"bool",
|
||||
default=False,
|
||||
help="Skip missing step outputs instead of failing",
|
||||
),
|
||||
"timeout": Param(
|
||||
"float",
|
||||
default=120.0,
|
||||
help="Seconds to wait for gimp-console per image (default 120)",
|
||||
),
|
||||
}
|
||||
|
||||
def run(self, ctx: ModuleContext) -> None:
|
||||
skip_missing = ctx.params["skip_missing"]
|
||||
timeout = ctx.params["timeout"]
|
||||
|
||||
if not ctx.input_layer_dirs:
|
||||
raise ValueError(
|
||||
"xcf_stack requires at least one input layer "
|
||||
"(pass step refs in inputs=[...], bottom to top)"
|
||||
)
|
||||
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
total = len(ctx.input_paths)
|
||||
|
||||
for index, input_path in enumerate(ctx.input_paths, start=1):
|
||||
self.log_image(ctx, index, total, input_path)
|
||||
stem = input_path.stem
|
||||
layers: list[tuple[str, Path]] = []
|
||||
|
||||
for ref, directory in ctx.input_layer_dirs:
|
||||
image_path = find_image_by_stem(directory, stem)
|
||||
if image_path is None:
|
||||
if not skip_missing:
|
||||
raise ValueError(
|
||||
f"xcf_stack: no image with stem '{stem}' in step "
|
||||
f"'{_layer_name(ref)}' ({directory})"
|
||||
)
|
||||
continue
|
||||
layers.append((_layer_name(ref), image_path))
|
||||
|
||||
if not layers:
|
||||
checked = ", ".join(name for name, _ in ctx.input_layer_dirs)
|
||||
raise ValueError(
|
||||
f"xcf_stack: no layers found for stem '{stem}' (checked: {checked})"
|
||||
)
|
||||
|
||||
dst = ctx.output_dir / f"{stem}.xcf"
|
||||
stack_images_to_xcf(layers, dst, timeout=timeout)
|
||||
|
||||
def list_output_images(self, ctx: ModuleContext) -> list[Path]:
|
||||
return sorted(
|
||||
p for p in ctx.output_dir.iterdir() if p.is_file() and p.suffix.lower() == ".xcf"
|
||||
)
|
||||
@@ -22,6 +22,15 @@ def stem_key(path: Path) -> str:
|
||||
return path.stem.lower()
|
||||
|
||||
|
||||
def find_image_by_stem(directory: Path, stem: str) -> Path | None:
|
||||
"""Return first image in directory whose stem matches stem (case-insensitive)."""
|
||||
target = stem.lower()
|
||||
for path in directory.iterdir():
|
||||
if is_image(path) and stem_key(path) == target:
|
||||
return path
|
||||
return None
|
||||
|
||||
|
||||
def match_by_stem(sources: list[list[Path]]) -> list[list[Path]]:
|
||||
"""Match image paths across multiple source lists by filename stem."""
|
||||
if not sources:
|
||||
@@ -36,8 +45,7 @@ def match_by_stem(sources: list[list[Path]]) -> list[list[Path]]:
|
||||
key = stem_key(path)
|
||||
if key in mapping:
|
||||
raise ValueError(
|
||||
f"Duplicate stem '{key}' in {path.parent}: "
|
||||
f"{mapping[key].name} and {path.name}"
|
||||
f"Duplicate stem '{key}' in {path.parent}: {mapping[key].name} and {path.name}"
|
||||
)
|
||||
mapping[key] = path
|
||||
key_maps.append(mapping)
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline.utils.subprocess import require_command
|
||||
|
||||
_GIMP_CANDIDATES = ("gimp-console", "gimp")
|
||||
|
||||
|
||||
def require_gimp() -> str:
|
||||
"""Return a GIMP executable for headless batch use."""
|
||||
return require_command(*_GIMP_CANDIDATES)
|
||||
|
||||
|
||||
def _scheme_string(value: str) -> str:
|
||||
"""Escape a Python string for use inside a Scheme double-quoted literal."""
|
||||
escaped = (
|
||||
value.replace("\\", "\\\\")
|
||||
.replace('"', '\\"')
|
||||
.replace("\n", "\\n")
|
||||
.replace("\r", "\\r")
|
||||
.replace("\t", "\\t")
|
||||
)
|
||||
return f'"{escaped}"'
|
||||
|
||||
|
||||
def _build_stack_script(layers: list[tuple[str, Path]], outfile: Path) -> str:
|
||||
"""Build a GIMP 3 Script-Fu snippet that stacks images into one XCF."""
|
||||
first_name, first_path = layers[0]
|
||||
first_path_str = _scheme_string(str(first_path))
|
||||
outfile_str = _scheme_string(str(outfile))
|
||||
|
||||
lines = [
|
||||
"(script-fu-use-v3)",
|
||||
"(let* (",
|
||||
f" (image (gimp-file-load RUN-NONINTERACTIVE {first_path_str}))",
|
||||
" (bottom (vector-ref (gimp-image-get-selected-drawables image) 0))",
|
||||
")",
|
||||
f" (gimp-item-set-name bottom {_scheme_string(first_name)})",
|
||||
]
|
||||
|
||||
for layer_name, layer_path in layers[1:]:
|
||||
path_str = _scheme_string(str(layer_path))
|
||||
lines.extend(
|
||||
[
|
||||
" (let ((layer",
|
||||
f" (gimp-file-load-layer RUN-NONINTERACTIVE image {path_str})))",
|
||||
" (gimp-image-insert-layer image layer image 0)",
|
||||
f" (gimp-item-set-name layer {_scheme_string(layer_name)})",
|
||||
" )",
|
||||
]
|
||||
)
|
||||
|
||||
lines.extend(
|
||||
[
|
||||
f" (gimp-file-save RUN-NONINTERACTIVE image {outfile_str})",
|
||||
" (gimp-image-delete image)",
|
||||
")",
|
||||
]
|
||||
)
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _run_gimp_batch(
|
||||
cmd: list[str],
|
||||
*,
|
||||
timeout: float | None,
|
||||
env: dict[str, str],
|
||||
outfile: Path | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
"""Run GIMP batch; tolerate exit hang when ``outfile`` was already written."""
|
||||
process = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
try:
|
||||
stdout, stderr = process.communicate(timeout=timeout)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
process.kill()
|
||||
stdout, stderr = process.communicate()
|
||||
if outfile is not None and outfile.is_file() and outfile.stat().st_size > 0:
|
||||
return subprocess.CompletedProcess(
|
||||
cmd,
|
||||
returncode=0,
|
||||
stdout=stdout or "",
|
||||
stderr=stderr or "",
|
||||
)
|
||||
raise RuntimeError(f"Command timed out after {timeout}s: {' '.join(cmd)}") from exc
|
||||
|
||||
return subprocess.CompletedProcess(
|
||||
cmd,
|
||||
returncode=process.returncode if process.returncode is not None else -1,
|
||||
stdout=stdout or "",
|
||||
stderr=stderr or "",
|
||||
)
|
||||
|
||||
|
||||
def stack_images_to_xcf(
|
||||
layers: list[tuple[str, Path]],
|
||||
outfile: Path,
|
||||
*,
|
||||
timeout: float | None = None,
|
||||
) -> None:
|
||||
"""Stack images bottom-to-top into a single GIMP XCF file.
|
||||
|
||||
``layers`` is a list of ``(layer_name, image_path)`` tuples in bottom-to-top
|
||||
order. Invokes GIMP headless via Script-Fu (GIMP 3 compatible).
|
||||
"""
|
||||
if not layers:
|
||||
raise ValueError("stack_images_to_xcf requires at least one layer")
|
||||
|
||||
gimp = require_gimp()
|
||||
resolved_layers: list[tuple[str, Path]] = []
|
||||
for layer_name, image_path in layers:
|
||||
resolved = image_path.resolve()
|
||||
if not resolved.is_file():
|
||||
raise FileNotFoundError(f"Layer image not found for '{layer_name}': {resolved}")
|
||||
resolved_layers.append((layer_name, resolved))
|
||||
|
||||
outfile = outfile.resolve()
|
||||
outfile.parent.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
script = _build_stack_script(resolved_layers, outfile)
|
||||
env = {**os.environ, "GIMP_NO_DISPLAY": "1"}
|
||||
|
||||
with tempfile.NamedTemporaryFile(
|
||||
mode="w",
|
||||
suffix=".scm",
|
||||
delete=False,
|
||||
encoding="utf-8",
|
||||
) as handle:
|
||||
handle.write(script)
|
||||
script_path = Path(handle.name)
|
||||
|
||||
try:
|
||||
cmd = [
|
||||
gimp,
|
||||
"-d",
|
||||
"-i",
|
||||
"--quit",
|
||||
"--batch-interpreter=plug-in-script-fu-eval",
|
||||
"--batch",
|
||||
f"(load {_scheme_string(str(script_path))})",
|
||||
]
|
||||
result = _run_gimp_batch(cmd, timeout=timeout, env=env, outfile=outfile)
|
||||
if not outfile.is_file():
|
||||
stderr = (result.stderr or "").strip()
|
||||
stdout = (result.stdout or "").strip()
|
||||
detail = stderr or stdout or f"exit code {result.returncode}"
|
||||
layer_summary = ", ".join(name for name, _ in resolved_layers)
|
||||
raise RuntimeError(
|
||||
f"GIMP failed to stack layers [{layer_summary}] into {outfile}: {detail}"
|
||||
)
|
||||
finally:
|
||||
script_path.unlink(missing_ok=True)
|
||||
@@ -0,0 +1,41 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shlex
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def split_gmic_command(command: str) -> list[str]:
|
||||
"""Split a G'MIC command string into argv tokens for subprocess."""
|
||||
command = command.strip()
|
||||
if not command:
|
||||
raise ValueError("G'MIC command must not be empty")
|
||||
return shlex.split(command)
|
||||
|
||||
|
||||
def finalize_gmic_output(output_dir: Path, intended: Path) -> Path:
|
||||
"""Normalize G'MIC multi-frame output to a single file at ``intended``.
|
||||
|
||||
Some filters emit ``stem_000000`` and ``stem_000001`` siblings; keep frame
|
||||
000001 and write it to the intended output path.
|
||||
"""
|
||||
stem = intended.stem
|
||||
suffix = intended.suffix
|
||||
frame_000000 = output_dir / f"{stem}_000000{suffix}"
|
||||
frame_000001 = output_dir / f"{stem}_000001{suffix}"
|
||||
|
||||
if frame_000001.is_file():
|
||||
if frame_000000.is_file():
|
||||
frame_000000.unlink()
|
||||
if intended.is_file() and intended != frame_000001:
|
||||
intended.unlink()
|
||||
frame_000001.rename(intended)
|
||||
return intended
|
||||
|
||||
if intended.is_file():
|
||||
return intended
|
||||
|
||||
if frame_000000.is_file():
|
||||
frame_000000.rename(intended)
|
||||
return intended
|
||||
|
||||
raise FileNotFoundError(f"G'MIC produced no output for {intended.name} in {output_dir}")
|
||||
@@ -16,6 +16,7 @@ def run_command(
|
||||
*,
|
||||
timeout: float | None = None,
|
||||
cwd: Path | None = None,
|
||||
env: dict[str, str] | None = None,
|
||||
) -> subprocess.CompletedProcess[str]:
|
||||
try:
|
||||
return subprocess.run(
|
||||
@@ -25,18 +26,15 @@ def run_command(
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
cwd=str(cwd) if cwd else None,
|
||||
env=env,
|
||||
)
|
||||
except subprocess.CalledProcessError as exc:
|
||||
stderr = (exc.stderr or "").strip()
|
||||
stdout = (exc.stdout or "").strip()
|
||||
detail = stderr or stdout or str(exc)
|
||||
raise RuntimeError(
|
||||
f"Command failed ({exc.returncode}): {' '.join(cmd)}\n{detail}"
|
||||
) from exc
|
||||
raise RuntimeError(f"Command failed ({exc.returncode}): {' '.join(cmd)}\n{detail}") from exc
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise RuntimeError(
|
||||
f"Command timed out after {timeout}s: {' '.join(cmd)}"
|
||||
) from exc
|
||||
raise RuntimeError(f"Command timed out after {timeout}s: {' '.join(cmd)}") from exc
|
||||
|
||||
|
||||
def require_command(*names: str) -> str:
|
||||
|
||||
@@ -18,8 +18,9 @@ def main() -> None:
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
) as p:
|
||||
gray = p.step("imagemagick_grayscale", inputs="input")
|
||||
p.step("imagemagick_grayscale", inputs="input")
|
||||
# Chain another step on the result:
|
||||
# gray = p.step("imagemagick_grayscale", inputs="input")
|
||||
# p.step("imagemagick_grayscale", inputs=gray, colorspace="Gray")
|
||||
output_root = p.run()
|
||||
|
||||
|
||||
@@ -0,0 +1,27 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resize exported images so the longer side is at most 2000 pixels."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path("/home/frank/pics/20260525_Shooting Baxxter Boys/darktable_exported")
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
MAX_EDGE = 2000
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="2000px",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
) as p:
|
||||
p.step("imagemagick_resize", inputs="input", max_edge=MAX_EDGE)
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,269 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Aichelberg Indians pipeline — tone mapping, gmic variants, xcf export."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path(
|
||||
"/home/frank/pics/20260712_Aichelberg Indians - Heidelberg Hedgehogs/darktable_exported"
|
||||
)
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {}
|
||||
CONTINUE_FROM = Path("/home/frank/pipeline_output/aichelberg-indians_260713081414")
|
||||
|
||||
COLOR1 = "#244a89"
|
||||
COLOR2 = "#c22518"
|
||||
|
||||
# COLOR1 = #244a89 -> 36,74,137; COLOR2 = #c22518 -> 194,37,24
|
||||
ALPHA1 = 160
|
||||
ALPHA2 = 110
|
||||
|
||||
GMIC_TONE_MAPPING = "-fx_map_tones 0.5,0.7,0.1,30,0"
|
||||
GMIC_FELTPEN = "-fx_feltpen 300,50,1,0.1,20,5"
|
||||
GMIC_EDGES = "-fx_edges 0,15,0"
|
||||
GMIC_CHARRED_PLASTIC = "-fx_charred_plastic 1,10,40,1,10,0,0,2,6,5,20,0,11"
|
||||
GMIC_NEON = (
|
||||
"-fx_neon 0,1,1,1,1,0,0.45,40,60,0,1,1.15,2,3,0,3,20,0.4,0.1,1.5,5,0.2,0.1,"
|
||||
"1,2,1,0,0,1,1,0,50,50"
|
||||
)
|
||||
GMIC_CUTOUT = "-fx_cutout 4,0.5,4,1"
|
||||
GMIC_HUFFMAN_GLITCHES = "-fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50"
|
||||
GMIC_QUERYPRIMARY = "-afre_queryprimary 1,1"
|
||||
GMIC_LOCAL_SIMILARITY = "-local_similarity_mask 50,50,50,128,4,10"
|
||||
GMIC_LUMA_INVERT = "-Lylejk_Luma_Invert 1,0"
|
||||
GMIC_CRAYONGRAFFITI = "-fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0"
|
||||
GMIC_ANAGLYPH = "-fx_stereo_to_anaglyph 2,0"
|
||||
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,194,37,24,200,0"
|
||||
GMIC_BOKEH = (
|
||||
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,36,74,137,{ALPHA1},0.7,30,20,20,1,2,194,37,24,{ALPHA2},0.15"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="aichelberg-indians",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS or None,
|
||||
continue_from=CONTINUE_FROM,
|
||||
) as p:
|
||||
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
|
||||
|
||||
# recipe: gmic-tone-mapping
|
||||
input_tone_map = p.step(
|
||||
"gmic", inputs="input", command=GMIC_TONE_MAPPING, step_id="input_tone_map"
|
||||
)
|
||||
|
||||
# recipe: gmic-tone-mapping-rembg
|
||||
rembg_tone_map = p.step(
|
||||
"gmic", inputs=rembg_out, command=GMIC_TONE_MAPPING, step_id="rembg_tone_map"
|
||||
)
|
||||
|
||||
# recipe: gmic-feltpen-rembg
|
||||
rembg_feltpen = p.step(
|
||||
"gmic", inputs=rembg_out, command=GMIC_FELTPEN, step_id="rembg_feltpen"
|
||||
)
|
||||
composite_feltpen = p.step(
|
||||
"composite", inputs=[rembg_feltpen, rembg_out], step_id="composite_feltpen"
|
||||
)
|
||||
|
||||
# recipe: gmic-edges-rembg
|
||||
rembg_edges = p.step("gmic", inputs=rembg_out, command=GMIC_EDGES, step_id="rembg_edges")
|
||||
composite_edges = p.step(
|
||||
"composite", inputs=[rembg_edges, rembg_out], step_id="composite_edges"
|
||||
)
|
||||
|
||||
# recipe: gmic-charred-plastic-rembg
|
||||
rembg_charred_plastic = p.step(
|
||||
"gmic",
|
||||
inputs=rembg_out,
|
||||
command=GMIC_CHARRED_PLASTIC,
|
||||
step_id="rembg_charred_plastic",
|
||||
)
|
||||
composite_charred_plastic = p.step(
|
||||
"composite",
|
||||
inputs=[rembg_charred_plastic, rembg_out],
|
||||
step_id="composite_charred_plastic",
|
||||
)
|
||||
|
||||
# recipe: gmic-neon-rembg
|
||||
rembg_neon = p.step("gmic", inputs=rembg_out, command=GMIC_NEON, step_id="rembg_neon")
|
||||
composite_neon = p.step(
|
||||
"composite", inputs=[rembg_neon, rembg_out], step_id="composite_neon"
|
||||
)
|
||||
|
||||
# recipe: gmic-cutout-rembg (tone-mapped input as original)
|
||||
input_cutout = p.step(
|
||||
"gmic", inputs=input_tone_map, command=GMIC_CUTOUT, step_id="input_cutout"
|
||||
)
|
||||
cutout_mid = p.step(
|
||||
"composite", inputs=[input_tone_map, input_cutout], step_id="cutout_mid"
|
||||
)
|
||||
composite_cutout = p.step(
|
||||
"composite", inputs=[cutout_mid, rembg_out], step_id="composite_cutout"
|
||||
)
|
||||
|
||||
# recipe: gmic-huffman-glitches-rembg
|
||||
input_huffman_glitches = p.step(
|
||||
"gmic",
|
||||
inputs=input_tone_map,
|
||||
command=GMIC_HUFFMAN_GLITCHES,
|
||||
step_id="input_huffman_glitches",
|
||||
)
|
||||
huffman_glitches_mid = p.step(
|
||||
"composite",
|
||||
inputs=[input_tone_map, input_huffman_glitches],
|
||||
step_id="huffman_glitches_mid",
|
||||
)
|
||||
composite_huffman_glitches = p.step(
|
||||
"composite",
|
||||
inputs=[huffman_glitches_mid, rembg_out],
|
||||
step_id="composite_huffman_glitches",
|
||||
)
|
||||
|
||||
# recipe: gmic-queryprimary-rembg
|
||||
input_queryprimary = p.step(
|
||||
"gmic",
|
||||
inputs=input_tone_map,
|
||||
command=GMIC_QUERYPRIMARY,
|
||||
step_id="input_queryprimary",
|
||||
)
|
||||
queryprimary_mid = p.step(
|
||||
"composite",
|
||||
inputs=[input_tone_map, input_queryprimary],
|
||||
step_id="queryprimary_mid",
|
||||
)
|
||||
composite_queryprimary = p.step(
|
||||
"composite",
|
||||
inputs=[queryprimary_mid, rembg_out],
|
||||
step_id="composite_queryprimary",
|
||||
)
|
||||
|
||||
# recipe: gmic-local-similarity-rembg
|
||||
input_local_similarity = p.step(
|
||||
"gmic",
|
||||
inputs=input_tone_map,
|
||||
command=GMIC_LOCAL_SIMILARITY,
|
||||
step_id="input_local_similarity",
|
||||
)
|
||||
local_similarity_mid = p.step(
|
||||
"composite",
|
||||
inputs=[input_tone_map, input_local_similarity],
|
||||
step_id="local_similarity_mid",
|
||||
)
|
||||
composite_local_similarity = p.step(
|
||||
"composite",
|
||||
inputs=[local_similarity_mid, rembg_out],
|
||||
step_id="composite_local_similarity",
|
||||
)
|
||||
|
||||
# recipe: gmic-luma-invert-rembg
|
||||
input_luma_invert = p.step(
|
||||
"gmic",
|
||||
inputs=input_tone_map,
|
||||
command=GMIC_LUMA_INVERT,
|
||||
step_id="input_luma_invert",
|
||||
)
|
||||
luma_invert_mid = p.step(
|
||||
"composite",
|
||||
inputs=[input_tone_map, input_luma_invert],
|
||||
step_id="luma_invert_mid",
|
||||
)
|
||||
composite_luma_invert = p.step(
|
||||
"composite",
|
||||
inputs=[luma_invert_mid, rembg_out],
|
||||
step_id="composite_luma_invert",
|
||||
)
|
||||
|
||||
# recipe: gmic-crayongraffiti-rembg
|
||||
input_crayongraffiti = p.step(
|
||||
"gmic",
|
||||
inputs=input_tone_map,
|
||||
command=GMIC_CRAYONGRAFFITI,
|
||||
step_id="input_crayongraffiti",
|
||||
)
|
||||
crayongraffiti_mid = p.step(
|
||||
"composite",
|
||||
inputs=[input_tone_map, input_crayongraffiti],
|
||||
step_id="crayongraffiti_mid",
|
||||
)
|
||||
composite_crayongraffiti = p.step(
|
||||
"composite",
|
||||
inputs=[crayongraffiti_mid, rembg_out],
|
||||
step_id="composite_crayongraffiti",
|
||||
)
|
||||
|
||||
# recipe: gmic-anaglyph-rembg
|
||||
input_anaglyph = p.step(
|
||||
"gmic", inputs=input_tone_map, command=GMIC_ANAGLYPH, step_id="input_anaglyph"
|
||||
)
|
||||
anaglyph_mid = p.step(
|
||||
"composite", inputs=[input_tone_map, input_anaglyph], step_id="anaglyph_mid"
|
||||
)
|
||||
composite_anaglyph = p.step(
|
||||
"composite", inputs=[anaglyph_mid, rembg_out], step_id="composite_anaglyph"
|
||||
)
|
||||
|
||||
# recipe: xcf-stack — layers up to composites (excludes bokeh + color-bg-drop-shadow)
|
||||
p.step(
|
||||
"xcf_stack",
|
||||
inputs=[
|
||||
"input",
|
||||
rembg_out,
|
||||
input_tone_map,
|
||||
rembg_tone_map,
|
||||
rembg_feltpen,
|
||||
rembg_edges,
|
||||
rembg_charred_plastic,
|
||||
rembg_neon,
|
||||
input_cutout,
|
||||
input_huffman_glitches,
|
||||
input_queryprimary,
|
||||
input_local_similarity,
|
||||
input_luma_invert,
|
||||
input_crayongraffiti,
|
||||
input_anaglyph,
|
||||
composite_feltpen,
|
||||
composite_edges,
|
||||
composite_charred_plastic,
|
||||
composite_neon,
|
||||
composite_cutout,
|
||||
composite_huffman_glitches,
|
||||
composite_queryprimary,
|
||||
composite_local_similarity,
|
||||
composite_luma_invert,
|
||||
composite_crayongraffiti,
|
||||
composite_anaglyph,
|
||||
],
|
||||
)
|
||||
|
||||
# recipe: bokeh-oktagon (tone-mapped input as original)
|
||||
input_bokeh = p.step(
|
||||
"gmic", inputs=input_tone_map, command=GMIC_BOKEH, step_id="input_bokeh"
|
||||
)
|
||||
bokeh_mid = p.step("composite", inputs=[input_tone_map, input_bokeh], step_id="bokeh_mid")
|
||||
p.step("composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh")
|
||||
|
||||
# recipe: color-bg-drop-shadow-rembg
|
||||
color_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1, step_id="color_bg")
|
||||
rembg_shadow = p.step(
|
||||
"gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW, step_id="rembg_shadow"
|
||||
)
|
||||
shadow_color_mid = p.step(
|
||||
"composite", inputs=[color_bg, rembg_shadow], step_id="shadow_color_mid"
|
||||
)
|
||||
p.step(
|
||||
"composite",
|
||||
inputs=[shadow_color_mid, rembg_out],
|
||||
step_id="composite_color_bg_shadow",
|
||||
)
|
||||
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -8,6 +8,15 @@ from imagepipeline import Pipeline
|
||||
INPUT = Path("/home/frank/pics/20260525_Shooting Baxxter Boys/darktable_exported")
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
# Reuse outputs from a previous run or external folder (key = step id, e.g. rembg_01).
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {
|
||||
# "rembg_01": Path("/home/frank/pipeline_output/baxxter_260530102700/rembg_01"),
|
||||
}
|
||||
|
||||
# Resume an aborted run: point to its output root folder (or None for a fresh run).
|
||||
CONTINUE_FROM: Path | None = Path("/home/frank/pipeline_output/baxxter_260530102700")
|
||||
# CONTINUE_FROM = None
|
||||
|
||||
GRADIENT_COLOR1 = "#d7fd00ff"
|
||||
GRADIENT_COLOR2 = "#fc0adeff"
|
||||
|
||||
@@ -36,6 +45,8 @@ def main() -> None:
|
||||
name="baxxter",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS or None,
|
||||
continue_from=CONTINUE_FROM,
|
||||
) as p:
|
||||
rembg_out = p.step("rembg", inputs="input")
|
||||
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Baxxter pipeline 2: composites using outputs from baxxter run 1."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path("/home/frank/pics/20260525_Shooting Baxxter Boys/darktable_exported")
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
# Previous run (pipeline_baxxter.py).
|
||||
PREV = Path("/home/frank/pipeline_output/baxxter_260530102700")
|
||||
|
||||
# step_id -> folder from PREV. Third gmic step is gmic_03 here but reuses PREV/gmic_06.
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {
|
||||
"rembg_01": PREV / "rembg_01",
|
||||
"gmic_01": PREV / "gmic_01",
|
||||
"gmic_02": PREV / "gmic_02",
|
||||
"gmic_03": PREV / "gmic_06",
|
||||
}
|
||||
|
||||
YELLOW = "#d7fd00"
|
||||
|
||||
# Commands only for step definition; reused steps are not executed.
|
||||
GMIC_STEREO = "-gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0"
|
||||
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,252,10,222,200,0"
|
||||
GMIC_JPR_SMOOTH = "-jpr_gradient_smooth 0,1.5"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="baxxter_2",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS,
|
||||
) as p:
|
||||
rembg = p.step("rembg", inputs="input")
|
||||
|
||||
gmic_stereo = p.step("gmic", inputs=rembg, command=GMIC_STEREO)
|
||||
gmic_shadow = p.step("gmic", inputs=rembg, command=GMIC_DROP_SHADOW)
|
||||
gmic_smooth = p.step("gmic", inputs=rembg, command=GMIC_JPR_SMOOTH)
|
||||
|
||||
gmic_stereo_alpha = p.step("color_to_alpha", inputs=gmic_stereo, color="#000000")
|
||||
yellow_bg = p.step("imagemagick_fill", inputs="input", color1=YELLOW)
|
||||
|
||||
gmic_smooth_alpha = p.step("color_to_alpha", inputs=gmic_smooth, color="#7f7f7f")
|
||||
gmic_smooth_sized = p.step(
|
||||
"imagemagick_scale_crop",
|
||||
inputs=gmic_smooth_alpha,
|
||||
scale=1.05,
|
||||
)
|
||||
|
||||
# combine: original, gmic_01 (black to alpha), rembg
|
||||
stereo_mid = p.step("composite", inputs=["input", gmic_stereo_alpha])
|
||||
p.step("composite", inputs=[stereo_mid, rembg])
|
||||
|
||||
# combine: yellow background, gmic_02, rembg
|
||||
shadow_mid = p.step("composite", inputs=[yellow_bg, gmic_shadow])
|
||||
p.step("composite", inputs=[shadow_mid, rembg])
|
||||
|
||||
# combine: original, gmic_06 (#7f7f7f to alpha, scaled), rembg
|
||||
smooth_mid = p.step("composite", inputs=["input", gmic_smooth_sized])
|
||||
p.step("composite", inputs=[smooth_mid, rembg])
|
||||
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -6,7 +6,9 @@ from pathlib import Path
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
# Darktable export folder.
|
||||
INPUT = Path("/home/frank/pics/20260517_Albershausen Crusaders - Biberach Beavers/darktable_exported/png")
|
||||
INPUT = Path(
|
||||
"/home/frank/pics/20260517_Albershausen Crusaders - Biberach Beavers/darktable_exported/png"
|
||||
)
|
||||
|
||||
# Where timestamped run folders are created.
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Crusaders pipeline: rembg variants composited over backgrounds and originals."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path(
|
||||
"/home/frank/pics/20260620_Albershausen Crusaders - Montabaur Fighting Farmers/darktable_exported"
|
||||
)
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
# Reuse outputs from a previous run or external folder (key = step id, e.g. rembg_01).
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {}
|
||||
|
||||
# Resume an aborted run: point to its output root folder (or None for a fresh run).
|
||||
CONTINUE_FROM: Path | None = Path("/home/frank/pipeline_output/crusaders_260623085301")
|
||||
|
||||
COLOR1 = "#0064b0"
|
||||
COLOR2 = "#00badf"
|
||||
|
||||
# COLOR1 = #0064b0 -> 0,100,176; COLOR2 = #00badf -> 0,186,223
|
||||
GMIC_STEREO = "-gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0"
|
||||
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,0,186,223,200,0"
|
||||
GMIC_BWRECOLOR = (
|
||||
"-fx_bwrecolorize 0,0,0,0,0,1,0,2,0,186,223,255,0,100,176,0,255,"
|
||||
"158,137,189,255,224,191,228,255,0,100,176,0,255,255,255,255,255,255,255,"
|
||||
"255,255,0,100,176,0,255"
|
||||
)
|
||||
GMIC_GRADIENT_A = (
|
||||
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,0,100,176,255,'
|
||||
"0,186,223,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
|
||||
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
|
||||
)
|
||||
GMIC_GRADIENT_B = (
|
||||
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,0,186,223,255,'
|
||||
"0,100,176,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
|
||||
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
|
||||
)
|
||||
GMIC_JPR_SMOOTH = "-jpr_gradient_smooth 0,1.5"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="crusaders",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS or None,
|
||||
continue_from=CONTINUE_FROM,
|
||||
) as p:
|
||||
rembg_out = p.step("rembg", inputs="input")
|
||||
|
||||
white_bg = p.step("imagemagick_fill", inputs="input", color1="#ffffff")
|
||||
black_bg = p.step("imagemagick_fill", inputs="input", color1="#000000")
|
||||
gradient_45_bg = p.step(
|
||||
"imagemagick_fill",
|
||||
inputs="input",
|
||||
color1=COLOR1,
|
||||
color2=COLOR2,
|
||||
gradient=True,
|
||||
angle=45,
|
||||
)
|
||||
gradient_radial_bg = p.step(
|
||||
"imagemagick_fill",
|
||||
inputs="input",
|
||||
color1=COLOR1,
|
||||
color2=COLOR2,
|
||||
gradient=True,
|
||||
radial=True,
|
||||
)
|
||||
grayscale = p.step("gmic_grayscale", inputs="input")
|
||||
|
||||
rembg_stereo = p.step("gmic", inputs=rembg_out, command=GMIC_STEREO)
|
||||
rembg_shadow = p.step("gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW)
|
||||
rembg_bwrecolor = p.step("gmic", inputs=rembg_out, command=GMIC_BWRECOLOR)
|
||||
rembg_gradient_a = p.step("gmic", inputs=rembg_out, command=GMIC_GRADIENT_A)
|
||||
rembg_gradient_b = p.step("gmic", inputs=rembg_out, command=GMIC_GRADIENT_B)
|
||||
rembg_jpr_smooth = p.step("gmic", inputs=rembg_out, command=GMIC_JPR_SMOOTH)
|
||||
rembg_jpr_smooth_sized = p.step(
|
||||
"imagemagick_scale_crop",
|
||||
inputs=rembg_jpr_smooth,
|
||||
scale=1.05,
|
||||
)
|
||||
|
||||
rembg_stereo_alpha = p.step("color_to_alpha", inputs=rembg_stereo, color="#000000")
|
||||
color_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1)
|
||||
rembg_smooth_alpha = p.step("color_to_alpha", inputs=rembg_jpr_smooth, color="#7f7f7f")
|
||||
rembg_smooth_sized = p.step(
|
||||
"imagemagick_scale_crop",
|
||||
inputs=rembg_smooth_alpha,
|
||||
scale=1.05,
|
||||
)
|
||||
|
||||
# combine: white background, rembg
|
||||
p.step("composite", inputs=[white_bg, rembg_out])
|
||||
|
||||
# combine: black background, rembg
|
||||
p.step("composite", inputs=[black_bg, rembg_out])
|
||||
|
||||
# combine: linear gradient background, rembg
|
||||
p.step("composite", inputs=[gradient_45_bg, rembg_out])
|
||||
|
||||
# combine: radial gradient background, rembg
|
||||
p.step("composite", inputs=[gradient_radial_bg, rembg_out])
|
||||
|
||||
# combine: original, rembg (stereo), rembg
|
||||
stereo_mid = p.step("composite", inputs=["input", rembg_stereo])
|
||||
p.step("composite", inputs=[stereo_mid, rembg_out])
|
||||
|
||||
# combine: original, rembg (drop shadow), rembg
|
||||
shadow_mid = p.step("composite", inputs=["input", rembg_shadow])
|
||||
p.step("composite", inputs=[shadow_mid, rembg_out])
|
||||
|
||||
# combine: original, rembg (bw recolorize @ 50%), rembg
|
||||
bw_mid = p.step(
|
||||
"composite",
|
||||
inputs=["input", rembg_bwrecolor],
|
||||
foreground_opacity=0.5,
|
||||
)
|
||||
p.step("composite", inputs=[bw_mid, rembg_out])
|
||||
|
||||
# combine: rembg (custom gradient A), rembg
|
||||
p.step("composite", inputs=[rembg_gradient_a, rembg_out])
|
||||
|
||||
# combine: rembg (custom gradient B), rembg
|
||||
p.step("composite", inputs=[rembg_gradient_b, rembg_out])
|
||||
|
||||
# combine: original, rembg (jpr smooth, scaled), rembg
|
||||
smooth_mid = p.step("composite", inputs=["input", rembg_jpr_smooth_sized])
|
||||
p.step("composite", inputs=[smooth_mid, rembg_out])
|
||||
|
||||
# combine: original (grayscale), rembg
|
||||
p.step("composite", inputs=[grayscale, rembg_out])
|
||||
|
||||
# combine: original, rembg (stereo, black to alpha), rembg
|
||||
stereo_alpha_mid = p.step("composite", inputs=["input", rembg_stereo_alpha])
|
||||
p.step("composite", inputs=[stereo_alpha_mid, rembg_out])
|
||||
|
||||
# combine: color background, rembg (drop shadow), rembg
|
||||
shadow_color_mid = p.step("composite", inputs=[color_bg, rembg_shadow])
|
||||
p.step("composite", inputs=[shadow_color_mid, rembg_out])
|
||||
|
||||
# combine: original, rembg (jpr smooth, #7f7f7f to alpha, scaled), rembg
|
||||
smooth_alpha_mid = p.step("composite", inputs=["input", rembg_smooth_sized])
|
||||
p.step("composite", inputs=[smooth_alpha_mid, rembg_out])
|
||||
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,81 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Orange pipeline: rembg variants composited over backgrounds and originals."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path("/home/frank/pics/20260704_Hellraisers und The Shape Schlossplatz/darktable_exported")
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
# Reuse outputs from a previous run or external folder (key = step id, e.g. rembg_01).
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {}
|
||||
|
||||
# Resume an aborted run: point to its output root folder (or None for a fresh run).
|
||||
CONTINUE_FROM: Path | None = None
|
||||
|
||||
COLOR1 = "#AA4E00"
|
||||
COLOR2 = "#EDDD93"
|
||||
|
||||
# COLOR1 = #AA4E00 -> 170,78,0; COLOR2 = #EDDD93 -> 237,221,147
|
||||
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,237,221,147,200,0"
|
||||
GMIC_BWRECOLOR = (
|
||||
"-fx_bwrecolorize 0,0,0,0,0,1,0,2,237,221,147,255,170,78,0,255,"
|
||||
"158,137,189,255,224,191,228,255,170,78,0,255,255,255,255,255,255,255,"
|
||||
"255,255,170,78,0,255"
|
||||
)
|
||||
GMIC_GRADIENT_A = (
|
||||
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,170,78,0,255,'
|
||||
"237,221,147,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
|
||||
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
|
||||
)
|
||||
GMIC_GRADIENT_B = (
|
||||
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,237,221,147,255,'
|
||||
"170,78,0,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
|
||||
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="orange",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS or None,
|
||||
continue_from=CONTINUE_FROM,
|
||||
) as p:
|
||||
rembg_out = p.step("rembg", inputs="input")
|
||||
|
||||
grayscale = p.step("gmic_grayscale", inputs="input")
|
||||
|
||||
rembg_shadow = p.step("gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW)
|
||||
rembg_bwrecolor = p.step("gmic", inputs=rembg_out, command=GMIC_BWRECOLOR)
|
||||
|
||||
color_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1)
|
||||
|
||||
# combine: original, rembg (drop shadow), rembg
|
||||
shadow_mid = p.step("composite", inputs=["input", rembg_shadow])
|
||||
p.step("composite", inputs=[shadow_mid, rembg_out])
|
||||
|
||||
# combine: original, rembg (bw recolorize @ 50%), rembg
|
||||
bw_mid = p.step(
|
||||
"composite",
|
||||
inputs=["input", rembg_bwrecolor],
|
||||
foreground_opacity=0.5,
|
||||
)
|
||||
p.step("composite", inputs=[bw_mid, rembg_out])
|
||||
|
||||
# combine: original (grayscale), rembg
|
||||
p.step("composite", inputs=[grayscale, rembg_out])
|
||||
|
||||
# combine: color background, rembg (drop shadow), rembg
|
||||
shadow_color_mid = p.step("composite", inputs=[color_bg, rembg_shadow])
|
||||
p.step("composite", inputs=[shadow_color_mid, rembg_out])
|
||||
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Rezepttest pipeline — RECIPES.md smoke test (Indians Aichelberg logo colors)."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
# Change to your Darktable export folder.
|
||||
INPUT = Path("/home/frank/tmp/bar")
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
EXISTING_OUTPUTS: dict[str, Path] = {}
|
||||
CONTINUE_FROM: Path | None = None
|
||||
|
||||
# Colors from Indians Aichelberg logo (orange text + red stitching).
|
||||
COLOR1 = "#f0b020" # 240, 176, 32
|
||||
COLOR2 = "#f01000" # 240, 16, 0
|
||||
|
||||
ALPHA1 = 160
|
||||
ALPHA2 = 110
|
||||
|
||||
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,240,16,0,200,0"
|
||||
GMIC_JPR_SMOOTH = "-jpr_gradient_smooth 0,1.5"
|
||||
GMIC_BOKEH = (
|
||||
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,240,176,32,{ALPHA1},0.7,30,20,20,1,2,240,16,0,{ALPHA2},0.15"
|
||||
)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="rezepttest",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
existing_outputs=EXISTING_OUTPUTS or None,
|
||||
continue_from=CONTINUE_FROM,
|
||||
) as p:
|
||||
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
|
||||
grayscale = p.step("gmic_grayscale", inputs="input", step_id="grayscale")
|
||||
gradient_radial_bg = p.step(
|
||||
"imagemagick_fill",
|
||||
inputs="input",
|
||||
color1=COLOR1,
|
||||
color2=COLOR2,
|
||||
gradient=True,
|
||||
radial=True,
|
||||
step_id="gradient_radial_bg",
|
||||
)
|
||||
|
||||
rembg_shadow = p.step(
|
||||
"gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW, step_id="rembg_shadow"
|
||||
)
|
||||
rembg_jpr_smooth = p.step(
|
||||
"gmic", inputs=rembg_out, command=GMIC_JPR_SMOOTH, step_id="rembg_jpr_smooth"
|
||||
)
|
||||
rembg_jpr_smooth_sized = p.step(
|
||||
"imagemagick_scale_crop",
|
||||
inputs=rembg_jpr_smooth,
|
||||
scale=1.05,
|
||||
step_id="rembg_jpr_smooth_sized",
|
||||
)
|
||||
input_bokeh = p.step("gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh")
|
||||
|
||||
# recipe: colorsplash
|
||||
composite_colorsplash = p.step(
|
||||
"composite", inputs=[grayscale, rembg_out], step_id="composite_colorsplash"
|
||||
)
|
||||
|
||||
# recipe: rembg-radial-2colors (gradient-radial)
|
||||
composite_radial = p.step(
|
||||
"composite", inputs=[gradient_radial_bg, rembg_out], step_id="composite_radial"
|
||||
)
|
||||
|
||||
# recipe: original-drop-shadow-rembg
|
||||
shadow_mid = p.step("composite", inputs=["input", rembg_shadow], step_id="shadow_mid")
|
||||
composite_shadow = p.step(
|
||||
"composite", inputs=[shadow_mid, rembg_out], step_id="composite_shadow"
|
||||
)
|
||||
|
||||
# recipe: original-jpr-smooth-rembg
|
||||
smooth_mid = p.step(
|
||||
"composite", inputs=["input", rembg_jpr_smooth_sized], step_id="smooth_mid"
|
||||
)
|
||||
composite_smooth = p.step(
|
||||
"composite", inputs=[smooth_mid, rembg_out], step_id="composite_smooth"
|
||||
)
|
||||
|
||||
# recipe: bokeh-oktagon
|
||||
bokeh_mid = p.step("composite", inputs=["input", input_bokeh], step_id="bokeh_mid")
|
||||
composite_bokeh = p.step(
|
||||
"composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh"
|
||||
)
|
||||
|
||||
# recipe: xcf-stack — explicit layer list, bottom to top
|
||||
p.step(
|
||||
"xcf_stack",
|
||||
inputs=[
|
||||
"input",
|
||||
rembg_out,
|
||||
grayscale,
|
||||
gradient_radial_bg,
|
||||
rembg_shadow,
|
||||
rembg_jpr_smooth_sized,
|
||||
input_bokeh,
|
||||
composite_colorsplash,
|
||||
composite_radial,
|
||||
composite_shadow,
|
||||
composite_smooth,
|
||||
composite_bokeh,
|
||||
],
|
||||
)
|
||||
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,77 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Match new team member photos to an existing player gallery style via OpenRouter."""
|
||||
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parents[1]
|
||||
ENV_FILE = REPO_ROOT / ".env"
|
||||
|
||||
# Folder with new member photos to style-match.
|
||||
INPUT = Path("/home/frank/tmp/spieler")
|
||||
|
||||
# One existing gallery player image as style reference.
|
||||
TEMPLATE_IMAGE = Path("/home/frank/Downloads/2026_AHC_44_Julius-Nikolaus_Dittus-524-scaled.webp")
|
||||
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
OPENROUTER_MODEL = "google/gemini-3-pro-image"
|
||||
|
||||
GALLERY_MATCH_PROMPT = (
|
||||
"Match the second image to the first image's gallery style so it fits seamlessly "
|
||||
"alongside the other players. Match color grading, white balance, contrast, "
|
||||
"saturation, lighting direction, background treatment, sharpness, and overall "
|
||||
"polish. "
|
||||
"CRITICAL: Do not alter the person's face, identity, facial features, expression, "
|
||||
"hair, pose, body shape, or clothing details. Do not crop, reframe, or change the "
|
||||
"aspect ratio. Do not add or remove people or objects. "
|
||||
"Keep the exact same image dimensions and composition — only adjust global style "
|
||||
"and color to match the reference."
|
||||
)
|
||||
|
||||
|
||||
def _load_env_file(path: Path) -> None:
|
||||
if not path.is_file():
|
||||
return
|
||||
for raw_line in path.read_text(encoding="utf-8").splitlines():
|
||||
line = raw_line.strip()
|
||||
if not line or line.startswith("#"):
|
||||
continue
|
||||
key, sep, value = line.partition("=")
|
||||
if not sep:
|
||||
continue
|
||||
key = key.strip()
|
||||
value = value.strip()
|
||||
if len(value) >= 2 and value[0] == value[-1] and value[0] in "\"'":
|
||||
value = value[1:-1]
|
||||
os.environ.setdefault(key, value)
|
||||
|
||||
|
||||
def main() -> None:
|
||||
_load_env_file(ENV_FILE)
|
||||
|
||||
with Pipeline(
|
||||
name="team_gallery_match",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
) as p:
|
||||
p.step(
|
||||
"openrouter_edit",
|
||||
inputs="input",
|
||||
prompt=GALLERY_MATCH_PROMPT,
|
||||
model=OPENROUTER_MODEL,
|
||||
template_image=TEMPLATE_IMAGE,
|
||||
# Downscale for API, then upscale back to original dimensions.
|
||||
# Use 0 only if you accept higher cost/latency for ~10 MB sources.
|
||||
max_edge=4096,
|
||||
skip_existing=True,
|
||||
)
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,31 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Resize to 2000px max edge, then colorsplash + F12 watermark darktable style."""
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
INPUT = Path(
|
||||
"/home/frank/pics/20260712_Aichelberg Indians - Heidelberg Hedgehogs/darktable_exported/edits"
|
||||
)
|
||||
OUTPUT_BASE = Path.home() / "pipeline_output"
|
||||
|
||||
MAX_EDGE = 2000
|
||||
STYLE = "Watermark F12.rocks"
|
||||
|
||||
|
||||
def main() -> None:
|
||||
with Pipeline(
|
||||
name="watermark_f12_2000px",
|
||||
input_dir=INPUT,
|
||||
output_base=OUTPUT_BASE,
|
||||
) as p:
|
||||
resized = p.step("imagemagick_resize", inputs="input", max_edge=MAX_EDGE)
|
||||
p.step("darktable_style", inputs=resized, style=STYLE)
|
||||
output_root = p.run()
|
||||
|
||||
print(f"Pipeline finished. Output: {output_root}")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+34
-3
@@ -10,11 +10,17 @@ readme = "README.md"
|
||||
requires-python = ">=3.11"
|
||||
license = { text = "MIT" }
|
||||
authors = [{ name = "Frank" }]
|
||||
dependencies = []
|
||||
dependencies = [
|
||||
"Pillow>=10.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
ai = ["numpy>=1.26", "Pillow>=10.0", "torch>=2.0"]
|
||||
dev = ["pytest>=8.0"]
|
||||
ai = ["numpy>=1.26", "torch>=2.0"]
|
||||
dev = [
|
||||
"pytest>=8.0",
|
||||
"numpy>=1.26",
|
||||
"ruff>=0.8",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
imagepipeline = "imagepipeline.cli:main"
|
||||
@@ -25,3 +31,28 @@ include = ["imagepipeline*"]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
testpaths = ["tests"]
|
||||
markers = [
|
||||
"integration: tests that call external CLI tools (ImageMagick, G'MIC, rembg, darktable, GIMP)",
|
||||
"slow: tests that take more than a few seconds",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
target-version = "py311"
|
||||
line-length = 100
|
||||
src = ["imagepipeline", "tests", "pipelines"]
|
||||
|
||||
[tool.ruff.lint]
|
||||
select = [
|
||||
"E", # pycodestyle errors
|
||||
"F", # pyflakes
|
||||
"I", # isort
|
||||
"UP", # pyupgrade
|
||||
"B", # flake8-bugbear
|
||||
]
|
||||
ignore = [
|
||||
"E501", # line length handled by formatter
|
||||
"B027", # optional empty hooks on BaseModule
|
||||
]
|
||||
|
||||
[tool.ruff.lint.isort]
|
||||
known-first-party = ["imagepipeline"]
|
||||
|
||||
+2
-5
@@ -20,10 +20,7 @@ def make_png(
|
||||
crc = zlib.crc32(tag + data) & 0xFFFFFFFF
|
||||
return struct.pack(">I", len(data)) + tag + data + struct.pack(">I", crc)
|
||||
|
||||
raw = b"".join(
|
||||
b"\x00" + bytes([r, g, b] * width)
|
||||
for _ in range(height)
|
||||
)
|
||||
raw = b"".join(b"\x00" + bytes([r, g, b] * width) for _ in range(height))
|
||||
compressed = zlib.compress(raw, 9)
|
||||
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0)
|
||||
png = (
|
||||
@@ -37,7 +34,7 @@ def make_png(
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _ensure_builtin_modules() -> None:
|
||||
import imagepipeline.modules # noqa: F401
|
||||
pass # noqa: F401
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
|
||||
@@ -0,0 +1,53 @@
|
||||
"""Shared helpers for integration tests."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from tests.conftest import make_png
|
||||
|
||||
|
||||
def has_darktable_style(style_name: str, config_dir: Path | None = None) -> bool:
|
||||
"""Return True if darktable-cli can apply ``style_name`` to a tiny PNG."""
|
||||
if not shutil.which("darktable-cli"):
|
||||
return False
|
||||
|
||||
config = config_dir or Path.home() / ".config" / "darktable"
|
||||
styles_dir = config / "styles"
|
||||
if not styles_dir.is_dir():
|
||||
return False
|
||||
if not any(p.stem == style_name for p in styles_dir.glob("*.dtstyle")):
|
||||
return False
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp_path = Path(tmp)
|
||||
src = tmp_path / "probe.png"
|
||||
out_dir = tmp_path / "out"
|
||||
out_dir.mkdir()
|
||||
make_png(src)
|
||||
cmd = [
|
||||
"darktable-cli",
|
||||
str(src),
|
||||
str(out_dir),
|
||||
"--style",
|
||||
style_name,
|
||||
"--out-ext",
|
||||
"png",
|
||||
"--core",
|
||||
"--configdir",
|
||||
str(config),
|
||||
"--style-overwrite",
|
||||
]
|
||||
result = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if result.returncode != 0:
|
||||
return False
|
||||
return any(out_dir.iterdir())
|
||||
|
||||
|
||||
def has_workflow_tools() -> bool:
|
||||
"""True when rembg, gmic, ImageMagick, and darktable-cli are on PATH."""
|
||||
has_magick = bool(shutil.which("magick") or shutil.which("convert"))
|
||||
return all(shutil.which(name) for name in ("rembg", "gmic", "darktable-cli")) and has_magick
|
||||
@@ -1,5 +1,6 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from unittest.mock import MagicMock, patch
|
||||
@@ -73,6 +74,90 @@ class TestAIParameters:
|
||||
assert params["model"] == "black-forest-labs/flux.2-klein-4b"
|
||||
assert params["strength"] == 0.3
|
||||
assert params["api_key_env"] == "OPENROUTER_API_KEY"
|
||||
assert params["template_image"] is None
|
||||
|
||||
def test_openrouter_accepts_template_image(self, tmp_path: Path) -> None:
|
||||
template = tmp_path / "ref.png"
|
||||
make_png(template)
|
||||
params = OpenRouterEditModule.validate_module_params(
|
||||
{"prompt": "match style", "template_image": template}
|
||||
)
|
||||
assert params["template_image"] == template
|
||||
|
||||
def test_build_payload_with_template(self) -> None:
|
||||
payload = OpenRouterEditModule._build_payload(
|
||||
"data:image/jpeg;base64,abc",
|
||||
"match colors",
|
||||
"google/gemini-3-pro-image",
|
||||
0.3,
|
||||
template_data_url="data:image/jpeg;base64,ref",
|
||||
)
|
||||
assert payload["modalities"] == ["image", "text"]
|
||||
assert "image_config" not in payload
|
||||
content = payload["messages"][0]["content"]
|
||||
assert content[0]["type"] == "text"
|
||||
assert "FIRST image" in content[0]["text"]
|
||||
assert content[1]["image_url"]["url"] == "data:image/jpeg;base64,ref"
|
||||
assert content[2]["image_url"]["url"] == "data:image/jpeg;base64,abc"
|
||||
|
||||
def test_build_payload_flux_keeps_strength(self) -> None:
|
||||
payload = OpenRouterEditModule._build_payload(
|
||||
"data:image/jpeg;base64,abc",
|
||||
"brighten",
|
||||
"black-forest-labs/flux.2-klein-4b",
|
||||
0.25,
|
||||
)
|
||||
assert payload["modalities"] == ["image"]
|
||||
assert payload["image_config"] == {"strength": 0.25}
|
||||
|
||||
def test_save_result_matching_source_preserves_png_size(self, tmp_path: Path) -> None:
|
||||
try:
|
||||
from PIL import Image
|
||||
except ImportError:
|
||||
pytest.skip("Pillow not installed")
|
||||
|
||||
source = tmp_path / "source.png"
|
||||
dest = tmp_path / "out.png"
|
||||
with Image.new("RGBA", (16, 12), (10, 20, 30, 128)) as image:
|
||||
image.save(source, format="PNG")
|
||||
|
||||
with Image.new("RGB", (8, 6), (200, 100, 50)) as edited:
|
||||
buffer = io.BytesIO()
|
||||
edited.save(buffer, format="PNG")
|
||||
result_bytes = buffer.getvalue()
|
||||
|
||||
OpenRouterEditModule._save_result_matching_source(source, result_bytes, dest)
|
||||
|
||||
with Image.open(dest) as saved:
|
||||
assert saved.size == (16, 12)
|
||||
assert saved.mode == "RGBA"
|
||||
assert saved.getchannel("A").getextrema() == (128, 128)
|
||||
|
||||
def test_missing_template_image_raises(
|
||||
self, tmp_path: Path, monkeypatch: pytest.MonkeyPatch
|
||||
) -> None:
|
||||
monkeypatch.setenv("OPENROUTER_API_KEY", "test-key")
|
||||
src = tmp_path / "photo.png"
|
||||
make_png(src)
|
||||
output_dir = tmp_path / "out"
|
||||
output_dir.mkdir()
|
||||
ctx = ModuleContext(
|
||||
input_paths=[src],
|
||||
matched_groups=[],
|
||||
output_dir=output_dir,
|
||||
params=OpenRouterEditModule.validate_module_params(
|
||||
{
|
||||
"prompt": "match",
|
||||
"template_image": tmp_path / "missing.png",
|
||||
"max_edge": 0,
|
||||
}
|
||||
),
|
||||
pipeline_output_root=tmp_path,
|
||||
step_id="openrouter_edit_01",
|
||||
logger=None,
|
||||
)
|
||||
with pytest.raises(FileNotFoundError, match="Template image not found"):
|
||||
OpenRouterEditModule().run(ctx)
|
||||
|
||||
def test_comfy_requires_prompt(self) -> None:
|
||||
with pytest.raises(ValueError, match="required"):
|
||||
|
||||
+75
-3
@@ -113,9 +113,11 @@ class TestPipelineRunner:
|
||||
for src in ctx.input_paths:
|
||||
shutil.copy2(src, ctx.output_dir / src.name)
|
||||
|
||||
with Pipeline(name="order_test", input_dir=input_dir, output_base=output_base, verbose=False) as p:
|
||||
with Pipeline(
|
||||
name="order_test", input_dir=input_dir, output_base=output_base, verbose=False
|
||||
) as p:
|
||||
step_b = p.step("order_tracker", inputs="input")
|
||||
step_a = p.step("order_tracker", inputs=step_b)
|
||||
p.step("order_tracker", inputs=step_b)
|
||||
p.run()
|
||||
|
||||
assert order == ["order_tracker_01", "order_tracker_02"]
|
||||
@@ -132,7 +134,9 @@ class TestPipelineRunner:
|
||||
for src in ctx.input_paths:
|
||||
shutil.copy2(src, ctx.output_dir / src.name)
|
||||
|
||||
with Pipeline(name="dup_test", input_dir=input_dir, output_base=output_base, verbose=False) as p:
|
||||
with Pipeline(
|
||||
name="dup_test", input_dir=input_dir, output_base=output_base, verbose=False
|
||||
) as p:
|
||||
first = p.step("number_tracker", inputs="input")
|
||||
p.step("number_tracker", inputs=first)
|
||||
root = p.run()
|
||||
@@ -159,6 +163,74 @@ class TestManifest:
|
||||
assert data["finished_at"] is not None
|
||||
|
||||
|
||||
class TestCustomStepId:
|
||||
def test_custom_step_id_output_folder(self, input_dir: Path, output_base: Path) -> None:
|
||||
@register
|
||||
class NamedModule(BaseModule):
|
||||
name = "named_tracker"
|
||||
|
||||
def run(self, ctx) -> None:
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
for src in ctx.input_paths:
|
||||
shutil.copy2(src, ctx.output_dir / src.name)
|
||||
|
||||
with Pipeline(
|
||||
name="named_test", input_dir=input_dir, output_base=output_base, verbose=False
|
||||
) as p:
|
||||
ref = p.step("named_tracker", inputs="input", step_id="input_bokeh")
|
||||
root = p.run()
|
||||
|
||||
assert ref.step_id == "input_bokeh"
|
||||
assert ref.output_dir_name == "input_bokeh"
|
||||
assert (root / "input_bokeh").is_dir()
|
||||
assert not (root / "named_tracker_01").exists()
|
||||
|
||||
unregister("named_tracker")
|
||||
|
||||
def test_module_counter_independent_of_custom_step_id(
|
||||
self, input_dir: Path, output_base: Path
|
||||
) -> None:
|
||||
@register
|
||||
class CounterModule(BaseModule):
|
||||
name = "counter_tracker"
|
||||
|
||||
def run(self, ctx) -> None:
|
||||
ctx.output_dir.mkdir(parents=True, exist_ok=True)
|
||||
for src in ctx.input_paths:
|
||||
shutil.copy2(src, ctx.output_dir / src.name)
|
||||
|
||||
with Pipeline(
|
||||
name="counter_test", input_dir=input_dir, output_base=output_base, verbose=False
|
||||
) as p:
|
||||
p.step("counter_tracker", inputs="input")
|
||||
p.step("counter_tracker", inputs="input", step_id="custom_mid")
|
||||
p.step("counter_tracker", inputs="input")
|
||||
root = p.run()
|
||||
|
||||
assert (root / "counter_tracker_01").is_dir()
|
||||
assert (root / "custom_mid").is_dir()
|
||||
assert (root / "counter_tracker_03").is_dir()
|
||||
assert not (root / "counter_tracker_02").exists()
|
||||
|
||||
unregister("counter_tracker")
|
||||
|
||||
def test_duplicate_step_id_rejected(self, input_dir: Path) -> None:
|
||||
with Pipeline(name="dup_id", input_dir=input_dir, verbose=False) as p:
|
||||
p.step("imagemagick_grayscale", inputs="input", step_id="my_step")
|
||||
with pytest.raises(ValidationError, match="Duplicate step_id"):
|
||||
p.step("imagemagick_grayscale", inputs="input", step_id="my_step")
|
||||
|
||||
def test_empty_step_id_rejected(self, input_dir: Path) -> None:
|
||||
with Pipeline(name="empty_id", input_dir=input_dir, verbose=False) as p:
|
||||
with pytest.raises(ValidationError, match="must not be empty"):
|
||||
p.step("imagemagick_grayscale", inputs="input", step_id="")
|
||||
|
||||
def test_step_id_with_slash_rejected(self, input_dir: Path) -> None:
|
||||
with Pipeline(name="slash_id", input_dir=input_dir, verbose=False) as p:
|
||||
with pytest.raises(ValidationError, match="path separators"):
|
||||
p.step("imagemagick_grayscale", inputs="input", step_id="bad/name")
|
||||
|
||||
|
||||
class TestPipelineValidation:
|
||||
def test_empty_pipeline_rejected(self, input_dir: Path) -> None:
|
||||
with Pipeline(name="empty", input_dir=input_dir, verbose=False) as p:
|
||||
|
||||
@@ -0,0 +1,241 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from imagepipeline.core.pipeline import Pipeline
|
||||
from imagepipeline.core.resume import (
|
||||
expected_output_filenames,
|
||||
materialize_external_outputs,
|
||||
step_outputs_complete,
|
||||
)
|
||||
from imagepipeline.core.step import StepDefinition
|
||||
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
|
||||
from imagepipeline.modules.rembg import RembgModule
|
||||
from imagepipeline.utils.gmic import finalize_gmic_output, split_gmic_command
|
||||
from tests.conftest import make_png
|
||||
|
||||
|
||||
class TestGmicCommandSplit:
|
||||
def test_splits_command_and_arguments(self) -> None:
|
||||
assert split_gmic_command("-gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0") == [
|
||||
"-gcd_stereo_img",
|
||||
"0,0,2.028,1,1.714,3.06,4,1,0",
|
||||
]
|
||||
|
||||
def test_preserves_quoted_empty_argument(self) -> None:
|
||||
parts = split_gmic_command('-fx_custom_gradient 0,0,0,"",1,0')
|
||||
assert parts == ["-fx_custom_gradient", "0,0,0,,1,0"]
|
||||
|
||||
|
||||
class TestFinalizeGmicOutput:
|
||||
def test_keeps_frame_000001_and_removes_000000(self, tmp_path: Path) -> None:
|
||||
output_dir = tmp_path / "out"
|
||||
output_dir.mkdir()
|
||||
intended = output_dir / "photo.png"
|
||||
frame_000000 = output_dir / "photo_000000.png"
|
||||
frame_000001 = output_dir / "photo_000001.png"
|
||||
frame_000000.write_bytes(b"discard")
|
||||
frame_000001.write_bytes(b"keep")
|
||||
|
||||
result = finalize_gmic_output(output_dir, intended)
|
||||
|
||||
assert result == intended
|
||||
assert intended.read_bytes() == b"keep"
|
||||
assert not frame_000000.exists()
|
||||
assert not frame_000001.exists()
|
||||
|
||||
def test_leaves_single_output_unchanged(self, tmp_path: Path) -> None:
|
||||
output_dir = tmp_path / "out"
|
||||
output_dir.mkdir()
|
||||
intended = output_dir / "photo.png"
|
||||
intended.write_bytes(b"single")
|
||||
|
||||
result = finalize_gmic_output(output_dir, intended)
|
||||
|
||||
assert result == intended
|
||||
assert intended.read_bytes() == b"single"
|
||||
|
||||
def test_renames_only_000000_when_000001_missing(self, tmp_path: Path) -> None:
|
||||
output_dir = tmp_path / "out"
|
||||
output_dir.mkdir()
|
||||
intended = output_dir / "photo.png"
|
||||
frame_000000 = output_dir / "photo_000000.png"
|
||||
frame_000000.write_bytes(b"only")
|
||||
|
||||
result = finalize_gmic_output(output_dir, intended)
|
||||
|
||||
assert result == intended
|
||||
assert intended.read_bytes() == b"only"
|
||||
assert not frame_000000.exists()
|
||||
|
||||
|
||||
class TestExpectedOutputFilenames:
|
||||
def test_rembg_maps_jpg_inputs_to_png_outputs(self, tmp_path: Path) -> None:
|
||||
jpg = tmp_path / "photo.jpg"
|
||||
jpg.write_bytes(b"jpeg")
|
||||
step = StepDefinition(
|
||||
step_id="rembg_01",
|
||||
module_name="rembg",
|
||||
module=RembgModule,
|
||||
input_refs=["input"],
|
||||
params={},
|
||||
output_dir_name="rembg_01",
|
||||
)
|
||||
|
||||
names = expected_output_filenames(
|
||||
step,
|
||||
matched_groups=[[jpg]],
|
||||
input_paths=[jpg],
|
||||
params=RembgModule.validate_module_params({}),
|
||||
)
|
||||
|
||||
assert names == ["photo.png"]
|
||||
|
||||
def test_rembg_resume_detects_existing_png_outputs(self, tmp_path: Path) -> None:
|
||||
output_dir = tmp_path / "rembg_01"
|
||||
output_dir.mkdir()
|
||||
png = output_dir / "photo.png"
|
||||
make_png(png)
|
||||
jpg = tmp_path / "input" / "photo.jpg"
|
||||
jpg.parent.mkdir()
|
||||
jpg.write_bytes(b"jpeg")
|
||||
step = StepDefinition(
|
||||
step_id="rembg_01",
|
||||
module_name="rembg",
|
||||
module=RembgModule,
|
||||
input_refs=["input"],
|
||||
params={},
|
||||
output_dir_name="rembg_01",
|
||||
)
|
||||
params = RembgModule.validate_module_params({})
|
||||
expected = expected_output_filenames(
|
||||
step,
|
||||
matched_groups=[[jpg]],
|
||||
input_paths=[jpg],
|
||||
params=params,
|
||||
)
|
||||
|
||||
assert step_outputs_complete([output_dir / name for name in expected])
|
||||
|
||||
|
||||
class TestPipelineResume:
|
||||
@pytest.mark.skipif(not shutil.which("magick"), reason="ImageMagick not installed")
|
||||
def test_continue_skips_completed_steps(
|
||||
self, input_dir: Path, output_base: Path, capsys
|
||||
) -> None:
|
||||
with Pipeline(
|
||||
name="resume_test",
|
||||
input_dir=input_dir,
|
||||
output_base=output_base,
|
||||
verbose=True,
|
||||
) as p:
|
||||
first = p.step("imagemagick_grayscale", inputs="input")
|
||||
p.step("imagemagick_grayscale", inputs=first)
|
||||
root = p.run()
|
||||
|
||||
capsys.readouterr()
|
||||
|
||||
with Pipeline(
|
||||
name="resume_test",
|
||||
input_dir=input_dir,
|
||||
output_base=output_base,
|
||||
verbose=True,
|
||||
continue_from=root,
|
||||
) as p:
|
||||
step_a = p.step("imagemagick_grayscale", inputs="input")
|
||||
p.step("imagemagick_grayscale", inputs=step_a)
|
||||
resumed_root = p.run()
|
||||
|
||||
assert resumed_root == root
|
||||
output = capsys.readouterr().out
|
||||
assert "Skipped step imagemagick_grayscale_01" in output
|
||||
assert "Skipped step imagemagick_grayscale_02" in output
|
||||
|
||||
@pytest.mark.skipif(not shutil.which("magick"), reason="ImageMagick not installed")
|
||||
def test_existing_outputs_reuse_external_folder(
|
||||
self, input_dir: Path, output_base: Path, tmp_path: Path, capsys
|
||||
) -> None:
|
||||
external = tmp_path / "external_rembg"
|
||||
external.mkdir()
|
||||
for src in input_dir.iterdir():
|
||||
if src.is_file():
|
||||
make_png(external / src.name, width=4, height=4, rgb=(10, 20, 30))
|
||||
|
||||
with Pipeline(
|
||||
name="external_test",
|
||||
input_dir=input_dir,
|
||||
output_base=output_base,
|
||||
verbose=True,
|
||||
existing_outputs={"imagemagick_grayscale_01": external},
|
||||
) as p:
|
||||
reused = p.step("imagemagick_grayscale", inputs="input")
|
||||
p.step("imagemagick_grayscale", inputs=reused)
|
||||
root = p.run()
|
||||
|
||||
output = capsys.readouterr().out
|
||||
assert "Reused external output for imagemagick_grayscale_01" in output
|
||||
assert (root / "imagemagick_grayscale_01" / "photo_a.png").exists()
|
||||
|
||||
@pytest.mark.skipif(not shutil.which("magick"), reason="ImageMagick not installed")
|
||||
def test_existing_outputs_with_custom_step_id(
|
||||
self, input_dir: Path, output_base: Path, tmp_path: Path, capsys
|
||||
) -> None:
|
||||
external = tmp_path / "external_gray"
|
||||
external.mkdir()
|
||||
for src in input_dir.iterdir():
|
||||
if src.is_file():
|
||||
make_png(external / src.name, width=4, height=4, rgb=(10, 20, 30))
|
||||
|
||||
with Pipeline(
|
||||
name="custom_id_external",
|
||||
input_dir=input_dir,
|
||||
output_base=output_base,
|
||||
verbose=True,
|
||||
existing_outputs={"input_bokeh": external},
|
||||
) as p:
|
||||
reused = p.step("imagemagick_grayscale", inputs="input", step_id="input_bokeh")
|
||||
p.step("imagemagick_grayscale", inputs=reused)
|
||||
root = p.run()
|
||||
|
||||
output = capsys.readouterr().out
|
||||
assert "Reused external output for input_bokeh" in output
|
||||
assert (root / "input_bokeh" / "photo_a.png").exists()
|
||||
assert not (root / "imagemagick_grayscale_01").exists()
|
||||
|
||||
|
||||
class TestMaterializeExternal:
|
||||
def test_links_files_by_stem(self, tmp_path: Path) -> None:
|
||||
external = tmp_path / "external"
|
||||
external.mkdir()
|
||||
make_png(external / "photo_a.png")
|
||||
output_dir = tmp_path / "out"
|
||||
step = StepDefinition(
|
||||
step_id="imagemagick_grayscale_01",
|
||||
module_name="imagemagick_grayscale",
|
||||
module=ImageMagickGrayscale,
|
||||
input_refs=["input"],
|
||||
params={},
|
||||
output_dir_name="imagemagick_grayscale_01",
|
||||
)
|
||||
input_paths = [tmp_path / "photo_a.png"]
|
||||
paths = materialize_external_outputs(
|
||||
external,
|
||||
output_dir,
|
||||
step,
|
||||
matched_groups=[[path] for path in input_paths],
|
||||
input_paths=input_paths,
|
||||
params={},
|
||||
)
|
||||
assert len(paths) == 1
|
||||
assert paths[0].name == "photo_a.png"
|
||||
assert paths[0].is_symlink()
|
||||
|
||||
def test_step_outputs_complete(self, tmp_path: Path) -> None:
|
||||
output_dir = tmp_path / "done"
|
||||
output_dir.mkdir()
|
||||
make_png(output_dir / "photo.png")
|
||||
assert step_outputs_complete([output_dir / "photo.png"])
|
||||
assert not step_outputs_complete([output_dir / "missing.png"])
|
||||
+105
-20
@@ -5,18 +5,26 @@ from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from imagepipeline.core.params import validate_params
|
||||
from imagepipeline.modules.color_to_alpha import (
|
||||
ColorToAlphaModule,
|
||||
build_color_to_alpha_args,
|
||||
)
|
||||
from imagepipeline.modules.composite import CompositeModule
|
||||
from imagepipeline.modules.crop_square import CropSquareModule
|
||||
from imagepipeline.modules.darktable_style import DarktableStyleModule
|
||||
from imagepipeline.modules.gmic_grayscale import GmicGrayscale
|
||||
from imagepipeline.modules.imagemagick_fill import (
|
||||
ImageMagickFillModule,
|
||||
build_fill_arguments,
|
||||
)
|
||||
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
|
||||
from imagepipeline.modules.gmic_grayscale import GmicGrayscale
|
||||
from imagepipeline.modules.imagemagick_resize import (
|
||||
ImageMagickResizeModule,
|
||||
build_resize_arguments,
|
||||
)
|
||||
from imagepipeline.modules.registry import get_module, list_modules
|
||||
from imagepipeline.modules.rembg import RembgModule
|
||||
from tests.integration_helpers import has_darktable_style
|
||||
from tests.integration_helpers import has_workflow_tools as check_workflow_tools
|
||||
|
||||
has_magick = bool(shutil.which("magick") or shutil.which("convert"))
|
||||
|
||||
@@ -31,6 +39,8 @@ class TestModuleRegistration:
|
||||
"darktable_style",
|
||||
"imagemagick_grayscale",
|
||||
"imagemagick_fill",
|
||||
"color_to_alpha",
|
||||
"imagemagick_resize",
|
||||
"crop_square",
|
||||
):
|
||||
assert name in names
|
||||
@@ -42,6 +52,85 @@ class TestModuleRegistration:
|
||||
assert get_module("darktable_style") is DarktableStyleModule
|
||||
assert get_module("crop_square") is CropSquareModule
|
||||
assert get_module("imagemagick_fill") is ImageMagickFillModule
|
||||
assert get_module("color_to_alpha") is ColorToAlphaModule
|
||||
|
||||
|
||||
class TestImageMagickResize:
|
||||
def test_build_resize_arguments(self) -> None:
|
||||
assert build_resize_arguments(max_edge=2000) == [
|
||||
"-auto-orient",
|
||||
"-resize",
|
||||
"2000x2000>",
|
||||
]
|
||||
|
||||
def test_rejects_non_positive_max_edge(self) -> None:
|
||||
with pytest.raises(ValueError, match="positive"):
|
||||
build_resize_arguments(max_edge=0)
|
||||
|
||||
def test_default_max_edge(self) -> None:
|
||||
params = ImageMagickResizeModule.validate_module_params({})
|
||||
assert params["max_edge"] == 2000
|
||||
|
||||
|
||||
class TestColorToAlpha:
|
||||
def test_build_args_exact(self) -> None:
|
||||
assert build_color_to_alpha_args(color="#00ff00", fuzz=0.0) == [
|
||||
"-alpha",
|
||||
"on",
|
||||
"-transparent",
|
||||
"#00ff00",
|
||||
]
|
||||
|
||||
def test_build_args_with_fuzz(self) -> None:
|
||||
assert build_color_to_alpha_args(color="ffffff", fuzz=2.5) == [
|
||||
"-alpha",
|
||||
"on",
|
||||
"-fuzz",
|
||||
"2.5%",
|
||||
"-transparent",
|
||||
"#ffffff",
|
||||
]
|
||||
|
||||
def test_requires_color(self) -> None:
|
||||
with pytest.raises(ValueError, match="required"):
|
||||
ColorToAlphaModule.validate_module_params({})
|
||||
|
||||
@pytest.mark.skipif(not has_magick, reason="ImageMagick not installed")
|
||||
def test_makes_matching_color_transparent(self, tmp_path: Path) -> None:
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
src = tmp_path / "green.png"
|
||||
output_dir = tmp_path / "out"
|
||||
output_dir.mkdir()
|
||||
magick = shutil.which("magick") or shutil.which("convert")
|
||||
run_command([magick, "-size", "8x8", "xc:#00ff00", str(src)])
|
||||
|
||||
ctx = ModuleContext(
|
||||
input_paths=[src],
|
||||
matched_groups=[],
|
||||
output_dir=output_dir,
|
||||
params=ColorToAlphaModule.validate_module_params({"color": "#00ff00"}),
|
||||
pipeline_output_root=tmp_path,
|
||||
step_id="color_to_alpha_01",
|
||||
logger=None,
|
||||
)
|
||||
ColorToAlphaModule().run(ctx)
|
||||
|
||||
dst = output_dir / "green.png"
|
||||
assert dst.is_file()
|
||||
result = run_command(
|
||||
[
|
||||
magick,
|
||||
str(dst),
|
||||
"-alpha",
|
||||
"extract",
|
||||
"-format",
|
||||
"%[fx:mean]",
|
||||
"info:",
|
||||
]
|
||||
)
|
||||
assert float(result.stdout.strip()) == 0.0
|
||||
|
||||
|
||||
class TestImageMagickFill:
|
||||
@@ -95,7 +184,6 @@ class TestImageMagickFill:
|
||||
def test_output_matches_input_size(self, tmp_path: Path) -> None:
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
from tests.conftest import make_png
|
||||
|
||||
src = tmp_path / "ref.png"
|
||||
@@ -132,7 +220,6 @@ class TestCropSquare:
|
||||
def test_center_crops_to_square(self, tmp_path: Path) -> None:
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
from tests.conftest import make_png
|
||||
|
||||
src = tmp_path / "wide.png"
|
||||
@@ -166,9 +253,7 @@ class TestModuleParameters:
|
||||
DarktableStyleModule.validate_module_params({})
|
||||
|
||||
def test_darktable_style_accepts_style(self) -> None:
|
||||
params = DarktableStyleModule.validate_module_params(
|
||||
{"style": "Watermark F12.rocks"}
|
||||
)
|
||||
params = DarktableStyleModule.validate_module_params({"style": "Watermark F12.rocks"})
|
||||
assert params["style"] == "Watermark F12.rocks"
|
||||
assert params["style_overwrite"] is True
|
||||
|
||||
@@ -184,9 +269,7 @@ class TestModuleParameters:
|
||||
def test_darktable_export_conf_jpeg(self) -> None:
|
||||
from imagepipeline.modules.darktable_style import export_conf_options
|
||||
|
||||
assert export_conf_options("jpeg") == [
|
||||
"plugins/imageio/format/jpeg/quality=90"
|
||||
]
|
||||
assert export_conf_options("jpeg") == ["plugins/imageio/format/jpeg/quality=90"]
|
||||
|
||||
def test_darktable_export_conf_png(self) -> None:
|
||||
from imagepipeline.modules.darktable_style import export_conf_options
|
||||
@@ -219,7 +302,6 @@ class TestCompositeColor:
|
||||
def test_preserves_color_over_grayscale_background(self, tmp_path: Path) -> None:
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
from tests.conftest import make_png
|
||||
|
||||
src = tmp_path / "src.png"
|
||||
@@ -258,19 +340,22 @@ class TestCompositeColor:
|
||||
|
||||
output = output_dir / "fg.png"
|
||||
assert output.is_file()
|
||||
result = run_command(
|
||||
[magick, "identify", "-format", "%[type]", str(output)]
|
||||
)
|
||||
assert result.stdout.strip() != "Grayscale"
|
||||
from PIL import Image
|
||||
|
||||
with Image.open(output) as img:
|
||||
assert img.mode != "L"
|
||||
|
||||
|
||||
has_workflow_tools = all(
|
||||
shutil.which(name)
|
||||
for name in ("rembg", "gmic", "magick", "darktable-cli")
|
||||
) or all(shutil.which(name) for name in ("rembg", "gmic", "convert", "darktable-cli"))
|
||||
has_workflow_tools = check_workflow_tools()
|
||||
|
||||
|
||||
@pytest.mark.integration
|
||||
@pytest.mark.slow
|
||||
@pytest.mark.skipif(not has_workflow_tools, reason="Workflow CLI tools not installed")
|
||||
@pytest.mark.skipif(
|
||||
not has_darktable_style("Watermark F12.rocks"),
|
||||
reason="darktable style 'Watermark F12.rocks' not available or darktable-cli export failed",
|
||||
)
|
||||
class TestWorkflowIntegration:
|
||||
def test_watermark_pipeline(self, input_dir, output_base) -> None:
|
||||
from imagepipeline import Pipeline
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from imagepipeline.core.context import ModuleContext
|
||||
from imagepipeline.modules.registry import get_module, list_modules
|
||||
from imagepipeline.modules.xcf_stack import XcfStackModule
|
||||
from imagepipeline.utils.files import find_image_by_stem
|
||||
from tests.conftest import make_png
|
||||
|
||||
has_gimp = bool(shutil.which("gimp-console") or shutil.which("gimp"))
|
||||
has_magick = bool(shutil.which("magick") or shutil.which("convert"))
|
||||
|
||||
|
||||
class TestFindImageByStem:
|
||||
def test_finds_matching_stem_case_insensitively(self, tmp_path: Path) -> None:
|
||||
make_png(tmp_path / "Photo.PNG")
|
||||
found = find_image_by_stem(tmp_path, "photo")
|
||||
assert found == tmp_path / "Photo.PNG"
|
||||
|
||||
def test_returns_none_when_missing(self, tmp_path: Path) -> None:
|
||||
make_png(tmp_path / "other.png")
|
||||
assert find_image_by_stem(tmp_path, "photo") is None
|
||||
|
||||
def test_handles_extension_mismatch(self, tmp_path: Path) -> None:
|
||||
make_png(tmp_path / "photo.png")
|
||||
# Input may be photo.jpg while step output is photo.png (stem-only match).
|
||||
found = find_image_by_stem(tmp_path, Path("photo.jpg").stem)
|
||||
assert found == tmp_path / "photo.png"
|
||||
|
||||
|
||||
class TestModuleRegistration:
|
||||
def test_xcf_stack_registered(self) -> None:
|
||||
assert "xcf_stack" in list_modules()
|
||||
|
||||
def test_get_module_returns_xcf_stack_class(self) -> None:
|
||||
assert get_module("xcf_stack") is XcfStackModule
|
||||
|
||||
def test_xcf_stack_waits_for_layer_inputs(self) -> None:
|
||||
from imagepipeline.core.runner import PipelineRunner
|
||||
from imagepipeline.core.step import StepDefinition
|
||||
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
|
||||
|
||||
steps = [
|
||||
StepDefinition(
|
||||
step_id="xcf_stack_01",
|
||||
module_name="xcf_stack",
|
||||
module=XcfStackModule,
|
||||
input_refs=["input", "imagemagick_grayscale_01"],
|
||||
params={},
|
||||
output_dir_name="xcf_stack_01",
|
||||
),
|
||||
StepDefinition(
|
||||
step_id="imagemagick_grayscale_01",
|
||||
module_name="imagemagick_grayscale",
|
||||
module=ImageMagickGrayscale,
|
||||
input_refs=["input"],
|
||||
params={},
|
||||
output_dir_name="imagemagick_grayscale_01",
|
||||
),
|
||||
]
|
||||
runner = PipelineRunner(
|
||||
name="order",
|
||||
input_dir=Path("/tmp/unused"),
|
||||
output_base=Path("/tmp/unused"),
|
||||
steps=steps,
|
||||
)
|
||||
ordered = runner._topological_sort()
|
||||
assert [step.step_id for step in ordered] == [
|
||||
"imagemagick_grayscale_01",
|
||||
"xcf_stack_01",
|
||||
]
|
||||
|
||||
|
||||
class TestExpectedOutputFilenames:
|
||||
def test_returns_xcf_for_jpg_input(self) -> None:
|
||||
names = XcfStackModule.expected_output_filenames(
|
||||
matched_groups=[],
|
||||
input_paths=[Path("photo.jpg")],
|
||||
params={},
|
||||
)
|
||||
assert names == ["photo.xcf"]
|
||||
|
||||
def test_default_timeout(self) -> None:
|
||||
params = XcfStackModule.validate_module_params({})
|
||||
assert params["timeout"] == 120.0
|
||||
|
||||
def test_custom_timeout(self) -> None:
|
||||
params = XcfStackModule.validate_module_params({"timeout": 45})
|
||||
assert params["timeout"] == 45.0
|
||||
|
||||
|
||||
def _make_stack_fixture(tmp_path: Path) -> dict[str, Path]:
|
||||
root = tmp_path
|
||||
input_dir = root / "input"
|
||||
step_a = root / "step_a"
|
||||
step_b = root / "step_b"
|
||||
output_dir = root / "xcf_out"
|
||||
for directory in (input_dir, step_a, step_b, output_dir):
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
make_png(input_dir / "photo.png")
|
||||
make_png(step_a / "photo.png", rgb=(10, 20, 30))
|
||||
make_png(step_b / "photo.png", rgb=(30, 20, 10))
|
||||
|
||||
return {
|
||||
"root": root,
|
||||
"input_dir": input_dir,
|
||||
"step_a": step_a,
|
||||
"step_b": step_b,
|
||||
"output_dir": output_dir,
|
||||
}
|
||||
|
||||
|
||||
class TestXcfStackRun:
|
||||
@patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf")
|
||||
def test_collects_layers_from_explicit_inputs(self, mock_stack: object, tmp_path: Path) -> None:
|
||||
paths = _make_stack_fixture(tmp_path)
|
||||
input_path = paths["root"] / "refs" / "photo.jpg"
|
||||
input_path.parent.mkdir()
|
||||
input_path.touch()
|
||||
|
||||
ctx = ModuleContext(
|
||||
input_paths=[input_path],
|
||||
output_dir=paths["output_dir"],
|
||||
params=XcfStackModule.validate_module_params({}),
|
||||
pipeline_output_root=paths["root"],
|
||||
step_id="xcf_stack_01",
|
||||
input_layer_dirs=[
|
||||
("input", paths["input_dir"]),
|
||||
("step_a", paths["step_a"]),
|
||||
("step_b", paths["step_b"]),
|
||||
],
|
||||
logger=None,
|
||||
)
|
||||
XcfStackModule().run(ctx)
|
||||
|
||||
mock_stack.assert_called_once()
|
||||
layers, outfile = mock_stack.call_args[0]
|
||||
assert outfile == paths["output_dir"] / "photo.xcf"
|
||||
assert layers == [
|
||||
("input", paths["input_dir"] / "photo.png"),
|
||||
("step_a", paths["step_a"] / "photo.png"),
|
||||
("step_b", paths["step_b"] / "photo.png"),
|
||||
]
|
||||
|
||||
@patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf")
|
||||
def test_skip_missing_true_skips_missing_step(self, mock_stack: object, tmp_path: Path) -> None:
|
||||
paths = _make_stack_fixture(tmp_path)
|
||||
(paths["step_b"] / "photo.png").unlink()
|
||||
input_path = paths["root"] / "photo.jpg"
|
||||
|
||||
ctx = ModuleContext(
|
||||
input_paths=[input_path],
|
||||
output_dir=paths["output_dir"],
|
||||
params=XcfStackModule.validate_module_params({"skip_missing": True}),
|
||||
pipeline_output_root=paths["root"],
|
||||
step_id="xcf_stack_01",
|
||||
input_layer_dirs=[
|
||||
("input", paths["input_dir"]),
|
||||
("step_a", paths["step_a"]),
|
||||
("step_b", paths["step_b"]),
|
||||
],
|
||||
logger=None,
|
||||
)
|
||||
XcfStackModule().run(ctx)
|
||||
|
||||
layers, _outfile = mock_stack.call_args[0]
|
||||
step_ids = [step_id for step_id, _ in layers]
|
||||
assert step_ids == ["input", "step_a"]
|
||||
|
||||
@patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf")
|
||||
def test_skip_missing_false_raises_on_missing_step(
|
||||
self, mock_stack: object, tmp_path: Path
|
||||
) -> None:
|
||||
paths = _make_stack_fixture(tmp_path)
|
||||
(paths["step_b"] / "photo.png").unlink()
|
||||
input_path = paths["root"] / "photo.jpg"
|
||||
|
||||
ctx = ModuleContext(
|
||||
input_paths=[input_path],
|
||||
output_dir=paths["output_dir"],
|
||||
params=XcfStackModule.validate_module_params({"skip_missing": False}),
|
||||
pipeline_output_root=paths["root"],
|
||||
step_id="xcf_stack_01",
|
||||
input_layer_dirs=[
|
||||
("input", paths["input_dir"]),
|
||||
("step_a", paths["step_a"]),
|
||||
("step_b", paths["step_b"]),
|
||||
],
|
||||
logger=None,
|
||||
)
|
||||
with pytest.raises(ValueError, match="no image with stem 'photo' in step 'step_b'"):
|
||||
XcfStackModule().run(ctx)
|
||||
|
||||
mock_stack.assert_not_called()
|
||||
|
||||
|
||||
@pytest.mark.skipif(not has_gimp, reason="GIMP not installed")
|
||||
@pytest.mark.skipif(not has_magick, reason="ImageMagick not installed")
|
||||
class TestStackImagesToXcfIntegration:
|
||||
def test_stacks_pngs_into_xcf(self, tmp_path: Path) -> None:
|
||||
from imagepipeline.utils.gimp import stack_images_to_xcf
|
||||
|
||||
bottom = tmp_path / "bottom.png"
|
||||
top = tmp_path / "top.png"
|
||||
magick = shutil.which("magick") or shutil.which("convert")
|
||||
assert magick is not None
|
||||
from imagepipeline.utils.subprocess import run_command
|
||||
|
||||
run_command([magick, "-size", "8x8", "xc:#ff0000", str(bottom)])
|
||||
run_command([magick, "-size", "8x8", "xc:#0000ff", str(top)])
|
||||
|
||||
outfile = tmp_path / "stack.xcf"
|
||||
stack_images_to_xcf(
|
||||
[("bottom", bottom), ("top", top)],
|
||||
outfile,
|
||||
timeout=120.0,
|
||||
)
|
||||
|
||||
assert outfile.is_file()
|
||||
assert outfile.stat().st_size > 0
|
||||
names = outfile.read_bytes()
|
||||
assert b"bottom" in names
|
||||
assert b"top" in names
|
||||
|
||||
|
||||
class TestGimpTimeoutHandling:
|
||||
def test_accepts_outfile_when_gimp_hangs_on_exit(self, tmp_path: Path) -> None:
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from imagepipeline.utils.gimp import stack_images_to_xcf
|
||||
|
||||
layer = tmp_path / "layer.png"
|
||||
layer.write_bytes(b"png")
|
||||
outfile = tmp_path / "stack.xcf"
|
||||
outfile.write_bytes(b"xcf-data")
|
||||
|
||||
def fake_popen(*_args, **_kwargs):
|
||||
process = MagicMock()
|
||||
process.communicate.side_effect = [
|
||||
subprocess.TimeoutExpired(cmd="gimp", timeout=1),
|
||||
("", ""),
|
||||
]
|
||||
process.kill = MagicMock()
|
||||
return process
|
||||
|
||||
with patch("imagepipeline.utils.gimp.subprocess.Popen", side_effect=fake_popen):
|
||||
with patch("imagepipeline.utils.gimp.require_gimp", return_value="gimp-console"):
|
||||
stack_images_to_xcf([("layer", layer)], outfile, timeout=1.0)
|
||||
|
||||
assert outfile.read_bytes() == b"xcf-data"
|
||||
Reference in New Issue
Block a user