Compare commits

..

12 Commits

Author SHA1 Message Date
Frank Schwenk 2da66dcb78 Now added delete for real
Test / pytest (push) Failing after 14s
2026-07-27 22:48:16 +02:00
Frank Schwenk a138e33c04 fix(tests): assert composite color via Pillow for IM6 CI
Test / pytest (push) Successful in 10s
Test / pytest (pull_request) Successful in 11s
ImageMagick 6 convert cannot run `identify` as a subcommand; use Pillow
mode so Gitea runners without magick pass the composite color check.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:49:16 +02:00
Frank Schwenk 5920766f42 feat(pipelines): add watermark f12 2000px resize pipeline
Test / pytest (push) Failing after 35s
Resize to 2000px max edge then apply F12 watermark darktable style.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:19 +02:00
Frank Schwenk ffc28914e1 docs: expand README and add architecture/contributing notes
Document resume, dependencies, external tools, dev commands, and MIT license.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:19 +02:00
Frank Schwenk 3443c8826d ci: add Gitea Actions workflow for ruff and unit tests
Run ruff check/format and pytest -m "not integration" on push/PR.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:19 +02:00
Frank Schwenk 92337b7dbc test: skip watermark integration when darktable export fails
Add darktable style probe and pytest integration/slow markers so CI and
machines without a working darktable-cli setup skip the full workflow test.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:19 +02:00
Frank Schwenk fd4658434a refactor: clarify module bootstrap and G'MIC output handling
Import all built-in modules explicitly, remove dead ImageMagick branch,
and call finalize_gmic_output in gmic_grayscale for resume parity with gmic.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:18 +02:00
Frank Schwenk a60a18a253 chore: add Ruff and apply formatting across codebase
Introduce ruff lint/format config, expand .gitignore, and reformat Python sources.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:14 +02:00
Frank Schwenk 84447d1d2c docs: add cleanup roadmap for quality pass
Phase 0 analysis and prioritized roadmap for the cleanup/quality-pass branch.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-18 17:34:12 +02:00
Frank Schwenk 9b6c80da42 feat: add aichelberg-indians pipeline and rebuild RECIPES.md
Replace the verbose recipe catalog with a compact agent reference (modules,
ImageMagick ops, G'MIC commands, combines) and add the Indians Aichelberg
tone-mapping pipeline with gmic variants and xcf export.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-13 10:15:28 +02:00
Frank Schwenk 765ebbe3da feat: optional step_id for custom output folder names
Allow readable step folders and xcf_stack layer labels via p.step(..., step_id=...)
while keeping the per-module auto counter unchanged.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-12 22:21:01 +02:00
Frank Schwenk 0daf3e2315 refactor(xcf_stack): use explicit inputs for layer list and scheduling
Replace prior_steps and runs_last with inputs=[...] step refs so GIMP
export waits only on listed layers. Add rezepttest pipeline and
bokeh-oktagon recipe.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-12 12:51:35 +02:00
57 changed files with 1968 additions and 815 deletions
+28
View File
@@ -0,0 +1,28 @@
name: Test
on:
push:
branches: [main, cleanup/**]
pull_request:
jobs:
pytest:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: "3.12"
- name: Install package and dev dependencies
run: pip install -e ".[dev]"
- name: Ruff
run: |
pip install ruff
ruff check .
ruff format --check .
- name: Pytest (unit; skip integration)
run: pytest -m "not integration"
+8
View File
@@ -5,6 +5,14 @@ __pycache__/
dist/
build/
.pytest_cache/
.ruff_cache/
.mypy_cache/
.coverage
htmlcov/
.venv/
venv/
.env
*.log
# Local pipeline run output (if created inside repo)
pipeline_output/
+57
View File
@@ -0,0 +1,57 @@
# Cleanup Plan — imagepipeline quality pass
Branch: `cleanup/quality-pass`
Started: 2026-07-18
## Phase 0 findings (summary)
| Area | Risk | Notes |
|------|------|-------|
| Core (runner, resume, pipeline) | Medium | Solid tests; `xcf_stack` special-case in runner — defer refactor |
| Modules | LowMedium | `gmic_grayscale` missing `finalize_gmic_output` (resume parity) |
| Dependencies | High | `dependencies = []` but Pillow/numpy imported at module load |
| Pipelines | Low | Machine-local paths intentional (SOUL); stale `CONTINUE_FROM` left as-is |
| Tests | Medium | 87/88 pass; watermark integration fails on darktable-cli 5.6 |
| Tooling | Safe | No linter/formatter/CI |
## Roadmap
### Phase 1 — Low-risk quick wins
- [x] Add Ruff (lint + format) and apply once
- [x] Expand `.gitignore`
- [x] Remove dead duplicate branch in `imagemagick_grayscale`
- [x] Fix flaky `test_watermark_pipeline` (darktable probe skip)
### Phase 2 — Structure & config
- [x] Explicit `import imagepipeline.modules` in `pipeline.py`
- [x] `gmic_grayscale`: call `finalize_gmic_output` (parity with `gmic`)
- [ ] Pipeline `CONTINUE_FROM` reset — **skipped** (machine-local resume state)
- [ ] `xcf_stack` runner hook — **deferred** (medium refactor risk)
### Phase 3 — Code quality
- [x] Pytest markers (`integration`, `slow`)
- [ ] Broad dedup of pipeline scripts — **deferred** (domain-specific, RECIPES first)
### Phase 4 — Dependencies & CI
- [x] Declare Pillow in core; numpy in dev for import-time ai_tone_map
- [x] Add Ruff to dev extras
- [x] Gitea Actions: pytest on push
### Phase 5 — Docs
- [x] README sync (resume, `.env`, `[ai]`, external tools)
- [x] `docs/ARCHITECTURE.md`, `CONTRIBUTING.md`
- [x] `LICENSE` (MIT, matches pyproject)
## Known bugs (not fixed — behavior change or out of scope)
1. **darktable-cli 5.6** — integration test fails locally; style file exists but CLI exits 1 on PNG export (possible upstream CLI change).
2. **Eager module imports**`ai_tone_map` pulls numpy at import; mitigated via dev dep, not lazy-import refactor.
## Verification
```bash
pip install -e ".[dev,ai]"
ruff check .
ruff format --check .
pytest
```
+36
View File
@@ -0,0 +1,36 @@
# Contributing
## Setup
```bash
pip install -e ".[dev,ai]"
```
Use `.[dev]` only for core tests; AI module tests need `.[ai]` (torch). OpenRouter/Comfy steps need `OPENROUTER_API_KEY` in `.env` (see `.env.example`).
## Code style
- **Ruff** for lint and format (`ruff check .`, `ruff format .`)
- Python **3.11+**, type hints on public APIs
- CLI messages: direct English, no marketing tone (see `SOUL.md`)
## Tests
```bash
pytest # full suite (may call local CLIs)
pytest -m "not integration" # fast subset for CI
```
Mark new external-tool tests with `@pytest.mark.integration`.
## Adding a module
See [MODULE_DEVELOPMENT.md](MODULE_DEVELOPMENT.md). Register in `imagepipeline/modules/__init__.py` and add param/behavior tests.
## Pipelines
Scripts in `pipelines/` are examples and shoot-specific recipes. Machine-local `INPUT` paths are intentional. Reset `CONTINUE_FROM` before sharing a script.
## Commits
Conventional Commits (`feat:`, `fix:`, `chore:`, …). One logical change per commit.
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Frank
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+45 -5
View File
@@ -7,15 +7,19 @@ Each pipeline is a Python script that defines a DAG of processing steps. Every s
## Requirements
- Python 3.11+
- [ImageMagick](https://imagemagick.org/) (`magick` or `convert` on PATH)
- [ImageMagick](https://imagemagick.org/) (`magick` or `convert` on PATH) — most pipelines
- Optional CLIs per module: [G'MIC](https://gmic.eu/), [rembg](https://github.com/danielgatis/rembg), [darktable-cli](https://www.darktable.org/), GIMP (`gimp-console`)
## Installation
```bash
cd /path/to/imagepipeline
pip install -e ".[dev]"
pip install -e ".[dev]" # core + tests
pip install -e ".[dev,ai]" # + torch/numpy for AI modules
```
Copy `.env.example` to `.env` and set `OPENROUTER_API_KEY` when using `openrouter_edit` or Comfy-related workflows.
## Quick Start
Edit the input path in `pipelines/example_grayscale.py`, then run:
@@ -35,9 +39,11 @@ with Pipeline(name="my_run", input_dir=Path("/path/to/export")) as p:
p.run()
```
List registered modules: `imagepipeline list-modules`
## Output Structure
Each run creates a folder like `my_run_20260527143022/`:
Each run creates a folder like `my_run_20260527143022/` under `~/pipeline_output/` (or `output_base`):
```
my_run_20260527143022/
@@ -48,7 +54,26 @@ my_run_20260527143022/
└── ...
```
Step folders are named `{module_name}_{nn}` (two-digit counter per module name).
Step folders are named `{module_name}_{nn}` by default (two-digit counter per module name). Pass optional `step_id="input_bokeh"` to `p.step()` for a custom folder name and step reference (see [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md#step-folder-naming)).
## Resume
Pipelines support resuming interrupted runs:
```python
CONTINUE_FROM = Path("~/pipeline_output/my_run_260718120000")
EXISTING_OUTPUTS = {"rembg_01": CONTINUE_FROM / "rembg_01"}
with Pipeline(
name="my_run",
input_dir=INPUT,
continue_from=CONTINUE_FROM,
existing_outputs=EXISTING_OUTPUTS,
) as p:
...
```
Modules that change file extensions must implement `expected_output_filenames` so skip logic works (e.g. `rembg``.png`).
## Writing Pipelines
@@ -67,12 +92,27 @@ with Pipeline(name="colorsplash", input_dir=INPUT) as p:
- Parameters are passed as kwargs and validated against each module's schema
- Multiple uses of the same module get separate numbered folders
Declarative building blocks for agents and humans: [RECIPES.md](RECIPES.md).
## Adding Modules
See [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md).
See [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md). Architecture overview: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).
## Development
```bash
ruff check .
ruff format .
pytest # full suite (uses local CLIs when present)
pytest -m "not integration" # fast subset (CI default)
```
See [CONTRIBUTING.md](CONTRIBUTING.md).
## Tests
```bash
pytest
```
Optional AI tests require `pip install -e ".[ai]"`.
+236 -541
View File
@@ -1,615 +1,310 @@
# RECIPES.md — Declarative Pipeline Recipes
# RECIPES.md
*Open this file when you forget how recipes work — rules first, catalog below.*
Not executable code. Named building blocks for creating or editing pipelines in `pipelines/<name>.py`. Load via `@RECIPES.md` in Cursor when working on pipelines.
Reference for agents building `pipelines/<name>.py`. Not executable — expand to `p.step(...)` calls.
---
## 1. Rules cheat sheet
## Rules
1. **What a recipe is** — a named list of declarative lines. Agents turn it into Python using modules from `imagepipeline/modules/`.
2. **Layer order** — in every `combine` / composite recipe, lines are listed **bottom layer → top layer** (first line = background, last line = foreground on top).
3. **Indentation** — recipe id on its own line; indented lines below are the layers/steps of that recipe.
4. **`combine` = composite** — a multi-line recipe produces one output image by stacking those layers.
5. **Two-layer vs three-layer** — the `composite` module accepts exactly 2 inputs. Three or more layers need chained composites (bottom pair first, then add the next layer on top):
1. **combine** = one output image. Layers **bottom → top** (first line = background).
2. **composite** takes exactly **2** inputs. For 3+ layers: `composite(A,B)``composite(mid, C)` → …
3. **rembg** once per pipeline — reuse the same step ref in every combine.
4. **G'MIC in Python:** `command="-filter args"` (leading `-`). Multi-frame output: framework keeps frame `000001`.
5. **Colors:** hex (`#RRGGBB`) for `imagemagick_fill` / `color_to_alpha`. G'MIC color slots: `R,G,B` ints 0255 from hex bytes.
6. **Placeholders** at pipeline top: `COLOR1`, `COLOR2`, `R1,G1,B1`, `R2,G2,B2`, `ALPHA1` (default 160), `ALPHA2` (default 110), `STYLE`, `MAX_EDGE`, `PROMPT`, `MODEL`, `TEMPLATE_IMAGE`, `YELLOW`.
7. **xcf_stack:** separate step, not a combine layer. `inputs=[...]` bottom → top; include `"input"` for originals. Needs `gimp` on PATH.
8. Lines starting with `#` in user prompts = human notes — ignore.
```text
# 3 layers: A (bottom), B (middle), C (top)
combine
A
B
C
# → composite(A, B) then composite(result, C)
```
6. **Shared expensive steps** — `rembg` runs once per pipeline and is reused. Recipe lines say `rembg`, but the agent must not re-run it for every composite.
7. **`xcf_stack` (GIMP export)** — stacks **all prior step outputs** (plus `input/` by default) into one `.xcf` per image with layers named after step ids. Usually the **last step** after all composites. Requires `gimp` on PATH. Not a layer inside a `combine` block — a separate finishing recipe.
8. **Placeholders** — `COLOR1`, `COLOR2`, `COLOR`, `STYLE`, `MAX_EDGE`, `PROMPT`, `MODEL`, `TEMPLATE_IMAGE`, etc. are filled in at pipeline creation time (from your prompt or constants at the top of the script).
9. **Colors** — hex for backgrounds (`#RRGGBB` or `#RRGGBBAA`). G'MIC filters need `R,G,B` tuples (see [GMIC color derivation](#6-gmic-color-derivation) below).
10. **Single-step vs composite vs pipeline** — one declarative line = one module step. Multiple indented lines in a **composite** recipe = layers (bottom → top). Multiple indented lines in a **pipeline** recipe = sequential steps (first step first, output feeds the next).
11. **Recipe inclusion** — an indented line that matches a **recipe id** from this file expands that recipe instead of inlining its steps:
- In a **pipeline** recipe (e.g. `colorsplash-watermark`): resolve the referenced recipe, then chain the next line on its output.
- In a **composite** recipe: layers stay declarative (layer descriptions, not recipe ids). Do not nest composite recipes as layers.
- Expand recursively; share expensive steps (`rembg`, etc.) once across all expanded recipes in the same pipeline.
12. **Notes (human only)** — freeform reminders for you. **Agents must ignore them entirely** when building pipelines:
- **Full-line note:** line starts with `#` (optional leading spaces) — not a layer, not a recipe id.
- **Inline note:** `# …` after a recipe id on the same line (everything from `#` onward is ignored).
- Notes may be German or English. They never become code, constants, or prompts.
13. **Quick read examples:**
```text
original-stereo-rembg # wie horseland 3d effekt
original
rembg with gmic: gcd_stereo_img …
rembg
```
```text
colorsplash
# team gallery default look
original as greyscale
rembg
```
```text
colorsplash-watermark
colorsplash
darktable style STYLE
```
**Hex → RGB:** `#244a89``36,74,137`. Strip `#RRGGBBAA` alpha for G'MIC tuples.
---
## 2. How agents use this file
## Layer lines (example)
- Recipes are **names + declarative steps**, not Python.
- When creating or editing a pipeline: read `@RECIPES.md`, resolve named recipes, substitute placeholders, emit real `Pipeline` code.
- **Recipe inclusion:** if an indented line is a known recipe id (e.g. `colorsplash` inside `colorsplash-watermark`), look up that recipe, expand it, and use its output as the input for the next step. Expand recursively. Do not duplicate shared steps — one `rembg` (etc.) per pipeline when multiple included recipes need it.
- **Pipeline vs composite:** multiple indented lines that are **sequential steps or recipe refs** = pipeline recipe (chain). Multiple indented **layer** lines (original, rembg, backgrounds, gmic-on-rembg, …) = composite recipe (combine). When unsure: if any line is another recipe id, treat the parent as a pipeline recipe.
- **Ignore notes:** skip any line that is only a `#` comment (after indent trim). Strip inline `# …` suffixes on recipe-id lines. Do not copy note text into Python comments unless Fränky asks.
- **`xcf_stack`:** when a pipeline includes recipe `xcf-stack` (or the user asks for GIMP layers), append as the final step: `p.step("xcf_stack", inputs="input")`. The module auto-collects `prior_steps` from the runner — do not list every composite as explicit inputs. See `imagepipeline/modules/xcf_stack.py`.
- **Shared steps:** define `rembg` (and other expensive steps) **once** per pipeline and reference the step in composites — mirror `pipelines/pipeline_baxxter.py`.
- Follow the **Rules cheat sheet** above — especially layer order and chained composites.
- **3+ layers:** chain `composite` steps (see baxxter, crusaders, orange).
---
## 3. Vocabulary
| Recipe term | Maps to |
|-------------|---------|
| `original` | `inputs="input"` |
| `original as greyscale` | `gmic_grayscale` on input |
| `grayscale` | `imagemagick_grayscale` on input |
| `rembg` | `rembg` on input (outputs `.png`) |
| `white background` / `black background` | `imagemagick_fill` solid `#ffffff` / `#000000` |
| `gradient background COLOR1 COLOR2 45 degree` | `imagemagick_fill` linear, `angle=45` |
| `gradient background COLOR1 COLOR2 radial` | `imagemagick_fill` radial |
| `COLOR background` | `imagemagick_fill` solid `color1=COLOR` |
| `rembg with gmic: FILTER` | `gmic` on rembg output; command = `-FILTER` |
| `layer opacity N%` | `composite` `foreground_opacity=N/100` on that layer |
| `make layer 5% bigger then crop to original size` | `imagemagick_scale_crop` `scale=1.05` |
| `COLOR to alpha` | `color_to_alpha` `color=COLOR` |
| `resize max edge MAX_EDGE` | `imagemagick_resize` `max_edge=MAX_EDGE` |
| `crop square` | `crop_square` |
| `darktable style STYLE` | `darktable_style` `style=STYLE` |
| `xcf stack all steps` | `xcf_stack` — GIMP layer export of all prior steps |
| `xcf stack without input` | `xcf_stack` `include_input=false` |
| `xcf stack skip missing` | `xcf_stack` `skip_missing=true` |
| `openrouter edit PROMPT MODEL` | `openrouter_edit` |
| `openrouter gallery match TEMPLATE_IMAGE` | `openrouter_edit` with `template_image` |
**Placeholders:** `COLOR1`, `COLOR2`, `COLOR`, `STYLE`, `MAX_EDGE`, `PROMPT`, `MODEL`, `TEMPLATE_IMAGE`, `YELLOW` — substituted from the user prompt or pipeline constants.
---
## 4. Single-step recipes
One entry per built-in module. Format: recipe id, declarative line(s), Python mapping.
### rembg
One combine, bottom → top. Other phrases: `grayscale` / `original as greyscale`, `white`/`black background`, `original with gmic: ID`, `#COLOR to alpha`, `resize max edge N`, `crop square`, `darktable style STYLE`, `openrouter edit …`, `xcf stack layers: a, b, c`.
```text
rembg
rembg
```
- **Module:** `rembg` — `p.step("rembg", inputs="input")`
- **Note:** outputs `.png`; downstream steps match by stem.
### rembg-alpha-matting-off
```text
rembg-alpha-matting-off
rembg
```
- **Module:** `rembg` — `alpha_matting=False`
### grayscale-gmic
```text
grayscale-gmic
original as greyscale
```
- **Module:** `gmic_grayscale` — default command `-to_gray`
### grayscale-imagemagick
```text
grayscale-imagemagick
grayscale
```
- **Module:** `imagemagick_grayscale`
### resize-max-edge
```text
resize-max-edge
resize max edge MAX_EDGE
```
- **Module:** `imagemagick_resize` — e.g. `max_edge=2000`
- **Source:** `pipelines/pipeline_2000px.py`
### scale-crop-5pct
```text
scale-crop-5pct
make layer 5% bigger then crop to original size
```
- **Module:** `imagemagick_scale_crop` — `scale=1.05`
### solid-fill
```text
solid-fill
COLOR background
```
- **Module:** `imagemagick_fill` — `color1=COLOR`
### gradient-linear-45
```text
gradient-linear-45
gradient background COLOR1 COLOR2 45 degree
```
- **Module:** `imagemagick_fill` — `gradient=True`, `angle=45`
### gradient-radial
```text
gradient-radial
demo-combine
gradient background COLOR1 COLOR2 radial
original
rembg with gmic: gmic-drop-shadow
make layer 5% bigger then crop to original size
rembg
```
- **Module:** `imagemagick_fill` — `gradient=True`, `radial=True`
### gmic
```text
gmic
gmic: COMMAND
```
- **Module:** `gmic` — `command="-COMMAND"` (leading `-` as in existing pipelines)
### color-to-alpha
```text
color-to-alpha
COLOR to alpha
```
- **Module:** `color_to_alpha` — outputs `.png`
### darktable-style
```text
darktable-style
darktable style STYLE
```
- **Module:** `darktable_style` — style must exist in `~/.config/darktable/styles/`
### crop-square
```text
crop-square
crop square
```
- **Module:** `crop_square` — center-crop to largest square
### xcf-stack
```text
xcf-stack # GIMP layer export — one .xcf per input image
xcf stack all steps
```
- **Module:** `xcf_stack` — `p.step("xcf_stack", inputs="input")` as **last step**
- **Layer order in XCF:** `input/` (bottom, default) then prior steps in pipeline definition order
- **Matching:** layers matched by filename stem (extension may differ, e.g. rembg `.png` on `.jpg` input)
- **Output:** `{stem}.xcf` per input image
- **Requires:** `gimp` on PATH
- **No existing pipeline uses this yet** — append to any multi-output pipeline (e.g. baxxter variants + `xcf-stack`)
### xcf-stack-no-input
```text
xcf-stack-no-input
xcf stack without input
```
- **Module:** `xcf_stack` — `include_input=False`
### xcf-stack-skip-missing
```text
xcf-stack-skip-missing
xcf stack skip missing
```
- **Module:** `xcf_stack` — `skip_missing=True`
### openrouter-edit
```text
openrouter-edit
openrouter edit PROMPT MODEL
```
- **Module:** `openrouter_edit` — needs `OPENROUTER_API_KEY` in `.env`
- **Source:** `pipelines/example_ai.py`
### openrouter-gallery-match
```text
openrouter-gallery-match
openrouter gallery match TEMPLATE_IMAGE
```
- **Module:** `openrouter_edit` with `template_image=TEMPLATE_IMAGE` and gallery-match prompt
- **Source:** `pipelines/pipeline_team_gallery_match.py`
### ai-exposure
```text
ai-exposure
ai exposure strength STRENGTH
```
- **Module:** `ai_exposure` — optional `[ai]` extra; `max_edge`, `strength`
### ai-tone-map
```text
ai-tone-map
ai tone map strength STRENGTH
```
- **Module:** `ai_tone_map` — optional `[ai]` extra
### comfy-flux-edit
```text
comfy-flux-edit
comfy flux edit PROMPT
```
- **Module:** `comfy_flux_edit` — local ComfyUI; very slow on CPU
---
## 5. Composite and pipeline recipes
## Modules
**Composite** recipes produce one output image. Layers listed bottom → top.
**Pipeline** recipes chain steps or other recipes in order (output of step *n* → input of step *n+1*).
**Parameterized** recipes use `COLOR1` / `COLOR2` (hex) — derive G'MIC RGB via [section 6](#6-gmic-color-derivation).
| name | parameters (defaults) | notes |
|------|----------------------|-------|
| `rembg` | `model`=`birefnet-general`, `alpha_matting`=`True` | output `.png` |
| `gmic` | `command` (required) | timeout 300s |
| `gmic_grayscale` | `command`=`-to_gray` | |
| `composite` | `mode`=`over`, `output_ext`=`.png`, `foreground_opacity`=`1.0` | 2 inputs: bg, fg |
| `imagemagick_fill` | `color1`, `color2`=`""`, `gradient`=`False`, `radial`=`False`, `angle`=`None` | sized to input |
| `imagemagick_grayscale` | `colorspace`=`Gray` | |
| `imagemagick_resize` | `max_edge`=`2000` | no upscale |
| `imagemagick_scale_crop` | `scale`=`1.05` | center crop after scale |
| `color_to_alpha` | `color` (required), `fuzz`=`0.0` | output `.png` |
| `crop_square` | — | center square crop |
| `darktable_style` | `style` (required), `style_overwrite`=`True`, `out_ext`=`""`, `config_dir`=`~/.config/darktable` | jpeg/png only |
| `xcf_stack` | `skip_missing`=`False`, `timeout`=`120` | multi-input step |
| `openrouter_edit` | `prompt`, `model`, `strength`=`0.3`, `template_image`=`None`, `api_key_env`=`OPENROUTER_API_KEY` + AI common | needs `.env` |
| `ai_exposure` | `strength`=`1.0` + AI common | `[ai]` extra |
| `ai_tone_map` | `checkpoint`=`""`, `strength`=`1.0`, `net_input_size`=`256` + AI common | `[ai]` extra |
| `comfy_flux_edit` | `prompt`, `denoise`=`0.35`, `seed`=`-1`, `server_url`, `workflow_path`, `poll_interval`=`2.0` + AI common | local ComfyUI |
### colorsplash
**AI common** (`ai_exposure`, `ai_tone_map`, `openrouter_edit`, `comfy_flux_edit`): `skip_existing`=`True`, `max_edge`=`2048`, `device`=`cpu`.
---
## ImageMagick operations
Used via modules above — not free-form CLI in pipelines. `W×H` = size of reference input image.
| id | module | operation |
|----|--------|-----------|
| `im-solid` | `imagemagick_fill` | `-size W×H xc:COLOR` |
| `im-gradient-linear` | `imagemagick_fill` | `-size W×H -define gradient:angle=N gradient:COLOR1-COLOR2` |
| `im-gradient-radial` | `imagemagick_fill` | `-size W×H radial-gradient:COLOR1-COLOR2` |
| `im-grayscale` | `imagemagick_grayscale` | `-colorspace Gray` |
| `im-resize-max-edge` | `imagemagick_resize` | `-auto-orient -resize {N}x{N}>` (no upscale) |
| `im-scale-crop` | `imagemagick_scale_crop` | `-resize {w×scale}x{h×scale}! -gravity Center -crop w×h+0+0 +repage` |
| `im-crop-square` | `crop_square` | `-auto-orient -crop S×S+X+Y +repage` (center square) |
| `im-color-to-alpha` | `color_to_alpha` | `-alpha on [-fuzz N%] -transparent COLOR` |
| `im-composite-over` | `composite` | `(bg) (fg) -compose over -composite` |
| `im-composite-multiply` | `composite` | `mode=multiply` |
| `im-composite-screen` | `composite` | `mode=screen` |
| `im-composite-opacity` | `composite` | fg: `-channel A -evaluate multiply {0..1}` then compose |
**Layer line → id:** `COLOR background``im-solid`; `gradient … 45 degree``im-gradient-linear`; `gradient … radial``im-gradient-radial`; `grayscale``im-grayscale`; `resize max edge N``im-resize-max-edge`; `make layer 5% bigger …``im-scale-crop` (`scale=1.05`); `crop square``im-crop-square`; `COLOR to alpha``im-color-to-alpha`.
---
## G'MIC commands
Python string = `-` + filter + args. Placeholders `R1,G1,B1` / `R2,G2,B2` from `COLOR1` / `COLOR2`.
| id | command |
|----|---------|
| `gmic-tone-mapping` | `fx_map_tones 0.5,0.7,0.1,30,0` |
| `gmic-grayscale` | `to_gray` (via `gmic_grayscale`) |
| `gmic-feltpen` | `fx_feltpen 300,50,1,0.1,20,5` |
| `gmic-edges` | `fx_edges 0,15,0` |
| `gmic-charred-plastic` | `fx_charred_plastic 1,10,40,1,10,0,0,2,6,5,20,0,11` |
| `gmic-neon` | `fx_neon 0,1,1,1,1,0,0.45,40,60,0,1,1.15,2,3,0,3,20,0.4,0.1,1.5,5,0.2,0.1,1,2,1,0,0,1,1,0,50,50` |
| `gmic-cutout` | `fx_cutout 4,0.5,4,1` |
| `gmic-poster` | `fx_poster_hope 0,3` |
| `gmic-color-abstraction` | `fx_color_abstraction 1,10,0.2` |
| `gmic-huffman-glitches` | `fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50` |
| `gmic-queryprimary` | `afre_queryprimary 1,1` |
| `gmic-local-similarity` | `local_similarity_mask 50,50,50,128,4,10` |
| `gmic-luma-invert` | `Lylejk_Luma_Invert 1,0` |
| `gmic-crayongraffiti` | `fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0` |
| `gmic-anaglyph` | `fx_stereo_to_anaglyph 2,0` |
| `gmic-stereo` | `gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0` |
| `gmic-drop-shadow` | `fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,R2,G2,B2,200,0` |
| `gmic-jpr-smooth` | `jpr_gradient_smooth 0,1.5` |
| `gmic-bokeh` | `fx_bokeh 3,5,0,30,8,4,0.3,0.2,R1,G1,B1,ALPHA1,0.7,30,20,20,1,2,R2,G2,B2,ALPHA2,0.15` |
| `gmic-bwrecolor` | `fx_bwrecolorize 0,0,0,0,0,1,0,2,R2,G2,B2,255,R1,G1,B1,255,158,137,189,255,224,191,228,255,R1,G1,B1,0,255,255,255,255,255,255,255,255,255,R1,G1,B1,0,255` |
| `gmic-custom-gradient-a` | `fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R1,G1,B1,0,255,R2,G2,B2,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0` |
| `gmic-custom-gradient-b` | `fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R2,G2,B2,255,R1,G1,B1,0,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0` |
**Typical input:** `original with gmic:` = on `"input"`; `rembg with gmic:` = on rembg step.
---
## Combines
Format: recipe id, then layers bottom → top. `→` = chained composites.
### 2 layers
```text
colorsplash
original as greyscale
rembg
```
- **Layers:** grayscale background, rembg cutout on top
- **Source:** baxxter, orange, crusaders, `pipeline_colorsplash_watermark_f12.py`
### colorsplash-watermark
```text
colorsplash-watermark
colorsplash
darktable style STYLE
```
- **Type:** pipeline recipe (includes `colorsplash`, then chains `darktable_style`)
- **Python:** expand `colorsplash` → composite step ref → `p.step("darktable_style", inputs=combined, style=STYLE)`
- **Source:** `pipelines/pipeline_colorsplash_watermark_f12.py` (`STYLE = "Watermark F12.rocks"`)
### rembg-white-bg
```text
rembg-white-bg
white background
rembg
```
- **Source:** baxxter, crusaders
### rembg-black-bg
```text
rembg-black-bg
black background
rembg
```
- **Source:** baxxter, crusaders
### rembg-gradient-45
```text
rembg-gradient-45
gradient background COLOR1 COLOR2 45 degree
rembg
```
- **Source:** baxxter, crusaders
### rembg-radial-2colors
```text
rembg-radial-2colors
gradient background COLOR1 COLOR2 radial
rembg
```
- **Source:** baxxter, crusaders
rembg-custom-gradient-a
rembg with gmic: gmic-custom-gradient-a
rembg
### original-stereo-rembg
rembg-custom-gradient-b
rembg with gmic: gmic-custom-gradient-b
rembg
```text
original-stereo-rembg # wie horseland 3d effekt
original
rembg with gmic: gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0
gmic-feltpen-rembg
rembg with gmic: gmic-feltpen
rembg
gmic-edges-rembg
rembg with gmic: gmic-edges
rembg
gmic-charred-plastic-rembg
rembg with gmic: gmic-charred-plastic
rembg
gmic-neon-rembg
rembg with gmic: gmic-neon
rembg
```
- **3 layers:** composite(original, rembg_stereo) → composite(result, rembg)
- **Source:** baxxter, crusaders
### 3 layers
### original-stereo-black-alpha-rembg
```text
original-stereo-rembg
original
rembg with gmic: gmic-stereo
rembg
→ composite(original, rembg_stereo) → composite(_, rembg)
original-drop-shadow-rembg
original
rembg with gmic: gmic-drop-shadow
rembg
original-bwrecolor-rembg
original
rembg with gmic: gmic-bwrecolor
rembg
→ first composite: foreground_opacity=0.5
original-jpr-smooth-rembg
original
rembg with gmic: gmic-jpr-smooth
make layer 5% bigger then crop to original size
rembg
→ gmic → scale_crop on middle layer, then 3-layer chain
bokeh-oktagon
original
original with gmic: gmic-bokeh
rembg
color-bg-drop-shadow-rembg
COLOR1 background
rembg with gmic: gmic-drop-shadow
rembg
yellow-bg-drop-shadow-rembg
YELLOW background
rembg with gmic: gmic-drop-shadow
rembg
gmic-cutout-rembg
original
original with gmic: gmic-cutout
rembg
gmic-poster-rembg
original
original with gmic: gmic-poster
rembg
gmic-color-abstraction-rembg
original
original with gmic: gmic-color-abstraction
rembg
gmic-huffman-glitches-rembg
original
original with gmic: gmic-huffman-glitches
rembg
gmic-queryprimary-rembg
original
original with gmic: gmic-queryprimary
rembg
gmic-local-similarity-rembg
original
original with gmic: gmic-local-similarity
rembg
gmic-luma-invert-rembg
original
original with gmic: gmic-luma-invert
rembg
gmic-crayongraffiti-rembg
original
original with gmic: gmic-crayongraffiti
rembg
gmic-anaglyph-rembg
original
original with gmic: gmic-anaglyph
rembg
```
### 4 layers (middle pipeline)
```text
original-stereo-black-alpha-rembg
original
rembg with gmic: gcd_stereo_img 0,0,2.028,1,1.714,3.06,4,1,0
rembg with gmic: gmic-stereo
#000000 to alpha
rembg
```
- **Middle layer:** stereo gmic, then `#000000 to alpha` on that output
- **Source:** crusaders, baxxter_2
### original-drop-shadow-rembg
```text
original-drop-shadow-rembg
original
rembg with gmic: fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,R2,G2,B2,200,0
rembg
```
- **Shadow color:** RGB from `COLOR2` (see [GMIC color derivation](#6-gmic-color-derivation))
- **Source:** baxxter, orange, crusaders
### original-bwrecolor-rembg
```text
original-bwrecolor-rembg
original
rembg with gmic: fx_bwrecolorize 0,0,0,0,0,1,0,2,R2,G2,B2,255,R1,G1,B1,255,158,137,189,255,224,191,228,255,R1,G1,B1,0,255,255,255,255,255,255,255,255,255,R1,G1,B1,0,255
rembg
```
- **Middle layer opacity:** 50% (`foreground_opacity=0.5` on first composite)
- **Palette:** `R1,G1,B1` from `COLOR1`, `R2,G2,B2` from `COLOR2`
- **Source:** baxxter, orange, crusaders
### rembg-custom-gradient-a
```text
rembg-custom-gradient-a
rembg with gmic: fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R1,G1,B1,0,255,R2,G2,B2,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0
rembg
```
- **Gradient direction:** COLOR1 → COLOR2 on the rembg cutout
- **Source:** baxxter, orange, crusaders
### rembg-custom-gradient-b
```text
rembg-custom-gradient-b
rembg with gmic: fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,R2,G2,B2,255,R1,G1,B1,0,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0
rembg
```
- **Gradient direction:** COLOR2 → COLOR1 on the rembg cutout
- **Source:** baxxter, orange, crusaders
### original-jpr-smooth-rembg
```text
original-jpr-smooth-rembg
original
rembg with gmic: jpr_gradient_smooth 0,1.5
make layer 5% bigger then crop to original size
rembg
```
- **Middle layer:** jpr smooth on rembg, then scale 5% + center crop
- **Source:** baxxter, crusaders
### original-jpr-smooth-grey-alpha-rembg
```text
original-jpr-smooth-grey-alpha-rembg
original
rembg with gmic: jpr_gradient_smooth 0,1.5
rembg with gmic: gmic-jpr-smooth
#7f7f7f to alpha
make layer 5% bigger then crop to original size
rembg
```
- **Middle layer:** jpr smooth → `#7f7f7f to alpha` → scale 5% + crop
- **Source:** crusaders, baxxter_2
### color-bg-drop-shadow-rembg
```text
color-bg-drop-shadow-rembg
COLOR1 background
rembg with gmic: fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,R2,G2,B2,200,0
rembg
```
- **3 layers:** solid COLOR1 bg, drop-shadow rembg in middle, rembg on top
- **Source:** orange (`COLOR1 = #AA4E00`), crusaders
### yellow-bg-drop-shadow-rembg
```text
yellow-bg-drop-shadow-rembg
YELLOW background
rembg with gmic: fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,R2,G2,B2,200,0
rembg
```
- **Source:** `pipelines/pipeline_baxxter_2.py` (`YELLOW = #d7fd00`, shadow RGB from baxxter COLOR2)
### resize-2000px
```text
resize-2000px
resize max edge 2000
```
- **Single-step preset** — not a composite
- **Source:** `pipelines/pipeline_2000px.py`
### openrouter-enhance
```text
openrouter-enhance
openrouter edit PROMPT MODEL
```
- **Single-step** — subtle enhancement prompt from `pipelines/example_ai.py`
- **Typical:** `max_edge=2048`
### team-gallery-match
```text
team-gallery-match
openrouter gallery match TEMPLATE_IMAGE
```
- **Single-step** — style-match new photos to existing gallery reference
- **Source:** `pipelines/pipeline_team_gallery_match.py`
---
## 6. GMIC color derivation
G'MIC commands in parameterized recipes use **comma-separated RGB integers**, not hex.
### Hex → RGB
1. Take `#RRGGBB` or strip alpha from `#RRGGBBAA` (last two hex digits = alpha, ignored for GMIC tuples).
2. Split into three byte pairs → decimal 0255.
| Hex | R,G,B |
|-----|-------|
| `#AA4E00` | 170, 78, 0 |
| `#EDDD93` | 237, 221, 147 |
| `#d7fd00` | 215, 253, 0 |
| `#fc0ade` | 252, 10, 222 |
| `#0064b0` | 0, 100, 176 |
| `#00badf` | 0, 186, 223 |
### Where colors go
| Recipe | Color usage |
|--------|-------------|
| `original-drop-shadow-rembg` | shadow tint: `R2,G2,B2` from **COLOR2** in `fx_drop_shadow3d …,R2,G2,B2,200,0` |
| `original-bwrecolor-rembg` | highlight `R2,G2,B2`, accent `R1,G1,B1` in `fx_bwrecolorize` |
| `rembg-custom-gradient-a` | starts with `R1,G1,B1`, transitions via `R2,G2,B2` |
| `rembg-custom-gradient-b` | starts with `R2,G2,B2`, transitions via `R1,G1,B1` |
| `imagemagick_fill` backgrounds | use full hex including alpha if needed (`#d7fd00ff`) |
### Agent workflow
1. Define `COLOR1` and `COLOR2` as constants at the top of the pipeline script.
2. Add a short comment with derived RGB tuples (as in `pipeline_orange.py`).
3. Build GMIC command strings using those integers.
---
## 7. Usage examples
### Compose by recipe name
```text
Create pipeline "foo" with recipes colorsplash and rembg-radial-2colors, COLOR1=#123456, COLOR2=#654321
```
Agent: shared `rembg` step, `gmic_grayscale`, two `imagemagick_fill` gradients, two `composite` outputs (separate variants, not chained).
### Pipeline recipe with inclusion
```text
Create pipeline "watermark" with recipe colorsplash-watermark, STYLE="Watermark F12.rocks"
```
Agent: expand `colorsplash` inside `colorsplash-watermark`, then chain `darktable_style` on the composite output.
### Pipeline chain (not a combine)
```text
colorsplash-watermark
colorsplash
colorsplash # expand combine first
darktable style STYLE
```
### With GIMP layer export
---
```text
Create pipeline "foo" with recipes colorsplash, rembg-radial-2colors, and xcf-stack
## Agent example
Request: *pipeline with colorsplash + rembg-radial-2colors, COLOR1=#244a89, COLOR2=#c22518*
```python
rembg_out = p.step("rembg", inputs="input")
grayscale = p.step("gmic_grayscale", inputs="input")
gradient_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1, color2=COLOR2, gradient=True, radial=True)
p.step("composite", inputs=[grayscale, rembg_out]) # colorsplash
p.step("composite", inputs=[gradient_bg, rembg_out]) # rembg-radial-2colors
```
Agent: all composite outputs first, then `p.step("xcf_stack", inputs="input")` as the final step.
3-layer combine (drop shadow on original):
### Full combine list (baxxter style)
Paste a raw list of `combine` blocks (bottom → top). Agent maps each block to a recipe id from section 5 or expands inline steps. Reuse one `rembg` step for the whole pipeline.
### Extend existing pipeline
```text
Add recipe color-bg-drop-shadow-rembg to pipeline_orange.py with current COLOR1
```python
rembg_shadow = p.step("gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW)
mid = p.step("composite", inputs=["input", rembg_shadow])
p.step("composite", inputs=[mid, rembg_out])
```
Agent: read existing constants, append new composite steps using the same `rembg_out` and `rembg_shadow` pattern.
xcf export:
### Human notes in prompts
You may add `# notizen` in recipe blocks in this file anytime — agents ignore them per section 1 rule 11.
```python
p.step("xcf_stack", inputs=["input", rembg_out, grayscale, composite_colorsplash, ...])
```
+59
View File
@@ -0,0 +1,59 @@
# Architecture
Imagepipeline is a small Python framework for defining **batch image pipelines** as DAGs. Each step is a registered module; each run writes numbered folders under a timestamped output root.
## Layout
```
imagepipeline/
├── core/ # Pipeline, runner, resume, manifest, params, logging
├── modules/ # Processing steps (@register)
├── utils/ # files, subprocess, gmic, gimp helpers
├── ai/ # Optional torch models (HDRNet, Zero-DCE)
└── cli.py # `imagepipeline list-modules`
pipelines/ # Runnable scripts (machine-local INPUT paths)
tests/ # pytest suite
docs/ # Developer docs
workflows/comfy/ # ComfyUI workflow JSON (optional AI path)
```
## Execution flow
1. **Define**`Pipeline(name=..., input_dir=...)` collects `step()` calls (`StepDefinition` DAG).
2. **Run**`PipelineRunner` topologically sorts steps, matches inputs by filename stem, builds `ModuleContext`.
3. **Resume**`continue_from` / `existing_outputs` reuse prior run folders; modules declare `expected_output_filenames` when extensions change.
4. **Manifest**`pipeline_manifest.json` records steps, params, and paths.
## Module contract
Every module subclasses `BaseModule` or `SubprocessModule`:
- `name`, `description`, `parameters()` schema
- `run(ctx: ModuleContext)` writes into `ctx.output_dir`
- Optional `expected_output_filenames()` for resume when output names differ from inputs
- `check_dependencies()` for external CLI tools
Registration happens via `@register` and eager import in `imagepipeline/modules/__init__.py`.
## External tools
Many modules shell out to CLIs (not Python packages):
| Tool | Modules |
|------|---------|
| ImageMagick (`magick`/`convert`) | `imagemagick_*`, `composite`, `color_to_alpha`, `crop_square` |
| G'MIC | `gmic`, `gmic_grayscale` |
| rembg | `rembg` |
| darktable-cli | `darktable_style` |
| GIMP | `xcf_stack` |
AI modules need `pip install -e ".[ai]"` (torch, numpy) and optionally API keys in `.env`.
## Design choices
- **Plain Python pipelines** — no YAML DSL; full control and easy resume constants in script.
- **Stem matching** — multi-input steps align files by basename across step folders.
- **Symlink input** — default run copies/symlinks source images into `input/` for reproducibility.
See [MODULE_DEVELOPMENT.md](MODULE_DEVELOPMENT.md) for adding modules.
+12 -2
View File
@@ -167,7 +167,7 @@ class MyCliModule(SubprocessModule):
## Step Folder Naming
The runner assigns output folders automatically: `{module_name}_{nn}`.
By default the runner assigns output folders automatically: `{module_name}_{nn}` (two-digit counter per module name).
Using the same module twice in one pipeline produces separate folders:
@@ -178,7 +178,17 @@ darktable_style_01/
darktable_style_02/
```
You do not choose folder names in the module.
In pipeline scripts you can pass an optional `step_id` to `p.step()` for readable folder names and GIMP layer labels (`xcf_stack` uses the step id as the layer name):
```python
input_bokeh = p.step("gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh")
```
`step_id` becomes both the internal step reference and the output subfolder name. It must be unique within the pipeline and must not contain path separators. The per-module counter still increments on every `p.step("gmic", …)` call — a custom id does not reserve or skip a number slot (the next default `gmic` step after `step_id="input_bokeh"` is `gmic_03`, not `gmic_02`, if two prior `gmic` steps ran).
Use the same ids in `EXISTING_OUTPUTS` when resuming with external folders.
You do not choose folder names inside the module implementation.
## Format Warnings
-1
View File
@@ -1,6 +1,5 @@
from __future__ import annotations
import hashlib
import urllib.request
from pathlib import Path
+1 -1
View File
@@ -49,7 +49,7 @@ def _lab_to_rgb(lab: np.ndarray) -> np.ndarray:
fz = fy - lab[..., 2] / 200
def finv(t):
t3 = t ** 3
t3 = t**3
return np.where(t3 > 216 / 24389, t3, (116 * t - 16) / kappa)
kappa = 24389 / 27
+20 -20
View File
@@ -1,11 +1,8 @@
from __future__ import annotations
import math
import numpy as np
import torch
import torch.nn as nn
import torch.nn.functional as F
from imagepipeline.ai.hdrnet.slice import batch_bilateral_slice
@@ -70,22 +67,27 @@ class Slice(nn.Module):
class ApplyCoeffs(nn.Module):
def forward(self, coeff, full_res_input):
r = torch.sum(full_res_input * coeff[:, 0:3, :, :], dim=1, keepdim=True) + coeff[
:, 9:10, :, :
]
g = torch.sum(full_res_input * coeff[:, 3:6, :, :], dim=1, keepdim=True) + coeff[
:, 10:11, :, :
]
b = torch.sum(full_res_input * coeff[:, 6:9, :, :], dim=1, keepdim=True) + coeff[
:, 11:12, :, :
]
r = (
torch.sum(full_res_input * coeff[:, 0:3, :, :], dim=1, keepdim=True)
+ coeff[:, 9:10, :, :]
)
g = (
torch.sum(full_res_input * coeff[:, 3:6, :, :], dim=1, keepdim=True)
+ coeff[:, 10:11, :, :]
)
b = (
torch.sum(full_res_input * coeff[:, 6:9, :, :], dim=1, keepdim=True)
+ coeff[:, 11:12, :, :]
)
return torch.cat([r, g, b], dim=1)
class GuideNN(nn.Module):
def __init__(self, params) -> None:
super().__init__()
self.conv1 = ConvBlock(3, params["guide_complexity"], kernel_size=1, padding=0, batch_norm=True)
self.conv1 = ConvBlock(
3, params["guide_complexity"], kernel_size=1, padding=0, batch_norm=True
)
self.conv2 = ConvBlock(
params["guide_complexity"], 1, kernel_size=1, padding=0, activation=nn.Sigmoid
)
@@ -112,9 +114,7 @@ class Coeffs(nn.Module):
for index in range(n_layers_splat):
use_bn = bn if index > 0 else False
out_ch = cm * (2**index) * lb
self.splat_features.append(
ConvBlock(prev_ch, out_ch, 3, stride=2, batch_norm=use_bn)
)
self.splat_features.append(ConvBlock(prev_ch, out_ch, 3, stride=2, batch_norm=use_bn))
prev_ch = out_ch
splat_ch = prev_ch
@@ -131,7 +131,9 @@ class Coeffs(nn.Module):
prev_ch = int(prev_ch * (nsize / 2**n_total) ** 2)
self.global_features_fc.append(FC(prev_ch, 32 * cm * lb, batch_norm=bn))
self.global_features_fc.append(FC(32 * cm * lb, 16 * cm * lb, batch_norm=bn))
self.global_features_fc.append(FC(16 * cm * lb, 8 * cm * lb, activation=None, batch_norm=bn))
self.global_features_fc.append(
FC(16 * cm * lb, 8 * cm * lb, activation=None, batch_norm=bn)
)
self.local_features = nn.ModuleList(
[
@@ -139,9 +141,7 @@ class Coeffs(nn.Module):
ConvBlock(8 * cm * lb, 8 * cm * lb, 3, activation=None, use_bias=False),
]
)
self.conv_out = ConvBlock(
8 * cm * lb, lb * nout * nin, 1, padding=0, activation=None
)
self.conv_out = ConvBlock(8 * cm * lb, lb * nout * nin, 1, padding=0, activation=None)
self.relu = nn.ReLU()
def forward(self, lowres_input):
+2 -6
View File
@@ -82,12 +82,8 @@ def _bilateral_slice(grid, guide):
grid_val_110 = grid[gi1c, gj1c, gk0c, :]
grid_val_111 = grid[gi1c, gj1c, gk1c, :]
w_000, w_001, w_010, w_011 = map(
torch.atleast_3d, (w_000, w_001, w_010, w_011)
)
w_100, w_101, w_110, w_111 = map(
torch.atleast_3d, (w_100, w_101, w_110, w_111)
)
w_000, w_001, w_010, w_011 = map(torch.atleast_3d, (w_000, w_001, w_010, w_011))
w_100, w_101, w_110, w_111 = map(torch.atleast_3d, (w_100, w_101, w_110, w_111))
return (
torch.multiply(w_000, grid_val_000)
+2 -6
View File
@@ -60,9 +60,7 @@ class EnhanceNetNoPool(nn.Module):
if self.scale_factor == 1:
x_down = x
else:
x_down = F.interpolate(
x, scale_factor=1 / self.scale_factor, mode="bilinear"
)
x_down = F.interpolate(x, scale_factor=1 / self.scale_factor, mode="bilinear")
x1 = self.relu(self.e_conv1(x_down))
x2 = self.relu(self.e_conv2(x1))
@@ -105,7 +103,5 @@ def enhance_image(
if strength < 1.0:
enhanced = tensor * (1.0 - strength) + enhanced * strength
enhanced = torch.clamp(enhanced, 0.0, 1.0)
out = (enhanced.squeeze(0).permute(1, 2, 0).cpu().numpy() * 255.0).astype(
np.uint8
)
out = (enhanced.squeeze(0).permute(1, 2, 0).cpu().numpy() * 255.0).astype(np.uint8)
return Image.fromarray(out)
-1
View File
@@ -2,7 +2,6 @@ from __future__ import annotations
import argparse
import sys
from pathlib import Path
from imagepipeline.modules.registry import list_modules
+3 -2
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, TYPE_CHECKING
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from imagepipeline.core.log import PipelineLogger
@@ -18,7 +18,8 @@ class ModuleContext:
pipeline_output_root: Path
step_id: str
matched_groups: list[list[Path]] = field(default_factory=list)
prior_steps: list[tuple[str, Path]] = field(default_factory=list)
input_refs: list[str] = field(default_factory=list)
input_layer_dirs: list[tuple[str, Path]] = field(default_factory=list)
logger: PipelineLogger | None = None
@property
+3 -9
View File
@@ -36,23 +36,17 @@ class PipelineLogger:
inputs: list[str],
params: dict,
) -> None:
self.info(
f"Step {step_index}/{step_total}: {step_id} ({module_name})"
)
self.info(f"Step {step_index}/{step_total}: {step_id} ({module_name})")
self.info(f" inputs: {', '.join(inputs)}")
if params:
rendered = ", ".join(f"{key}={value!r}" for key, value in params.items())
self.info(f" params: {rendered}")
def image(self, module_name: str, index: int, total: int, filename: str) -> None:
self.info(
f" Applying module {module_name} to image [{index}/{total}]: {filename}"
)
self.info(f" Applying module {module_name} to image [{index}/{total}]: {filename}")
def skipped(self, module_name: str, index: int, total: int, filename: str) -> None:
self.info(
f" Skipped module {module_name} [{index}/{total}] {filename} (output exists)"
)
self.info(f" Skipped module {module_name} [{index}/{total}] {filename} (output exists)")
def image_done(
self,
+2 -2
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
import json
from dataclasses import asdict, dataclass, field
from datetime import datetime, timezone
from datetime import UTC, datetime
from pathlib import Path
from typing import Any
@@ -39,4 +39,4 @@ def write_manifest(path: Path, manifest: PipelineManifest) -> None:
def utc_now_iso() -> str:
return datetime.now(timezone.utc).replace(microsecond=0).isoformat()
return datetime.now(UTC).replace(microsecond=0).isoformat()
+3 -7
View File
@@ -1,6 +1,6 @@
from __future__ import annotations
from dataclasses import dataclass, field
from dataclasses import dataclass
from typing import Any
@@ -22,9 +22,7 @@ class Param:
if self.choices is not None and value not in self.choices:
allowed = ", ".join(repr(c) for c in self.choices)
raise ValueError(
f"Parameter '{name}' must be one of [{allowed}], got {value!r}"
)
raise ValueError(f"Parameter '{name}' must be one of [{allowed}], got {value!r}")
if self.type == "string":
if not isinstance(value, str):
@@ -56,9 +54,7 @@ class Param:
raise ValueError(f"Unknown parameter type '{self.type}' for '{name}'")
def validate_params(
schema: dict[str, Param], raw: dict[str, Any]
) -> dict[str, Any]:
def validate_params(schema: dict[str, Param], raw: dict[str, Any]) -> dict[str, Any]:
unknown = set(raw) - set(schema)
if unknown:
names = ", ".join(sorted(unknown))
+17 -9
View File
@@ -4,14 +4,13 @@ from collections import defaultdict
from pathlib import Path
from typing import Any
# Register all built-in modules (side effect of submodule imports).
import imagepipeline.modules # noqa: F401
from imagepipeline.core.exceptions import ValidationError
from imagepipeline.core.runner import PipelineRunner
from imagepipeline.core.step import INPUT_SOURCE, StepDefinition, StepRef
from imagepipeline.modules.registry import get_module
# Import built-in modules so they register on package load.
import imagepipeline.modules.imagemagick_grayscale # noqa: F401
class Pipeline:
"""Define and run an image processing pipeline."""
@@ -45,20 +44,23 @@ class Pipeline:
module_name: str,
*,
inputs: StepRef | str | list[StepRef | str],
step_id: str | None = None,
**params: Any,
) -> StepRef:
module_cls = get_module(module_name)
self._module_counters[module_name] += 1
counter = self._module_counters[module_name]
if step_id is None:
output_dir_name = f"{module_name}_{counter:02d}"
step_id = output_dir_name
else:
self._validate_step_id(step_id)
output_dir_name = step_id
input_refs = self._normalize_inputs(inputs)
reserved = {"inputs", "input"}
if reserved & set(params):
raise ValidationError(
"Do not pass 'inputs' or 'input' as module parameters"
)
raise ValidationError("Do not pass 'inputs' or 'input' as module parameters")
definition = StepDefinition(
step_id=step_id,
@@ -92,15 +94,21 @@ class Pipeline:
def output_root(self) -> Path | None:
return self._output_root
def _normalize_inputs(
self, inputs: StepRef | str | list[StepRef | str]
) -> list[str]:
def _normalize_inputs(self, inputs: StepRef | str | list[StepRef | str]) -> list[str]:
if isinstance(inputs, list):
if not inputs:
raise ValidationError("inputs must not be an empty list")
return [self._input_ref(item) for item in inputs]
return [self._input_ref(inputs)]
def _validate_step_id(self, step_id: str) -> None:
if not step_id:
raise ValidationError("step_id must not be empty")
if "/" in step_id or "\\" in step_id:
raise ValidationError(f"step_id must not contain path separators: {step_id!r}")
if any(step.step_id == step_id for step in self._steps):
raise ValidationError(f"Duplicate step_id: {step_id!r}")
@staticmethod
def _input_ref(value: StepRef | str) -> str:
if isinstance(value, StepRef):
+22 -18
View File
@@ -49,12 +49,9 @@ class PipelineRunner:
self.symlink_input = symlink_input
self.logger = PipelineLogger(verbose=verbose)
self.existing_outputs = {
key: Path(value).resolve()
for key, value in (existing_outputs or {}).items()
key: Path(value).resolve() for key, value in (existing_outputs or {}).items()
}
self.continue_from = (
Path(continue_from).resolve() if continue_from is not None else None
)
self.continue_from = Path(continue_from).resolve() if continue_from is not None else None
self.skip_completed = skip_completed
self.output_root = self._build_output_root()
self._input_link_dir = self.output_root / "input"
@@ -63,9 +60,7 @@ class PipelineRunner:
def _build_output_root(self) -> Path:
if self.continue_from is not None:
if not self.continue_from.is_dir():
raise ValidationError(
f"continue_from directory not found: {self.continue_from}"
)
raise ValidationError(f"continue_from directory not found: {self.continue_from}")
return self.continue_from
timestamp = datetime.now().strftime("%y%m%d%H%M%S")
@@ -151,9 +146,23 @@ class PipelineRunner:
step.module.check_dependencies()
validated = step.module.validate_module_params(step.params)
if step.module_name == "xcf_stack":
if not step.input_refs:
raise ValidationError("xcf_stack requires at least one input layer")
source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs]
input_paths = source_lists[0]
matched_groups = [[path] for path in input_paths]
input_layer_dirs: list[tuple[str, Path]] = []
for ref in step.input_refs:
if ref == INPUT_SOURCE:
input_layer_dirs.append(("input", self._input_link_dir))
else:
input_layer_dirs.append((ref, self._results[ref].output_dir))
else:
source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs]
matched_groups = match_by_stem(source_lists)
input_paths = flatten_matched(matched_groups)
input_layer_dirs = []
output_dir = self.output_root / step.output_dir_name
output_dir.mkdir(parents=True, exist_ok=True)
@@ -220,7 +229,8 @@ class PipelineRunner:
pipeline_output_root=self.output_root,
step_id=step.step_id,
matched_groups=matched_groups,
prior_steps=[(sid, res.output_dir) for sid, res in self._results.items()],
input_refs=list(step.input_refs),
input_layer_dirs=input_layer_dirs,
logger=self.logger,
)
@@ -234,8 +244,7 @@ class PipelineRunner:
output_paths = step.module().list_output_images(ctx)
if not output_paths:
raise StepError(
f"Step '{step.output_dir_name}' ({step.module_name}) "
"produced no output images"
f"Step '{step.output_dir_name}' ({step.module_name}) produced no output images"
)
self.logger.step_done(step.output_dir_name, step.output_dir_name, len(output_paths))
@@ -264,9 +273,7 @@ class PipelineRunner:
raise ValidationError(f"Step '{step.step_id}' references unknown step '{dep}'")
dependents[dep].append(step.step_id)
queue = deque(
step_id for step_id, degree in in_degree.items() if degree == 0
)
queue = deque(step_id for step_id, degree in in_degree.items() if degree == 0)
ordered_ids: list[str] = []
while queue:
@@ -280,7 +287,4 @@ class PipelineRunner:
if len(ordered_ids) != len(self.steps):
raise CycleError("Pipeline contains a cycle in step dependencies")
ordered = [step_by_id[step_id] for step_id in ordered_ids]
deferred = [step for step in ordered if step.module.runs_last]
regular = [step for step in ordered if not step.module.runs_last]
return regular + deferred
return [step_by_id[step_id] for step_id in ordered_ids]
+1 -1
View File
@@ -2,7 +2,7 @@ from __future__ import annotations
from dataclasses import dataclass, field
from pathlib import Path
from typing import Any, TYPE_CHECKING
from typing import TYPE_CHECKING, Any
if TYPE_CHECKING:
from imagepipeline.modules.base import BaseModule
+1 -1
View File
@@ -2,8 +2,8 @@
import imagepipeline.modules.ai_exposure # noqa: F401
import imagepipeline.modules.ai_tone_map # noqa: F401
import imagepipeline.modules.comfy_flux_edit # noqa: F401
import imagepipeline.modules.color_to_alpha # noqa: F401
import imagepipeline.modules.comfy_flux_edit # noqa: F401
import imagepipeline.modules.composite # noqa: F401
import imagepipeline.modules.crop_square # noqa: F401
import imagepipeline.modules.darktable_style # noqa: F401
+1 -3
View File
@@ -115,9 +115,7 @@ class AIModule(BaseModule):
avg = sum(elapsed_times) / len(elapsed_times)
eta = avg * remaining if remaining else 0.0
if ctx.logger is not None:
ctx.logger.image_done(
self.name, index, total, dst.name, elapsed, eta_seconds=eta
)
ctx.logger.image_done(self.name, index, total, dst.name, elapsed, eta_seconds=eta)
@staticmethod
def _image_size(path: Path) -> tuple[int, int]:
-1
View File
@@ -14,7 +14,6 @@ class BaseModule(ABC):
name: ClassVar[str]
description: ClassVar[str] = ""
runs_last: ClassVar[bool] = False
supported_input_formats: ClassVar[tuple[str, ...]] = (
".jpg",
".jpeg",
+2 -6
View File
@@ -24,8 +24,7 @@ def build_color_to_alpha_args(*, color: str, fuzz: float) -> list[str]:
class ColorToAlphaModule(SubprocessModule):
name = "color_to_alpha"
description = (
"Make a solid color transparent (GIMP-style color to alpha). "
"Outputs PNG with alpha."
"Make a solid color transparent (GIMP-style color to alpha). Outputs PNG with alpha."
)
command_candidates = ("magick", "convert")
@@ -50,10 +49,7 @@ class ColorToAlphaModule(SubprocessModule):
"fuzz": Param(
"float",
default=0.0,
help=(
"Match tolerance in percent (ImageMagick -fuzz); "
"0 = exact color only"
),
help=("Match tolerance in percent (ImageMagick -fuzz); 0 = exact color only"),
),
}
+3 -9
View File
@@ -76,9 +76,7 @@ class ComfyFluxEditModule(AIModule):
server_url = ctx.params["server_url"].rstrip("/")
self._ensure_server(server_url)
if ctx.logger is not None:
ctx.logger.warning(
"ComfyUI on CPU: expect hours per full-resolution image"
)
ctx.logger.warning("ComfyUI on CPU: expect hours per full-resolution image")
workflow_template = json.loads(workflow_path.read_text(encoding="utf-8"))
denoise = ctx.params["denoise"]
@@ -98,9 +96,7 @@ class ComfyFluxEditModule(AIModule):
seed=seed,
)
prompt_id = self._queue_prompt(server_url, workflow)
output_info = self._wait_for_output(
server_url, prompt_id, poll_interval=poll_interval
)
output_info = self._wait_for_output(server_url, prompt_id, poll_interval=poll_interval)
image_bytes = self._download_view(server_url, output_info)
dst.write_bytes(image_bytes)
@@ -111,9 +107,7 @@ class ComfyFluxEditModule(AIModule):
try:
urllib.request.urlopen(f"{server_url}/system_stats", timeout=5)
except urllib.error.URLError as exc:
raise DependencyError(
f"ComfyUI server not reachable at {server_url}: {exc}"
) from exc
raise DependencyError(f"ComfyUI server not reachable at {server_url}: {exc}") from exc
@staticmethod
def _upload_image(server_url: str, src: Path) -> str:
+10 -5
View File
@@ -35,7 +35,7 @@ class CompositeModule(SubprocessModule):
"mode": Param(
"string",
default="over",
choices=("over", "multiply", "screen"),
choices=("over", "multiply", "screen", "linearburn", "hardmix"),
help="ImageMagick -compose mode",
),
"output_ext": Param(
@@ -55,7 +55,14 @@ class CompositeModule(SubprocessModule):
raise ValueError("composite requires at least one matched input group")
command = self.resolve_command()
mode = ctx.params["mode"]
# ImageMagick accepts these case-insensitively; keep canonical spellings.
mode = {
"over": "Over",
"multiply": "Multiply",
"screen": "Screen",
"linearburn": "LinearBurn",
"hardmix": "HardMix",
}[ctx.params["mode"]]
output_ext = ctx.params["output_ext"]
foreground_opacity = ctx.params["foreground_opacity"]
ctx.output_dir.mkdir(parents=True, exist_ok=True)
@@ -63,9 +70,7 @@ class CompositeModule(SubprocessModule):
for index, group in enumerate(ctx.matched_groups, start=1):
if len(group) < 2:
raise ValueError(
"composite requires at least two input sources per image"
)
raise ValueError("composite requires at least two input sources per image")
background, foreground = group[0], group[1]
self.log_image(ctx, index, total, foreground)
+9 -2
View File
@@ -90,6 +90,9 @@ class DarktableStyleModule(SubprocessModule):
)
self.log_image(ctx, index, total, src)
format_name, conf_options = resolve_export_format(src, out_ext)
# darktable-cli options must come before --core; anything after
# --core is parsed as darktable GUI flags (e.g. --style-overwrite
# after --core makes darktable 5.x print usage and fail).
cmd = [
"darktable-cli",
str(src),
@@ -98,12 +101,16 @@ class DarktableStyleModule(SubprocessModule):
style,
"--out-ext",
format_name,
]
if style_overwrite:
cmd.append("--style-overwrite")
cmd.extend(
[
"--core",
"--configdir",
str(config_dir),
]
)
for conf in conf_options:
cmd.extend(["--conf", conf])
if style_overwrite:
cmd.append("--style-overwrite")
run_command(cmd, timeout=self.default_timeout)
+2 -1
View File
@@ -4,7 +4,7 @@ from imagepipeline.core.context import ModuleContext
from imagepipeline.core.params import Param
from imagepipeline.modules.base import SubprocessModule
from imagepipeline.modules.registry import register
from imagepipeline.utils.gmic import split_gmic_command
from imagepipeline.utils.gmic import finalize_gmic_output, split_gmic_command
from imagepipeline.utils.subprocess import run_command
@@ -35,3 +35,4 @@ class GmicGrayscale(SubprocessModule):
dst = ctx.output_dir / src.name
cmd = ["gmic", str(src), *split_gmic_command(gmic_command), "-output", str(dst)]
run_command(cmd, timeout=self.default_timeout)
finalize_gmic_output(ctx.output_dir, dst)
+1 -3
View File
@@ -50,9 +50,7 @@ def build_fill_arguments(
@register
class ImageMagickFillModule(SubprocessModule):
name = "imagemagick_fill"
description = (
"Create solid-color or gradient images sized to match each input image"
)
description = "Create solid-color or gradient images sized to match each input image"
command_candidates = ("magick", "convert")
@classmethod
@@ -1,7 +1,5 @@
from __future__ import annotations
from pathlib import Path
from imagepipeline.core.context import ModuleContext
from imagepipeline.core.params import Param
from imagepipeline.modules.base import SubprocessModule
@@ -34,8 +32,5 @@ class ImageMagickGrayscale(SubprocessModule):
for index, src in enumerate(ctx.input_paths, start=1):
self.log_image(ctx, index, total, src)
dst = ctx.output_dir / src.name
if command == "magick":
cmd = [command, str(src), "-colorspace", colorspace, str(dst)]
else:
cmd = [command, str(src), "-colorspace", colorspace, str(dst)]
run_command(cmd)
@@ -12,9 +12,7 @@ from imagepipeline.utils.subprocess import run_command
@register
class ImageMagickScaleCropModule(SubprocessModule):
name = "imagemagick_scale_crop"
description = (
"Scale an image then center-crop back to its original dimensions"
)
description = "Scale an image then center-crop back to its original dimensions"
command_candidates = ("magick", "convert")
@classmethod
+6 -18
View File
@@ -70,9 +70,7 @@ class OpenRouterEditModule(AIModule):
@classmethod
def check_dependencies(cls) -> None:
if not os.environ.get("OPENROUTER_API_KEY"):
raise DependencyError(
"OPENROUTER_API_KEY environment variable is not set"
)
raise DependencyError("OPENROUTER_API_KEY environment variable is not set")
def run(self, ctx: ModuleContext) -> None:
api_key_env = ctx.params["api_key_env"]
@@ -155,17 +153,11 @@ class OpenRouterEditModule(AIModule):
*,
template_data_url: str | None = None,
) -> dict:
full_prompt = (
f"{TEMPLATE_PROMPT_PREFIX}{prompt}" if template_data_url else prompt
)
full_prompt = f"{TEMPLATE_PROMPT_PREFIX}{prompt}" if template_data_url else prompt
content: list[dict] = [{"type": "text", "text": full_prompt}]
if template_data_url is not None:
content.append(
{"type": "image_url", "image_url": {"url": template_data_url}}
)
content.append(
{"type": "image_url", "image_url": {"url": source_data_url}}
)
content.append({"type": "image_url", "image_url": {"url": template_data_url}})
content.append({"type": "image_url", "image_url": {"url": source_data_url}})
payload: dict = {
"model": model,
"modalities": cls._modalities_for_model(model),
@@ -176,9 +168,7 @@ class OpenRouterEditModule(AIModule):
return payload
@classmethod
def _save_result_matching_source(
cls, source: Path, result_bytes: bytes, dest: Path
) -> None:
def _save_result_matching_source(cls, source: Path, result_bytes: bytes, dest: Path) -> None:
with Image.open(source) as original:
orig_format = original.format
orig_size = original.size
@@ -229,9 +219,7 @@ class OpenRouterEditModule(AIModule):
return json.loads(response.read().decode("utf-8"))
except urllib.error.HTTPError as exc:
detail = exc.read().decode("utf-8", errors="replace")
raise RuntimeError(
f"OpenRouter API error ({exc.code}): {detail}"
) from exc
raise RuntimeError(f"OpenRouter API error ({exc.code}): {detail}") from exc
@staticmethod
def _extract_image_bytes(response: dict) -> bytes:
+14 -24
View File
@@ -4,19 +4,21 @@ from pathlib import Path
from imagepipeline.core.context import ModuleContext
from imagepipeline.core.params import Param
from imagepipeline.core.step import INPUT_SOURCE
from imagepipeline.modules.base import SubprocessModule
from imagepipeline.modules.registry import register
from imagepipeline.utils.files import find_image_by_stem
from imagepipeline.utils.gimp import stack_images_to_xcf
def _layer_name(ref: str) -> str:
return "input" if ref == INPUT_SOURCE else ref
@register
class XcfStackModule(SubprocessModule):
name = "xcf_stack"
runs_last = True
description = (
"Stack all prior pipeline step outputs as GIMP layers into one XCF per image"
)
description = "Stack listed pipeline step outputs as GIMP layers into one XCF per image"
command_candidates = ("gimp-console", "gimp")
@classmethod
@@ -32,11 +34,6 @@ class XcfStackModule(SubprocessModule):
@classmethod
def parameters(cls) -> dict[str, Param]:
return {
"include_input": Param(
"bool",
default=True,
help="Include pipeline input/ directory as bottom layer",
),
"skip_missing": Param(
"bool",
default=False,
@@ -50,19 +47,13 @@ class XcfStackModule(SubprocessModule):
}
def run(self, ctx: ModuleContext) -> None:
include_input = ctx.params["include_input"]
skip_missing = ctx.params["skip_missing"]
timeout = ctx.params["timeout"]
layer_sources: list[tuple[str, Path]] = []
if include_input:
layer_sources.append(("input", ctx.pipeline_output_root / "input"))
layer_sources.extend(ctx.prior_steps)
if not layer_sources:
if not ctx.input_layer_dirs:
raise ValueError(
"xcf_stack has no layer sources "
"(include_input=False and no prior_steps)"
"xcf_stack requires at least one input layer "
"(pass step refs in inputs=[...], bottom to top)"
)
ctx.output_dir.mkdir(parents=True, exist_ok=True)
@@ -73,22 +64,21 @@ class XcfStackModule(SubprocessModule):
stem = input_path.stem
layers: list[tuple[str, Path]] = []
for step_id, directory in layer_sources:
for ref, directory in ctx.input_layer_dirs:
image_path = find_image_by_stem(directory, stem)
if image_path is None:
if not skip_missing:
raise ValueError(
f"xcf_stack: no image with stem '{stem}' in step "
f"'{step_id}' ({directory})"
f"'{_layer_name(ref)}' ({directory})"
)
continue
layers.append((step_id, image_path))
layers.append((_layer_name(ref), image_path))
if not layers:
checked = ", ".join(step_id for step_id, _ in layer_sources)
checked = ", ".join(name for name, _ in ctx.input_layer_dirs)
raise ValueError(
f"xcf_stack: no layers found for stem '{stem}' "
f"(checked: {checked})"
f"xcf_stack: no layers found for stem '{stem}' (checked: {checked})"
)
dst = ctx.output_dir / f"{stem}.xcf"
+1 -2
View File
@@ -45,8 +45,7 @@ def match_by_stem(sources: list[list[Path]]) -> list[list[Path]]:
key = stem_key(path)
if key in mapping:
raise ValueError(
f"Duplicate stem '{key}' in {path.parent}: "
f"{mapping[key].name} and {path.name}"
f"Duplicate stem '{key}' in {path.parent}: {mapping[key].name} and {path.name}"
)
mapping[key] = path
key_maps.append(mapping)
+3 -8
View File
@@ -91,9 +91,7 @@ def _run_gimp_batch(
stdout=stdout or "",
stderr=stderr or "",
)
raise RuntimeError(
f"Command timed out after {timeout}s: {' '.join(cmd)}"
) from exc
raise RuntimeError(f"Command timed out after {timeout}s: {' '.join(cmd)}") from exc
return subprocess.CompletedProcess(
cmd,
@@ -122,9 +120,7 @@ def stack_images_to_xcf(
for layer_name, image_path in layers:
resolved = image_path.resolve()
if not resolved.is_file():
raise FileNotFoundError(
f"Layer image not found for '{layer_name}': {resolved}"
)
raise FileNotFoundError(f"Layer image not found for '{layer_name}': {resolved}")
resolved_layers.append((layer_name, resolved))
outfile = outfile.resolve()
@@ -159,8 +155,7 @@ def stack_images_to_xcf(
detail = stderr or stdout or f"exit code {result.returncode}"
layer_summary = ", ".join(name for name, _ in resolved_layers)
raise RuntimeError(
f"GIMP failed to stack layers [{layer_summary}] into {outfile}: "
f"{detail}"
f"GIMP failed to stack layers [{layer_summary}] into {outfile}: {detail}"
)
finally:
script_path.unlink(missing_ok=True)
+1 -3
View File
@@ -38,6 +38,4 @@ def finalize_gmic_output(output_dir: Path, intended: Path) -> Path:
frame_000000.rename(intended)
return intended
raise FileNotFoundError(
f"G'MIC produced no output for {intended.name} in {output_dir}"
)
raise FileNotFoundError(f"G'MIC produced no output for {intended.name} in {output_dir}")
+2 -6
View File
@@ -32,13 +32,9 @@ def run_command(
stderr = (exc.stderr or "").strip()
stdout = (exc.stdout or "").strip()
detail = stderr or stdout or str(exc)
raise RuntimeError(
f"Command failed ({exc.returncode}): {' '.join(cmd)}\n{detail}"
) from exc
raise RuntimeError(f"Command failed ({exc.returncode}): {' '.join(cmd)}\n{detail}") from exc
except subprocess.TimeoutExpired as exc:
raise RuntimeError(
f"Command timed out after {timeout}s: {' '.join(cmd)}"
) from exc
raise RuntimeError(f"Command timed out after {timeout}s: {' '.join(cmd)}") from exc
def require_command(*names: str) -> str:
+2 -1
View File
@@ -18,8 +18,9 @@ def main() -> None:
input_dir=INPUT,
output_base=OUTPUT_BASE,
) as p:
gray = p.step("imagemagick_grayscale", inputs="input")
p.step("imagemagick_grayscale", inputs="input")
# Chain another step on the result:
# gray = p.step("imagemagick_grayscale", inputs="input")
# p.step("imagemagick_grayscale", inputs=gray, colorspace="Gray")
output_root = p.run()
+269
View File
@@ -0,0 +1,269 @@
#!/usr/bin/env python3
"""Aichelberg Indians pipeline — tone mapping, gmic variants, xcf export."""
from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path(
"/home/frank/pics/20260712_Aichelberg Indians - Heidelberg Hedgehogs/darktable_exported"
)
OUTPUT_BASE = Path.home() / "pipeline_output"
EXISTING_OUTPUTS: dict[str, Path] = {}
CONTINUE_FROM = Path("/home/frank/pipeline_output/aichelberg-indians_260713081414")
COLOR1 = "#244a89"
COLOR2 = "#c22518"
# COLOR1 = #244a89 -> 36,74,137; COLOR2 = #c22518 -> 194,37,24
ALPHA1 = 160
ALPHA2 = 110
GMIC_TONE_MAPPING = "-fx_map_tones 0.5,0.7,0.1,30,0"
GMIC_FELTPEN = "-fx_feltpen 300,50,1,0.1,20,5"
GMIC_EDGES = "-fx_edges 0,15,0"
GMIC_CHARRED_PLASTIC = "-fx_charred_plastic 1,10,40,1,10,0,0,2,6,5,20,0,11"
GMIC_NEON = (
"-fx_neon 0,1,1,1,1,0,0.45,40,60,0,1,1.15,2,3,0,3,20,0.4,0.1,1.5,5,0.2,0.1,"
"1,2,1,0,0,1,1,0,50,50"
)
GMIC_CUTOUT = "-fx_cutout 4,0.5,4,1"
GMIC_HUFFMAN_GLITCHES = "-fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50"
GMIC_QUERYPRIMARY = "-afre_queryprimary 1,1"
GMIC_LOCAL_SIMILARITY = "-local_similarity_mask 50,50,50,128,4,10"
GMIC_LUMA_INVERT = "-Lylejk_Luma_Invert 1,0"
GMIC_CRAYONGRAFFITI = "-fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0"
GMIC_ANAGLYPH = "-fx_stereo_to_anaglyph 2,0"
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,194,37,24,200,0"
GMIC_BOKEH = (
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,36,74,137,{ALPHA1},0.7,30,20,20,1,2,194,37,24,{ALPHA2},0.15"
)
def main() -> None:
with Pipeline(
name="aichelberg-indians",
input_dir=INPUT,
output_base=OUTPUT_BASE,
existing_outputs=EXISTING_OUTPUTS or None,
continue_from=CONTINUE_FROM,
) as p:
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
# recipe: gmic-tone-mapping
input_tone_map = p.step(
"gmic", inputs="input", command=GMIC_TONE_MAPPING, step_id="input_tone_map"
)
# recipe: gmic-tone-mapping-rembg
rembg_tone_map = p.step(
"gmic", inputs=rembg_out, command=GMIC_TONE_MAPPING, step_id="rembg_tone_map"
)
# recipe: gmic-feltpen-rembg
rembg_feltpen = p.step(
"gmic", inputs=rembg_out, command=GMIC_FELTPEN, step_id="rembg_feltpen"
)
composite_feltpen = p.step(
"composite", inputs=[rembg_feltpen, rembg_out], step_id="composite_feltpen"
)
# recipe: gmic-edges-rembg
rembg_edges = p.step("gmic", inputs=rembg_out, command=GMIC_EDGES, step_id="rembg_edges")
composite_edges = p.step(
"composite", inputs=[rembg_edges, rembg_out], step_id="composite_edges"
)
# recipe: gmic-charred-plastic-rembg
rembg_charred_plastic = p.step(
"gmic",
inputs=rembg_out,
command=GMIC_CHARRED_PLASTIC,
step_id="rembg_charred_plastic",
)
composite_charred_plastic = p.step(
"composite",
inputs=[rembg_charred_plastic, rembg_out],
step_id="composite_charred_plastic",
)
# recipe: gmic-neon-rembg
rembg_neon = p.step("gmic", inputs=rembg_out, command=GMIC_NEON, step_id="rembg_neon")
composite_neon = p.step(
"composite", inputs=[rembg_neon, rembg_out], step_id="composite_neon"
)
# recipe: gmic-cutout-rembg (tone-mapped input as original)
input_cutout = p.step(
"gmic", inputs=input_tone_map, command=GMIC_CUTOUT, step_id="input_cutout"
)
cutout_mid = p.step(
"composite", inputs=[input_tone_map, input_cutout], step_id="cutout_mid"
)
composite_cutout = p.step(
"composite", inputs=[cutout_mid, rembg_out], step_id="composite_cutout"
)
# recipe: gmic-huffman-glitches-rembg
input_huffman_glitches = p.step(
"gmic",
inputs=input_tone_map,
command=GMIC_HUFFMAN_GLITCHES,
step_id="input_huffman_glitches",
)
huffman_glitches_mid = p.step(
"composite",
inputs=[input_tone_map, input_huffman_glitches],
step_id="huffman_glitches_mid",
)
composite_huffman_glitches = p.step(
"composite",
inputs=[huffman_glitches_mid, rembg_out],
step_id="composite_huffman_glitches",
)
# recipe: gmic-queryprimary-rembg
input_queryprimary = p.step(
"gmic",
inputs=input_tone_map,
command=GMIC_QUERYPRIMARY,
step_id="input_queryprimary",
)
queryprimary_mid = p.step(
"composite",
inputs=[input_tone_map, input_queryprimary],
step_id="queryprimary_mid",
)
composite_queryprimary = p.step(
"composite",
inputs=[queryprimary_mid, rembg_out],
step_id="composite_queryprimary",
)
# recipe: gmic-local-similarity-rembg
input_local_similarity = p.step(
"gmic",
inputs=input_tone_map,
command=GMIC_LOCAL_SIMILARITY,
step_id="input_local_similarity",
)
local_similarity_mid = p.step(
"composite",
inputs=[input_tone_map, input_local_similarity],
step_id="local_similarity_mid",
)
composite_local_similarity = p.step(
"composite",
inputs=[local_similarity_mid, rembg_out],
step_id="composite_local_similarity",
)
# recipe: gmic-luma-invert-rembg
input_luma_invert = p.step(
"gmic",
inputs=input_tone_map,
command=GMIC_LUMA_INVERT,
step_id="input_luma_invert",
)
luma_invert_mid = p.step(
"composite",
inputs=[input_tone_map, input_luma_invert],
step_id="luma_invert_mid",
)
composite_luma_invert = p.step(
"composite",
inputs=[luma_invert_mid, rembg_out],
step_id="composite_luma_invert",
)
# recipe: gmic-crayongraffiti-rembg
input_crayongraffiti = p.step(
"gmic",
inputs=input_tone_map,
command=GMIC_CRAYONGRAFFITI,
step_id="input_crayongraffiti",
)
crayongraffiti_mid = p.step(
"composite",
inputs=[input_tone_map, input_crayongraffiti],
step_id="crayongraffiti_mid",
)
composite_crayongraffiti = p.step(
"composite",
inputs=[crayongraffiti_mid, rembg_out],
step_id="composite_crayongraffiti",
)
# recipe: gmic-anaglyph-rembg
input_anaglyph = p.step(
"gmic", inputs=input_tone_map, command=GMIC_ANAGLYPH, step_id="input_anaglyph"
)
anaglyph_mid = p.step(
"composite", inputs=[input_tone_map, input_anaglyph], step_id="anaglyph_mid"
)
composite_anaglyph = p.step(
"composite", inputs=[anaglyph_mid, rembg_out], step_id="composite_anaglyph"
)
# recipe: xcf-stack — layers up to composites (excludes bokeh + color-bg-drop-shadow)
p.step(
"xcf_stack",
inputs=[
"input",
rembg_out,
input_tone_map,
rembg_tone_map,
rembg_feltpen,
rembg_edges,
rembg_charred_plastic,
rembg_neon,
input_cutout,
input_huffman_glitches,
input_queryprimary,
input_local_similarity,
input_luma_invert,
input_crayongraffiti,
input_anaglyph,
composite_feltpen,
composite_edges,
composite_charred_plastic,
composite_neon,
composite_cutout,
composite_huffman_glitches,
composite_queryprimary,
composite_local_similarity,
composite_luma_invert,
composite_crayongraffiti,
composite_anaglyph,
],
)
# recipe: bokeh-oktagon (tone-mapped input as original)
input_bokeh = p.step(
"gmic", inputs=input_tone_map, command=GMIC_BOKEH, step_id="input_bokeh"
)
bokeh_mid = p.step("composite", inputs=[input_tone_map, input_bokeh], step_id="bokeh_mid")
p.step("composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh")
# recipe: color-bg-drop-shadow-rembg
color_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1, step_id="color_bg")
rembg_shadow = p.step(
"gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW, step_id="rembg_shadow"
)
shadow_color_mid = p.step(
"composite", inputs=[color_bg, rembg_shadow], step_id="shadow_color_mid"
)
p.step(
"composite",
inputs=[shadow_color_mid, rembg_out],
step_id="composite_color_bg_shadow",
)
output_root = p.run()
print(f"Pipeline finished. Output: {output_root}")
if __name__ == "__main__":
main()
+2 -6
View File
@@ -40,14 +40,10 @@ def main() -> None:
gmic_shadow = p.step("gmic", inputs=rembg, command=GMIC_DROP_SHADOW)
gmic_smooth = p.step("gmic", inputs=rembg, command=GMIC_JPR_SMOOTH)
gmic_stereo_alpha = p.step(
"color_to_alpha", inputs=gmic_stereo, color="#000000"
)
gmic_stereo_alpha = p.step("color_to_alpha", inputs=gmic_stereo, color="#000000")
yellow_bg = p.step("imagemagick_fill", inputs="input", color1=YELLOW)
gmic_smooth_alpha = p.step(
"color_to_alpha", inputs=gmic_smooth, color="#7f7f7f"
)
gmic_smooth_alpha = p.step("color_to_alpha", inputs=gmic_smooth, color="#7f7f7f")
gmic_smooth_sized = p.step(
"imagemagick_scale_crop",
inputs=gmic_smooth_alpha,
@@ -6,7 +6,9 @@ from pathlib import Path
from imagepipeline import Pipeline
# Darktable export folder.
INPUT = Path("/home/frank/pics/20260517_Albershausen Crusaders - Biberach Beavers/darktable_exported/png")
INPUT = Path(
"/home/frank/pics/20260517_Albershausen Crusaders - Biberach Beavers/darktable_exported/png"
)
# Where timestamped run folders are created.
OUTPUT_BASE = Path.home() / "pipeline_output"
+5 -7
View File
@@ -5,7 +5,9 @@ from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path("/home/frank/pics/20260620_Albershausen Crusaders - Montabaur Fighting Farmers/darktable_exported")
INPUT = Path(
"/home/frank/pics/20260620_Albershausen Crusaders - Montabaur Fighting Farmers/darktable_exported"
)
OUTPUT_BASE = Path.home() / "pipeline_output"
# Reuse outputs from a previous run or external folder (key = step id, e.g. rembg_01).
@@ -80,13 +82,9 @@ def main() -> None:
scale=1.05,
)
rembg_stereo_alpha = p.step(
"color_to_alpha", inputs=rembg_stereo, color="#000000"
)
rembg_stereo_alpha = p.step("color_to_alpha", inputs=rembg_stereo, color="#000000")
color_bg = p.step("imagemagick_fill", inputs="input", color1=COLOR1)
rembg_smooth_alpha = p.step(
"color_to_alpha", inputs=rembg_jpr_smooth, color="#7f7f7f"
)
rembg_smooth_alpha = p.step("color_to_alpha", inputs=rembg_jpr_smooth, color="#7f7f7f")
rembg_smooth_sized = p.step(
"imagemagick_scale_crop",
inputs=rembg_smooth_alpha,
+359
View File
@@ -0,0 +1,359 @@
#!/usr/bin/env python3
"""GoBlue — curated Crusaders looks for sideline / blog edits.
Keeps vortex / ink_press / cyan_punch, pulls proven combines from older
pipelines (aichelberg, crusaders, orange, rezepttest) recolored to club blues,
plus a set of new G'MIC mid-layer looks.
"""
from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path(
"/home/frank/pics/20260719_Biberach Beavers - Albershausen Crusaders/"
"darktable_exported/png"
)
OUTPUT_BASE = Path.home() / "pipeline_output"
EXISTING_OUTPUTS: dict[str, Path] = {}
CONTINUE_FROM: Path | None = None
# Club blues (#GoBlue)
COLOR1 = "#0064b0" # 0,100,176
COLOR2 = "#00badf" # 0,186,223
R1, G1, B1 = 0, 100, 176
R2, G2, B2 = 0, 186, 223
ALPHA1 = 160
ALPHA2 = 110
# --- shared / kept from previous goblue ---
GMIC_TONE_MAPPING = "-fx_map_tones 0.5,0.7,0.1,30,0"
GMIC_ANGULAR = "-fx_blur_angular 2.5,50,50,0,0,7,0"
GMIC_INK_WASH = "-fx_ink_wash 0.14,23,0,0.5,0.54,2.25,0,0,0,0,0,0,0,0,0"
GMIC_DROP_SHADOW = f"-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,{R2},{G2},{B2},200,0"
# --- from aichelberg (on tone-mapped original) ---
GMIC_CUTOUT = "-fx_cutout 4,0.5,4,1"
GMIC_HUFFMAN_GLITCHES = "-fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50"
GMIC_LOCAL_SIMILARITY = "-local_similarity_mask 50,50,50,128,4,10"
GMIC_LUMA_INVERT = "-Lylejk_Luma_Invert 1,0"
GMIC_CRAYONGRAFFITI = "-fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0"
# --- from crusaders (custom gradient B on rembg) ---
GMIC_GRADIENT_B = (
f'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,'
f"{R2},{G2},{B2},255,{R1},{G1},{B1},255,255,255,0,255,255,255,255,255,0,255,"
f"255,255,0,255,0,255,0,0,255,255,128,128,128,255,255,0,255,255,0,0,0,0"
)
# --- from rezepttest bokeh, crusaders colors ---
GMIC_BOKEH = (
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,{R1},{G1},{B1},{ALPHA1},0.7,30,20,20,1,2,"
f"{R2},{G2},{B2},{ALPHA2},0.15"
)
# --- new mid-layer filters ---
GMIC_LOOSE_PHOTOS = (
"-fx_loose_photos 60,40,50,100,50,360,0,2,255,255,255,50,25,0,0,0,0,0,0,50,1,1,1"
)
GMIC_BLUR_LINEAR = "-fx_blur_linear 10,0.5,0,0,2,7,0"
GMIC_TUNNEL = "-fx_tunnel 4,80,50,50,0.2,0"
GMIC_BARBOUILLAGE = "-samj_Barbouillage_Paint_Daub 2,2,100,0.2,1,4,1,0,8"
# Multi Thresholds with club-blue palette (dark → COLOR1 → COLOR2 → light → near-white)
GMIC_MULTI_THRESHOLD = (
f"-tran_multi_threshold 50,100,150,200,"
f"0,20,40,{R1},{G1},{B1},{R2},{G2},{B2},126,200,227,232,244,252"
)
GMIC_OLDSCHOOL_8BITS = "-fx_8bits 25,800,16"
def main() -> None:
with Pipeline(
name="crusaders-goblue",
input_dir=INPUT,
output_base=OUTPUT_BASE,
existing_outputs=EXISTING_OUTPUTS or None,
continue_from=CONTINUE_FROM,
) as p:
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
tone = p.step("gmic", inputs="input", command=GMIC_TONE_MAPPING, step_id="tone")
# ========== kept from previous goblue ==========
angular = p.step(
"gmic", inputs="input", command=GMIC_ANGULAR, step_id="angular"
)
vortex = p.step("composite", inputs=[angular, rembg_out], step_id="vortex")
ink = p.step("gmic", inputs=tone, command=GMIC_INK_WASH, step_id="ink")
ink_press = p.step("composite", inputs=[ink, rembg_out], step_id="ink_press")
radial_bg = p.step(
"imagemagick_fill",
inputs="input",
color1=COLOR1,
color2=COLOR2,
gradient=True,
radial=True,
step_id="radial_bg",
)
rembg_shadow = p.step(
"gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW, step_id="rembg_shadow"
)
punch_mid = p.step(
"composite", inputs=[radial_bg, rembg_shadow], step_id="punch_mid"
)
cyan_punch = p.step(
"composite", inputs=[punch_mid, rembg_out], step_id="cyan_punch"
)
# ========== from aichelberg (tone → filter → mid → rembg) ==========
input_cutout = p.step(
"gmic", inputs=tone, command=GMIC_CUTOUT, step_id="input_cutout"
)
cutout_mid = p.step(
"composite", inputs=[tone, input_cutout], step_id="cutout_mid"
)
composite_cutout = p.step(
"composite", inputs=[cutout_mid, rembg_out], step_id="composite_cutout"
)
input_huffman = p.step(
"gmic",
inputs=tone,
command=GMIC_HUFFMAN_GLITCHES,
step_id="input_huffman_glitches",
)
huffman_mid = p.step(
"composite",
inputs=[tone, input_huffman],
step_id="huffman_glitches_mid",
)
composite_huffman_glitches = p.step(
"composite",
inputs=[huffman_mid, rembg_out],
step_id="composite_huffman_glitches",
)
input_local_sim = p.step(
"gmic",
inputs=tone,
command=GMIC_LOCAL_SIMILARITY,
step_id="input_local_similarity",
)
local_sim_mid = p.step(
"composite",
inputs=[tone, input_local_sim],
step_id="local_similarity_mid",
)
composite_local_similarity = p.step(
"composite",
inputs=[local_sim_mid, rembg_out],
step_id="composite_local_similarity",
)
input_luma_invert = p.step(
"gmic", inputs=tone, command=GMIC_LUMA_INVERT, step_id="input_luma_invert"
)
luma_invert_mid = p.step(
"composite",
inputs=[tone, input_luma_invert],
step_id="luma_invert_mid",
)
composite_luma_invert = p.step(
"composite",
inputs=[luma_invert_mid, rembg_out],
step_id="composite_luma_invert",
)
input_crayon = p.step(
"gmic",
inputs=tone,
command=GMIC_CRAYONGRAFFITI,
step_id="input_crayongraffiti",
)
crayon_mid = p.step(
"composite",
inputs=[tone, input_crayon],
step_id="crayongraffiti_mid",
)
composite_crayongraffiti = p.step(
"composite",
inputs=[crayon_mid, rembg_out],
step_id="composite_crayongraffiti",
)
# ========== from crusaders ==========
# composite_04: radial club-blue gradient + rembg
composite_04 = p.step(
"composite", inputs=[radial_bg, rembg_out], step_id="composite_04"
)
# composite_12: rembg custom gradient B + rembg
rembg_gradient_b = p.step(
"gmic", inputs=rembg_out, command=GMIC_GRADIENT_B, step_id="rembg_gradient_b"
)
composite_12 = p.step(
"composite", inputs=[rembg_gradient_b, rembg_out], step_id="composite_12"
)
# ========== from orange (composite_02: original + drop shadow + rembg) ==========
shadow_mid = p.step(
"composite", inputs=["input", rembg_shadow], step_id="shadow_mid"
)
composite_02 = p.step(
"composite", inputs=[shadow_mid, rembg_out], step_id="composite_02"
)
# ========== from rezepttest (composite_bokeh, club blues) ==========
input_bokeh = p.step(
"gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh"
)
bokeh_mid = p.step(
"composite", inputs=["input", input_bokeh], step_id="bokeh_mid"
)
composite_bokeh = p.step(
"composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh"
)
# ========== new compositions ==========
# original / loose photos (linearburn 50%) / rembg
loose = p.step(
"gmic", inputs="input", command=GMIC_LOOSE_PHOTOS, step_id="loose_photos"
)
loose_mid = p.step(
"composite",
inputs=["input", loose],
mode="linearburn",
foreground_opacity=0.5,
step_id="loose_photos_mid",
)
composite_loose_photos = p.step(
"composite",
inputs=[loose_mid, rembg_out],
step_id="composite_loose_photos",
)
# original with blur linear / rembg
blur_linear = p.step(
"gmic", inputs="input", command=GMIC_BLUR_LINEAR, step_id="blur_linear"
)
composite_blur_linear = p.step(
"composite",
inputs=[blur_linear, rembg_out],
step_id="composite_blur_linear",
)
# original with tunnel / rembg
tunnel = p.step("gmic", inputs="input", command=GMIC_TUNNEL, step_id="tunnel")
composite_tunnel = p.step(
"composite", inputs=[tunnel, rembg_out], step_id="composite_tunnel"
)
# original with barbouillage / rembg
barbouillage = p.step(
"gmic",
inputs="input",
command=GMIC_BARBOUILLAGE,
step_id="barbouillage",
)
composite_barbouillage = p.step(
"composite",
inputs=[barbouillage, rembg_out],
step_id="composite_barbouillage",
)
# original / luma invert (multiply 50%) / rembg
luma_on_input = p.step(
"gmic",
inputs="input",
command=GMIC_LUMA_INVERT,
step_id="luma_invert_on_input",
)
luma_mul_mid = p.step(
"composite",
inputs=["input", luma_on_input],
mode="multiply",
foreground_opacity=0.5,
step_id="luma_invert_multiply_mid",
)
composite_luma_invert_multiply = p.step(
"composite",
inputs=[luma_mul_mid, rembg_out],
step_id="composite_luma_invert_multiply",
)
# original / multi thresholds (hardmix 50%) / rembg
multi_threshold = p.step(
"gmic",
inputs="input",
command=GMIC_MULTI_THRESHOLD,
step_id="multi_threshold",
)
multi_mid = p.step(
"composite",
inputs=["input", multi_threshold],
mode="hardmix",
foreground_opacity=0.5,
step_id="multi_threshold_mid",
)
composite_multi_threshold = p.step(
"composite",
inputs=[multi_mid, rembg_out],
step_id="composite_multi_threshold",
)
# original / oldschool 8bits (screen) / rembg
oldschool = p.step(
"gmic",
inputs="input",
command=GMIC_OLDSCHOOL_8BITS,
step_id="oldschool_8bits",
)
oldschool_mid = p.step(
"composite",
inputs=["input", oldschool],
mode="screen",
step_id="oldschool_8bits_mid",
)
composite_oldschool_8bits = p.step(
"composite",
inputs=[oldschool_mid, rembg_out],
step_id="composite_oldschool_8bits",
)
p.step(
"xcf_stack",
inputs=[
"input",
rembg_out,
vortex,
ink_press,
cyan_punch,
composite_cutout,
composite_huffman_glitches,
composite_local_similarity,
composite_luma_invert,
composite_crayongraffiti,
composite_04,
composite_12,
composite_02,
composite_bokeh,
composite_loose_photos,
composite_blur_linear,
composite_tunnel,
composite_barbouillage,
composite_luma_invert_multiply,
composite_multi_threshold,
composite_oldschool_8bits,
],
step_id="xcf_looks",
)
output_root = p.run()
print(f"Pipeline finished. Output: {output_root}")
if __name__ == "__main__":
main()
+13 -12
View File
@@ -5,7 +5,7 @@ from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path("/home/frank/pics/20260704_Hellraisers und The Shape Schlossplatz/darktable_exported")
INPUT = Path("/home/frank/pics/20260726_Hellraisers Schlossplatz die Zweite/darktable_exported/png")
OUTPUT_BASE = Path.home() / "pipeline_output"
# Reuse outputs from a previous run or external folder (key = step id, e.g. rembg_01).
@@ -14,27 +14,28 @@ EXISTING_OUTPUTS: dict[str, Path] = {}
# Resume an aborted run: point to its output root folder (or None for a fresh run).
CONTINUE_FROM: Path | None = None
COLOR1 = "#AA4E00"
COLOR2 = "#EDDD93"
COLOR1 = "#732f74"
COLOR2 = "#552577"
# COLOR1 = #AA4E00 -> 170,78,0; COLOR2 = #EDDD93 -> 237,221,147
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,237,221,147,200,0"
# COLOR1 = #732f74 -> 45,18,45; COLOR2 = #552577 -> 33,14,46
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,33,14,46,200,0"
GMIC_BWRECOLOR = (
"-fx_bwrecolorize 0,0,0,0,0,1,0,2,237,221,147,255,170,78,0,255,"
"158,137,189,255,224,191,228,255,170,78,0,255,255,255,255,255,255,255,"
"255,255,170,78,0,255"
"-fx_bwrecolorize 0,0,0,0,0,1,0,2,33,14,46,255,45,18,45,255,"
"158,137,189,255,224,191,228,255,45,18,45,255,255,255,255,255,255,255,"
"255,255,45,18,45,255"
)
GMIC_GRADIENT_A = (
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,170,78,0,255,'
"237,221,147,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,45,18,45,255,'
"33,14,46,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
)
GMIC_GRADIENT_B = (
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,237,221,147,255,'
"170,78,0,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
'-fx_custom_gradient 0,0,0,1,2,1,0,128,100,100,2,0,1,0,"",1,0,33,14,46,255,'
"45,18,45,255,255,255,0,255,255,255,255,255,0,255,255,255,0,255,0,255,0,0,"
"255,255,128,128,128,255,255,0,255,255,0,0,0,0"
)
def main() -> None:
with Pipeline(
name="orange",
+118
View File
@@ -0,0 +1,118 @@
#!/usr/bin/env python3
"""Rezepttest pipeline — RECIPES.md smoke test (Indians Aichelberg logo colors)."""
from pathlib import Path
from imagepipeline import Pipeline
# Change to your Darktable export folder.
INPUT = Path("/home/frank/tmp/bar")
OUTPUT_BASE = Path.home() / "pipeline_output"
EXISTING_OUTPUTS: dict[str, Path] = {}
CONTINUE_FROM: Path | None = None
# Colors from Indians Aichelberg logo (orange text + red stitching).
COLOR1 = "#f0b020" # 240, 176, 32
COLOR2 = "#f01000" # 240, 16, 0
ALPHA1 = 160
ALPHA2 = 110
GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,240,16,0,200,0"
GMIC_JPR_SMOOTH = "-jpr_gradient_smooth 0,1.5"
GMIC_BOKEH = (
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,240,176,32,{ALPHA1},0.7,30,20,20,1,2,240,16,0,{ALPHA2},0.15"
)
def main() -> None:
with Pipeline(
name="rezepttest",
input_dir=INPUT,
output_base=OUTPUT_BASE,
existing_outputs=EXISTING_OUTPUTS or None,
continue_from=CONTINUE_FROM,
) as p:
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
grayscale = p.step("gmic_grayscale", inputs="input", step_id="grayscale")
gradient_radial_bg = p.step(
"imagemagick_fill",
inputs="input",
color1=COLOR1,
color2=COLOR2,
gradient=True,
radial=True,
step_id="gradient_radial_bg",
)
rembg_shadow = p.step(
"gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW, step_id="rembg_shadow"
)
rembg_jpr_smooth = p.step(
"gmic", inputs=rembg_out, command=GMIC_JPR_SMOOTH, step_id="rembg_jpr_smooth"
)
rembg_jpr_smooth_sized = p.step(
"imagemagick_scale_crop",
inputs=rembg_jpr_smooth,
scale=1.05,
step_id="rembg_jpr_smooth_sized",
)
input_bokeh = p.step("gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh")
# recipe: colorsplash
composite_colorsplash = p.step(
"composite", inputs=[grayscale, rembg_out], step_id="composite_colorsplash"
)
# recipe: rembg-radial-2colors (gradient-radial)
composite_radial = p.step(
"composite", inputs=[gradient_radial_bg, rembg_out], step_id="composite_radial"
)
# recipe: original-drop-shadow-rembg
shadow_mid = p.step("composite", inputs=["input", rembg_shadow], step_id="shadow_mid")
composite_shadow = p.step(
"composite", inputs=[shadow_mid, rembg_out], step_id="composite_shadow"
)
# recipe: original-jpr-smooth-rembg
smooth_mid = p.step(
"composite", inputs=["input", rembg_jpr_smooth_sized], step_id="smooth_mid"
)
composite_smooth = p.step(
"composite", inputs=[smooth_mid, rembg_out], step_id="composite_smooth"
)
# recipe: bokeh-oktagon
bokeh_mid = p.step("composite", inputs=["input", input_bokeh], step_id="bokeh_mid")
composite_bokeh = p.step(
"composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh"
)
# recipe: xcf-stack — explicit layer list, bottom to top
p.step(
"xcf_stack",
inputs=[
"input",
rembg_out,
grayscale,
gradient_radial_bg,
rembg_shadow,
rembg_jpr_smooth_sized,
input_bokeh,
composite_colorsplash,
composite_radial,
composite_shadow,
composite_smooth,
composite_bokeh,
],
)
output_root = p.run()
print(f"Pipeline finished. Output: {output_root}")
if __name__ == "__main__":
main()
+294
View File
@@ -0,0 +1,294 @@
#!/usr/bin/env python3
"""Sports — generic red/yellow looks for sideline / blog edits.
Keeps proven composites from crusaders-goblue (recolored), plus four new
BG/FG G'MIC blend recipes.
"""
from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path(
"/home/frank/pics/20260726_Bikepark Winnenden/darktable_exported/png"
)
OUTPUT_BASE = Path.home() / "pipeline_output"
EXISTING_OUTPUTS: dict[str, Path] = {}
CONTINUE_FROM: Path | None = Path(
"/home/frank/pipeline_output/sports_260726190157"
)
# Standard sports red / yellow
COLOR1 = "#e30613" # 227,6,19
COLOR2 = "#ffcc00" # 255,204,0
R1, G1, B1 = 227, 6, 19
R2, G2, B2 = 255, 204, 0
ALPHA1 = 160
ALPHA2 = 110
# --- shared / kept from goblue ---
GMIC_TONE_MAPPING = "-fx_map_tones 0.5,0.7,0.1,30,0"
GMIC_ANGULAR = "-fx_blur_angular 2.5,50,50,0,0,7,0"
GMIC_INK_WASH = "-fx_ink_wash 0.14,23,0,0.5,0.54,2.25,0,0,0,0,0,0,0,0,0"
GMIC_CUTOUT = "-fx_cutout 4,0.5,4,1"
GMIC_HUFFMAN_GLITCHES = "-fx_huffman_glitches 30,0,25,0,0,0,0,0,50,50"
GMIC_LOCAL_SIMILARITY = "-local_similarity_mask 50,50,50,128,4,10"
GMIC_LUMA_INVERT = "-Lylejk_Luma_Invert 1,0"
GMIC_CRAYONGRAFFITI = "-fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0"
GMIC_BOKEH = (
f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,{R1},{G1},{B1},{ALPHA1},0.7,30,20,20,1,2,"
f"{R2},{G2},{B2},{ALPHA2},0.15"
)
GMIC_LOOSE_PHOTOS = (
"-fx_loose_photos 60,40,50,100,50,360,0,2,255,255,255,50,25,0,0,0,0,0,0,50,1,1,1"
)
GMIC_BLUR_LINEAR = "-fx_blur_linear 10,0.5,0,0,2,7,0"
GMIC_TUNNEL = "-fx_tunnel 4,80,50,50,0.2,0"
# --- new BG/FG recipes (filter on copy → G'MIC blend → next layer) ---
# Rainbowify + lchlightness / Autofill Coloring Book + interpolation
GMIC_RAINBOWIFY_AUTOFILL = (
"+fx_rep_rainbowify 0,0,100 blend lchlightness "
"+gui_rep_acb 180,0,1,0,250000 blend interpolation"
)
# Hard Sketch + burn / Jpr Line Edges + add
GMIC_HARDSKETCH_LINE_EDGES = (
"+fx_hardsketchbw 300,50,1,0.1,20,0,4 blend burn "
"+jpr_line_edges 2,2,4,1 blend add"
)
# Blur [Linear] + shapeaverage / Charred Plastic + blue
GMIC_BLUR_CHARRED = (
"+fx_blur_linear 10,0.5,0,0,2,7,0 blend shapeaverage "
"+fx_charred_plastic 1,10,40,1,10,0,0,2,6,5,20,0,11 blend blue"
)
# Colored Pencils + hue / Black Crayon Graffiti + green
GMIC_PENCILS_CRAYON = (
"+fx_cpencil 1.3,50,20,2,2,1 blend hue "
"+fx_crayongraffiti2 300,50,1,0.4,12,1,2,2,0 blend green"
)
def main() -> None:
with Pipeline(
name="sports",
input_dir=INPUT,
output_base=OUTPUT_BASE,
existing_outputs=EXISTING_OUTPUTS or None,
continue_from=CONTINUE_FROM,
) as p:
rembg_out = p.step("rembg", inputs="input", step_id="rembg_out")
tone = p.step("gmic", inputs="input", command=GMIC_TONE_MAPPING, step_id="tone")
# ========== kept from goblue ==========
angular = p.step(
"gmic", inputs="input", command=GMIC_ANGULAR, step_id="angular"
)
vortex = p.step("composite", inputs=[angular, rembg_out], step_id="vortex")
ink = p.step("gmic", inputs=tone, command=GMIC_INK_WASH, step_id="ink")
ink_press = p.step("composite", inputs=[ink, rembg_out], step_id="ink_press")
input_cutout = p.step(
"gmic", inputs=tone, command=GMIC_CUTOUT, step_id="input_cutout"
)
cutout_mid = p.step(
"composite", inputs=[tone, input_cutout], step_id="cutout_mid"
)
composite_cutout = p.step(
"composite", inputs=[cutout_mid, rembg_out], step_id="composite_cutout"
)
input_huffman = p.step(
"gmic",
inputs=tone,
command=GMIC_HUFFMAN_GLITCHES,
step_id="input_huffman_glitches",
)
huffman_mid = p.step(
"composite",
inputs=[tone, input_huffman],
step_id="huffman_glitches_mid",
)
composite_huffman_glitches = p.step(
"composite",
inputs=[huffman_mid, rembg_out],
step_id="composite_huffman_glitches",
)
input_local_sim = p.step(
"gmic",
inputs=tone,
command=GMIC_LOCAL_SIMILARITY,
step_id="input_local_similarity",
)
local_sim_mid = p.step(
"composite",
inputs=[tone, input_local_sim],
step_id="local_similarity_mid",
)
composite_local_similarity = p.step(
"composite",
inputs=[local_sim_mid, rembg_out],
step_id="composite_local_similarity",
)
input_crayon = p.step(
"gmic",
inputs=tone,
command=GMIC_CRAYONGRAFFITI,
step_id="input_crayongraffiti",
)
crayon_mid = p.step(
"composite",
inputs=[tone, input_crayon],
step_id="crayongraffiti_mid",
)
composite_crayongraffiti = p.step(
"composite",
inputs=[crayon_mid, rembg_out],
step_id="composite_crayongraffiti",
)
input_bokeh = p.step(
"gmic", inputs="input", command=GMIC_BOKEH, step_id="input_bokeh"
)
bokeh_mid = p.step(
"composite", inputs=["input", input_bokeh], step_id="bokeh_mid"
)
composite_bokeh = p.step(
"composite", inputs=[bokeh_mid, rembg_out], step_id="composite_bokeh"
)
loose = p.step(
"gmic", inputs="input", command=GMIC_LOOSE_PHOTOS, step_id="loose_photos"
)
loose_mid = p.step(
"composite",
inputs=["input", loose],
mode="linearburn",
foreground_opacity=0.5,
step_id="loose_photos_mid",
)
composite_loose_photos = p.step(
"composite",
inputs=[loose_mid, rembg_out],
step_id="composite_loose_photos",
)
blur_linear = p.step(
"gmic", inputs="input", command=GMIC_BLUR_LINEAR, step_id="blur_linear"
)
composite_blur_linear = p.step(
"composite",
inputs=[blur_linear, rembg_out],
step_id="composite_blur_linear",
)
tunnel = p.step("gmic", inputs="input", command=GMIC_TUNNEL, step_id="tunnel")
composite_tunnel = p.step(
"composite", inputs=[tunnel, rembg_out], step_id="composite_tunnel"
)
luma_on_input = p.step(
"gmic",
inputs="input",
command=GMIC_LUMA_INVERT,
step_id="luma_invert_on_input",
)
luma_mul_mid = p.step(
"composite",
inputs=["input", luma_on_input],
mode="multiply",
foreground_opacity=0.5,
step_id="luma_invert_multiply_mid",
)
composite_luma_invert_multiply = p.step(
"composite",
inputs=[luma_mul_mid, rembg_out],
step_id="composite_luma_invert_multiply",
)
# ========== new BG/FG recipes ==========
rainbowify_autofill = p.step(
"gmic",
inputs="input",
command=GMIC_RAINBOWIFY_AUTOFILL,
step_id="rainbowify_autofill",
)
composite_rainbowify_autofill = p.step(
"composite",
inputs=[rainbowify_autofill, rembg_out],
step_id="composite_rainbowify_autofill",
)
hardsketch_line_edges = p.step(
"gmic",
inputs="input",
command=GMIC_HARDSKETCH_LINE_EDGES,
step_id="hardsketch_line_edges",
)
composite_hardsketch_line_edges = p.step(
"composite",
inputs=[hardsketch_line_edges, rembg_out],
step_id="composite_hardsketch_line_edges",
)
blur_charred = p.step(
"gmic",
inputs="input",
command=GMIC_BLUR_CHARRED,
step_id="blur_charred",
)
composite_blur_charred = p.step(
"composite",
inputs=[blur_charred, rembg_out],
step_id="composite_blur_charred",
)
pencils_crayon = p.step(
"gmic",
inputs="input",
command=GMIC_PENCILS_CRAYON,
step_id="pencils_crayon",
)
composite_pencils_crayon = p.step(
"composite",
inputs=[pencils_crayon, rembg_out],
step_id="composite_pencils_crayon",
)
p.step(
"xcf_stack",
inputs=[
"input",
rembg_out,
vortex,
ink_press,
composite_cutout,
composite_huffman_glitches,
composite_local_similarity,
composite_crayongraffiti,
composite_bokeh,
composite_loose_photos,
composite_blur_linear,
composite_tunnel,
composite_luma_invert_multiply,
composite_rainbowify_autofill,
composite_hardsketch_line_edges,
composite_blur_charred,
composite_pencils_crayon,
],
step_id="xcf_looks",
)
output_root = p.run()
print(f"Pipeline finished. Output: {output_root}")
if __name__ == "__main__":
main()
@@ -0,0 +1,31 @@
#!/usr/bin/env python3
"""Resize to 2000px max edge, then colorsplash + F12 watermark darktable style."""
from pathlib import Path
from imagepipeline import Pipeline
INPUT = Path(
"/home/frank/pics/20260726_Hellraisers Schlossplatz die Zweite/darktable_exported/gimp"
)
OUTPUT_BASE = Path.home() / "pipeline_output"
MAX_EDGE = 2000
STYLE = "Watermark F12.rocks"
def main() -> None:
with Pipeline(
name="watermark_f12_2000px",
input_dir=INPUT,
output_base=OUTPUT_BASE,
) as p:
resized = p.step("imagemagick_resize", inputs="input", max_edge=MAX_EDGE)
p.step("darktable_style", inputs=resized, style=STYLE, out_ext="jpg")
output_root = p.run()
print(f"Pipeline finished. Output: {output_root}")
if __name__ == "__main__":
main()
+34 -3
View File
@@ -10,11 +10,17 @@ readme = "README.md"
requires-python = ">=3.11"
license = { text = "MIT" }
authors = [{ name = "Frank" }]
dependencies = []
dependencies = [
"Pillow>=10.0",
]
[project.optional-dependencies]
ai = ["numpy>=1.26", "Pillow>=10.0", "torch>=2.0"]
dev = ["pytest>=8.0"]
ai = ["numpy>=1.26", "torch>=2.0"]
dev = [
"pytest>=8.0",
"numpy>=1.26",
"ruff>=0.8",
]
[project.scripts]
imagepipeline = "imagepipeline.cli:main"
@@ -25,3 +31,28 @@ include = ["imagepipeline*"]
[tool.pytest.ini_options]
testpaths = ["tests"]
markers = [
"integration: tests that call external CLI tools (ImageMagick, G'MIC, rembg, darktable, GIMP)",
"slow: tests that take more than a few seconds",
]
[tool.ruff]
target-version = "py311"
line-length = 100
src = ["imagepipeline", "tests", "pipelines"]
[tool.ruff.lint]
select = [
"E", # pycodestyle errors
"F", # pyflakes
"I", # isort
"UP", # pyupgrade
"B", # flake8-bugbear
]
ignore = [
"E501", # line length handled by formatter
"B027", # optional empty hooks on BaseModule
]
[tool.ruff.lint.isort]
known-first-party = ["imagepipeline"]
+2 -5
View File
@@ -20,10 +20,7 @@ def make_png(
crc = zlib.crc32(tag + data) & 0xFFFFFFFF
return struct.pack(">I", len(data)) + tag + data + struct.pack(">I", crc)
raw = b"".join(
b"\x00" + bytes([r, g, b] * width)
for _ in range(height)
)
raw = b"".join(b"\x00" + bytes([r, g, b] * width) for _ in range(height))
compressed = zlib.compress(raw, 9)
ihdr = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0)
png = (
@@ -37,7 +34,7 @@ def make_png(
@pytest.fixture(autouse=True)
def _ensure_builtin_modules() -> None:
import imagepipeline.modules # noqa: F401
pass # noqa: F401
@pytest.fixture
+53
View File
@@ -0,0 +1,53 @@
"""Shared helpers for integration tests."""
from __future__ import annotations
import shutil
import subprocess
import tempfile
from pathlib import Path
from tests.conftest import make_png
def has_darktable_style(style_name: str, config_dir: Path | None = None) -> bool:
"""Return True if darktable-cli can apply ``style_name`` to a tiny PNG."""
if not shutil.which("darktable-cli"):
return False
config = config_dir or Path.home() / ".config" / "darktable"
styles_dir = config / "styles"
if not styles_dir.is_dir():
return False
if not any(p.stem == style_name for p in styles_dir.glob("*.dtstyle")):
return False
with tempfile.TemporaryDirectory() as tmp:
tmp_path = Path(tmp)
src = tmp_path / "probe.png"
out_dir = tmp_path / "out"
out_dir.mkdir()
make_png(src)
cmd = [
"darktable-cli",
str(src),
str(out_dir),
"--style",
style_name,
"--out-ext",
"png",
"--style-overwrite",
"--core",
"--configdir",
str(config),
]
result = subprocess.run(cmd, capture_output=True, text=True)
if result.returncode != 0:
return False
return any(out_dir.iterdir())
def has_workflow_tools() -> bool:
"""True when rembg, gmic, ImageMagick, and darktable-cli are on PATH."""
has_magick = bool(shutil.which("magick") or shutil.which("convert"))
return all(shutil.which(name) for name in ("rembg", "gmic", "darktable-cli")) and has_magick
+1 -3
View File
@@ -110,9 +110,7 @@ class TestAIParameters:
assert payload["modalities"] == ["image"]
assert payload["image_config"] == {"strength": 0.25}
def test_save_result_matching_source_preserves_png_size(
self, tmp_path: Path
) -> None:
def test_save_result_matching_source_preserves_png_size(self, tmp_path: Path) -> None:
try:
from PIL import Image
except ImportError:
+75 -3
View File
@@ -113,9 +113,11 @@ class TestPipelineRunner:
for src in ctx.input_paths:
shutil.copy2(src, ctx.output_dir / src.name)
with Pipeline(name="order_test", input_dir=input_dir, output_base=output_base, verbose=False) as p:
with Pipeline(
name="order_test", input_dir=input_dir, output_base=output_base, verbose=False
) as p:
step_b = p.step("order_tracker", inputs="input")
step_a = p.step("order_tracker", inputs=step_b)
p.step("order_tracker", inputs=step_b)
p.run()
assert order == ["order_tracker_01", "order_tracker_02"]
@@ -132,7 +134,9 @@ class TestPipelineRunner:
for src in ctx.input_paths:
shutil.copy2(src, ctx.output_dir / src.name)
with Pipeline(name="dup_test", input_dir=input_dir, output_base=output_base, verbose=False) as p:
with Pipeline(
name="dup_test", input_dir=input_dir, output_base=output_base, verbose=False
) as p:
first = p.step("number_tracker", inputs="input")
p.step("number_tracker", inputs=first)
root = p.run()
@@ -159,6 +163,74 @@ class TestManifest:
assert data["finished_at"] is not None
class TestCustomStepId:
def test_custom_step_id_output_folder(self, input_dir: Path, output_base: Path) -> None:
@register
class NamedModule(BaseModule):
name = "named_tracker"
def run(self, ctx) -> None:
ctx.output_dir.mkdir(parents=True, exist_ok=True)
for src in ctx.input_paths:
shutil.copy2(src, ctx.output_dir / src.name)
with Pipeline(
name="named_test", input_dir=input_dir, output_base=output_base, verbose=False
) as p:
ref = p.step("named_tracker", inputs="input", step_id="input_bokeh")
root = p.run()
assert ref.step_id == "input_bokeh"
assert ref.output_dir_name == "input_bokeh"
assert (root / "input_bokeh").is_dir()
assert not (root / "named_tracker_01").exists()
unregister("named_tracker")
def test_module_counter_independent_of_custom_step_id(
self, input_dir: Path, output_base: Path
) -> None:
@register
class CounterModule(BaseModule):
name = "counter_tracker"
def run(self, ctx) -> None:
ctx.output_dir.mkdir(parents=True, exist_ok=True)
for src in ctx.input_paths:
shutil.copy2(src, ctx.output_dir / src.name)
with Pipeline(
name="counter_test", input_dir=input_dir, output_base=output_base, verbose=False
) as p:
p.step("counter_tracker", inputs="input")
p.step("counter_tracker", inputs="input", step_id="custom_mid")
p.step("counter_tracker", inputs="input")
root = p.run()
assert (root / "counter_tracker_01").is_dir()
assert (root / "custom_mid").is_dir()
assert (root / "counter_tracker_03").is_dir()
assert not (root / "counter_tracker_02").exists()
unregister("counter_tracker")
def test_duplicate_step_id_rejected(self, input_dir: Path) -> None:
with Pipeline(name="dup_id", input_dir=input_dir, verbose=False) as p:
p.step("imagemagick_grayscale", inputs="input", step_id="my_step")
with pytest.raises(ValidationError, match="Duplicate step_id"):
p.step("imagemagick_grayscale", inputs="input", step_id="my_step")
def test_empty_step_id_rejected(self, input_dir: Path) -> None:
with Pipeline(name="empty_id", input_dir=input_dir, verbose=False) as p:
with pytest.raises(ValidationError, match="must not be empty"):
p.step("imagemagick_grayscale", inputs="input", step_id="")
def test_step_id_with_slash_rejected(self, input_dir: Path) -> None:
with Pipeline(name="slash_id", input_dir=input_dir, verbose=False) as p:
with pytest.raises(ValidationError, match="path separators"):
p.step("imagemagick_grayscale", inputs="input", step_id="bad/name")
class TestPipelineValidation:
def test_empty_pipeline_rejected(self, input_dir: Path) -> None:
with Pipeline(name="empty", input_dir=input_dir, verbose=False) as p:
+29 -2
View File
@@ -13,7 +13,6 @@ from imagepipeline.core.resume import (
)
from imagepipeline.core.step import StepDefinition
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
from imagepipeline.modules.registry import get_module
from imagepipeline.modules.rembg import RembgModule
from imagepipeline.utils.gmic import finalize_gmic_output, split_gmic_command
from tests.conftest import make_png
@@ -124,7 +123,9 @@ class TestExpectedOutputFilenames:
class TestPipelineResume:
@pytest.mark.skipif(not shutil.which("magick"), reason="ImageMagick not installed")
def test_continue_skips_completed_steps(self, input_dir: Path, output_base: Path, capsys) -> None:
def test_continue_skips_completed_steps(
self, input_dir: Path, output_base: Path, capsys
) -> None:
with Pipeline(
name="resume_test",
input_dir=input_dir,
@@ -178,6 +179,32 @@ class TestPipelineResume:
assert "Reused external output for imagemagick_grayscale_01" in output
assert (root / "imagemagick_grayscale_01" / "photo_a.png").exists()
@pytest.mark.skipif(not shutil.which("magick"), reason="ImageMagick not installed")
def test_existing_outputs_with_custom_step_id(
self, input_dir: Path, output_base: Path, tmp_path: Path, capsys
) -> None:
external = tmp_path / "external_gray"
external.mkdir()
for src in input_dir.iterdir():
if src.is_file():
make_png(external / src.name, width=4, height=4, rgb=(10, 20, 30))
with Pipeline(
name="custom_id_external",
input_dir=input_dir,
output_base=output_base,
verbose=True,
existing_outputs={"input_bokeh": external},
) as p:
reused = p.step("imagemagick_grayscale", inputs="input", step_id="input_bokeh")
p.step("imagemagick_grayscale", inputs=reused)
root = p.run()
output = capsys.readouterr().out
assert "Reused external output for input_bokeh" in output
assert (root / "input_bokeh" / "photo_a.png").exists()
assert not (root / "imagemagick_grayscale_01").exists()
class TestMaterializeExternal:
def test_links_files_by_stem(self, tmp_path: Path) -> None:
+20 -20
View File
@@ -5,7 +5,6 @@ from pathlib import Path
import pytest
from imagepipeline.core.params import validate_params
from imagepipeline.modules.color_to_alpha import (
ColorToAlphaModule,
build_color_to_alpha_args,
@@ -13,18 +12,19 @@ from imagepipeline.modules.color_to_alpha import (
from imagepipeline.modules.composite import CompositeModule
from imagepipeline.modules.crop_square import CropSquareModule
from imagepipeline.modules.darktable_style import DarktableStyleModule
from imagepipeline.modules.gmic_grayscale import GmicGrayscale
from imagepipeline.modules.imagemagick_fill import (
ImageMagickFillModule,
build_fill_arguments,
)
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
from imagepipeline.modules.imagemagick_resize import (
ImageMagickResizeModule,
build_resize_arguments,
)
from imagepipeline.modules.gmic_grayscale import GmicGrayscale
from imagepipeline.modules.registry import get_module, list_modules
from imagepipeline.modules.rembg import RembgModule
from tests.integration_helpers import has_darktable_style
from tests.integration_helpers import has_workflow_tools as check_workflow_tools
has_magick = bool(shutil.which("magick") or shutil.which("convert"))
@@ -184,7 +184,6 @@ class TestImageMagickFill:
def test_output_matches_input_size(self, tmp_path: Path) -> None:
from imagepipeline.core.context import ModuleContext
from imagepipeline.utils.subprocess import run_command
from tests.conftest import make_png
src = tmp_path / "ref.png"
@@ -221,7 +220,6 @@ class TestCropSquare:
def test_center_crops_to_square(self, tmp_path: Path) -> None:
from imagepipeline.core.context import ModuleContext
from imagepipeline.utils.subprocess import run_command
from tests.conftest import make_png
src = tmp_path / "wide.png"
@@ -255,15 +253,17 @@ class TestModuleParameters:
DarktableStyleModule.validate_module_params({})
def test_darktable_style_accepts_style(self) -> None:
params = DarktableStyleModule.validate_module_params(
{"style": "Watermark F12.rocks"}
)
params = DarktableStyleModule.validate_module_params({"style": "Watermark F12.rocks"})
assert params["style"] == "Watermark F12.rocks"
assert params["style_overwrite"] is True
def test_composite_mode_choices(self) -> None:
with pytest.raises(ValueError, match="must be one of"):
CompositeModule.validate_module_params({"mode": "invalid"})
params = CompositeModule.validate_module_params({"mode": "linearburn"})
assert params["mode"] == "linearburn"
params = CompositeModule.validate_module_params({"mode": "hardmix"})
assert params["mode"] == "hardmix"
def test_rembg_defaults(self) -> None:
params = RembgModule.validate_module_params({})
@@ -273,9 +273,7 @@ class TestModuleParameters:
def test_darktable_export_conf_jpeg(self) -> None:
from imagepipeline.modules.darktable_style import export_conf_options
assert export_conf_options("jpeg") == [
"plugins/imageio/format/jpeg/quality=90"
]
assert export_conf_options("jpeg") == ["plugins/imageio/format/jpeg/quality=90"]
def test_darktable_export_conf_png(self) -> None:
from imagepipeline.modules.darktable_style import export_conf_options
@@ -308,7 +306,6 @@ class TestCompositeColor:
def test_preserves_color_over_grayscale_background(self, tmp_path: Path) -> None:
from imagepipeline.core.context import ModuleContext
from imagepipeline.utils.subprocess import run_command
from tests.conftest import make_png
src = tmp_path / "src.png"
@@ -347,19 +344,22 @@ class TestCompositeColor:
output = output_dir / "fg.png"
assert output.is_file()
result = run_command(
[magick, "identify", "-format", "%[type]", str(output)]
)
assert result.stdout.strip() != "Grayscale"
from PIL import Image
with Image.open(output) as img:
assert img.mode != "L"
has_workflow_tools = all(
shutil.which(name)
for name in ("rembg", "gmic", "magick", "darktable-cli")
) or all(shutil.which(name) for name in ("rembg", "gmic", "convert", "darktable-cli"))
has_workflow_tools = check_workflow_tools()
@pytest.mark.integration
@pytest.mark.slow
@pytest.mark.skipif(not has_workflow_tools, reason="Workflow CLI tools not installed")
@pytest.mark.skipif(
not has_darktable_style("Watermark F12.rocks"),
reason="darktable style 'Watermark F12.rocks' not available or darktable-cli export failed",
)
class TestWorkflowIntegration:
def test_watermark_pipeline(self, input_dir, output_base) -> None:
from imagepipeline import Pipeline
+14 -12
View File
@@ -41,7 +41,7 @@ class TestModuleRegistration:
def test_get_module_returns_xcf_stack_class(self) -> None:
assert get_module("xcf_stack") is XcfStackModule
def test_xcf_stack_runs_last(self) -> None:
def test_xcf_stack_waits_for_layer_inputs(self) -> None:
from imagepipeline.core.runner import PipelineRunner
from imagepipeline.core.step import StepDefinition
from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale
@@ -51,7 +51,7 @@ class TestModuleRegistration:
step_id="xcf_stack_01",
module_name="xcf_stack",
module=XcfStackModule,
input_refs=["input"],
input_refs=["input", "imagemagick_grayscale_01"],
params={},
output_dir_name="xcf_stack_01",
),
@@ -71,7 +71,10 @@ class TestModuleRegistration:
steps=steps,
)
ordered = runner._topological_sort()
assert ordered[-1].module_name == "xcf_stack"
assert [step.step_id for step in ordered] == [
"imagemagick_grayscale_01",
"xcf_stack_01",
]
class TestExpectedOutputFilenames:
@@ -116,9 +119,7 @@ def _make_stack_fixture(tmp_path: Path) -> dict[str, Path]:
class TestXcfStackRun:
@patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf")
def test_collects_layers_from_input_and_prior_steps(
self, mock_stack: object, tmp_path: Path
) -> None:
def test_collects_layers_from_explicit_inputs(self, mock_stack: object, tmp_path: Path) -> None:
paths = _make_stack_fixture(tmp_path)
input_path = paths["root"] / "refs" / "photo.jpg"
input_path.parent.mkdir()
@@ -130,7 +131,8 @@ class TestXcfStackRun:
params=XcfStackModule.validate_module_params({}),
pipeline_output_root=paths["root"],
step_id="xcf_stack_01",
prior_steps=[
input_layer_dirs=[
("input", paths["input_dir"]),
("step_a", paths["step_a"]),
("step_b", paths["step_b"]),
],
@@ -148,9 +150,7 @@ class TestXcfStackRun:
]
@patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf")
def test_skip_missing_true_skips_missing_step(
self, mock_stack: object, tmp_path: Path
) -> None:
def test_skip_missing_true_skips_missing_step(self, mock_stack: object, tmp_path: Path) -> None:
paths = _make_stack_fixture(tmp_path)
(paths["step_b"] / "photo.png").unlink()
input_path = paths["root"] / "photo.jpg"
@@ -161,7 +161,8 @@ class TestXcfStackRun:
params=XcfStackModule.validate_module_params({"skip_missing": True}),
pipeline_output_root=paths["root"],
step_id="xcf_stack_01",
prior_steps=[
input_layer_dirs=[
("input", paths["input_dir"]),
("step_a", paths["step_a"]),
("step_b", paths["step_b"]),
],
@@ -187,7 +188,8 @@ class TestXcfStackRun:
params=XcfStackModule.validate_module_params({"skip_missing": False}),
pipeline_output_root=paths["root"],
step_id="xcf_stack_01",
prior_steps=[
input_layer_dirs=[
("input", paths["input_dir"]),
("step_a", paths["step_a"]),
("step_b", paths["step_b"]),
],