diff --git a/RECIPES.md b/RECIPES.md index 530317d..506c9f5 100644 --- a/RECIPES.md +++ b/RECIPES.md @@ -24,7 +24,7 @@ Not executable code. Named building blocks for creating or editing pipelines in ``` 6. **Shared expensive steps** — `rembg` runs once per pipeline and is reused. Recipe lines say `rembg`, but the agent must not re-run it for every composite. -7. **`xcf_stack` (GIMP export)** — stacks **all prior step outputs** (plus `input/` by default) into one `.xcf` per image with layers named after step ids. Usually the **last step** after all composites. Requires `gimp` on PATH. Not a layer inside a `combine` block — a separate finishing recipe. +7. **`xcf_stack` (GIMP export)** — stacks **listed step outputs** into one `.xcf` per image. Pass step refs in `inputs=[...]` (bottom layer → top). The runner waits until all listed steps finish. Requires `gimp` on PATH. Not a layer inside a `combine` block — a separate recipe. 8. **Placeholders** — `COLOR1`, `COLOR2`, `COLOR`, `STYLE`, `MAX_EDGE`, `PROMPT`, `MODEL`, `TEMPLATE_IMAGE`, etc. are filled in at pipeline creation time (from your prompt or constants at the top of the script). 9. **Colors** — hex for backgrounds (`#RRGGBB` or `#RRGGBBAA`). G'MIC filters need `R,G,B` tuples (see [GMIC color derivation](#6-gmic-color-derivation) below). 10. **Single-step vs composite vs pipeline** — one declarative line = one module step. Multiple indented lines in a **composite** recipe = layers (bottom → top). Multiple indented lines in a **pipeline** recipe = sequential steps (first step first, output feeds the next). @@ -67,7 +67,7 @@ Not executable code. Named building blocks for creating or editing pipelines in - **Recipe inclusion:** if an indented line is a known recipe id (e.g. `colorsplash` inside `colorsplash-watermark`), look up that recipe, expand it, and use its output as the input for the next step. Expand recursively. Do not duplicate shared steps — one `rembg` (etc.) per pipeline when multiple included recipes need it. - **Pipeline vs composite:** multiple indented lines that are **sequential steps or recipe refs** = pipeline recipe (chain). Multiple indented **layer** lines (original, rembg, backgrounds, gmic-on-rembg, …) = composite recipe (combine). When unsure: if any line is another recipe id, treat the parent as a pipeline recipe. - **Ignore notes:** skip any line that is only a `#` comment (after indent trim). Strip inline `# …` suffixes on recipe-id lines. Do not copy note text into Python comments unless Fränky asks. -- **`xcf_stack`:** when a pipeline includes recipe `xcf-stack` (or the user asks for GIMP layers), append as the final step: `p.step("xcf_stack", inputs="input")`. The module auto-collects `prior_steps` from the runner — do not list every composite as explicit inputs. See `imagepipeline/modules/xcf_stack.py`. +- **`xcf_stack`:** list every layer source in `inputs=[...]` (bottom → top). Include `"input"` for the original. The runner schedules `xcf_stack` after all listed steps complete. See `imagepipeline/modules/xcf_stack.py` and `pipelines/pipeline_rezepttest.py`. - **Shared steps:** define `rembg` (and other expensive steps) **once** per pipeline and reference the step in composites — mirror `pipelines/pipeline_baxxter.py`. - Follow the **Rules cheat sheet** above — especially layer order and chained composites. - **3+ layers:** chain `composite` steps (see baxxter, crusaders, orange). @@ -87,14 +87,14 @@ Not executable code. Named building blocks for creating or editing pipelines in | `gradient background COLOR1 COLOR2 radial` | `imagemagick_fill` radial | | `COLOR background` | `imagemagick_fill` solid `color1=COLOR` | | `rembg with gmic: FILTER` | `gmic` on rembg output; command = `-FILTER` | +| `original with gmic: FILTER` | `gmic` on input; command = `-FILTER` | | `layer opacity N%` | `composite` `foreground_opacity=N/100` on that layer | | `make layer 5% bigger then crop to original size` | `imagemagick_scale_crop` `scale=1.05` | | `COLOR to alpha` | `color_to_alpha` `color=COLOR` | | `resize max edge MAX_EDGE` | `imagemagick_resize` `max_edge=MAX_EDGE` | | `crop square` | `crop_square` | | `darktable style STYLE` | `darktable_style` `style=STYLE` | -| `xcf stack all steps` | `xcf_stack` — GIMP layer export of all prior steps | -| `xcf stack without input` | `xcf_stack` `include_input=false` | +| `xcf stack layers: LAYER1, LAYER2, …` | `xcf_stack` with `inputs=[...]` in that order | | `xcf stack skip missing` | `xcf_stack` `skip_missing=true` | | `openrouter edit PROMPT MODEL` | `openrouter_edit` | | `openrouter gallery match TEMPLATE_IMAGE` | `openrouter_edit` with `template_image` | @@ -230,24 +230,15 @@ crop-square ```text xcf-stack # GIMP layer export — one .xcf per input image - xcf stack all steps + xcf stack layers: input, rembg, composite_01, composite_02, … ``` -- **Module:** `xcf_stack` — `p.step("xcf_stack", inputs="input")` as **last step** -- **Layer order in XCF:** `input/` (bottom, default) then prior steps in pipeline definition order +- **Module:** `xcf_stack` — `p.step("xcf_stack", inputs=["input", step_a, step_b, …])` +- **Layer order in XCF:** same as `inputs` list (bottom → top); GIMP layer names = step ids (`input` for originals) - **Matching:** layers matched by filename stem (extension may differ, e.g. rembg `.png` on `.jpg` input) - **Output:** `{stem}.xcf` per input image - **Requires:** `gimp` on PATH -- **No existing pipeline uses this yet** — append to any multi-output pipeline (e.g. baxxter variants + `xcf-stack`) - -### xcf-stack-no-input - -```text -xcf-stack-no-input - xcf stack without input -``` - -- **Module:** `xcf_stack` — `include_input=False` +- **Source:** `pipelines/pipeline_rezepttest.py` ### xcf-stack-skip-missing @@ -461,6 +452,19 @@ original-jpr-smooth-rembg - **Middle layer:** jpr smooth on rembg, then scale 5% + center crop - **Source:** baxxter, crusaders +### bokeh-oktagon + +```text +bokeh-oktagon + original + original with gmic: fx_bokeh 3,5,0,30,8,4,0.3,0.2,R1,G1,B1,ALPHA1,0.7,30,20,20,1,2,R2,G2,B2,ALPHA2,0.15 + rembg +``` + +- **3 layers:** composite(original, input_bokeh) → composite(result, rembg) +- **Colors:** `R1,G1,B1` from **COLOR1**; `R2,G2,B2` from **COLOR2**; `ALPHA1` / `ALPHA2` are 0–255 (defaults 160 / 110) +- **G'MIC:** octagonal bokeh discs — first color pair `R1,G1,B1,ALPHA1`, second `R2,G2,B2,ALPHA2` + ### original-jpr-smooth-grey-alpha-rembg ```text @@ -556,6 +560,7 @@ G'MIC commands in parameterized recipes use **comma-separated RGB integers**, no | `original-bwrecolor-rembg` | highlight `R2,G2,B2`, accent `R1,G1,B1` in `fx_bwrecolorize` | | `rembg-custom-gradient-a` | starts with `R1,G1,B1`, transitions via `R2,G2,B2` | | `rembg-custom-gradient-b` | starts with `R2,G2,B2`, transitions via `R1,G1,B1` | +| `bokeh-oktagon` | bokeh tints `R1,G1,B1,ALPHA1` and `R2,G2,B2,ALPHA2` in `fx_bokeh` | | `imagemagick_fill` backgrounds | use full hex including alpha if needed (`#d7fd00ff`) | ### Agent workflow @@ -596,7 +601,12 @@ colorsplash-watermark Create pipeline "foo" with recipes colorsplash, rembg-radial-2colors, and xcf-stack ``` -Agent: all composite outputs first, then `p.step("xcf_stack", inputs="input")` as the final step. +Agent: build all composite steps, then `p.step("xcf_stack", inputs=["input", rembg_out, …, composite_final])` listing every layer bottom → top. + +```text +xcf-stack + xcf stack layers: input, rembg, composite_colorsplash, composite_radial, … +``` ### Full combine list (baxxter style) diff --git a/imagepipeline/core/context.py b/imagepipeline/core/context.py index f90e389..dcb7a0c 100644 --- a/imagepipeline/core/context.py +++ b/imagepipeline/core/context.py @@ -18,7 +18,8 @@ class ModuleContext: pipeline_output_root: Path step_id: str matched_groups: list[list[Path]] = field(default_factory=list) - prior_steps: list[tuple[str, Path]] = field(default_factory=list) + input_refs: list[str] = field(default_factory=list) + input_layer_dirs: list[tuple[str, Path]] = field(default_factory=list) logger: PipelineLogger | None = None @property diff --git a/imagepipeline/core/runner.py b/imagepipeline/core/runner.py index 62c40ae..7376be2 100644 --- a/imagepipeline/core/runner.py +++ b/imagepipeline/core/runner.py @@ -151,9 +151,23 @@ class PipelineRunner: step.module.check_dependencies() validated = step.module.validate_module_params(step.params) - source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs] - matched_groups = match_by_stem(source_lists) - input_paths = flatten_matched(matched_groups) + if step.module_name == "xcf_stack": + if not step.input_refs: + raise ValidationError("xcf_stack requires at least one input layer") + source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs] + input_paths = source_lists[0] + matched_groups = [[path] for path in input_paths] + input_layer_dirs: list[tuple[str, Path]] = [] + for ref in step.input_refs: + if ref == INPUT_SOURCE: + input_layer_dirs.append(("input", self._input_link_dir)) + else: + input_layer_dirs.append((ref, self._results[ref].output_dir)) + else: + source_lists = [self._resolve_source_paths(ref) for ref in step.input_refs] + matched_groups = match_by_stem(source_lists) + input_paths = flatten_matched(matched_groups) + input_layer_dirs = [] output_dir = self.output_root / step.output_dir_name output_dir.mkdir(parents=True, exist_ok=True) @@ -220,7 +234,8 @@ class PipelineRunner: pipeline_output_root=self.output_root, step_id=step.step_id, matched_groups=matched_groups, - prior_steps=[(sid, res.output_dir) for sid, res in self._results.items()], + input_refs=list(step.input_refs), + input_layer_dirs=input_layer_dirs, logger=self.logger, ) @@ -280,7 +295,4 @@ class PipelineRunner: if len(ordered_ids) != len(self.steps): raise CycleError("Pipeline contains a cycle in step dependencies") - ordered = [step_by_id[step_id] for step_id in ordered_ids] - deferred = [step for step in ordered if step.module.runs_last] - regular = [step for step in ordered if not step.module.runs_last] - return regular + deferred + return [step_by_id[step_id] for step_id in ordered_ids] diff --git a/imagepipeline/modules/base.py b/imagepipeline/modules/base.py index a083cdb..c4527e8 100644 --- a/imagepipeline/modules/base.py +++ b/imagepipeline/modules/base.py @@ -14,7 +14,6 @@ class BaseModule(ABC): name: ClassVar[str] description: ClassVar[str] = "" - runs_last: ClassVar[bool] = False supported_input_formats: ClassVar[tuple[str, ...]] = ( ".jpg", ".jpeg", diff --git a/imagepipeline/modules/xcf_stack.py b/imagepipeline/modules/xcf_stack.py index c4efdda..4957ad1 100644 --- a/imagepipeline/modules/xcf_stack.py +++ b/imagepipeline/modules/xcf_stack.py @@ -4,18 +4,22 @@ from pathlib import Path from imagepipeline.core.context import ModuleContext from imagepipeline.core.params import Param +from imagepipeline.core.step import INPUT_SOURCE from imagepipeline.modules.base import SubprocessModule from imagepipeline.modules.registry import register from imagepipeline.utils.files import find_image_by_stem from imagepipeline.utils.gimp import stack_images_to_xcf +def _layer_name(ref: str) -> str: + return "input" if ref == INPUT_SOURCE else ref + + @register class XcfStackModule(SubprocessModule): name = "xcf_stack" - runs_last = True description = ( - "Stack all prior pipeline step outputs as GIMP layers into one XCF per image" + "Stack listed pipeline step outputs as GIMP layers into one XCF per image" ) command_candidates = ("gimp-console", "gimp") @@ -32,11 +36,6 @@ class XcfStackModule(SubprocessModule): @classmethod def parameters(cls) -> dict[str, Param]: return { - "include_input": Param( - "bool", - default=True, - help="Include pipeline input/ directory as bottom layer", - ), "skip_missing": Param( "bool", default=False, @@ -50,19 +49,13 @@ class XcfStackModule(SubprocessModule): } def run(self, ctx: ModuleContext) -> None: - include_input = ctx.params["include_input"] skip_missing = ctx.params["skip_missing"] timeout = ctx.params["timeout"] - layer_sources: list[tuple[str, Path]] = [] - if include_input: - layer_sources.append(("input", ctx.pipeline_output_root / "input")) - layer_sources.extend(ctx.prior_steps) - - if not layer_sources: + if not ctx.input_layer_dirs: raise ValueError( - "xcf_stack has no layer sources " - "(include_input=False and no prior_steps)" + "xcf_stack requires at least one input layer " + "(pass step refs in inputs=[...], bottom to top)" ) ctx.output_dir.mkdir(parents=True, exist_ok=True) @@ -73,19 +66,19 @@ class XcfStackModule(SubprocessModule): stem = input_path.stem layers: list[tuple[str, Path]] = [] - for step_id, directory in layer_sources: + for ref, directory in ctx.input_layer_dirs: image_path = find_image_by_stem(directory, stem) if image_path is None: if not skip_missing: raise ValueError( f"xcf_stack: no image with stem '{stem}' in step " - f"'{step_id}' ({directory})" + f"'{_layer_name(ref)}' ({directory})" ) continue - layers.append((step_id, image_path)) + layers.append((_layer_name(ref), image_path)) if not layers: - checked = ", ".join(step_id for step_id, _ in layer_sources) + checked = ", ".join(name for name, _ in ctx.input_layer_dirs) raise ValueError( f"xcf_stack: no layers found for stem '{stem}' " f"(checked: {checked})" diff --git a/pipelines/pipeline_rezepttest.py b/pipelines/pipeline_rezepttest.py new file mode 100644 index 0000000..4751801 --- /dev/null +++ b/pipelines/pipeline_rezepttest.py @@ -0,0 +1,101 @@ +#!/usr/bin/env python3 +"""Rezepttest pipeline — RECIPES.md smoke test (Indians Aichelberg logo colors).""" + +from pathlib import Path + +from imagepipeline import Pipeline + +# Change to your Darktable export folder. +INPUT = Path("/home/frank/tmp/bar") +OUTPUT_BASE = Path.home() / "pipeline_output" + +EXISTING_OUTPUTS: dict[str, Path] = {} +CONTINUE_FROM: Path | None = None + +# Colors from Indians Aichelberg logo (orange text + red stitching). +COLOR1 = "#f0b020" # 240, 176, 32 +COLOR2 = "#f01000" # 240, 16, 0 + +ALPHA1 = 160 +ALPHA2 = 110 + +GMIC_DROP_SHADOW = "-fx_drop_shadow3d 0,0,0,10,1,1,2,0.5,240,16,0,200,0" +GMIC_JPR_SMOOTH = "-jpr_gradient_smooth 0,1.5" +GMIC_BOKEH = ( + f"-fx_bokeh 3,5,0,30,8,4,0.3,0.2,240,176,32,{ALPHA1},0.7,30,20,20,1,2," + f"240,16,0,{ALPHA2},0.15" +) + + +def main() -> None: + with Pipeline( + name="rezepttest", + input_dir=INPUT, + output_base=OUTPUT_BASE, + existing_outputs=EXISTING_OUTPUTS or None, + continue_from=CONTINUE_FROM, + ) as p: + rembg_out = p.step("rembg", inputs="input") + grayscale = p.step("gmic_grayscale", inputs="input") + gradient_radial_bg = p.step( + "imagemagick_fill", + inputs="input", + color1=COLOR1, + color2=COLOR2, + gradient=True, + radial=True, + ) + + rembg_shadow = p.step("gmic", inputs=rembg_out, command=GMIC_DROP_SHADOW) + rembg_jpr_smooth = p.step("gmic", inputs=rembg_out, command=GMIC_JPR_SMOOTH) + rembg_jpr_smooth_sized = p.step( + "imagemagick_scale_crop", + inputs=rembg_jpr_smooth, + scale=1.05, + ) + input_bokeh = p.step("gmic", inputs="input", command=GMIC_BOKEH) + + # recipe: colorsplash + composite_colorsplash = p.step("composite", inputs=[grayscale, rembg_out]) + + # recipe: rembg-radial-2colors (gradient-radial) + composite_radial = p.step("composite", inputs=[gradient_radial_bg, rembg_out]) + + # recipe: original-drop-shadow-rembg + shadow_mid = p.step("composite", inputs=["input", rembg_shadow]) + composite_shadow = p.step("composite", inputs=[shadow_mid, rembg_out]) + + # recipe: original-jpr-smooth-rembg + smooth_mid = p.step("composite", inputs=["input", rembg_jpr_smooth_sized]) + composite_smooth = p.step("composite", inputs=[smooth_mid, rembg_out]) + + # recipe: bokeh-oktagon + bokeh_mid = p.step("composite", inputs=["input", input_bokeh]) + composite_bokeh = p.step("composite", inputs=[bokeh_mid, rembg_out]) + + # recipe: xcf-stack — explicit layer list, bottom to top + p.step( + "xcf_stack", + inputs=[ + "input", + rembg_out, + grayscale, + gradient_radial_bg, + rembg_shadow, + rembg_jpr_smooth_sized, + input_bokeh, + composite_colorsplash, + composite_radial, + composite_shadow, + composite_smooth, + composite_bokeh, + ], + ) + + output_root = p.run() + + print(f"Pipeline finished. Output: {output_root}") + + +if __name__ == "__main__": + main() diff --git a/tests/test_xcf_stack.py b/tests/test_xcf_stack.py index 29a3dd7..7ce41db 100644 --- a/tests/test_xcf_stack.py +++ b/tests/test_xcf_stack.py @@ -41,7 +41,7 @@ class TestModuleRegistration: def test_get_module_returns_xcf_stack_class(self) -> None: assert get_module("xcf_stack") is XcfStackModule - def test_xcf_stack_runs_last(self) -> None: + def test_xcf_stack_waits_for_layer_inputs(self) -> None: from imagepipeline.core.runner import PipelineRunner from imagepipeline.core.step import StepDefinition from imagepipeline.modules.imagemagick_grayscale import ImageMagickGrayscale @@ -51,7 +51,7 @@ class TestModuleRegistration: step_id="xcf_stack_01", module_name="xcf_stack", module=XcfStackModule, - input_refs=["input"], + input_refs=["input", "imagemagick_grayscale_01"], params={}, output_dir_name="xcf_stack_01", ), @@ -71,7 +71,10 @@ class TestModuleRegistration: steps=steps, ) ordered = runner._topological_sort() - assert ordered[-1].module_name == "xcf_stack" + assert [step.step_id for step in ordered] == [ + "imagemagick_grayscale_01", + "xcf_stack_01", + ] class TestExpectedOutputFilenames: @@ -116,7 +119,7 @@ def _make_stack_fixture(tmp_path: Path) -> dict[str, Path]: class TestXcfStackRun: @patch("imagepipeline.modules.xcf_stack.stack_images_to_xcf") - def test_collects_layers_from_input_and_prior_steps( + def test_collects_layers_from_explicit_inputs( self, mock_stack: object, tmp_path: Path ) -> None: paths = _make_stack_fixture(tmp_path) @@ -130,7 +133,8 @@ class TestXcfStackRun: params=XcfStackModule.validate_module_params({}), pipeline_output_root=paths["root"], step_id="xcf_stack_01", - prior_steps=[ + input_layer_dirs=[ + ("input", paths["input_dir"]), ("step_a", paths["step_a"]), ("step_b", paths["step_b"]), ], @@ -161,7 +165,8 @@ class TestXcfStackRun: params=XcfStackModule.validate_module_params({"skip_missing": True}), pipeline_output_root=paths["root"], step_id="xcf_stack_01", - prior_steps=[ + input_layer_dirs=[ + ("input", paths["input_dir"]), ("step_a", paths["step_a"]), ("step_b", paths["step_b"]), ], @@ -187,7 +192,8 @@ class TestXcfStackRun: params=XcfStackModule.validate_module_params({"skip_missing": False}), pipeline_output_root=paths["root"], step_id="xcf_stack_01", - prior_steps=[ + input_layer_dirs=[ + ("input", paths["input_dir"]), ("step_a", paths["step_a"]), ("step_b", paths["step_b"]), ],