Each pipeline is a Python script that defines a DAG of processing steps. Every step writes its output to a numbered subfolder inside a timestamped run directory.

Requirements

Python 3.11+
ImageMagick (magick or convert on PATH) — most pipelines
Optional CLIs per module: G'MIC, rembg, darktable-cli, GIMP (gimp-console)

Installation

cd /path/to/imagepipeline
pip install -e ".[dev]"      # core + tests
pip install -e ".[dev,ai]"   # + torch/numpy for AI modules

Copy .env.example to .env and set OPENROUTER_API_KEY when using openrouter_edit or Comfy-related workflows.

Quick Start

Edit the input path in pipelines/example_grayscale.py, then run:

python pipelines/example_grayscale.py

Or from Python:

from pathlib import Path
from imagepipeline import Pipeline

with Pipeline(name="my_run", input_dir=Path("/path/to/export")) as p:
    gray = p.step("imagemagick_grayscale", inputs="input")
    p.run()

List registered modules: imagepipeline list-modules

Output Structure

Each run creates a folder like my_run_20260527143022/ under ~/pipeline_output/ (or output_base):

my_run_20260527143022/
├── pipeline_manifest.json
├── input/                      # symlinks to source images
├── imagemagick_grayscale_01/
│   └── photo.jpg
└── ...

Step folders are named {module_name}_{nn} by default (two-digit counter per module name). Pass optional step_id="input_bokeh" to p.step() for a custom folder name and step reference (see docs/MODULE_DEVELOPMENT.md).

Resume

Pipelines support resuming interrupted runs:

CONTINUE_FROM = Path("~/pipeline_output/my_run_260718120000")
EXISTING_OUTPUTS = {"rembg_01": CONTINUE_FROM / "rembg_01"}

with Pipeline(
    name="my_run",
    input_dir=INPUT,
    continue_from=CONTINUE_FROM,
    existing_outputs=EXISTING_OUTPUTS,
) as p:
    ...

Modules that change file extensions must implement expected_output_filenames so skip logic works (e.g. rembg → .png).

Writing Pipelines

Pipelines are plain Python scripts. Reference previous steps via StepRef objects returned by p.step():

with Pipeline(name="colorsplash", input_dir=INPUT) as p:
    rembg_out = p.step("rembg", inputs="input")
    bw = p.step("imagemagick_grayscale", inputs="input")
    combined = p.step("composite", inputs=[bw, rembg_out], mode="foreground_over")
    p.step("darktable_style", inputs=combined, style="vintage.dtstyle")
    p.run()

"input" refers to the original input directory
Parameters are passed as kwargs and validated against each module's schema
Multiple uses of the same module get separate numbered folders

Declarative building blocks for agents and humans: RECIPES.md.

Adding Modules

See docs/MODULE_DEVELOPMENT.md. Architecture overview: docs/ARCHITECTURE.md.

Development

ruff check .
ruff format .
pytest                      # full suite (uses local CLIs when present)
pytest -m "not integration" # fast subset (CI default)

See CONTRIBUTING.md.

Tests

pytest

Optional AI tests require pip install -e ".[ai]".