# Image Pipeline

Modular Python framework for chaining image processing steps after Darktable export.

Each pipeline is a Python script that defines a DAG of processing steps. Every step writes its output to a numbered subfolder inside a timestamped run directory.

## Requirements

- Python 3.11+
- [ImageMagick](https://imagemagick.org/) (`magick` or `convert` on PATH) — most pipelines
- Optional CLIs per module: [G'MIC](https://gmic.eu/), [rembg](https://github.com/danielgatis/rembg), [darktable-cli](https://www.darktable.org/), GIMP (`gimp-console`)

## Installation

```bash
cd /path/to/imagepipeline
pip install -e ".[dev]"      # core + tests
pip install -e ".[dev,ai]"   # + torch/numpy for AI modules
```

Copy `.env.example` to `.env` and set `OPENROUTER_API_KEY` when using `openrouter_edit` or Comfy-related workflows.

## Quick Start

Edit the input path in `pipelines/example_grayscale.py`, then run:

```bash
python pipelines/example_grayscale.py
```

Or from Python:

```python
from pathlib import Path
from imagepipeline import Pipeline

with Pipeline(name="my_run", input_dir=Path("/path/to/export")) as p:
    gray = p.step("imagemagick_grayscale", inputs="input")
    p.run()
```

List registered modules: `imagepipeline list-modules`

## Output Structure

Each run creates a folder like `my_run_20260527143022/` under `~/pipeline_output/` (or `output_base`):

```
my_run_20260527143022/
├── pipeline_manifest.json
├── input/                      # symlinks to source images
├── imagemagick_grayscale_01/
│   └── photo.jpg
└── ...
```

Step folders are named `{module_name}_{nn}` by default (two-digit counter per module name). Pass optional `step_id="input_bokeh"` to `p.step()` for a custom folder name and step reference (see [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md#step-folder-naming)).

## Resume

Pipelines support resuming interrupted runs:

```python
CONTINUE_FROM = Path("~/pipeline_output/my_run_260718120000")
EXISTING_OUTPUTS = {"rembg_01": CONTINUE_FROM / "rembg_01"}

with Pipeline(
    name="my_run",
    input_dir=INPUT,
    continue_from=CONTINUE_FROM,
    existing_outputs=EXISTING_OUTPUTS,
) as p:
    ...
```

Modules that change file extensions must implement `expected_output_filenames` so skip logic works (e.g. `rembg` → `.png`).

## Writing Pipelines

Pipelines are plain Python scripts. Reference previous steps via `StepRef` objects returned by `p.step()`:

```python
with Pipeline(name="colorsplash", input_dir=INPUT) as p:
    rembg_out = p.step("rembg", inputs="input")
    bw = p.step("imagemagick_grayscale", inputs="input")
    combined = p.step("composite", inputs=[bw, rembg_out], mode="foreground_over")
    p.step("darktable_style", inputs=combined, style="vintage.dtstyle")
    p.run()
```

- `"input"` refers to the original input directory
- Parameters are passed as kwargs and validated against each module's schema
- Multiple uses of the same module get separate numbered folders

Declarative building blocks for agents and humans: [RECIPES.md](RECIPES.md).

## Adding Modules

See [docs/MODULE_DEVELOPMENT.md](docs/MODULE_DEVELOPMENT.md). Architecture overview: [docs/ARCHITECTURE.md](docs/ARCHITECTURE.md).

## Development

```bash
ruff check .
ruff format .
pytest                      # full suite (uses local CLIs when present)
pytest -m "not integration" # fast subset (CI default)
```

See [CONTRIBUTING.md](CONTRIBUTING.md).

## Tests

```bash
pytest
```

Optional AI tests require `pip install -e ".[ai]"`.