Image algorithms

Python in the browser covers most needs, but sometimes your best code is C++, Rust or a model with its own stack. An image algorithm lets you bring it as a container. The platform treats it like any other step: same categories, same ports, same checks on what it writes. Only the packaging differs.

The arrangement is simple. The container reads a job file and its inputs from fixed folders, writes its outputs to fixed folders and exits with a code. Since there is no decorator to read, you write the interface by hand as algorithm.yaml, and embodiflow algorithm push bakes it into the image label io.embodiflow.algorithm.

What the container does

• Read /embodiflow/config/job.json.
• Read inputs under /embodiflow/input/<port>/ and write outputs under /embodiflow/output/<port>/, in the layouts below.
• Optionally report progress in /embodiflow/output/progress.json: { "done", "total", "message" }.
• On a refusal, write /embodiflow/output/error.json: { "code", "message" }, with a code in capital letters.
• Use /embodiflow/tmp/ for scratch space.
• Exit with one of the codes below.

job.json

The job file tells your container what it's working on: the run, the item, the parameters, the facts and where each port lives. This one is for an episode-level enrich step.

{
  "contract": "embodiflow-runtime/v1",
  "category": "enrich", "level": "episode",
  "runId": "<uuid>", "stepId": "success", "recordingId": "<uuid>",
  "item": { "episodeIndex": 3, "startS": 120.4, "endS": 161.0, "task": "pick up the cup" },
  "params": { "threshold": 0.5 },
  "facts": { "kind": "human", "stereo": true, "imu": true, "privacy_required": true, "duration_s": 40.6 },
  "inputs":  { "recording": { "type": "recording", "root": "/embodiflow/input/recording" },
               "episodes":  { "type": "episodes",  "root": "/embodiflow/input/episodes" } },
  "outputs": { "features": { "type": "features", "root": "/embodiflow/output/features" } }
}

An optional input that is absent is simply missing from inputs.

Layouts at a port root

A per-frame feature needs exactly one row per frame in frames.parquet. A per-episode value is keyed by episodeIndex, its position in the connected episodes document. Times in checks.json and episodes.json are seconds on the recording timeline.

Exit codes

Before exiting with 85 or 87, write the code and message to error.json. The step message then reads <code>: <message>.

algorithm.yaml

This file is the same document a Python decorator produces, with the schema embodiflow-algorithm/v1. Here is the motion energy algorithm from Your first algorithm, declared for an image:

apiVersion: embodiflow-algorithm/v1
name: motion-energy
title: Motion energy
category: enrich
features:
  - name: motion_energy
    type: scalar
    per: frame
    dtype: float32
    unit: none
params:
  step: { type: integer, default: 1, minimum: 1, title: Frame step }
resources: { cpu: 4, memory: 8Gi, timeout: 30m }

Test locally, then register

You don't have to start from an empty folder. embodiflow algorithm init <dir> --category <category> --lang image writes an algorithm.yaml, a Dockerfile with an exec-form ENTRYPOINT and an entry script that already writes the outputs of the category. From there the loop looks like this:

# write my-algo/algorithm.yaml (above) and a Dockerfile by hand
embodiflow algorithm validate my-algo
embodiflow algorithm test my-algo --build --input recording=./rec
docker build -t my-algo my-algo && docker save my-algo -o my-algo.tar
embodiflow algorithm push my-algo --archive my-algo.tar --registry <registry-host>

validate checks algorithm.yaml with the same parser the platform uses. test runs the container on your machine and checks its outputs with the platform's rules. push imports the image archive into your team's registry and registers the algorithm with an image reference pinned to its digest.

Next

• Command line
• Troubleshooting