平台本身就能读取 MCAP、LeRobot 和视频上传。如果你的录制是一个 HDF5 文件、一个 rosbag,或者一组视频加 CSV,标准化算法就是那座桥:它把你的格式的一次上传转成标准录制。从这以后,质检、脱敏、位姿和导出都能像处理其他数据一样处理你的数据。
标准化算法每条录制运行一次。它读取 raw,也就是上传的文件,并通过 out(一个 RecordingBuilder)构建录制:
| 方法 | 作用 |
|---|---|
out.fps(30) | 设置录制的帧率。 |
out.task(text) | 设置任务文本。 |
out.camera(key, role, frames=… 或 video=path, intrinsics=…, distortion=…, model=…) | 用解码后的帧或视频文件添加一路相机。 |
out.column(name, values, type, unit, frame, source="recorded") | 添加一列表格数据,每帧一行。 |
out.timestamps(seconds) | 帧的时间不均匀时,给出每一帧的时间。 |
out.stereo_pair(left, right) | 把两路相机标记为双目对。 |
out.dropped(what, reason) | 说明丢弃了什么、为什么丢弃。 |
函数返回后,平台会按标准规则 R1 到 R7 检查结果。违反任何一条的录制都会被 CANONICAL_OUTPUT_INVALID 拒绝,提示里会写出是哪条规则,你可以直接对着改。
示例:一个视频加一个 CSV
这是编辑器给你的起步代码。它读取一个 .mp4 和一个 .csv,CSV 有表头,每个视频帧对应一行数值。文件是 packages/sdk-python/examples/v1/standardize-video-csv/main.py。
"""Standardize: one MP4 and a per-frame CSV become the canonical recording.
In: the uploaded files (raw): one .mp4 from the camera and one .csv with a header row of joint names and
one row of joint values per video frame, recorded at 30 Hz.
Out: the canonical recording: one camera, one "observation.state" column, a task text.
Why: every later step (checks, curation, export, training) reads the canonical recording, never the raw files.
Anything that does not fit (no video, two CSVs) is "not applicable", never guessed at.
"""
import csv
from dataclasses import dataclass
import numpy as np
from embodiflow import Context, RawFiles, RecordingBuilder, algorithm, param
@dataclass
class Params:
task: str = param("move the arm", title="Task text", description="Recorded as the task of the episode")
@algorithm.standardize(
name="video-csv",
title="Video and CSV episode",
description="Reads one .mp4 and one .csv with a header row and one numeric row per video frame, at 30 Hz.",
params=Params,
)
def run(raw: RawFiles, out: RecordingBuilder, ctx: Context[Params]) -> None:
videos = raw.find("**/*.mp4")
tables = raw.find("**/*.csv")
if len(videos) != 1 or len(tables) != 1:
ctx.not_applicable("NOT_ONE_VIDEO_AND_CSV", f"expected one .mp4 and one .csv, found {len(videos)} and {len(tables)}")
with open(tables[0].local_path, newline="", encoding="utf-8") as handle:
reader = csv.reader(handle)
names = next(reader)
states = np.asarray([[float(cell) for cell in row] for row in reader], dtype=np.float32)
out.fps(30)
out.task(ctx.params.task)
out.camera("observation.images.top", "top", video=videos[0].local_path)
out.column("observation.state", states, "vector", names=names)
ctx.progress(1, 1, "converted")它的测试是 packages/sdk-python/examples/v1/standardize-video-csv/test_main.py:用一份很小的测试数据,通过真实的运行器运行这段代码,并检查写出的值。想从它开始,运行 embodiflow algorithm init my-algorithm --category standardize,它会写出 main.py 和 test_main.py;再运行 embodiflow algorithm test my-algorithm(需要装有 pytest 和 SDK 的 Python)。
示例:一个 HDF5 文件
这个例子从一个 .h5 文件里读取 /frames 和 /joint_positions。它需要 h5py 包,所以把它写在 requirements 里。文件是 packages/sdk-python/examples/v1/standardize-hdf5/main.py。
"""Standardize: one HDF5 file (frames, joint positions) becomes the canonical recording.
In: the uploaded files (raw): one .h5 file with /frames (N,H,W,3 uint8) and /joint_positions (N,7).
Out: the canonical recording: one camera, one "observation.state" column in radians, a task text.
Why: teleoperation tools often dump everything into one HDF5 file; this shows how to read one with h5py
and hand the arrays to the builder. Anything but exactly one .h5 file is "not applicable".
"""
from dataclasses import dataclass
import h5py
import numpy as np
from embodiflow import Context, RawFiles, RecordingBuilder, algorithm, param
@dataclass
class Params:
task: str = param("move the arm", title="Task text", description="Used when the file carries no task attribute")
@algorithm.standardize(
name="hdf5-arm",
title="HDF5 arm episode",
description="Reads /frames (N,H,W,3 uint8) and /joint_positions (N,7) from one .h5 file.",
params=Params,
requirements=["h5py>=3.10"],
)
def run(raw: RawFiles, out: RecordingBuilder, ctx: Context[Params]) -> None:
files = raw.find("**/*.h5")
if len(files) != 1:
ctx.not_applicable("NOT_ONE_HDF5", f"expected one .h5 file, found {len(files)}")
with h5py.File(files[0].local_path, "r") as h5:
frames = np.asarray(h5["frames"])
joints = np.asarray(h5["joint_positions"], dtype=np.float32)
task = h5.attrs.get("task", ctx.params.task)
out.fps(30)
out.task(str(task))
out.camera("observation.images.top", "top", frames=frames)
out.column("observation.state", joints, "action.joint", unit="rad", frame="base", names=[f"j{i}" for i in range(joints.shape[1])])
ctx.progress(1, 1, "converted")它的测试是 packages/sdk-python/examples/v1/standardize-hdf5/test_main.py:用一份很小的测试数据,通过真实的运行器运行这段代码,并检查写出的值。想从它开始,运行 embodiflow algorithm init my-algorithm --category standardize --example standardize-hdf5,它会写出 main.py 和 test_main.py;再运行 embodiflow algorithm test my-algorithm(需要装有 pytest 和 SDK 的 Python)。
值得知道
raw.find("**/*.h5") 列出匹配某个模式的上传文件,每个 RawFile.local_path 都是可以直接打开的路径。
上传的数据不归你处理时,要明确说出来。ctx.not_applicable(code, message) 让步骤以「不适用」结束,显示为灰色,不算失败。ctx.input_unusable(code, message) 用于已经损坏的数据:步骤失败,录制被隔离。
读取 raw 的步骤只知道上传时告诉平台的信息。对自有格式来说,只有 kind、format 和 privacy_required,其余都是未知,平台不会去猜。
在工作流里,标准化步骤可以写 uses: auto。平台会选用与录制数据格式对应的标准化算法,并把这个选择记录在运行里。auto 标准化算法有一个文本参数 task,所以 uses: auto 可以配合 with: { task: … } 使用;你自己的标准化算法也可以有这个参数,两个示例都是这样写的。