训练集成

相关源文件

生成此 wiki 页面时使用了以下文件作为上下文：

目的与范围

本文档说明 IB-Robot 如何集成外部 LeRobot training library，用于训练 ACT、Diffusion Policy 和 VLA 等具身智能策略。内容覆盖通过 patch-based 配置系统设置 lerobot 库、数据集兼容性，以及 weighted training、knowledge distillation 等专用训练能力。

有关通过遥操作采集示教数据，请参见 遥操作与数据采集。有关将 ROS2 bags 转换为 LeRobot dataset format，请参见 数据集转换（bag_to_lerobot）。

训练架构概览

IB-Robot 将模型训练委托给外部 LeRobot 库，该库以子模块形式维护在 libs/lerobot 中。为了兼容多种平台（Ubuntu、openEuler、OpenHarmony）和硬件（NVIDIA GPUs、Ascend NPUs），IB-Robot 使用定义在 third_party/patches/lerobot/ 中的 patch-based integration system third_party/patches/lerobot/v0.5.1/manifest.yaml:1-13。

训练生命周期与代码实体

        graph TB
    subgraph "IB-Robot Workspace"
        A["bag_to_lerobot.py"]
        B["robot_config YAML"]
    end
    
    subgraph "LeRobot Submodule (libs/lerobot)"
        C["lerobot_train.py"]
        D["modeling_act.py"]
        E["modeling_diffusion.py"]
        F["modeling_mt_act.py"]
    end

    subgraph "Patch System"
        P1["0010-weighted-training.patch"]
        P2["0011-knowledge-distillation.patch"]
        P3["0012-mt-act-model.patch"]
        P4["0014-training-logs.patch"]
    end
    
    A -->|"Generates v3 Dataset"| C
    B -->|"Defines Contract"| C
    P1 -.->|"Patches"| C
    P1 -.->|"Patches"| D
    P2 -.->|"Patches"| D
    P2 -.->|"Patches"| E
    P3 -.->|"Patches"| F
    P4 -.->|"Patches"| C
    C -->|"Saves"| CKPT["Policy Checkpoint"]

来源: third_party/patches/lerobot/v0.5.1/manifest.yaml:125-144, third_party/patches/lerobot/v0.5.1/0010-weighted-training.patch:1-15, third_party/patches/lerobot/v0.5.1/0011-knowledge-distillation.patch:1-15, third_party/patches/lerobot/v0.5.1/0012-mt-act-model.patch:1-18, third_party/patches/lerobot/v0.5.1/0014-training-logs.patch:1-10

LeRobot 库集成

基于 Patch 的兼容性

系统使用 manifest-driven patch applier，根据主机环境和训练需求修改 libs/lerobot 子模块 third_party/patches/lerobot/v0.5.1/manifest.yaml:3-9。

训练专用 Patches

为启用高级训练能力，以下 patches 会应用到 training profile：

来源: third_party/patches/lerobot/v0.5.1/manifest.yaml:28-45, third_party/patches/lerobot/v0.5.1/0010-weighted-training.patch:16-37, third_party/patches/lerobot/v0.5.1/0011-knowledge-distillation.patch:1-15, third_party/patches/lerobot/v0.5.1/0012-mt-act-model.patch:1-18, third_party/patches/lerobot/v0.5.1/0014-training-logs.patch:1-10

高级训练特性

1. Weighted Training

通过 0010-weighted-training.patch 修改后，训练脚本 lerobot_train.py 支持：

2. Knowledge Distillation (KD)

通过 0011-knowledge-distillation.patch 修改后，KD 允许 “student” 模型学习 “teacher” 模型输出：

3. Multi-Task ACT (MT-ACT)

Patch 0012-mt-act-model.patch 引入 mt_act policy family。

4. TensorBoard Logging

Patch 0014-training-logs.patch 通过 TensorBoard 集成增强训练流程：

来源: third_party/patches/lerobot/v0.5.1/0010-weighted-training.patch:16-147, third_party/patches/lerobot/v0.5.1/0011-knowledge-distillation.patch:18-41, third_party/patches/lerobot/v0.5.1/0012-mt-act-model.patch:90-157, third_party/patches/lerobot/v0.5.1/0014-training-logs.patch:36-50, third_party/patches/lerobot/v0.5.1/0014-training-logs.patch:81-82

平台特定训练环境

训练集成因目标平台不同而变化，并通过 lerobot_filter_series.py 回归测试工具验证 scripts/setup/tests/test_lerobot_filter.sh:2-7。

Platform

Profile

Python

Key Features

Ubuntu 22.04

core,ros,dev

3.10

标准 GPU training，可视化工具 scripts/setup/tests/test_lerobot_filter.sh:104-108

openEuler

core,ros,openeuler

3.11

Ascend NPU training 支持 scripts/setup/tests/test_lerobot_filter.sh:114-118

Ascend NPU

ascend,om,training

3.10/3.11

Weighted training、QAT、OM export 兼容性 third_party/patches/lerobot/v0.5.1/manifest.yaml:35-39

OpenHarmony

openharmony

3.12

仅推理，使用 0004-openharmony-lazy-import-policy-stack.patch scripts/setup/tests/test_lerobot_filter.sh:122-126

训练集成流（Ascend/NPU）

        graph LR
    subgraph "Training Host (Ascend/NPU)"
        A["lerobot_train.py"]
        B["0005-compat-add-npu-device-detection.patch"]
        C["0006-compat-add-ascend-om-config-fields.patch"]
    end
    
    subgraph "Model Config"
        D["is_ascend_om_enabled: bool"]
        E["om_model_path: str"]
    end

    A -->|"Uses"| B
    A -->|"Populates"| D
    A -->|"Populates"| E

来源: scripts/setup/tests/test_lerobot_filter.sh:104-142, third_party/patches/lerobot/v0.5.1/manifest.yaml:105-115

训练执行

使用 IB-Robot 修改运行训练会话：

  1. 初始化环境：确保正确 profiles 已启用，例如 training,ascend。

  2. 应用 Patches：运行 setup.sh，该脚本会调用 lerobot_filter_series.py 选择并应用 patches scripts/setup/tests/test_lerobot_filter.sh:20-25。

  3. 执行脚本：使用位于 libs/lerobot/src/lerobot/scripts/ 的 patched lerobot_train.py。

关键训练 CLI Flags（Patched）：

来源: third_party/patches/lerobot/v0.5.1/0010-weighted-training.patch:20-35, third_party/patches/lerobot/v0.5.1/0011-knowledge-distillation.patch:15-20, third_party/patches/lerobot/v0.5.1/0012-mt-act-model.patch:150-155