Back

large behavior model (LBM)

In robotics, a large behavior model is a high‑capacity, foundation‑style policy that maps rich sensory input (vision, proprioception, sometimes language) to extended sequences of actions across many tasks.
More concretely, in this context:
  1. It’s a single, usually neural, control model trained on large, diverse datasets of demonstrations or interactions, so it can generate many different behaviors rather than a single skill.
  2. Inputs typically include egocentric camera streams, robot state, and optional natural‑language instructions; outputs are time‑extended action trajectories (e.g., end‑effector poses, motor commands) rather than one‑step actions.
  3. The “large” refers both to scale (parameters, data, tasks) and to scope: the model is meant to generalize across environments and tasks as a generalist controller, similar in spirit to how large language models generalize across text tasks.
TRI (Toyota Research Institute) is one of the main originators and drivers of the “large behavior model” concept in robotics, and is actively building and scaling these models for real robots.

Key aspects of TRI’s role:
  1. Coining and framing: TRI explicitly framed LBMs as the robotics analog of large language models in the context of teaching robots many dexterous skills via a generative AI approach called Diffusion Policy.
  2. Core algorithms: TRI co‑developed Diffusion Policy for robot behavior learning and then scaled it into multitask, language‑conditioned LBMs that take in multimodal sensory streams (wrist/scene cameras, proprioception, language prompts) and output action chunks for manipulation tasks.
  3. Data and skills at scale: TRI has collected large demonstration datasets on custom dual‑arm platforms and reports training LBMs that can perform dozens of household‑style skills today, with targets of hundreds to thousands of skills.
  4. Benchmarking and theory: Through work like “A Careful Examination of Large Behavior Models for Multitask Dexterous Manipulation,” TRI has published systematic evaluations of how scaling data, model size, and task diversity affect generalization and transfer in LBMs.
  5. Ecosystem collaborations: TRI’s LBM architectures and datasets are being used in collaborations with companies such as Boston Dynamics to train generalist policies for humanoids like Atlas, extending TRI’s LBM work to new embodiments.


Share: