从模型记录，到支撑判断的原句，再到研究与发布的时间。
在线查阅整理结果，需要时再下载结构化文件。
名称、输入输出、开放声明和材料说明的限制。
在线查看版本标识随文件内容变化，不代表模型发布或来源阅读日期。
93784dfd18b2a2891276c66d29ee75932068555a5769644fbc22d8a9f5a49c74世界模型图谱，《模型记录》，https://worldmodelatlas.com/data/projects.json，SHA-256：93784dfd18b2a2891276c66d29ee75932068555a5769644fbc22d8a9f5a49c74。访问日期：请填写实际下载日期。
每项判断对应的来源原句、链接和阅读日期。
在线查看版本标识随文件内容变化，不代表模型发布或来源阅读日期。
b6e0c1be2fc2c2efcbecb25fbb3a9fe4015c606b67d2596507106e66f9a2c614世界模型图谱，《字段证据》，https://worldmodelatlas.com/data/evidence.json，SHA-256：b6e0c1be2fc2c2efcbecb25fbb3a9fe4015c606b67d2596507106e66f9a2c614。访问日期：请填写实际下载日期。
将研究与发布事件放回时间，保留日期精度。
在线查看版本标识随文件内容变化，不代表模型发布或来源阅读日期。
180da3ae79cacb4e8167d386eef32540833f510e51a27df4626e43e4072c213a世界模型图谱，《事件记录》，https://worldmodelatlas.com/data/timeline.json，SHA-256：180da3ae79cacb4e8167d386eef32540833f510e51a27df4626e43e4072c213a。访问日期：请填写实际下载日期。
打开模型可查看完整字段、分类依据、来源和未知项。记录不代表能力排名或今日服务可用性。当前分类数据：模型标签 · 分类定义。
在世界模型生成的梦境环境中训练智能体，再将策略迁回实际环境。
查看字段与来源 →OpenAI 在这篇技术报告中介绍可生成最长一分钟视频的 Sora。
查看字段与来源 →先将传感器观测编码为 token，再通过离散扩散预测未来的世界模型。
查看字段与来源 →根据文本提示生成可实时导航的动态世界。
查看字段与来源 →冻结预训练视频编码器后，在其表征之上另行训练的动作条件预测器。
查看字段与来源 →从多种输入生成 3D 世界，并支持交互编辑、扩展和组合。
查看字段与来源 →基于 Gen-4.5、逐帧生成且可通过动作控制的实时自回归模型。
查看字段与来源 →基于 Genie 3 的 Waymo 世界模型。
查看字段与来源 →联合处理和生成语言、图像、视频、音频及动作序列的世界模型家族。
查看字段与来源 →让 AI Driver 参与闭环模拟的 GAIA 世界模型。
查看字段与来源 →学习环境模型，通过想象未来情景改进智能体行为。
查看字段与来源 →将目标特征作为预测目标，通过优化动作序列进行视觉目标规划。
查看字段与来源 →在实体机器人上学习世界模型和行为策略，不依赖模拟器。
查看字段与来源 →通过扩散世界模型预测游戏下一帧，并在模型中训练强化学习智能体。
查看字段与来源 →可与 VLA 策略共同在想象空间中滚动生成的动作条件世界模型。
查看字段与来源 →联合建模游戏画面与控制器动作，可生成画面、动作或二者的序列。
查看字段与来源 →面向驾驶、强调通用性与多种控制条件的世界模型。
查看字段与来源 →以共享空间上下文生成多模态内容的世界模型。
查看字段与来源 →从像素端到端训练的 JEPA 世界模型。
查看字段与来源 →在各层潜在空间中，用历史状态与动作预测不同时间尺度的未来。
查看字段与来源 →联合学习语义理解、未来视觉预测与动作生成。
查看字段与来源 →预测动作引起的潜在状态增量，再更新状态并解码未来画面。
查看字段与来源 →以候选行驶轨迹与动作预测未来交通占据和动态流。
查看字段与来源 →由文本动作与连续相机输入驱动的自回归视听世界模型。
查看字段与来源 →以流式 3D 点缓存和重新渲染的空间条件生成视角可控视频。
查看字段与来源 →借助外部世界状态库，持续生成受相机和文本条件控制的视频。
查看字段与来源 →按用户提供的目标相机轨迹，从图像或视频生成视图视频。
查看字段与来源 →把用户交互作为条件，逐帧生成视觉界面。
查看字段与来源 →以历史观察、提示和动作持续预测可交互的视觉环境。
查看字段与来源 →以机器人动作轨迹为条件，预测多视角未来视频。
查看字段与来源 →从历史观察预测有助于驾驶规划的未来表征，再生成并选择轨迹。
查看字段与来源 →把长视觉历史与未来视频预测结合，用于生成机器人动作块。
查看字段与来源 →在有限跨度内并行预测有因果关联的未来表征轨迹，用于动作规划。
查看字段与来源 →在策略使用的 VLM 视觉 token 空间中，按动作预测未来状态。
查看字段与来源 →以历史状态和实时多模态输入，持续生成同步音视频环境。
查看字段与来源 →按模型展开，核对整理后的判断和对应原句。摘录存在，不自动证明它足以支持结论；未知字段没有证据条目。原阅读方向保留作历史字段，当前目录分类请见模型页。
表征预测
The MDN outputs the parameters of a mixture of Gaussian distribution used to sample a prediction of the next latent vector
在世界模型生成的梦境环境中训练智能体，再将策略迁回实际环境。
We can even train our agent entirely inside of its own dream environment generated by its world model, and transfer this policy back into the actual environment.
用于采样下一潜在向量预测的高斯混合分布参数。
The MDN outputs the parameters of a mixture of Gaussian distribution used to sample a prediction of the next latent vector
记忆模型 M 接收当前潜在向量和动作。
M will then take the current $z_{t}$zt and action $a_{t}$at as an input
是
M will then take the current $z_{t}$zt and action $a_{t}$at as an input
视觉生成
Our largest model, Sora, is capable of generating a minute of high fidelity video.
OpenAI 在这篇技术报告中介绍可生成最长一分钟视频的 Sora。
it can generate videos and images spanning diverse durations, aspect ratios and resolutions, up to a full minute of high definition video.
一分钟视频。
Our largest model, Sora, is capable of generating a minute of high fidelity video.
图像和提示词。
Sora is capable of generating videos provided an image and prompt as input.
该报告指出，Sora 不能准确建模一些基础物理交互，例如玻璃破碎。
Sora currently exhibits numerous limitations as a simulator. For example, it does not accurately model the physics of many basic interactions, like glass shattering.
表征预测
Use these BEV tokens to predict future BEV tokens with our world model auto-regressively, which leverages discrete diffusion.
先将传感器观测编码为 token，再通过离散扩散预测未来的世界模型。
Consequently, we propose Copilot4D, a novel world modeling approach that first tokenizes sensor observations with VQVAE, then predicts the future via discrete diffusion.
BEV token 和自车动作。
With those BEV tokens as well as the SDV action at t, it performs k steps of discrete diffusion to obtain the BEV tokens representing the next frame, t.
由预测的 BEV token 解码得到的未来点云。
Map the predicted futures from BEV tokens back to point clouds with the tokenizer decoder and differentiable neural depth rendering.
是
To make this possible, Copilot4D has the ability to be prompted with potential future actions, and thus it can be called a world model.
交互世界
Given a text prompt, Genie 3 can generate dynamic worlds that you can navigate in real time at 24 frames per second, retaining consistency for a few minutes at a resolution of 720p.
根据文本提示生成可实时导航的动态世界。
Given a text prompt, Genie 3 can generate dynamic worlds that you can navigate in real time at 24 frames per second, retaining consistency for a few minutes at a resolution of 720p.
导航输入和文本交互。
In addition to navigational inputs, Genie 3 also enables a more expressive form of text-based interaction, which we refer to as promptable world events.
可实时导航的动态视觉世界。
Given a text prompt, Genie 3 can generate dynamic worlds that you can navigate in real time at 24 frames per second, retaining consistency for a few minutes at a resolution of 720p.
是
Like any other environment, Genie 3 is not aware of the agent’s goal, instead it simulates the future based on the agent's actions.
有限开放
we are announcing Genie 3 as a limited research preview, providing early access to a small cohort of academics and creators.
该发布文称，连续交互只能维持几分钟。
Limited interaction duration. The model can currently support a few minutes of continuous interaction, rather than extended hours.
表征预测
post-training a latent action-conditioned world model, V-JEPA 2-AC, using less than 62 hours of unlabeled robot videos from the Droid dataset.
是
post-training a latent action-conditioned world model, V-JEPA 2-AC, using less than 62 hours of unlabeled robot videos from the Droid dataset.
冻结预训练视频编码器后，在其表征之上另行训练的动作条件预测器。
After pretraining, we freeze the video encoder and learn a new action-conditioned predictor, V-JEPA 2-AC, on top of the learned representation.
V-JEPA 2 基础视频表征模型
After pretraining, we freeze the video encoder and learn a new action-conditioned predictor, V-JEPA 2-AC, on top of the learned representation.
过去的视频帧、动作和末端执行器状态。
predicting the representations of future video frames conditioned on past video frames, actions, and end-effector states.
以控制动作和本体感知观测为条件，预测未来视频观测的表征。
To achieve this, we train V-JEPA 2-AC, an autoregressive model that predicts representations of future video observations conditioned on control actions and proprioceptive observations.
自回归预测会累积误差，限制长时程规划的可靠性。
Long horizon planning with world models is limited by a number of factors. First, autoregressive prediction suffers from error accumulation:
开放权重
[checkpoint](https://dl.fbaipublicfiles.com/vjepa2/vjepa2-ac-vitg.pt)
交互世界
Marble also lets you interactively edit, expand, and combine worlds.
从多种输入生成 3D 世界，并支持交互编辑、扩展和组合。
Marble can create 3D worlds from text, images, video, or coarse 3D layouts; Marble also lets you interactively edit, expand, and combine worlds.
文本、图像、视频或粗略 3D 布局。
Marble can create 3D worlds from text, images, video, or coarse 3D layouts;
可导出为 Gaussian splats、网格或视频的 3D 世界。
Once generated, 3D worlds can be exported as Gaussian splats, meshes, or videos.
面向公众开放
Today we are making Marble, a first-in-class generative multimodal world model, generally available for anyone to use.
碰撞网格为低保真网格，用于粗略物理模拟。
Marble can generate both collider meshes, which are low-fidelity meshes intended for coarse physics simulation;
基于 Gen-4.5、逐帧生成且可通过动作控制的实时自回归模型。
GWM-1 is an autoregressive model built on top of Gen-4.5. It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio.
交互世界
It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio.
家族所述动作控制包括相机位姿、机器人指令和音频。
It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio.
逐帧生成的视觉序列。
It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio.
是
It generates frame by frame, runs in real time, and can be controlled interactively with actions—camera pose, robot commands, audio.
单一、统一的通用模型
GWM Worlds for explorable environments, GWM Avatars for conversational characters, and GWM Robotics for robotic manipulation. Today, these are separate post-trained models.
基于 Genie 3 的 Waymo 世界模型。
The Waymo World Model is built upon Genie 3
交互世界
Driving action control allows us to have a responsive simulator that adheres to specific driving inputs.
是
Driving action control allows us to have a responsive simulator that adheres to specific driving inputs.
语言提示、驾驶输入和场景布局。
The model’s architecture offers high controllability, allowing our engineers to modify simulations with simple language prompts, driving inputs, and scene layouts.
包含相机与激光雷达数据的多传感器模拟输出。
Notably, the Waymo World Model generates high-fidelity, multi-sensor outputs that include both camera and lidar data.
模拟时间越长，计算和维持稳定质量越困难。
That’s harder to do because the longer the simulation, the tougher it is to compute and maintain stable quality.
物理 AI
NVIDIA today launched NVIDIA Cosmos™ 3, an open world foundation model for physical AI
联合处理和生成语言、图像、视频、音频及动作序列的世界模型家族。
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences
语言、图像、视频、音频和动作序列。
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences
语言、图像、视频、音频和动作序列。
We introduce Cosmos 3, a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences
开放权重
Cosmos 3 Edge, Cosmos 3 Nano and Cosmos 3 Super are available now on_[_Hugging Face_](https://huggingface.co/collections/nvidia/cosmos3)
是
Generate infinite plausible futures from text, image, video, ambient sound and action input.
让 AI Driver 参与闭环模拟的 GAIA 世界模型。
Now with GAIA-4, we can close the loop, turning a recorded scene into the starting point for a new simulated run with the AI Driver back in the loop:
交互世界
the AI Driver’s actions drive the simulation forward — its position, its viewpoint, and the sensor stream it receives
是
the AI Driver’s actions drive the simulation forward — its position, its viewpoint, and the sensor stream it receives
记录的传感器数据。
It generates the scene directly from recorded sensor data.
相机与雷达的多模态模拟数据。
GAIA-4 extends this capability with multimodal generation, bringing radar into the simulation alongside cameras, an industry-first for autonomous driving simulators.
World-on-rails 是可选回放约束，并非模型整体限制；不能把该模式等同于完全交互的交通模拟。
World-on-rails is a capability we can switch on, not a limitation of the world model.
world-on-rails 模式下完全响应自车的交通模拟
World-on-rails means every road user follows its logged trajectory, producing a conservative replay used for validation.
表征预测
The world model learns compact representations of sensory inputs through autoencoding24 and enables planning by predicting future representations and rewards for potential actions.
学习环境模型，通过想象未来情景改进智能体行为。
Dreamer learns a model of the environment and improves its behaviour by imagining future scenarios.
感官观测及动作。
The world model encodes sensory inputs xt using the encoder (enc) into discrete representations zt that are predicted by a sequence model with recurrent state ht given actions at.
未来内部表征和奖励。
enables planning by predicting future representations and rewards for potential actions.
是
the world model predicts the outcomes of potential actions
是
Source code for Dreamer and all presented experiments is available under an open-source license at https://github.com/danijar/dreamerv3.
表征预测
DINO-WM models the world dynamics on compact embeddings of the world, rather than the raw observations themselves.
将目标特征作为预测目标，通过优化动作序列进行视觉目标规划。
This allows DINO-WM to achieve observational goals through action sequence optimization, facilitating task-agnostic planning by treating goal features as prediction targets.
视觉表征和动作。
Given these visual embeddings and actions, DINO-WM uses the ViT architecture to predict future embeddings.
未来视觉 patch 特征。
DINO-WM leverages spatial patch features pre-trained with DINOv2, enabling it to learn from offline behavioral trajectories by predicting future patch features.
是
Given these visual embeddings and actions, DINO-WM uses the ViT architecture to predict future embeddings.
开放权重
We have uploaded our trained world model checkpoints for PointMaze, PushT, and Wall here under checkpoints.
表征预测
A recurrent neural network predicts the sequence of these representations given actions.
在实体机器人上学习世界模型和行为策略，不依赖模拟器。
DayDreamer learns a world model and an actor critic behavior to train robots from small amounts of experience in the real world, without using simulators.
本体感知和视觉观测。
To learn from proprioceptive and visual inputs alike, the world model fuses the sensory inputs of the same time step together into a compact discrete representation.
动作条件下预测的内部表征序列。
A recurrent neural network predicts the sequence of these representations given actions.
是
A recurrent neural network predicts the sequence of these representations given actions.
长时间在硬件上学习会造成磨损，可能需要人工干预或维修。
learning on hardware over many hours creates wear on robots that may require human intervention or repair.
视觉生成
We train a diffusion model to predict the next frame of the game.
通过扩散世界模型预测游戏下一帧，并在模型中训练强化学习智能体。
DIAMOND (DIffusion As a Model Of eNvironment Dreams) is a reinforcement learning agent trained entirely in a diffusion world model.
历史游戏帧和智能体动作。
The diffusion model takes into account the agent’s action and the previous frames to simulate the environment response.
下一游戏画面帧。
We train a diffusion model to predict the next frame of the game.
是
The diffusion world model takes into account the agent's action and previous frames to generate the next frame.
开放权重
Quick install to try our pretrained world models using miniconda:
交互世界
Ctrl-World is designed for policy-in-the-loop rollouts with generalist robot policies.
可与 VLA 策略共同在想象空间中滚动生成的动作条件世界模型。
Ctrl-World is an action-conditioned world model compatible with modern VLA policies and enables policy-in-the-loop rollouts entirely in imagination
初始画面和动作序列。
All videos are generated by passing in the initial frame and a different sequences of actions as input.
联合预测的多视角视频，包括腕部视角。
It generates joint multi-view predictions (including wrist views), enforces fine-grained action control via frame-level conditioning,
是
enforces fine-grained action control via frame-level conditioning
开放权重
This repo include the Ctrl-World model checkpoint trained on opensourced DROID dataset (~95k trajectories, 564 scenes).
模型对物理交互的建模仍可能不精确。
The model can evaluate a policy’s instruction-following capability but also can be imprecise in modeling physical interactions.
交互世界
WHAM can be used in world-model mode (predicting how the game evolves), action-model mode (generating plausible controller actions), or both.
联合建模游戏画面与控制器动作，可生成画面、动作或二者的序列。
WHAM can be used in world-model mode (predicting how the game evolves), action-model mode (generating plausible controller actions), or both.
游戏图像和控制器动作条件。
a predictive model that can be conditioned on (modified) images and/or controller actions.
预测的游戏画面和控制器动作序列，依具体模式而定。
WHAM was trained on human gameplay data to predict game visuals (‘frames’) and players’ controller actions
是
a predictive model that can be conditioned on (modified) images and/or controller actions.
开放权重
We provide model weights, sample data and the WHAM Demonstrator to support interpreting, validating and extending our results.
论文的游戏案例聚焦单一 3D 电子游戏。
Extrapolating from our use case focusing on a single 3D video game,
交互世界
Execute multi-modal actions (steering angles, speeds, commands, trajectories, goal points).
面向驾驶、强调通用性与多种控制条件的世界模型。
Vista: A Generalizable Driving World Model with High Fidelity and Versatile Controllability
历史帧与高层意图，包括指令和目标点。
historical frames as priors for coherent long-horizon rollouts. For action controllability, we incorporate a versatile set of controls from high-level intentions (command, goal point)
未来驾驶场景的视觉预测。
Simulated futures in a wide range of driving scenarios by Vista.
是
Execute multi-modal actions (steering angles, speeds, commands, trajectories, goal points).
开放权重
[2024/06/06] We released the model weights v1.0 at Hugging Face and Google Drive.
交互世界
Atlas takes one or more reference images and generates new views at any camera position and angle you specify.
以共享空间上下文生成多模态内容的世界模型。
Atlas is a multimodal autoregressive diffusion transformer. Its inputs are grounded in 3D space to form a spatial context, and it generates multimodal outputs conditioned on its context.
文本、图像、相机位姿和三维深度图；视频表示为图像序列。
At present it can operate on text, images, camera poses, and 3D depth maps; videos are represented as sequences of images.
图像帧与三维深度信息，可组成点云或 Gaussian splats。
Atlas natively operates on both 2D image frames and 3D depth maps, enabling it to output worlds as point clouds or 3D Gaussian splats.
有限开放
Atlas is entering early access with select partners.
当前 Marble 产品
Atlas will power future versions of Marble and other products from World Labs.
表征预测
The predictor models the environment dynamics in latent space by predicting the embedding of the next frame observation
从像素端到端训练的 JEPA 世界模型。
we introduce LeWorldModel (LeWM), the first JEPA that trains stably end-to-end from raw pixels using only two loss terms
当前图像的潜在表征和动作。
given the latent embedding \(\vz_t\) and an action \(\va_t\).
下一帧观测的潜在表征。
The predictor models the environment dynamics in latent space by predicting the embedding of the next frame observation
是
given the latent embedding \(\vz_t\) and an action \(\va_t\).
开放权重
Pretrained LeWM checkpoints for each environment are mirrored on the Hugging Face Hub (model repos), alongside the datasets (dataset repos) in the same collection:
表征预测
All levels predict future latent states from past latent states and actions.
在各层潜在空间中，用历史状态与动作预测不同时间尺度的未来。
All levels predict future latent states from past latent states and actions.
过去的潜在状态与动作。
All levels predict future latent states from past latent states and actions.
各层的未来潜在状态。
All levels predict future latent states from past latent states and actions.
是
All levels predict future latent states from past latent states and actions.
开放权重
Trained models of H-JEPA
视觉生成
visual prediction
联合学习语义理解、未来视觉预测与动作生成。
The hybrid model jointly supports semantic understanding, visual prediction, and action generation.
当前观测、本体感知状态与任务指令。
Given the current observation $o_{t}$, proprioceptive state $s_{t}$, and task instruction $I$
语言输出、未来观测与动作。
the joint distribution of language outputs, future observations, and actions
表征预测
Delta-LTM predicts the action-conditioned latent increment
预测动作引起的潜在状态增量，再更新状态并解码未来画面。
Delta-LTM predicts the action-conditioned latent increment
初始视觉上下文与机器人动作序列。
an initial visual context and a sequence of robot actions
解码后的未来 RGB 观测。
Each predicted latent state is decoded by the frozen decoder to generate the corresponding RGB observation
是
Delta-LTM predicts the action-conditioned latent increment
表征预测
future occupancy and dynamic flow explicitly conditioned on the selected trajectory and action sequence
以候选行驶轨迹与动作预测未来交通占据和动态流。
future occupancy and dynamic flow explicitly conditioned on the selected trajectory and action sequence
车路协同场景表征、所选轨迹与动作序列。
future occupancy and dynamic flow explicitly conditioned on the selected trajectory and action sequence
未来交通占据与动态流。
future occupancy and dynamic flow explicitly conditioned on the selected trajectory and action sequence
是
future occupancy and dynamic flow explicitly conditioned on the selected trajectory and action sequence
交互世界
The model supports both first-person and third-person navigation
由文本动作与连续相机输入驱动的自回归视听世界模型。
GWM Worlds 2 is an autoregressive diffusion video and audio model
场景初始提示与首帧、当前相机和文本动作、历史生成帧。
a global context (the genesis prompt and first frame), the current frame's inputs (camera input and the text actions that span the current frame) and the past generated frames
逐步生成的视频与音频。
GWM Worlds 2 is an autoregressive diffusion video and audio model
是
camera input and the text actions that span the current frame
快速相机旋转可能使细节、纹理和几何退化；长时程记忆仍不完善。
Difficult camera inputs such as very quick rotations can cause details, textures and geometry to degrade. Long-term memory is also imperfect
交互世界
viewpoint control is provided entirely through the re-rendered spatial condition
以流式 3D 点缓存和重新渲染的空间条件生成视角可控视频。
streaming 3D point-cache renderer
经重新渲染的空间记忆与运动感知视觉条件。
viewpoint control is provided entirely through the re-rendered spatial condition
按视角条件生成的视频。
temporal statistics consistent with those of the generated video
开放权重
Pretrained weights v1.1
交互世界
external, camera-indexed world state bank
借助外部世界状态库，持续生成受相机和文本条件控制的视频。
external, camera-indexed world state bank
时间变化的文本、相机轨迹与投影历史。
timed text schedule changes while the camera trajectory, warp history, and random seed remain fixed
交互生成的视频。
interactive world model that generates 384 × 640 @ 24 fps video
开放权重
Model weights for EVOKE
固定预算的循环结构本身不能保证长时程内容一致。
does not by itself ensure long-range content consistency
交互世界
The target trajectory is still supplied by the user.
按用户提供的目标相机轨迹，从图像或视频生成视图视频。
The target trajectory is still supplied by the user.
单张图像、四张图像或视频，以及用户提供的目标相机轨迹。
a single image, four images, or a video
源画面、渲染画面、遮罩与预测视频文件。
{source,render,mask,pred}.mp4
开放权重
Default v2v checkpoint — 1.3B
网页环境
Solaris generates the interface one frame at a time
把用户交互作为条件，逐帧生成视觉界面。
Solaris generates the interface one frame at a time
用户点击、拖拽等已发生的交互。
The model observes clicks, drags and other interactions as it generates
逐帧生成的视觉界面。
Solaris generates the interface one frame at a time
是
The model observes clicks, drags and other interactions as it generates
需要申请
Fill out the form below to request early access.
稳定、可读的文字仍是视频界面生成的困难。
Stable, legible text remains one of the hardest problems in video generation
交互世界
We build Odyssey-3 as a multi-step video diffusion transformer
以历史观察、提示和动作持续预测可交互的视觉环境。
training it to continue from preceding observations and predict future states conditioned on action inputs.
历史观察、文本提示和动作输入；预览支持第一/第三人称导航与独立相机移动。
Today’s preview provides first-person and third-person navigation alongside independent camera movement
随交互持续生成的未来环境视频。
We build Odyssey-3 as a multi-step video diffusion transformer
是
predict future states conditioned on action inputs.
有限开放
You can try Odyssey-3 in our research preview
视觉生成
physically plausible video prediction.
以机器人动作轨迹为条件，预测多视角未来视频。
we render action trajectories into image-space conditions
初始场景与机器人动作条件；轨迹经几何校准后渲染到图像空间。
Predictions from the same initial scene under multiple action conditions.
多视角未来机器人操作视频。
a multi-view, cross-embodiment robot world model for action-faithful and physically plausible video prediction.
是
we render action trajectories into image-space conditions
动作表示、生成器和奖励模型均在图像空间工作；遮挡或视角不足时，可能生成或奖励看似合理但物理错误的交互。
under occlusion or limited views, the framework may generate or reward visually plausible but physically incorrect interactions.
表征预测
A latent world model then predicts this planning-shaped future latent representation
从历史观察预测有助于驾驶规划的未来表征，再生成并选择轨迹。
The predicted future latent representation serves as planning context and guides trajectory generation and selection.
历史多视角相机观察、自车状态与导航信息。
denote the historical multi-view camera observations, $c_{t-n:t}$ the corresponding ego states and navigation information
面向规划的未来潜在表征，以及由其引导生成和选择的自车轨迹。
The predicted future latent representation serves as planning context and guides trajectory generation and selection.
视觉生成
The video expert $\theta$ predicts $K_{v}$ future latent steps
把长视觉历史与未来视频预测结合，用于生成机器人动作块。
The resulting policy conditions actions on both observed history and anticipated futures
观测视频历史、机器人状态与语言任务指令；具体格式随变体而定。
observed video latents, $q_{t}$ the robot state, $c$ the language instruction
未来视频潜变量与机器人动作块；IDM 先预测视频，再生成动作。
it predicts future visual latents independently of action tokens, then denoises actions conditioned on those predictions
开放权重
Long-WAM policy weights for LIBERO.
表征预测
All outputs within a span are directly supervised by their corresponding future latent targets
在有限跨度内并行预测有因果关联的未来表征轨迹，用于动作规划。
predicts a causal finite-horizon trajectory in parallel, allowing interaction among future representations
近期潜在状态与已执行动作组成的上下文，以及待预测的动作序列。
Given recent context and an action span
有限跨度内的多步未来潜在表征轨迹。
PPWM instead performs causal interaction among future hidden representations and decodes each future latent only after this interaction.
是
Given recent context and an action span
表征预测
model dynamics in a compact VLM visual-token space
在策略使用的 VLM 视觉 token 空间中，按动作预测未来状态。
an autoregressive action-conditioned world-model simulator that rolls out in compact policy-aligned VLM token space.
压缩的 VLM 视觉 token、本体感知与机器人动作历史。
Compact visual tokens, proprioception, and actions are projected into a shared model space
未来压缩 token 状态，可映射回策略使用的 VLM 视觉表征。
Future compact tokens are generated with a sliding temporal window and recurrent state.
是
conditioned on the current history and action.
交互世界
predicts the next synchronized audio-video segment from available history and live control.
以历史状态和实时多模态输入，持续生成同步音视频环境。
predicts the next synchronized audio-video segment from available history and live control.
历史环境状态，以及实时文本、参考材料、音频、WASD 等动作输入。
Four types of input enter one running world: text prompts, multimodal references, audio, and actions such as WASD or continuous controls.
连续更新的同步视频与音频片段。
predicts the next synchronized audio-video segment from available history and live control.
是
They can arrive while the model is generating, alter the next world state, and drive synchronized video and audio.
保留事件日期原有精度；只有年份时，不补造月日。
PlanWAM
Odyssey-3
DreamTrue
Long-WAM
Token-World
H-JEPA
DeltaWorld
V2X-WAM
GWM Worlds 2
Atlas（World Labs / 2026）
Solaris
PixVerse R2
Alaya-EVOKE
AlayaWorld v1.1
GAIA-4
Cosmos 3
Waymo World Model
LeWorldModel（LeWM / 2026 研究）
Ctrl-World（ICLR 2026）
GWM-1
Marble
DINO-WM（ICML 2025）
DreamerV3（Nature 2025）
WHAM（Nature 2025）
V-JEPA 2-AC
Genie 3
Vista（NeurIPS 2024）
Sora（2024 技术报告）
DIAMOND（Atari / NeurIPS 2024）
Copilot4D
DayDreamer（CoRL 2022）
World Models
说明更新于 2026-09-23 · 这是使用范围说明，尚未指定统一开源许可证。
本站提供结构化记录的在线查阅与文件下载，方便核对来源和进行研究。公开可访问不等于对网站全部内容授予统一的开源许可。
本站尚未选定统一的数据再利用许可证，商业复用的授权范围也尚未统一确定。涉及本站有权授权的文字表达或编排的转载、改编、再分发时，请先确认适用授权；本说明不限制客观事实的使用，也不影响法律允许的其他使用。
引用本站整理成果时，请标明“世界模型图谱”、相应页面链接和访问日期；使用下载文件时，建议同时记录文件版本标识。事实依据仍应链接对应的原始材料。
如对记录作了筛选、修改或重新发布，请明确标注改动及其时间，不要把修改后的内容呈现为本站原版或本站认可的结论。这里的引用建议不扩大本站对客观事实或第三方材料享有的权利。
厂商原句、论文摘录、第三方图片及其他外部材料的权利归相应权利人所有，不包含在本站可能授予的原创整理成果许可中。再次使用时，应单独核实相关权限；来源署名本身不替代授权判断。
站名、Logo 和品牌标识不随数据下载获得使用授权。引用或复用资料不代表与本站存在合作、认证或背书关系。原创文章、配图和网站代码的许可也不因数据公开而自动改变。
这些记录是基于所列材料的持续整理，可能存在遗漏、错误或过时内容。来源阅读日期不等于模型发布日期，也不代表我们在今天实测过相关服务。请在作出判断前回到原始材料核对。
发现问题时，请记录页面或模型名称、具体字段、原始来源链接和建议修改的内容，方便复核。
页面或模型名称：
需要核对的字段：
原始来源链接与阅读日期：
问题描述及建议修改：
纠错与授权咨询：[email protected]