hold the arrow keys (or WASD) to drive
The world is a participant: a MuJoCo scene publishing its camera as video and reading a commands input. The pilot is qwen-agent-v1 attached to it: a VLM that looks at the frame twice a second and publishes a command with a reason. Your keys publish to the same input; the world prefers you while your commands are fresh and hands back to the agent a second after you let go. A world model (Cosmos3) would be another participant with the same inputs and outputs.
world = session.add_source(rgb.Node("world", publishes=["video"], subscribes=["commands"]))
pilot = session.attach("qwen-agent-v1", to=world, as_="pilot", params={"prompt": "drive to the red crate"})
@world.inputs.commands.watch
def on_command(publisher, cmd): ... # the agent's, or a person's; the world decides who wins