WorldString
IEI Lab
Actionable World Representation
- 1Tsinghua University — IEI Lab
- 2UC San Diego
- 3CalTech
- 4NVIDIA
♦ Equal contribution
♦ Equal contribution
Real-world capture in the lab: RGB-D sensing and object-centric processing toward keypoint-conditioned WorldString training data.
Each row shows three panels, from left to right: input keypoints, learned token-colored shape, and error map. Drag to rotate and scroll to zoom; panning is disabled. The camera stays centered on the cloud or keypoints. Use Sync other panels to this view in the keypoint panel to copy its camera, frame, and play/pause state to the other two panels in the same row.
Error map: green — TP · red — FP · blue — FN (red/blue are softened toward greener tones in the panel for readability). Keypoints are colored by joint index (HSV) and drawn as shaded spheres; they use the same frame index as the adjacent shape panels.
Training process visualization for the Go2 and H1 models. Test the saved checkpoints on same keypoint states.