Although current digital video processing systems have developed rapidly, they still show obvious "lack of understanding" when facing complex spatial movements and physical laws. They may be able to describe the content of the scene, but they struggle to answer questions involving detailed physical logic, such as "whether the red car passed through the intersection before the blue car turned" or "where is the highest point of the ball's trajectory."
The root cause lies in the extreme scarcity of high-quality motion reference data. Existing reference information is limited in scale and highly dependent on costly manual identification, making it difficult to support computational systems in learning fine-grained physical motion in the real world. To address this challenge, a research team from institutions such as MIT, NVIDIA, and the University of California, Berkeley, has proposed FoundationMotion: an automated data generation pipeline that requires no human involvement.
The workflow of this pipeline operates like a fully automated "motion data factory," consisting of three main stages:
Track Extraction: The system uses advanced object tracking technology to convert objects such as pedestrians, vehicles, or robotic arms in videos into continuous spatiotemporal trajectory coordinates.
Semantic Conversion: It transforms abstract coordinate numbers into structured textual descriptions, combining video frame information to provide the system with a detailed "motion manual."
Automatic Quality Inspection and Generation: Finally, through logical integration, it generates refined question-and-answer data that includes speed, direction, temporal relationships, and spatial positions.
Surprisingly, experimental results show that after optimizing with data generated by this pipeline alone, a video analysis system with 15 billion parameters achieved an accuracy rate of 90.6% on motion understanding tasks. This performance not only surpasses a large open-source architecture with 72 billion parameters but also outperforms mainstream commercial closed-source systems in the market.
Researchers point out that this improvement is entirely attributed to the purity and accuracy of the data, proving that systems can develop intuition about the physical world through massive, high-quality automated data training in fields such as autonomous driving and robot collaboration. This marks a crucial step forward in the journey toward embodied technologies with "physical common sense."
