A research team from Carnegie Mellon University has released an innovative artificial intelligence project called LegoGPT. This model can automatically generate LEGO brick designs based on natural language text. The project is open-source on GitHub, allowing users to freely download the model and dataset for experimentation and extension.
LegoGPT is driven by a self-regressive large language model, trained on more than 47,000 LEGO bricks that form over 28,000 unique 3D objects. Users only need to input a text prompt like "a guitar shape," and the model will generate a structurally sound and stable LEGO building plan.
Its core highlights include the "validity check" and "physics-aware rollback" mechanisms during the building process, ensuring that the generated brick layout does not overlap or float. At the same time, it can also generate structured design diagrams that can be built by humans or robots, along with annotated textual instructions.
The research team also constructed a training dataset called StableText2Lego, using ShapeNetCore meshes and voxel layouts to generate initial shapes, which were then filtered and optimized to form the final training samples. In the future, this technology could be extended into a complete image-to-LEGO design workflow, allowing users to obtain creative building plans by uploading photos.