As AI continues to evolve toward multimodality and long-term capabilities, how to free agents from the trap of “forgetting as soon as they see something” or inefficient retrieval has long been a core pain point tackled by both academia and industry. Recently, a team from Zhejiang University, in collaboration with Lenovo, proposed a new event-centric multimodal long-term memory solution—EM²Mem. The related research has officially debuted at EMNLP2026.
Disrupting the Traditional Paradigm: From “Retrieve First, Align Later” to “Align First, Retrieve Later”
For a long time, existing multimodal agents have often adopted relatively simplified storage and retrieval methods when handling long-term memory. This traditional paradigm usually stores various types of information in a fragmented way, making it difficult for agents to effectively connect clues from different dimensions when facing complex problems.
To address this pain point, EM²Mem introduces a fundamental disruption in underlying logic: it innovatively shifts the traditional “retrieve first, align later” paradigm to “align first, retrieve later.” At the initial stage of memory construction, the solution directly links diverse clues such as people, objects, actions, and time within the same event, thereby building multi-scale event archives. On this basis, through the organic combination of episodic graphs and semantic graphs, it further connects different experiences and ultimately distills long-term patterns.
Significantly Improving Recall and Efficiency: Building the Multimodal Agent Memory System LightMem-Ego
Judging from actual testing and application results, EM²Mem demonstrates multidimensional performance advantages:
Both accuracy and recall increase: Experimental data show that, compared with existing conventional methods, EM²Mem achieves significant improvements in event-level evidence recall and question-answering accuracy.
Reduced time and compute consumption: The solution can greatly reduce the time and token usage of a single question-answering interaction. Since the memory can be reused multiple times after construction, its overall efficiency advantage becomes even more evident over long-term amortized use.
Supported by this core technology, the research team further built the multimodal agent memory system LightMem-Ego. In the future, this system can be widely deployed in diverse scenarios such as personal life assistants and scientific laboratory assistants, providing solid underlying support for the continuous learning and personalized development of multimodal agents.
