On September 2, Jakub Pachocki, Chief Scientist at OpenAI, posted on the X platform to respond to the controversy surrounding the AI model Astra being "unmonitored" and having "opaque reasoning."
The Origin of the Controversy: Deep Recurrence May Make the Chain of Thought Harder to Trace
The incident first emerged from a detailed report by The Information, which revealed that OpenAI is testing "deep recurrence" technology, potentially making it harder for researchers to explain the model's "chain of thought" reasoning process.
In terms of technical approaches, mainstream models such as ChatGPT, Claude, and Gemini mostly use the Transformer architecture. These models typically process information in linear steps and can output reasoning processes in natural language in some scenarios, but this does not necessarily accurately reflect the internal computations of the model. "Deep recurrence" allows the model to repeatedly call the same set of network layers, continuously updating internal representations. Compared to traditional linear structures, this approach may cause more of the model's actual reasoning to remain in internal states that are difficult to read directly, making it harder for researchers to trace decision paths using the chain of thought. After the report was published, it quickly became a hot topic in the AI community, with concerns that the implementation of this technology could make external monitoring tools less able to understand the sources of the model's conclusions, reducing transparency.
Pachocki: The Computation Graph Depth Is Less Than Twice That of GPT-4
Pachocki stated in his tweet that he hoped to avoid a "race toward an unmonitored AI state due to confusing media reports." He emphasized that the computation graph depth of OpenAI's current most advanced models (including Astra) is less than twice that of GPT-4, and the model architecture has not seen a jump in complexity as feared by the public.
He also said that since the initial reasoning models were introduced, OpenAI has been committed to maintaining and utilizing the chain of thought monitoring, and places great importance on this technology because it allows the team to understand how model alignment generalizes from the training distribution. However, he also admitted that the chain of thought monitoring is currently very fragile and is moving in a negative direction for reasons unrelated to architectural changes. The team will take measures to strengthen it — this is one of the core goals of the current research project. OpenAI said it will deploy additional chain of thought monitoring for Astra.
Computation graph depth is a key metric for measuring the complexity of neural network architectures, referring to the number of computational layers the model needs to execute sequentially from input to output. The higher the value, the more complex the structure.
