Ученые Цинхуа представили систему SAEVerbalizer для автоматической расшифровки миллионов функций нейросетей

Researchers at Tsinghua University have posted their preprint on arXiv addressing one of AI interpretability's most persistent scaling problems: the inability to label millions of sparse autoencoder features fast enough to make systematic model audits practical. Their system, SAEVerbalizer, bypasses the standard approach of collecting and summarizing text examples by instead injecting SAE decoder directions — the raw geometric vectors that encode what a feature represents — directly into a language model's internal representations. The paper, submitted to arXiv on August 13, 2026 by Weihan Meng, Hongzhu Guo, Yi Jing, Dewen Liu, Zijun Yao, Xiaozhi Wang, Lei Hou, and Juanzi Li, arrives as mechanistic interpretability has achieved a level of mainstream recognition that makes the field's bottlenecks newly consequential: MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies for 2026. Simultaneously, the commercial interpretability sector has attracted serious investment, with Goodfire — a lab whose work depends on automated feature analysis — raising $150 million Series B at a $1.25 billion valuation in February 2026. Why Labeling Millions of Features Is a Bottleneck Sparse autoencoders have become the dominant instrument in mechanistic interpretability over the past two years. The core idea is that language models represent knowledge not as discrete facts stored in individual neurons, but as distributed patterns across millions of parameters — patterns where a single neuron frequently responds to multiple unrelated concepts simultaneously, a phenomenon called polysemanticity. SAEs address this by re-expressing a model's internal activations as a much larger dictionary of "features," each of which tends to activate sparsely and represent a single, coherent concept. Anthropic's interpretability team demonstrated this vividly in work on Claude 3 Sonnet, finding features ranging from a "Golden Gate Bridge" feature and a "code error" feature to more abstract ones for "sycophantic praise" and "gender bias awareness." Google DeepMind released Gemma Scope, a publicly available suite of SAEs trained on the Gemma model family, contributing millions of feature vectors to the research community. Read more: New Study Delves Into World of Artificial Neural Networks, Unraveling How AI Brains Learn The problem is that each of those features is, by itself, just a direction — a vector of numbers in high-dimensional space. To know what a feature means, researchers have traditionally had to collect the text passages that most strongly activate it, read through those examples, and write a description by hand. When a single SAE dictionary for a large model can contain tens of thousands to millions of features, that process is neither consistent nor remotely scalable. How SAEVerbalizer Works SAEVerbalizer takes a different approach, working from the inside of the model rather than from observations of its behavior. An SAE produces, for each of its learned features, a decoder direction — a vector in the language model's activation space representing the concept that feature encodes. SAEVerbalizer injects these vectors directly into the model's representations during a fine-tuning process that teaches the model's downstream layers to generate natural-language descriptions of the injected geometric state. The key distinction from prior methods is what the system does not require at inference time: no text collection, no external LLM prompting, and no sampling of max-activating examples. Once the verbalizer is trained, it reads decoder directions and produces explanations directly — treating the SAE's geometry as the language it was trained to translate. This contrasts with what the authors call "external observation" approaches, which include both the original OpenAI AutoInterp method (collect max-activating texts → summarize with an external LLM) and Neuronpedia's current infrastructure, which hosts browsable feature explanations generated through similar behavioral prompting pipelines. A third approach, output-side analysis, examines what tokens or concepts a feature promotes in the model's predictions rather than what inputs activate it. Both directions share a structural limitation: they treat the model as a black box and reconstruct meaning from behavioral evidence, a process that is computationally expensive and, critics have argued, prone to superficiality. Anthropic published a competing Natural Language Autoencoders architecture in May 2026, which jointly trains an activation verbalizer and an activation reconstructor using reinforcement learning to reconstruct residual stream activations. Anthropic released training code and trained models for open-source use with that publication. SAEVerbalizer's design point differs: rather than training two components jointly from scratch, it fine-tunes a model's downstream layers to respond to SAE-derived decoder directions — a distinction that allows it to generalize across separately trained SAE dictionaries without full retraining. Generalization Is the Real Question A system that learns to describe features it has already seen during training would be a moderate improvement on existing pipelines. What would make SAEVerbalizer more significant is its ability to generalize — to describe features it has never encountered, features produced by different SAE training runs, and features from entirely different language models. The paper reports that the learned verbalization capability generalizes to unseen features, transfers across separately trained SAE dictionaries applied to the same model, and extends to SAE features from different LLMs using a lightweight adapter. The cross-dictionary transfer result matters because it would mean that a single trained verbalizer does not need to be retrained each time researchers produce a new set of SAE features — which happens constantly across the field as labs experiment with different architectures, layer choices, and training hyperparameters. If this generalization holds across a wider range of models and SAE configurations than those tested in the paper, it raises the possibility of a shared community labeling infrastructure — a trained verbalizer that researchers across different labs could use as a common tool, much as Neuronpedia serves as shared infrastructure for hosting features. That would represent a structural change in how interpretability research scales: from a per-project effort to a shared resource. The paper also includes intervention experiments specifically designed to probe whether the system has genuinely internalized the geometry of feature space or is simply producing plausible-sounding descriptions based on statistical patterns. Injecting multiple decoder directions simultaneously produces explanations that combine the meanings of those features, and reversing a direction produces corresponding shifts in the generated description. This compositionality and directionality in the outputs is encouraging evidence that the verbalizer is tracking geometric structure rather than pattern-matching on surface statistics. A Live Methodological Debate SAEVerbalizer enters a field that is actively debating whether automatic feature explanation methods actually explain what they claim to. A May 2026 critique paper titled "Do Activation Verbalization Methods Convey Privileged Information?" raises a fundamental concern: verbalization methods may generate confident, coherent descriptions that reflect the verbalizer model's own parametric knowledge rather than the target model's internal representations. If both the verbalizer and the target model were trained on similar data, the verbalizer can "explain" a feature correctly because it shares relevant knowledge — not because it has genuinely read the target model's geometry. SAEVerbalizer's intervention experiments are specifically designed to address this concern, but the authors do not claim the concern is fully resolved. A separate structural problem was documented in a May 2026 descriptive collision paper — the finding that in the largest publicly available dataset of human-annotated SAE features, 82.1% of features share their annotation with at least one other feature, and the most common annotation string ("plural nouns") was applied to 101 distinct features spanning 18 layers and four model components. The practical consequence: even if SAEVerbalizer generates accurate descriptions, those descriptions may not be discriminative — may not distinguish a feature from its neighbors in a way that is useful for targeted behavioral analysis or model editing. SAEExplainer published June 2026 by a separate team, addressed a similar bottleneck through a different mechanism: using activation scores as a reward signal to train a self-correcting model through iterative bootstrapping over two optimization rounds. These parallel efforts illustrate how quickly the auto-interpretability problem has attracted competing approaches, each with different architectural assumptions and generalization properties. What This Means For AI Safety and Compliance Mechanistic interpretability is not primarily a research-productivity problem. Its practical stakes are tied directly to AI safety: the ability to systematically audit what large language models have learned, detect misaligned or dangerous concepts, and verify that behavioral steering interventions have produced the intended effects rather than compensating workarounds. Those audits require readable features. A SAE dictionary with millions of features and no human-comprehensible labels is, for safety purposes, not much more useful than the raw activations the dictionary compressed. The EU AI Act, whose Article 13 transparency requirements for high-risk AI systems are phasing in from August 2026 through December 2027, creates regulatory pressure for enterprises deploying AI in high-risk domains to demonstrate that they can interpret and explain their systems' outputs. Automated feature labeling at scale is directly relevant to that compliance infrastructure. Goodfire, which has built a commercial "model design environment" on top of interpretability techniques, reached a $1.25 billion valuation in February 2026 after raising $150 million from B Capital, Salesforce Ventures, and investors including former Google CEO Eric Schmidt. The company's core value proposition depends on being able to identify, explain, and intervene on specific model features — work that currently requires expensive human annotation at scale. SAEVerbalizer's preprint does not include released code or trained models. The paper is not yet peer-reviewed, and its generalization claims have not been independently replicated across the wider range of models and SAE configurations the field has developed. Community evaluation of those claims is likely to follow quickly, given the pace at which the interpretability research community typically moves. Can a Model Learn to Describe Its Own Geometry? The conceptual question SAEVerbalizer tests — whether a model can be trained to read and describe the geometric structure of its own learned representations — has implications beyond the SAE labeling problem. If the answer is robustly yes, it suggests that the geometry of a trained model's activation space is itself interpretable to that model, in a way that can be extracted through targeted fine-tuning. That would open a path toward models that can participate in their own auditing, generating explanations of their internals that human researchers can then evaluate and challenge. If the answer is more qualified — if the intervention experiments show compositionality and directionality in controlled conditions, but generalization degrades as models and SAE configurations diverge — it would suggest that representation verbalization is a useful tool for specific contexts rather than a general-purpose auditing mechanism. The paper is a preprint. The community will provide the answer. Frequently Asked Questions How is SAEVerbalizer different from existing tools like Neuronpedia that already label AI features? Neuronpedia and similar platforms generate explanations by collecting text examples that strongly activate each feature, then prompting an external language model to summarize those examples — a process that is computationally expensive and must be repeated for every feature in every new SAE dictionary. SAEVerbalizer bypasses text collection entirely: it injects the geometric vector that represents a feature directly into the model's internal state, then reads out the description the model generates from that state. The practical consequence is that once the verbalizer is trained, labeling new features requires no additional data collection — and the paper's generalization experiments suggest the approach may transfer across different SAE training runs and different language models without full retraining. Whether those claims hold across the broader ecosystem of open SAE dictionaries is the key question for community evaluation. What is mechanistic interpretability, and why does the ability to label features matter for AI safety? Mechanistic interpretability is the effort to understand AI models not just by observing what they output, but by identifying the internal structures — features, circuits, and representations — that determine their behavior. Sparse autoencoders are the current leading tool for this: they decompose a model's internal activations into a large dictionary of "features," each representing a specific concept the model has learned. But those features are geometric objects (vectors), not text — and without human-readable labels, researchers cannot systematically search for dangerous concepts, verify that behavioral interventions worked, or audit a model before deployment. MIT Technology Review named mechanistic interpretability one of its 10 Breakthrough Technologies for 2026, reflecting how central this capability has become to AI safety research. Is SAEVerbalizer ready to use in production research pipelines? Not yet. At the time of posting, the paper was an unreviewed preprint, and neither code nor trained models were publicly released. The generalization claims — that the approach transfers across separately trained SAE dictionaries and across different language models — are encouraging but have not been independently replicated. Researchers interested in automated SAE feature labeling have several options available now, including Neuronpedia's existing AutoInterp infrastructure, Anthropic's Natural Language Autoencoders (released with code in May 2026), and SAEExplainer (released June 2026). SAEVerbalizer's architecture appears meaningfully distinct from all three — but "distinct" and "better in practice" are questions community evaluation will need to answer. Could SAEVerbalizer generate incorrect or misleading feature descriptions? Yes, and this is an active concern in the research community. A May 2026 paper raised the specific worry that verbalization methods may produce accurate-sounding descriptions that reflect the verbalizer model's own knowledge rather than genuinely reading the target model's internal state — meaning the system could produce confident labels that are wrong about what the feature actually represents. A separate study found that in existing annotated SAE feature datasets, most features share their label with at least one other feature, suggesting that even accurate descriptions may fail to discriminate between distinct concepts. SAEVerbalizer's intervention experiments are designed to probe for this failure mode, but the authors do not claim to have resolved it definitively. ⓒ 2026 TECHTIMES.com All rights reserved. Do not reproduce without permission.