Pangram verdict · v3.3
We believe that this document is primarily human-written, with a small amount of AI-assisted content detected
AI likelihood · overall
HumanArticle text · 1,411 words · 4 segments analyzed
Abstract Although transformers in large language models (LLMs) effectively implement a self-attention mechanism that has revolutionized natural language processing, they lack an explicit architecture for the executive control of attention found in humans, which is essential for resolving conflicts and selecting relevant information in the presence of competing computations and is critical for adaptive behavior. To investigate the impact of this limitation in LLMs, we employed the classic color Stroop task, widely regarded as the gold standard, to test the executive control of attention in these models. Our results revealed a typical conflict effect of underperformance in terms of accuracy in the incongruent condition (e.g. naming the color of the word RED in blue) compared with the congruent condition (e.g. naming the color of the word RED in red), in short word lists, similar to human performance. However, as the length of the word lists increased, performance on the incongruent condition degraded toward near-total performance collapse, even as accuracy in the congruent condition remained excellent, and word reading (e.g. reading the word RED [in red] or RED [in blue], ignoring the color) was near-perfect. These findings demonstrate that transformer attention mechanisms are fundamentally limited in their capacity for conflict resolution across extended contexts, and a failure to up-regulate control adaptively under rising interference. We suggest that incorporating executive control mechanisms akin to those in biological attention is crucial for achieving artificial general intelligence. Significance StatementWhile transformer-based large language models (LLMs) have achieved remarkable advances by implementing attentional mechanisms, it remains unclear whether these artificial systems capture the core mechanism of executive control of attention. Employing the classic color Stroop task, we demonstrated that state-of-the-art LLMs exhibit dramatic performance degradation in naming the color of a word under the incongruent condition (e.g. word RED in blue) as word list length increases. This study reveals a critical deficiency in conflict resolution within transformer attention architectures, potentially constraining AI's capacity to develop adaptive behavior. Introduction The introduction of the transformer model in their paper “Attention is All You Need” (1) marked a pivotal turning point in the accelerating progress of machine learning (ML) and AI.
Generalizing the mechanism that allowed neural translation models to selectively weight relevant input positions (2), the transformer, which relies on an attention mechanism rather than recurrent or convolutional layers to uncover essential spatiotemporal structures in input data, has become the foundation for state-of-the-art natural language processing (NLP) models. Since then, the transformer attention mechanism has been applied to various multimodal and large language models (LLMs) in tasks beyond NLP, including computer vision, speech recognition, and even video generation (3, 4). The revolutionary success of transformer-based LLMs naturally invites a critical question regarding the role of attention in ML and AI, including its nature, boundaries, and limitations. In particular, how transformer-based machine attention differs from human attention, a fundamental concept in psychology that has been actively studied, remains elusive.More than a century ago, William James famously stated that “Everyone knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought” (5). Despite this assertion, there is ongoing debate about how little we know of attention (6–8). Attention's explanatory power stems from its ability to serve as a universal explanation for understanding value, selection, and human agency (9). Modern neuroscience views attention not as a single straightforward mechanism but as a complex mental faculty supported by multiple neural networks. According to the attention network theory (ANT), there are three distinct attentional functions of alerting, orienting, and executive control, each of which is supported by a distinct and interactive neural network (10–13).The three networks are functionally integrated and interact to selectively focus cognitive resources on specific stimuli, tasks, or information while filtering out irrelevant inputs. The alerting network achieves and maintains an alert state, with brain areas such as the locus coeruleus involved. The orienting network selects specific information from sensory input, directing attention to a particular stimulus or location, with key areas including the thalamus and the superior colliculus, frontal eye fields, and parietal lobe for visual input. The executive control network is engaged in tasks requiring control over thoughts and actions amid competing information and is essential for adaptive behavior, recruiting regions of the frontoparietal network, including the anterior cingulate cortex (ACC), the anterior insular cortex, and the dorsolateral prefrontal cortex (DLPFC) (14, 15).
Behavioral and functional neuroimaging studies have demonstrated that these three networks function both independently and interactively (10, 11, 16, 17). The ANT was also computationally validated through connectionist modeling (18).Modern neural network–based AI systems efficiently process large amounts of spatially or temporally distributed input information to make decisions, such as image classification or next-word prediction. The cross-modal attention architecture typically employs a multihead attention mechanism, with some heads focusing on object–word associations while others attend to spatial relationships or abstract concepts (19, 20). This parallel processing enables the model to capture complex relationships among modalities, much as human perception integrates multiple sensory inputs (21). A key advance of the transformer architecture over traditional ML techniques is its attention mechanism, which prioritizes relevant input tokens by assigning weights based on latent semantic information rather than relying solely on spatial or temporal proximity. This selective weighting parallels the biological orienting function, where attentional resources are dynamically allocated to task-relevant content (22). In multihead self-attention, the softmax normalization establishes competitive selection dynamics in which input elements compete for representational resources, with higher-scoring elements receiving proportionally greater weight while suppressing competitors (see Supplementary Introduction for multimodal LLM architecture). This mechanism resembles the biased competition framework (23) observed in human visual attention, in which neural representations compete for processing resources via mutual inhibition.Both biological and artificial systems implement attention through multiplicative scaling and competitive selection, core mechanisms that have been explicitly mapped between artificial and biological attention, including the orienting function that shifts focus through dynamic reweighting (24). Empirical evidence shows that transformer attention weights correlate strongly with human eye-tracking patterns during natural reading (25), suggesting that both systems may converge on similar solutions for selective information routing during language comprehension. The self-attention mechanism's capacity to model long-range dependencies through parallelizable computation has enabled unprecedented scaling, with empirical studies demonstrating predictable power-law improvements in performance as a function of model, dataset, and compute sizes (26, 27). However, a critical distinction is that human attention comprises not only orienting but also executive control, the capacity to maintain task goals, detect and resolve conflict of computation, and override prepotent responses. The softmax mechanism routes information competitively but lacks an architectural analog for executive control.
For transformers, backpropagation distributes gradient signals across millions of parameters without respect to compositional or task-relevant structure, resulting in a fundamental diagnostic limitation. There is no mechanism that evaluates whether current processing remains aligned with an overarching task goal, no conflict signal that triggers compensatory adjustments, and no persistent representation that sustains priority hierarchies across sequential demands. See Supplementary Introduction for rationale and significance of executive control.The Stroop (conflict) effect of the color Stroop task, first documented by John Ridley Stroop in 1935, is a robust psychological phenomenon that demonstrates interference in reaction time and accuracy when there is a mismatch between the physical color of a word and its meaning (28, 29). This interference occurs when individuals must name the color of a word while ignoring its meaning. For example, when participants are asked to name the color of the word “BLUE” printed in red (the incongruent condition), they typically respond more slowly and make more errors than when the word and color match such as “RED” printed in red (the congruent condition), which is called the Stroop or conflict effect. The traditional automaticity explanation (30) posits that the interference results from automatic reading processes that overwhelm controlled color naming and assumes that reading occurs involuntarily, cannot be suppressed, and proceeds independently of task demands (see Supplementary Introduction for challenges to automaticity theory). In the parallel distributed processing (PDP) account of the color Stroop effect (31), “automaticity” is a continuum that emerges through practice while remaining susceptible to top-down modulation; their Stroop model reproduces the graded interference by varying this attentional modulation. In studying biological attention, the conflict effect has been utilized to investigate conflict processing in the human brain (14, 32). Because the Stroop effect varies systematically across conditions and persists even with practice, human performance indicates that it reflects fundamental aspects of executive control of attention.