Speaker: Nikolaus Salvatore

Bio

Location : CoRE 305

Committee: 

Assistant Professor Qiong Zhang (Chair)

Assistant Professor Hao Wang

Assistant Ruixiang Tang

Professor Lirong Xia

Event Type: Qualifying Exam

Abstract: We introduce a cognitively grounded framework for understanding and improving memory in neural language models. First, we present a mechanistic connection between attention-based sequence-to-sequence models and cognitive models of human memory search. This work maps components of neural machine translation architectures onto context-based memory models and shows that learned attention mechanisms can support human-like retrieval behavior in free recall, including systematic effects of context and item history. Second, we study the “lost-in-the-middle” phenomenon in long-context language models, in which models retrieve information more reliably from the beginning and end of a context than from the middle. Through experiments on language models trained on tasks inspired by human short-term and long-term memory paradigms, we show that this U-shaped positional bias can emerge from competing retrieval demands, autoregressive model structure, and learned attention dynamics rather than from context length alone. Together, these results suggest that cognitive memory theory can help explain when and why neural language models succeed or fail at retrieval. Building on this foundation, our ongoing work investigates how memory-augmented LLM architectures can incorporate cognitive principles such as hierarchical organization, episodic structure, and memory encoding and retrieval. This research aims to develop more structured, adaptive, and reliable model memory mechanisms for long-context reasoning and retrieval-augmented generation.

Organization: 

Contact  Assistant Professor Qiong Zhang

Zoom Link: https://rutgers.zoom.us/j/92485197268?pwd=KnaZoXphrU9FQv3fL2zCBaKKE19VL1.1