Keynote lectures
Emergent Linguistic Structure in Agent Communication
Why is linguistic structure shaped the way it is? To answer this, we need to disentangle the many forces, cognitive, communicative, and social, that shape languages as they are learned and used. Agent-based simulations offer a powerful way to isolate pieces of this puzzle and can model the spontaneous development of a communication system through repeated interactions between individuals. While this set-up is certainly not new, it is increasingly implemented with deep learning-based neural agents trained via reinforcement learning and this comes with a challenge: these neural agents still often behave differently from human learners and their languages fail to exhibit human-like properties, including a tendency toward destabilizing language drift. I will present recent work drawing on developmental trajectories in human language acquisition, showing that age-based plasticity, where younger agents learn quickly while older agents maintain more stable representations, can substantially reduce drift and help populations sustain a shared language even as new learners are introduced. Directly inspired by human artificial language learning studies, I will also present a set-up in which agents first learn an artificial language and then use it to communicate, with the aim of studying the emergence of a well-known language phenomenon: the word-order/case-marking trade-off. Finally, I will discuss recent results that reveal what a novel artificial language looks like when it has emerged to adapt to preferences of both humans and LLMs in a hybrid language game experiment.
Tessa Verhoef is an assistant professor at the intersection of language, cognition, cultural evolution and computation at the Leiden Institute of Advanced Computer Science of Universiteit Leiden, and director of its Emergent Communication group.
Using Toy Models and Interpretability to Understand How Large Reasoning Models Reason
Large language models have evolved into large reasoning models, and have started to yield revolutionary results in mathematics and computer science. Meanwhile, Interpretability — the field trying to understand model internals — is struggling to keep up. Interpretability remains crucial, however, for ensuring safety, governability and crediting of sources, and new approaches to understanding reasoning models are therefore needed. I will show how experiments with toy transformers, surrogate models and a hypothesis-driven methodology shed light on how the Transformer architecture may implement reasoning strategies. Finally, using the planning puzzle ‘Tower of Hanoi’ as a case study, I will show that such a methodology can also be successful in frontier large reasoning models.
Willem Zuidema is an associate professor in natural language processing, explainable AI and cognitive modelling at the Institute for Logic, Language and Computation of the Universiteit van Amsterdam, and director of its Cognition, Language and Computation lab.