understanding how networks represent, reason, and learn
Research
My work follows three questions. For each I give the idea in a sentence or two and the papers that carry it. The complete record is under Publications.
Neural representations and computation
What the cells of a brain, or the units of a network, represent, and what that representation makes easy to compute.
The population of place cells forms an embedding of position whose inner products are normalized random-walk transition probabilities. In this embedding, planning follows a straight path.
(1)
Figure 1. Place fields at several time scales, and the paths planned from them in open fields and mazes.
A language model with explicit thought vectors, inferred for each sequence by fast learning of local parameters, which complements the slow learning of global ones.
Figure 2. Left: thought vectors condition every layer through cross-attention. Right: validation perplexity against inference steps.
Learning, reasoning, and planning
How a model learns from experience, and how it uses what it has learned to think ahead.