Deep Deep Learning

Major ideas. Fundamental questions. The mathematics underneath.

Deep Deep Learning is a graduate, student-led seminar for studying ideas you have heard about and want to understand properly. The candidate pool connects neural tangent kernels, μP and feature learning, Mamba, in-context learning, test-time training, hypernetworks, world models, online continual learning, model merging, mechanistic interpretability, new generative frameworks, graph neural networks, and the theory of why training works. Stop-Gradient, Games, and Learning Dynamics asks when learning optimizes a potential and when convergence needs a different argument. Training without backpropagation uses Dust to study perturbation-based gradient estimates and their computational cost.

Each selected topic gets two academic hours. A session builds from the minimum background to one central research paper and selected recent developments. We work through a derivation, examine an experiment, and ask what remains unresolved. Students lead the presentations and discussion.

Everyone completes one shared pre-reading before each selected meeting. Presenters choose it in consultation with the instructor and announce it one week before class: a tutorial, a short paper or specified sections that take 30–45 minutes. Two guiding questions accompany the reading; each student arrives with one question or point of confusion.

The reading pool is larger than the course. Week 10 asks whether neural networks are intrinsically entangled or whether our tools obscure their structure; Week 11 develops circuit tracing and the implications of choosing units of analysis. Students and the instructor choose the remaining topics. Dates and all presenter assignments are TBD.

Topic numbers identify reading bundles, independently of week numbers. Weeks 10 and 11 cover units of analysis and circuit tracing respectively. Students and the instructor choose the remaining topics and teaching order; dates and all presenters are TBD.