Theoretical Machine Learning (Fall 2026)

Presentation Overview

The final part of the course (30% of your grade) will consist of student presentations on recent research papers in theoretical machine learning, with an emphasis on papers related to current developments in AI. Students will work in a group of two. Each group will study an assigned paper in depth, give a presentation explaining its main results, key technical ideas, and their significance, and answer questions during the presentation. Each of you has been assigned a teammate, a paper, and a date for presentation, based on alphabetical order of last names.

Assigned Papers and Schedule

Date	Papers	Presenters
11/13	Size-Independent Sample Complexity of Neural Networks	Mohammad Amin Abbasfar and Faith Baca
	Bridging the Gap: Rademacher Complexity in Robust and Standard Generalization	Viraj Bansal and Xinjie Chen
	Sequence Length Independent Norm-Based Generalization Bounds for Transformers	Shupeng Cheng and Gong Chen
	Tight Sample Complexity of Transformers	Parth Gosar and Md Abrar Jahin
	Transformers Learn In-Context by Gradient Descent	Lohith Saradhi Kandukuri and Athan Li
	Transformers as Algorithms: Generalization and Stability in In-context Learning	Kaiwen Li and Xinyuan Li
11/20	Trained Transformer Classifiers Generalize and Exhibit Benign Overfitting In-Context	Beijie Liu and Yu-Chun Liu
	On Learning Verifiers and Implications to Chain-of-Thought Reasoning	Qi Pan and Xingen Wang
	What Fits (Into Few Tokens) Doesn’t Overfit: Compression and Generalization in ML Research Agents	Prabhu Pugalenthi and Kratika Rathi
	Adaptive Matrix Online Learning through Smoothing with Guarantees for Nonsmooth Nonconvex Optimization	Dexin Ren and Samruddhi Saoji
	Muon with Finite Newton–Schulz: The Smoothing Benefit in Nonsmooth Nonconvex Optimization	Ania Serbina and Adithya Sivanandham Srinivas
	Next-Token Prediction and Regret Minimization	Yung-Yi Tu and Jingxuan Wang
12/04	Tokenized Bandit for LLM Decoding and Alignment	Li-Heng Wang and Shangshang Wang
	Logarithmic Regret for Online KL-Regularized Reinforcement Learning	Satish Panda and Emily Weiss
	Efficient and Near-Optimal Algorithm for Contextual Dueling Bandits with Offline Regression Oracles	Shicheng Wen and Yifan Wu
	Correcting the Mythos of KL-Regularization: Direct Alignment without Overoptimization via Chi-Squared Preference Optimization	Tony Yang and Haobo Zhang
	Rethinking the Hardness of PbRL: A Provable General Regret Bound	Jiacheng Zhang and Konghao Zhao

Guidance on Reading the Papers

Most of these papers are closely related to and extend the materials covered in the lectures. So when reading them, try to find the connections with what you have learned from the lectures.
Some papers can be notation-heavy and get rather technical in some parts. You are by no means required to understand every single detail of the paper. Instead, try to focus on the key ideas/messages. It is fine that you only skim some parts of the paper, as long as you spend time carefully reading and understanding some other main parts.
You should discuss the paper with your teammates for better understanding of the paper!
You are also welcome to use AI tools to help understand the paper (but note that you are responsible for understanding everything you present and being able to answer questions about it).
If you need further help understanding the papers, feel free to ask on Piazza or come to our office hours.

Instructions on the Presentation

The goal of this part is to give you an opportunity to practice your presentation skills, which is an extremely important part of your study (but is often overlooked). Follow the instructions below to make sure you and your teammates give a smooth talk.

Each presentation is about 25 minutes long plus 5 minutes of Q&A and transition. Each member should present for roughly equal amounts of time, in whatever manner you think works best. It is recommended that you roughly break down the talk into the following parts:

(~5 minutes) introduction: what is the topic you are presenting? how is this related to the lectures? what are the main results of the paper in a few words? why are they significant? what will you cover in the rest of the talk?
(~6 minutes) setting up: describe the problem and introduce necessary notations.
(~12 minutes) main results: describe the main algorithms and/or theorems, with emphasis on explaining the key ideas.
(~2 minutes) conclusion: conclude your talk and highlight the key take-home message.

In addition, keep the following in mind when preparing the talks:

You can either give a board talk (just like our lectures) or use slides for the presentation. For a board talk, think carefully about how you are going to use the board: how much space do I need? what should I put in each part of the board? how do I avoid erasing stuff that I need later? If you are using slides instead, please let me know before the lecture so we can make sure there is no problem in setting up the projector.

Anticipate questions from the audience. Just like the lectures, everyone is encouraged to ask questions during the talk. Make sure you understand the question before answering, and try to answer it concisely so you can finish the talk in time. It is perfectly fine if you do not know the answer for tough questions — simply acknowledge it in this case and move on. But make sure you address simple clarification questions, especially at the beginning of the talk when setting up the problems.

You by no means need to cover everything in your assigned paper! In fact, most likely you will only have time to cover one or two main results. Keep this in mind and make the key message of your talk clear.

Even if the paper is notation-heavy, it does not mean your talk has to be so. Try to use a minimum amount of notations to convey what you want to say. It is common to make simplification in talks, even if that means sacrificing some generality or rigor.

If you have any other questions, feel free to ask on Piazza!