Foundations of Data Science
Weekly outline
-
Final Exam
- Friday, January 19, 2024: 15:15-18:15. Rooms GCA330 (surnames from A to L) and GCA331 (surnames from M to Z).
- Exam coverage and rules:
- The exam will be open book: all printed material is allowed, but electronic devices are not allowed.
- The exam will cover all class materials including lecture notes, homework, and the contents of live lectures.
- The exam will be open book: all printed material is allowed, but electronic devices are not allowed.
- Special exam office hours:
- Tuesday, Jan 9, 14:00-15:00, INR 136 (Marco)
- Wednesday, Jan 10, 14:00-15:00, INR 116 (Rüdiger)
- Thursday, Jan 11, 14:00-15:00, INR 030 (Thomas)
- Friday, Jan 12, 14:00-15:00, INR 130 (Michael)
- Monday, Jan 15, 14:00-15:00, INR 030 (Thomas)
- Tuesday, Jan 16, 14:00-15:00, INR 136 (Marco)
- Wednesday, Jan 17, 13:00-14:00, INR 116 (Rüdiger)
- Thursday, Jan 18, 14:00-15:00, INR 130 (Michael)
- Tuesday, Jan 9, 14:00-15:00, INR 136 (Marco)
SWITCHtube Channel
We will not make new recording this year. However you can access the videos from last year. The content is largely the same.
Summary
We discuss a set of topics that are important for the understanding of modern data science but that are typically not taught in an introductory ML course. In particular we discuss fundamental ideas and techniques that come from probability, information theory as well as signal processing.
Content
This class presents basic concepts of Information Theory and Signal Processing and their relevance to emerging problems in Data Science and Machine Learning.
A tentative list of topics covered is:
- Information Measures
- Signal Representations
- Detection and Estimation
- Multi-arm Bandits
- Distribution Estimation, Property Testing, and Property Estimation
- Exponential Families
- Compression and Dimensionality Reduction
- Information Measures and Generalization Error
Materials
- Lecture Notes (Version of Monday, September 18, 2023). Note: Check for updates on a semi-regular basis.
Additional Material:- T. M. Cover and J. A. Thomas, Elements of Information Theory (Click to get access to the full PDF via the EPFL library). New York: Wiley. Second Edition, 2006.
- T. Lattimore and C. Szepesvari, Bandit Algorithms
Schedule
Classes:
- Tuesday 11:15-13:00 (CE1105)
- Thursday 17:15-19:00 (INF1)
Exercise:- Tuesday 13:15-15:00 (CE1105)
Grading
- If you do not hand in your final exam your overall grade will be NA.
- Otherwise, if we can hold the Midterm Exam (on Thursday, November 16, 2023, 17:15-19:00), your grade will be determined based on the following weighted average: 10% for the Homework, 30% for the Midterm Exam, 60% for the Final Exam.
- If we cannot hold the Midterm Exam, your grade will be determined based on the following weighted average: 10% for the Homework, 90% for the Final Exam.
The Final Exam will take place on at some point between January 15, 2024 and February 3, 2024.
-
Sept 19 : General Introduction ; Probability Review
Sept 21 : Information Measures -
Sept 26 : Information Measures
Sept 28 : Information Measures ; Signal Representations -
Oct 3 : Signal Representations
Oct 5 : Signal Representations -
Oct 10 : Signal Representations
Oct 12 : Signal Representations; Estimation -
Oct 17 : Multi-arm Bandits explore-then-exploit
Oct 19 : Multi-arm Bandits : UCB -
Oct 24 : Multi-arm Bandits
Oct 26 : Multi-arm Bandits ; Distribution Estimation -
Oct 31 : Distribution Estimation
Nov 2 : Distribution Estimation -
Nov 7 : Distribution Estimation
Nov 9 : Property Testing and Estimation -
Nov 14 : Detection and Estimation
Nov 16 : Midterm Exam -
Nov 21 : Detection and Estimation
Nov 23 : Exponential Families -
Nov 28 : Exponential Families
Nov 30 : Exponential Families -
Dec 5 : Compression
Dec 7 : Compression -
Dec 12 : Compression
Dec 14 : Exploration Bias and Generalization Bounds -
Dec 19 : Generalization Bounds
Dec 21 : Review Session