Information Theory and Signal Processing

Instructor Michael Gastpar Instructor Emre Telatar Instructor Ruediger Urbanke
Office INR 130 Office INR 117 Office INR 116
Email michael.gastpar@epfl.ch Email emre.telatar@epfl.ch Email ruediger.urbanke@epfl.ch
Office Hours By appointment Office Hours By appointment Office Hours By appointment
Teaching Assistant Pierre Quinton
Email pierre.quinton@epfl.ch Office INR 030
Teaching Assistant Amedeo Esposito Email amedeo.esposito@epfl.ch Office INR o31
Admin Assistant Muriel Bardet Email muriel.bardet@epfl.ch Office INR 137
Lectures Monday 09:15 – 11:00  Room: INM200
Friday 08:15 – 10:00  Room: INM201
Exercises Friday 10:15 – 12:00 Room:  INM201
Language: English
Credits : 6 ECTS

Lecture notes (PDF file)

Official Prerequisites: COM-300 Modèles stochastiques pour les communications (or equivalent)

Here is a link to official coursebook information).

Homework:
Some Homework will be graded…

Grading:
If you do not hand in your final exam your overall grade will be NA. Otherwise, your grade will be determined based on the following weighted average:
10 % for the Homework, 90 % for the Final Exam.

Special Announcements

Last year’s Final Exam and Solutions.

Detailed Schedule

(tentative, subject to changes)

Date Topics Covered Lectures Exercises
Review HW0 Sol0
21/9 General Introduction ; Review Linear Algebra, Probability
Exercise: Review Session (Linear Algebra, Probability)   (MG)
Chapter 2 Handout
Information Measures
24/9 Basic Information Measures (ET) Handout
28/9 (ET) HW1
01/10 (ET)
05/10 (ET) Sol1
Estimation and Detection
08/10 Optimum Detection and Estimation ; MMSE (MG)
12/10 Parameter estimation ; Fisher information ; Cram`er-Rao bound (MG) HW2
15/10 Exploration bias and generalization guarantees (II) Handout1
19/10 via information measures (II) Handout2 Sol2
Exponential Families
22/10 Exponential families ; Max Entropy problems (RU) Chapter 4
26/10 Boltzmann distribution ; Exponential families (RU) HW3 Sol3
Compression and Quantization
29/10 Compression and Quantization (ET)
02/11 Compression and Quantization (ET) HW4
05/11 Compression and Quantization (ET)
09/11 Compression and Quantization (ET) Sol4
Multi-Arm Bandits
12/11 Multi-armed Bandits : Explore & Exploit (RU) Chapter 7
16/11 Multi-armed Bandits : UCB algorithm (RU) HW5
19/11 Multi-armed Bandits : Converse bound (RU) Sol5
23/11 Multi-armed Bandits : Variations (RU)
Signal Representations
26/11
Signal Representation : Fourier, Sparse Fourier, (MG)
Chapter 6
30/11 Wavelets, Compressed sensing? (MG) HW6
03/12 Wiener Filter (MG)
07/12 LMS Adaptive Filter (MG) Sol6
Distribution Estimation
10/12 Distribution Estimation ; Property Testing and Estimation (RU) Chapter 8
14/12 Distribution Estimation ; Property Testing and Estimation (RU) HW7
17/12 Distribution Estimation ; Property Testing and Estimation (RU)
21/12 Distribution Estimation ; Property Testing and Estimation (RU) Sol7

Textbooks