# Learning Theory CS-526

 Instructor Nicolas Macris Instructor Ruediger Urbanke Office INR 134 Office INR 116 Email nicolas.macris@epfl.ch Email ruediger.urbanke@epfl.ch Office Hours By appointment Office Hours By appointment
 Teaching Assistant Chan Chun Lam Email chunlam.chan@epfl.ch Office INR032 Teaching Assistant Kirill Ivanov Email kirill.ivanov@epfl.ch Office INR 030 Teaching Assistant Clement Luneau Email clement.luneau@epfl.ch Office INR 141
 Lectures Monday 08:15 – 10:00  Room: INM202 Exercises Tuesday 17:15 – 19:00 Room:  INR219
 Language: English Credits : 4 ECTS

Prerequisites:

• Analysis I, II, III
• Linear Algebra
• Machine learning
• Probability
• Algorithms (CS-250)

Here is a link to official coursebook information.

Homework:
Some homework will be graded.

If you do not hand in your final exam your overall grade will be NA. Otherwise, your grade will be determined based on the following weighted average: 10 % for the Homework, 90 % for the Final Exam. For the graded homeworks,  you can discuss the homework with other people. But you have to write down your own solution and note on the first page the set of people that you discussed with.

### Detailed Schedule

(tentative, subject to changes)

Date Lectures Exercises Solutions
18/2 Chap 3 and 4 (in UML) 3.1; 3.3; 3.7; 3.8; 4.1; 4.2 Solution 1
25/2 Chap 5  (in UML) idem ++

Exercise 2

Solution 2
4/3 Chap 6  (in UML) Graded: 5.1; 6.2; 6.5; 6.8; 6.9; 7.3 Solution 3
11/3 Chap 7  (in UML) idem
18/3 remaining of Chap 7 and Chap 14 start (in UML) Deadline for handing in graded homework 19/3 during exercise session

Exercise 4

Solution 4
25/3 remaining of Chap 14 (in UML) 2nd graded homework:

Exercise 5

Solution 5
1/4 Lecture notes on two-layer neural networks” by A. Montanari Hand-out of 1st graded homework

(lecture and exercise session)

8/4 Introduction to graphical probabilistic models

PGM-Lect-1.pdf

(Chap 3 and 4 in D. Barber and Chap 8 in C. Bishop)

2nd graded homework continued (exercise 5).

Deadline is 16 April

15/4 Factor graphs, Marginalization.

PGM-Lect-2.pdf

Notes on message passing for marginalization (sum-product algorithm)

(Chap 4, 5 in Barber, Chap 8 in Bishop)

Exercise 6

Solution-6
22/4 Vacations
29/4 Sampling: Ancestral sampling for belief Networks and MCMC.

PGM-Lect-3.pdf

Learning graphical models: (Barber paragraphs 9.3 and 9.6 mostly 9.6.1)

Exercises 6 continued. Use notes on message passing for problems 8, 9, 10
6/5 Variational bayes EM, standard EM

PGM-Lect-4.pdf

Learning graphical models: (Barber 11.2 mostly 11.2.1, 11.2.2 and 11.5.

Exercise 7

New Deadline: May 28.

Solution-7
13/5 Tensor methods: Next three classes based on the Review

Tens-Lect-1.pdf

Tensor product, Rank, Jennrich’s thm

Exercise 8

New Deadline: June 4 in  mailbox in IPG corridor (INR) or with the assistants.

Solution-8
20/5 Tens-Lect-2.pdf

ALS, multilinear rank, Tucker HOSVD

27/5 Tens-Lect-3.pdf

Applications: GMM, Topic models, multiview models.

If time permits: Power Method, Whitening