MSc in Computer Science - Specialization in Advanced Machine Learning and Data Science

Deep Learning for Visual Recognition

Course content

Deep learning has enabled today’s AI systems to drive cars autonomously, beat humans in computer games, and paint whatever you tell them to. The deep learning revolution is upon us, transforming many businesses and changing how we write software.

This course is a deep dive into the details of deep learning with a focus on solving computer vision tasks, like image classification, object detection, image segmentation, face recognition, human pose estimation, and image captioning. Computer vision is a prominent field within deep learning.

During the 14-week course, students will learn to implement, train and debug their neural networks and gain a theoretical understanding of neural network architectures and optimization strategies. The weekly hands-on exercises will involve setting up computer vision problems, like image recognition, and applying learning algorithms and practical engineering tricks for training and fine-tuning the networks.

The mandatory course allows students to apply what they have learned in class to a problem of their interest (in groups of 1-3 students).

Description of qualifications

Objectives of the course:

The participants will after the course have detailed knowledge of deep neural networks and will be able to solve complex visual recognition tasks with deep learning.

Learning outcomes and competencies:

At the end of the course, the participants will be able to:

• Identify and describe visual recognition tasks that can be solved with deep learning.
• Describe and compare different neural network architectures.
• Explain and compare techniques for training neural networks.
• Apply deep learning to standard visual recognition tasks and interpret the results.
• Design their course project, implement it, experiment with it, and analyse and relate the results to techniques and theories in deep learning for visual recognition.

Cluster Analysis

Course content

This course will introduce several clustering algorithms along with techniques by which we can analyze them. In the beginning, we will study center-based clustering algorithms such as the popular k-means objective. We will see approximation algorithms for these problems based on (basic) rounding for linear programs and sampling techniques. In addition, we will introduce several methods to compress data sets concerning these objectives.

In the next step, we move on to cut-based clustering problems. Here we will introduce both spectral methods (Cheeger's inequality), metric embeddings (Leighton-Rao), and semi-definite programming (Max-Cut).

Finally, we will move on to clustering instances with stability assumptions. We first motivate stability assumptions by showing how they challenge Kleinberg's impossibility axioms. Next, we will consider a stability assumption for which practical successful heuristics have provable guarantees. These include k-means++ and Llyods algorithm (cost-stability) and hierarchical clustering (perturbation resilience and planted partition models).

This course mainly focuses on those clustering algorithms that are important in both theory and practice. While the lectures will focus on the theoretical analysis, a greater (but not exclusive) focus on practical aspects is placed on the implementation of these algorithms for the hand-ins.

List of topics (some topics will take multiple lectures)

1. Introduction and Kleinberg’s Impossibility Theorem
2. k-Means: Lloyd’s Algorithm and k-Means++
3. Facility location/k-Median: LP-based approximation
4. Corsets
5. Cut-Based Clustering
6. Spectral Clustering
7. Dimension Reduction
8. Stability Conditions
9. Hierarchical Clustering

Description of qualifications

Objectives of the course

The course introduces theoretical aspects of cluster analysis, along with strong application ties justifying why these problems and algorithms are studied. We will survey most variants of algorithms that have widespread impact in practice and are simultaneously amenable to theoretical analysis. The course will also introduce approximation algorithms based on linear programming and spectral methods, as well as sampling techniques for clustering and compressing large data sets. 

Algorithms, Incentives, and Data

Course content

The course will cover recent advances in the area of algorithmic game theory, with a particular focus on the role of data in the design of algorithms that are simultaneously efficient and robust to the strategic behaviour of the entities involved in computation. It will first cover the theory of mechanism design for single-parameter settings, with a focus on optimizing social welfare and revenue in auctions. Applications to sponsored search will also be discussed. Then, the design principles for designing simple but approximately optimal auctions will be explored. In addition to the classical theory that assumes accurate statistical information about the input, scenarios in which optimal auctions have to be learned using available samples of input data will be considered. Other topics include mechanism design in multi-parameter settings (including the VCG mechanism for social welfare maximization), mechanism design with payment constraints, and mechanisms without money, with applications to matching problems, kidney exchange, and simple machine learning tasks. The last part of the course will focus on non-truthful mechanisms and the games they induce between the entities involved in computation. Issues such as game dynamics and efficiency of equilibria (price of anarchy) will be addressed in congestion games and auctions.

Indicative list of lectures

1. Basics of mechanism design, examples of rules that lead to undesired strategic behaviour, introduction to the inefficiency of selfish behaviour
2. Single-item auctions, sponsored search ad auctions, ad auction data
3. Myerson's monotonicity lemma
4. Algorithmic mechanism design, Knapsack auctions, revelation principle
5. Revenue-maximizing auctions
6. Simple near-optimal auctions, prophet inequalities
7. Learning optimal auctions from samples
8. Multi-parameter mechanism design, combinatorial auctions, spectrum auctions
9. Mechanism design with payment constraints
10. Mechanism design without money, stable matchings, kidney exchange
11. Mechanism design for machine learning tasks: truthful mean estimation, regression, classification
12. Games and equilibria, potential games, congestion games, action games
13. Price of anarchy (congestion games)
14. Price of anarchy (auctions)
15. Welfare guarantees from data
16. Best-response dynamics
17. No-regret dynamics
18. Swap regret and the minimax theorem

Description of qualifications

Purpose

After the course, the participants will have obtained a good knowledge of the basic design principles for truthful mechanisms in single and multi-parameter settings and of analysis techniques for non-truthful mechanisms. These include the general theoretical background necessary for solving problems when the use of monetary transfers (payments) between the computing entities is allowed as well as mechanism design without money. In addition, the participants will become familiar with mechanism design from sampled data, which is the most important design and analysis scenario in practice.