phy694 - Machine Learning II (Complete module description)

Attendance: 56 hrs, Self study: 124 hrs

The course requires the introductory course "Machine Learning - Probabilistic Unsupervised Learning" or equivalent courses. Furthermore, basic knowledge in higher Mathematics as taught as part of first degrees in Physics, Mathematics, Statistics, Engineering or Computer Science (basic linear algebra and analysis) is required. Additionally, programming skills are required (the course supports matlab
and python). Many relations to statistical physics, statistics, probability theory, stochastic exist but the course's content will be developed independently of detailed prior knowledge in these fields.

The students will deepen their knowledge on mathematical models of data and sensory signals. Building up on the previously acquired Machine Learning models and methods, the students will be lead closer to current research topics and will learn about models that currently represent the state-of-the-art. Based on these models, the students will be exposed to the typical theoretical and practical challenges in the development of current Machine Learning algorithms. Typical such challenges are analytical and computational intractabilities, or local optima problems. Based on concrete examples, the students will learn how to address such problems. Applications to different data will teach skills to use the appropriate model for a desired task and the ability to interpret an algorithm's result as well as ways for further improvements. Furthermore, the students will learn interpretations of biological and artificial intelligence
based on state-of-the-art Machine Learning models.

This course builds up on the basic models and methods introduced in introductory Machine Learning lectures. Advanced Machine Learning models will be introduced alongside methods for efficient parameter optimization. Analytical approximations for computationally intractable models will be defined and discussed as well as stochastic (Monte Carlo) approximations. Advantages of different approximations will be contrasted with their potential disadvantages. Advanced models in the lecture will include models for clustering, classification, recognition, denoising, compression, dimensionality reduction, deep learning, tracking etc. Typical application domains will be general pattern recognition, computational neuroscience and sensory data models including computer hearing and computer vision.

Pattern Recognition and Machine Learning, C. M. Bishop, Springer 2006. (best suited for lecture).;
Information Theory, Inference, and Learning Algorithms, D. MacKay, Cambridge University Press, 2003. (free online)

KL