In recent years, there has been a surge of interest in algorithms that can "learn" patterns from data. Whilst such algorithms have traditionally been studied in the area of statistics, modern computers and large datasets have led to a new field of Machine Learning (ML) which lies at the intersection of mathematics, statistics, computer science and engineering. Nowadays, many large-scale ML models (e.g. ChatGPT, Gemini and Stable Diffusion) are so impressive at generating and processing data, we often call them "Artificial Intelligence". In this unit, we will introduce some of the central topics and algorithms in machine learning, explore their underlying mathematics and develop practical implementations using Python. Topics covered in this unit include: • Statistical learning theory: Data as random variables, expected risk and cross validation. • Data preparation and retrieval: Numerical data as vectors and matrices (e.g. images) and non-numerical data (e.g. text). • Unsupervised learning: Clustering and Principal Component Analysis (PCA). • Supervised learning: Linear regression, logistic regression and maximum likelihood. • Optimization algorithms: Gradient Descent (GD) and Stochastic Gradient Descent (SGD). • Non-parametric models: Decision trees and K-nearest neighbours.
Lecture notes (pdf) Problem sheets can be found below and at github.com/james-m-foster/MA50290_24. • Problem Sheet 1, Oxford temperature data • Problem Sheet 2 • Problem Sheet 3 • Problem Sheet 4, Bath webpages text data • Problem Sheet 5 • Problem Sheet 6, Gradient descent example • Problem Sheet 7 • Problem Sheet 8 • Problem Sheet 9, Perceptron example