This graduate-level course will aim to cover various mathematical aspects of big and high-dimensional learning arising in data science and machine learning applications. The focus will be on building a principled understanding of randomized and relaxations methods via a mixture of empirical evaluations and mathematical modeling. Specifically, we will explore large-scale optimization algorithms for both convex and non-convex optimization, dimension reduction and random projection methods, large-scale numerical linear algebra, sparse recovery and compressed sensing, low-rank matrix recovery, convex geometry and linear inverse problems, empirical processes and generalization bounds, as well as theory and optimization landscape of deep neural networks, etc.

- Introduction
- Regularized Empirical Risk Minimization
- Linear Algebra and Matrix Computation
- Convex Analysis, Optimization, and Optimality Conditions
- Concentration
- First-order Methods for Large-scale Optimization
- Stochastic Gradient Descent and Acceleration
- Parallel and Distributed Optimization
- Accelerated Training and Inference in Deep Neural Netwroks
- Random Projections and Applications in Dimensionality Reduction and Learning
- Sketching and Randomized Linear Algebra
- Sparse Recovery: Theory and Algorithms
- Low-rank Matrix Recovery: Theory and Algorithms
- Empirical Processes and Generalization