This graduate-level course will aim to cover various mathematical aspects of big and high-dimensional learning arising in data science and machine learning applications. The focus will be on building a principled understanding of randomized and relaxations methods via a mixture of empirical evaluations and mathematical modeling. Specifically, we will explore large-scale optimization algorithms for both convex and non-convex optimization, dimension reduction and random projection methods, large-scale numerical linear algebra, sparse recovery and compressed sensing, low-rank matrix recovery, convex geometry and linear inverse problems, empirical processes and generalization bounds, as well as theory and optimization landscape of deep neural networks, etc.

• Introduction
• Regularized Empirical Risk Minimization
• Linear Algebra and Matrix Computation
• Convex Analysis, Optimization, and Optimality Conditions
• Concentration
• First-order Methods for Large-scale Optimization
• Stochastic Gradient Descent and Acceleration
• Parallel and Distributed Optimization
• Accelerated Training and Inference in Deep Neural Netwroks
• Random Projections and Applications in Dimensionality Reduction and Learning
• Sketching and Randomized Linear Algebra
• Sparse Recovery: Theory and Algorithms
• Low-rank Matrix Recovery: Theory and Algorithms
• Empirical Processes and Generalization

• Spring 2019
• Fall 2019