Large-scale Machine Learning:
Mathematical Foundations and Applications

CSE 588

This graduate-level course will aim to cover various mathematical aspects of big and high-dimensional learning arising in data science and machine learning applications. The focus will be on building a principled understanding of randomized and relaxations methods via a mixture of empirical evaluations and mathematical modeling. Specifically, we will explore large-scale optimization algorithms for both convex and non-convex optimization, dimension reduction and random projection methods, large-scale numerical linear algebra, sparse recovery and compressed sensing, low-rank matrix recovery, convex geometry and linear inverse problems, empirical processes and generalization bounds, as well as theory and optimization landscape of deep neural networks, etc.


  • Introduction
  • Regularized Empirical Risk Minimization
  • Linear Algebra and Matrix Computation
  • Convex Analysis, Optimization, and Optimality Conditions
  • Concentration
  • First-order Methods for Large-scale Optimization
  • Stochastic Gradient Descent and Acceleration
  • Parallel and Distributed Optimization
  • Accelerated Training and Inference in Deep Neural Netwroks
  • Random Projections and Applications in Dimensionality Reduction and Learning
  • Sketching and Randomized Linear Algebra
  • Sparse Recovery: Theory and Algorithms
  • Low-rank Matrix Recovery: Theory and Algorithms
  • Empirical Processes and Generalization

Course Info

  • Spring 2019
  • Fall 2019