Large-scale Machine Learning

CSE 597

This graduate-level course will aim to cover various mathematical aspects of big and high-dimensional learning arising in data science and machine learning applications. The focus will be on building a principled understanding of randomized and relaxations methods via a mixture of empirical evaluations and mathematical modeling. Specifically, we will explore large-scale optimization algorithms for both convex and non-convex optimization, dimension reduction and random projection methods, large-scale numerical linear algebra, sparse recovery and compressed sensing, low-rank matrix recovery, convex geometry and linear inverse problems, empirical processes and generalization bounds, as well as theory and optimization landscape of deep neural networks, etc.

Lectures

  • Introduction    
  • Regularized Empirical Risk Minimization    
  • Linear Algebra and Matrix Computation    
  • Convex Analysis, Optimization, and Optimality Conditions    
  • Concentration    
  • First-order Methods for Large-scale Optimization    
  • Stochastic Gradient Descent and Acceleration    
  • Parallel and Distributed Optimization    
  • Accelerated Training and Inference in Deep Neural Netwroks    
  • Random Projections and Applications in Dimensionality Reduction and Learning    
  • Sketching and Randomized Linear Algebra    
  • Sparse Recovery: Theory and Algorithms    
  • Low-rank Matrix Recovery: Theory and Algorithms    
  • Empirical Processes and Generalization