Decision, Optimization and Learning at the California Institute of Technology

dolcit

Courses

Courses offered at Caltech in topics relevant to DOLCIT are below. Do note that some courses are not offered every year; please check the Registrar's course schedule for current course availability.

Statistics, Machine Learning, Data Mining, and Analytics

  • CMS/CS/CNS/EE 155. Machine Learning Data Mining. 12 units (3-3-6); second term. Prerequisites: background in algorithms and statistics (CS/CNS/EE/NB 154 or CS/CNS/EE 156 a or instructor’s permission). This course will cover popular methods in machine learning and data mining, with an emphasis on developing a working understanding of how to apply these methods in practice. This course will also cover core foundational concepts underpinning and motivating modern machine learning and data mining approaches. This course will be research-oriented, and will cover recent research developments. Instructors: Yue.

  • CS/CNS/EE 156 ab. Learning Systems. 9 units (3-0-6); first, third terms. Prerequisites: Ma 2 and CS 2, or equivalent.Introduction to the theory, algorithms, and applications of automated learning. How much information is needed to learn a task, how much computation is involved, and how it can be accomplished. Special emphasis will be given to unifying the different approaches to the subject coming from statistics, function approximation, optimization, pattern recognition, and neural networks. Not offered 2015–16.

  • CS/CNS/EE 159. Advanced Topics in Machine Learning.     9 units (3-0-6); third term. Prerequisites: CS 155; strong background in statistics, probability theory, algorithms, and linear algebra; background in optimization is a plus as well. This course focuses on current topics in machine learning research. This is a paper reading course, and students are expected to understand material directly from research articles. Students are also expected to present in class, and to do a final project. Instructors: Yue.

  • BEM/Ec 150. Business Analytics. 9 units (3-0-6); third term. Prerequisites: GE/ESE 118 or Ec 122, and knowledge of R. This class teaches how to use very large, cross-media datasets to infer what variables influence choices and trends of economic and business interest. Topics include database management, cleaning and visualization of data, statistical and machine learning methods, natural language processing, social and conventional media, personal sensors and devices, sentiment analysis, and controlled collection of data (including experiments). Grades are based on hands-on data analysis homework assignments and detailed analysis of one dataset. Instructor: Camerer.

  • ACM 216. Markov Chains, Discrete Stochastic Processes and Applications. 9 units (3-0-6); second term. Prerequisite: ACM/EE 116 or equivalent. Stable laws, Markov chains, classification of states, ergodicity, von Neumann ergodic theorem, mixing rate, stationary/equilibrium distributions and convergence of Markov chains, Markov chain Monte Carlo and its applications to scientific computing, Metropolis Hastings algorithm, coupling from the past, martingale theory and discrete time martingales, rare events, law of large deviations, Chernoff bounds. Instructor: Owhadi.

  • ACM/CS/EE 218. Statistical Inference. 9 units (3-0-6); third term. Prerequisites: CMS/ACM 104 and CMS/ACM/EE 116, or instructor’s permission. Fundamentals of estimation theory and hypothesis testing; Bayesian and non-Bayesian approaches; minimax analysis, Cramer-Rao bounds, shrinkage in high dimensions; Kalman filtering, basics of graphical models; statistical model selection. Throughout the course, a computational viewpoint will be emphasized. Not offered 2015–16.

    • ↑ top

    Computer Vision and Signal Processing

  • CNS/Bi/EE/CS/NB 186. Vision: From Computational Theory to Neuronal Mechanisms. 12 units (4-4-4); second term.Lecture, laboratory, and project course aimed at understanding visual information processing, in both machines and the mammalian visual system. The course will emphasize an interdisciplinary approach aimed at understanding vision at several levels: computational theory, algorithms, psychophysics, and hardware (i.e., neuroanatomy and neurophysiology of the mammalian visual system). The course will focus on early vision processes, in particular motion analysis, binocular stereo, brightness, color and texture analysis, visual attention and boundary detection. Students will be required to hand in approximately three homework assignments as well as complete one project integrating aspects of mathematical analysis, modeling, physiology, psychophysics, and engineering. Given in alternate years; offered 2015–16.

  • ACM/EE 170. Mathematics of Signal Processing. 12 units (3-0-9); third term. Prerequisites: CMS/ACM 104, CMS/ACM 113, and CMS/ACM 116; or instructor’s permission. This course covers classical and modern approaches to problems in signal processing. Problems may include denoising, deconvolution, spectral estimation, direction-of-arrival estimation, array processing, independent component analysis, system identification, filter design, and transform coding. Methods rely heavily on linear algebra, convex optimization, and stochastic modeling. In particular, the class will cover techniques based on least-squares and on sparse modeling. Throughout the course, a computational viewpoint will be emphasized. Not offered 2015–16.

  • EE/Ma/CS 126 ab. Information Theory. 9 units (3-0-6); first, second terms. Prerequisites: Ma 2. Shannon’s mathematical theory of communication, 1948-present. Entropy, relative entropy, and mutual information for discrete and continuous random variables. Shannon’s source and channel coding theorems. Mathematical models for information sources and communication channels, including memoryless, first- order Markov, ergodic, and Gaussian. Calculation of capacity and rate-distortion functions. Kolmogorov complexity and universal source codes. Side information in source coding and communications. Network information theory, including multiuser data compression, multiple access channels, broadcast channels, and multiterminal networks. Discussion of philosophical and practical implications of the theory. This course, when combined with EE 112, EE/Ma/CS 127, EE 161, and/or EE 167 should prepare the student for research in information theory, coding theory, wireless communications, and/or data compression. Instructor: Effros.

  • EE 128 ab. Selected Topics in Digital Signal Processing. 9 units (3-0-6); second, third terms. Prerequisites: EE 111 and EE 160 or equivalent required, and EE 112 or equivalent recommended. The course focuses on several important topics that are basic to modern signal processing. Topics include multirate signal processing material such as decimation, interpolation, filter banks, polyphase filtering, advanced filtering structures and nonuniform sampling, optimal statistical signal processing material such as linear prediction and antenna array processing, and signal processing for communication including optimal transceivers. Not offered 2015–16.

  • EE/CNS/CS 148. Selected Topics in Computational Vision. 9 units (3-0-6); third term. Prerequisites: undergraduate calculus, linear algebra, geometry, statistics, computer programming . The class will focus on an advanced topic in computational vision: recognition, vision-based navigation, 3-D reconstruction. The class will include a tutorial introduction to the topic, an exploration of relevant recent literature, and a project involving the design, implementation, and testing of a vision system. Instructors: Perona.

  • EE 164. Stochastic and Adaptive Signal Processing. 9 units (3-0-6); third term. Prerequisite: ACM/EE 116 or equivalent.Fundamentals of linear estimation theory are studied, with applications to stochastic and adaptive signal processing. Topics include deterministic and stochastic least-squares estimation, the innovations process, Wiener filtering and spectral factorization, state-space structure and Kalman filters, array and fast array algorithms, displacement structure and fast algorithms, robust estimation theory and LMS and RLS adaptive fields. Given in alternate years; offered 2015–16. Instructor: Hassibi.

    • ↑ top


    Computational Neuroscience

  • CNS/Bi/Ph/CS/NB 187. Neural Computation. 9 units (3-0-6); first term. Prerequisites: familiarity with digital circuits, probability theory, linear algebra, and differential equations. Programming will be required. This course investigates computation by neurons. Of primary concern are models of neural computation and their neurological substrate, as well as the physics of collective computation. Thus, neurobiology is used as a motivating factor to introduce the relevant algorithms. Topics include rate-code neural networks, their differential equations, and equivalent circuits; stochastic models and their energy functions; associative memory; supervised and unsupervised learning; development; spike-based computing; single-cell computation; error and noise tolerance. Instructor: Perona.

    • ↑ top


    Linear Algebra, Optimization, and Convex Analysis

  • CMS/ACM 104. Linear Algebra and Applied Operator Theory. 12 units (3-0-9); first term. Prerequisites: Undergraduate prerequistes: Ma 1 abc (analytic track), Ma 2, and ACM 95 ab; or instructor’s permission. This course introduces the theory and applications of linear algebra and linear analysis. Lectures and homework will require the ability to understand and produce mathematical proofs. Theoretical topics may include topology of metric spaces, structure of Banach and Hilbert spaces, examples of normed spaces, duality, structure of linear operators, spectral theory, functional calculus for linear operators, and calculus in Banach spaces. Applications will be drawn from signal processing, numerical analysis, optimization, approximation, differential equations, control, and other areas. Emphasis will be placed on geometry and convexity. Instructors: Tropp.

  • CMS/ACM 113. Mathematical Optimization. 9 units (3-0-6); first term. Prerequisites: ACM 95/100 ab, ACM 11, or instructor’s permission. Corequisite: It is suggested that students take CMS/ACM 104 concurrently. This class studies mathematical optimization from the viewpoint of convexity. Topics covered include duality and representation of convex sets; linear and semidefinite programming; connections to discrete, network, and robust optimization; relaxation methods for intractable problems; as well as applications to problems arising in graphs and networks, information theory, control, signal processing, and other engineering disciplines. Instructors: Chandrasekaran.

  • ACM 204. Topics in Convexity. 9 units (3-0-6); second term. Prerequisites: CMS/ACM 104 and CMS/ACM 113; or instructor’s permission. The content of this course varies from year to year among advanced subjects in linear algebra, convex analysis, and related fields. Specific topics for the class include matrix analysis, operator theory, convex geometry, or convex algebraic geometry. Lectures and homework will require the ability to understand and produce mathematical proofs. Not overed 2015–16.

  • ACM 213. Topics in Optimization. 9 units (3-0-6); third term. Prerequisites: CMS/ACM 104, CMS/ACM 113. Material varies year-to-year. Example topics include discrete optimization, convex and computational algebraic geometry, numerical methods for large-scale optimization, and convex geometry. Instructors: Chandrasekaran.

    • ↑ top


    Design and Analysis of Algorithms

  • CMS/CS 139. Analysis and Design of Algorithms. 12 units (3-0-9); second term. Prerequisites: Ma 2, Ma 3, Ma/CS 6a, CS 21, CS 38/138, CMS/ACM/EE 116, or instructor’s permission. This course covers advanced topics in the design and analysis of algorithms. Topics are drawn from approximation algorithms, randomized algorithms, online algorithms, streaming algorithms, and other areas of current research interest in algorithms. Instructors: Vidick.

  • CS 150. Probability and Algorithms. 9 units (3-0-6); second term. Prerequisites: CS 38 a and Ma 5 abc. Elementary randomized algorithms and algebraic bounds in communication, hashing, and identity testing. Game tree evaluation. Topics may include randomized parallel computation; independence, k-wise independence and derandomization; rapidly mixing Markov chains; expander graphs and their applications; clustering algorithms. Not offered 2015–16.

    • ↑ top

    Algorithmic and Networked Economics

  • CMS/CS/EE 144. Networks: Structure Economics. 12 units (3-3-6); second term. Prerequisites: Ma 2, Ma 3, Ma/CS 6a, and CS 38, or instructor permission. Social networks, the web, and the internet are essential parts of our lives and we all depend on them every day, but do you really know what makes them work?This course studies the “big” ideas behind our networked lives. Things like, what do networks actually look like (and why do they all look the same)? How do search engines work? Why do memes spread the way they do? How does web advertising work? For all these questions and more, the course will provide a mixture of both mathematical analysis and hands-on labs. The course assumes students are comfortable with graph theory, probability, and basic programming. Instructors: Wierman.
  • SS/CS 149. Introduction to Algorithmic Economics. 9 units (3-0-6); first term. Prerequisites: Ma 3, CS 24 and CS 38, or instructor permission. This course will equip students to engage with current topics of active research at the intersection of social and information sciences, including: algorithmic mechanism design; auctions; existence and computation of equilibria; and learning and games. Not offered 2015–16.
  • CS/SS 152. Introduction to Data Privacy. 9 units (3-0-6); first term. Prerequisites: Ma 3, CS 24 and CS 38, or instructor’s permission. How should we define privacy? What are the tradeoffs between useful computation on large datasets and the privacy of those from whom the data is derived? This course will take a mathematically rigorous approach to addressing these and other questions at the frontier of research in data privacy. We will draw connections with a wide variety of topics, including economics, statistics, information theory, game theory, probability, learning theory, geometry, and approximation algorithms. Not offered 2015–16.
  • SS/CS 241. Topics in Algorithmic Economics. 9 units (3-0-6). Prerequisites: SS/CS 149. This is a graduate-level seminar covering recent topics at the intersection of computer science and economics. Topics will vary, but may include, e.g., dynamics in games, algorithmic mechanism design, and prediction markets. Instructors: EAS and HSS faculty. Not offered 2015–16.

    • ↑ top

    Control Theory

  • CDS 212. Introduction to Modern Control. 9 units (3-0-6); second term. Prerequisites: Ma2/102, CMS/ACM 104, CDS 110.Introduction to modern control systems with emphasis on the role of control in overall system analysis and design. Examples drawn from throughout engineering and science. Open versus closed loop control. State-space methods, time and frequency domain, stability and stabilization, realization theory. Time-varying and nonlinear models. Uncertainty and robustness. Instructor: Doyle.
↑ top