Decision, Optimization and Learning at the California Institute of Technology
Applied Random Matrix Theory
Random matrices now play a role in many areas of theoretical, applied, and computational mathematics. Therefore, it is desirable to have tools for studying random matrices that are flexible, easy to use, and powerful. The goal of this project is to extend, develop, and disseminate methods that allow researchers to analyze a wide range of random matrices. Joel Tropp is directing research in this area.
An important function of our visual system is watching people. We are social animals, and to be successful we need to know who is where and what is everyone doing. The same is true for machines -- they would be much more useful if they knew where people are, what are they up to, and infer goals and social interactions. We combine machine learning, computer vision, and models of cognition to approach this important and challenging topic. This area of research is led by Yisong Yue and Pietro Perona.
Community Sense and Respond Systems
We tackle a fundamental question in cyber-physical systems: What is
the ideal structure of systems that detect critical events, such as
earthquakes, by using data from large numbers of sensors held and
managed by ordinary people in the community? The approach is to
develop theory about widely-distributed sense and respond systems,
using dynamic, and possibly unreliable, networks using sensors and
responders installed and managed by ordinary citizens, and to apply
the theory to problems important to society. Current research is
developing sense and respond systems for earthquake detection and
radiation detection. This project is led by K. Mani Chandy. More details can be found on the Infospheres project page.
The Computational Power of Relative Entropy
The relative entropy
function plays a prominent role in information theory and in
statistics in the characterization of the performance of a variety of
inferential procedures as well as in proofs of a number of fundamental
inequalities. This function also has a number of attractive
computational properties that are useful for obtaining tractable
algorithms for addressing problems arising in combinatorial
optimization, dynamical systems, and quantum information. Venkat
Chandrasekaran is directing a research effort aimed at investigating
the computational properties of the relative entropy function and its
utility in a range of applications.
Ten years ago, computing power was purchased as a stock, and a key bottleneck for new tech startups was the cost of acquiring and scaling computational power as they grew. Today, computing power is purchased as a flow that can be scaled as needed via cloud providers like Amazon EC, Microsoft Azure, etc. Thus, today, a key bottleneck for many new tech startups has become data. For emerging entrepreneurs, data often are expensive, hard to obtain, and hard to harness. The problem is particularly sharp for start-up firms, but information management is a problem that nearly all businesses must confront and which is evolving rapidly. In ten years, firms will no longer have to treat data as a commodity; they will be able to get access as a service. Such a transition will pose challenging questions including how to design such "data markets." These can only be answered by a mix of learning, optimization, and economics.
Human-in-the-Loop Machine Learning
Many machine learning approaches are intended to learn from and make predictions for humans. In the former case, we must make sure that we properly account for human biases and interpret implicit human feedback, in order to learn a reliable model. In the latter case, we must make sure that the predictions of the resulting machine learning model are understandable or interpretable to human end users. This research is led by Yisong Yue and Pietro Perona.
Interactive Machine Learning
In many cases, the learning algorithm is not operating in a vacuum, but rather in a system that is constantly interacting with some environment, and potentially gathering data as it operates. Interactive machine learning pertains to any machine learning setting that requires the system to make decisions while simultaneously learning about the environment. Topics include multi-armed bandits, reinforcement learning, sequential experimental design, active learning, online algorithms, adaptive stochastic optimization. This research is led by Yisong Yue and Adam Wierman.
Latent-Variable Statistical Modeling
A central task in data
analysis is to learn simple or concisely described models that
characterize the statistical dependencies among a collection of
variables. Concisely specified models avoid problems associated with
overfitting, and they often provide useful interpretations about the
relationships inherent in the underlying variables. However, latent
or unobserved phenomena complicate this task significantly as latent
variables induce complicated and confounding relationships among the
observed variables that are not easily or concisely described. Venkat
Chandrasekaran is leading efforts to develop principled and
computationally tractable methods based on convex optimization to
address this challenge.
Precise Analysis of Statistical Estimation Problems
Regularized estimation problems are central to the practice of modern statistics, machine learning, and signal processing. Recent breakthroughs by Caltech faculty and students are leading to a comprehensive and precise understanding of the performance of statistical estimators based on convex optimization. Venkat Chandrasekaran, Babak Hassibi, and Joel Tropp are all leading projects in this area.
Randomized Algorithms for Practical Linear Algebra and Convex Optimization
Linear algebra and convex optimization problems are among the building blocks for more complicated procedures in machine learning, scientific computing, and other areas. Over the last decade, researchers have recognized that randomness can help us develop simple, efficient, and robust methods for solving these problems. This research aims to develop and analyze randomized methods that are useful in practice. Houman Owhadi and Joel Tropp are both leading projects in this area.
Social and Economic Networks Markets
The precise structure of social interactions can impact a variety of behaviors and outcomes. For example, learning a new computer or spoken language may depend on the number of acquaintances who already know it. Information about job openings may flow through word-of-mouth interactions. Financial investments and outcomes may depend on the underlying connections between firms. The efficiency and robustness of electricity markets depend on the structure of the power grid connecting generators and demand, and also on how consumers will respond to the incentives imbedded in the smart grid. Similarly, political organizations are becoming ever more adept at (and dependent on) building networks using new information technologies to weigh in on the political debate. These observations have opened the door to an array of theoretical and empirical questions: How do individuals and organizations strategically interact with neighbors on complex social and economic networks? What are network architectures that are more conducive to diffusion of behavior and financial outcomes? How do we quantify the impacts of these networks on outcomes using field and experimental data?
There is an ongoing explosion of spatiotemporal data being collected in a variety of domains, including animal behavioral studies, sports analytics, geospatial sensors, and medical imaging. We study questions regarding how to develop efficient and compact models from raw spatiotemporal data. This research is led by Yisong Yue and Pietro Perona.
Broadly speaking, structured prediction refers to any setting that involves multiple interdependent outputs. Examples include predicting parse trees, sequence alignment, submodular optimization, and routing. We study how to develop efficient and robust machine learning algorithms for training structured prediction models. This research is led by Yisong Yue.
A Theory for Privacy
Sensitive information is held by an enormous variety of entities, particularly in today's online world. Some of the challenges in handling it include developing principled models and definitions for privacy guarantees, along with privacy-preserving algorithms and provable bounds on how information privacy can be traded off against its usefulness. Integrating notions of privacy into utility theoretic and decision theoretic frameworks will provide us with more sophisticated means of reasoning about sensitive information. Work in this area is led by Katrina Ligett and Federico Echenique.
Visipedia is a joint project between Pietro Perona's Vision Group at Caltech and Serge Belongie's Vision Group at Cornell Tech. Visipedia, short for "Visual Encyclopedia," is an augmented version of Wikipedia, where pictures are first-class citizens alongside text. Goals of Visipedia include creation of hyperlinked, interactive images embedded in Wikipedia articles, scalable representations of visual knowledge, largescale machine vision datasets, and visual search capabilities. Toward achieving these goals, Visipedia advocates interaction and collaboration between machine vision and human users and experts.