Events

Events Search and Views Navigation

Event Views Navigation

Today

Filters

Changing any of the form inputs will cause the list of events to refresh with the filtered results.

  • Detection and recovery of low-rank signals under heteroskedastic noise

    Seminar Series

    A fundamental task in data analysis is to detect and recover a low-rank signal in a noisy data matrix. Typically, this task is addressed by inspecting and manipulating the spectrum of the observed data, e.g., thresholding the singular values of the data matrix at a certain critical level. This approach is well-established in the case of homoskedastic noise, where the noise variance is identical across the entries. However, in numerous applications, such as single-cell RNA sequencing (scRNA-seq), the noise can be heteroskedastic, where the noise characteristics vary considerably across the rows and columns of the data. In such scenarios, the noise spectrum can differ significantly from the homoskedastic case, posing various challenges for signal detection and recovery. In this talk, I will present a procedure for standardizing the noise spectrum by judiciously scaling the rows and columns of the data. Importantly, this procedure can provably enforce the standard spectral behavior of homoskedastic noise -- the Marchenko-Pastur law. I will describe methods for estimating the required scaling factors directly from the observed data with suitable theoretical justification, and demonstrate the advantages of the proposed approach for signal detection and recovery in simulations and on real scRNA-seq data.

  • Flat minima and generalization in deep learning: a case study in low rank matrix recovery

    Seminar Series

    Abstract: Recent advances in machine learning and artificial intelligence have relied on fitting highly overparameterized models, notably deep neural networks, to observed data. In such settings, the number of parameters of the model is much greater than the number of data samples, thereby resulting in a continuum of models with near-zero training error. Understanding which of these models generalize well and which do not is the central open question in deep learning. Recent empirical evidence suggests one mechanism for generalization: the shape of the training loss around a local minimizer seems to strongly impact the model’s performance. In particular, flat minima -- those around which the loss grows slowly -- appear to generalize well. Clarifying this phenomenon can shed new light on generalization in deep learning, which still largely remains a mystery.

    I will describe our recent work that takes a step towards this goal by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyze overparameterized matrix and bilinear sensing, robust PCA, covariance matrix estimation, and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. These results suggest (i) a theoretical basis for favoring methods that bias iterates towards flat solutions and (ii) use of Hessian trace as a good regularizer for some learning tasks. We end by discussing the impact of depth on the generalization properties of flat solutions, which surprisingly is not always beneficial.

  • Causal Inference for Complex Real-world Questions | Chan Park

    Special Seminar Series

    Abstract: Causal inference has gained popularity due to its ability to draw causal conclusions for real-world questions. While much of the existing work in causal inference revolves around determining average treatment effects under no interference and no unmeasured confounding assumptions, these approaches may not be suitable for answering causal questions in complex settings. In the talk, we focus on the following three topics, each accompanied by relevant applications.

    First, we discuss causal inference under interference and non-i.i.d. settings. Interference refers to a phenomenon where one’s outcome is affected by others’ treatment status; for instance, one’s own risk of flu infection is affected by family members’ flu vaccination status. We explore two estimators, a point-estimator and a bound-estimator, that are developed to infer causal effects in the presence of interference.

    Second, we explore causal inference under the presence of unmeasured confounders, variables that affect both treatment status and outcome. We demonstrate that ignoring the potential existence of unmeasured confounding can lead to incorrect causal conclusions using data on the Zika virus outbreak. Recently developed frameworks, including universal difference-in-differences and single proxy control, are discussed to address the challenges posed by unmeasured confounding.

  • Uncertainty Quantification for Interpretable Machine Learning | Lili Zheng

    Special Seminar Series

    Interpretable machine learning has been widely deployed for scientific discoveries and decision-making, while its reliability hinges on the critical role of uncertainty quantification (UQ). In this talk, I will discuss UQ in two challenging scenarios motivated by scientific and societal applications: selective inference for large-scale graph learning and UQ for model-agnostic machine learning interpretations. Specifically, the first part concerns graphical model inference when only irregular, patchwise observations are available, a common setting in neuroscience, healthcare, genomics, and econometrics. To filter out low-confidence edges due to the irregular measurements, I will present a novel inference method that quantifies the uneven edgewise uncertainty levels over the graph as well as an FDR control procedure; this is achieved by carefully disentangling the dependencies across the graph and consequently yields more reliable graph selection. In the second part, I will discuss the computational and statistical challenges associated with UQ for feature importance of any machine learning model. I will take inspiration from recent advances in conformal inference and utilize an ensemble framework to address these challenges. This leads to an almost computationally free, assumption-light, and statistically powerful inference approach for occlusion-based feature importance. For both parts of the talk, I will highlight the potential applications of my research in science and society as well as how it contributes to more reliable and trustworthy data science.

  • Canceled TILOS Seminar: Medical image reconstruction via deep learning: new architectures, data reduction and theoretical guarantees

    TILOS Seminar Series
    Computer Science & Engineering Building (CSE), Room 2154 3234 Matthews Ln, La Jolla, CA, United States

    In this talk I will discuss the challenges and opportunities for using deep learning in medical image reconstruction. Contemporary techniques in this field rely on convolutional architectures that are limited by the spatial invariance of their filters and have difficulty modeling long-range dependencies. To remedy this, I will discuss our work on designing new transformer-based architectures called HUMUS-Net that lead to state of the art performance and do not suffer from these limitations. In the next part of the talk I will report on techniques to significantly reduce the required data for training. Finally, I will briefly discuss our recent attempts to develop rigorous theory for simple end-to-end training methods used in image reconstruction problems which is surprisingly quite challenging even for simple target functions. Notability, our theory will be in the rich (or beyond NTK regime) that conforms with practical choice of hyperparameters. Time permitting I will discuss other exciting directions for the use of deep learning in MR.

  • Inference and Decision-Making amid Social Interactions | Shuangning Li

    Special Seminar Series
    Computer Science & Engineering Building (CSE), Room 4140 3234 Matthews Ln, La Jolla, CA, United States

    From social media trends to family dynamics, social interactions shape our daily lives. In this talk, I will present tools I have developed for statistical inference and decision-making in light of these social interactions.

    (1) Inference: I will talk about estimation of causal effects in the presence of interference. In causal inference, the term “interference” refers to a situation where, due to interactions between units, the treatment assigned to one unit affects the observed outcomes of others. I will discuss large-sample asymptotics for treatment effect estimation under network interference where the interference graph is a random draw from a graphon. When targeting the direct effect, we show that popular estimators in our setting are considerably more accurate than existing results suggest. Meanwhile, when targeting the indirect effect, we propose a consistent estimator in a setting where no other consistent estimators are currently available.

    (2) Decision-Making: Turning to reinforcement learning amid social interactions, I will focus on a problem inspired by a specific class of mobile health trials involving both target individuals and their care partners. These trials feature two types of interventions: those targeting individuals directly and those aimed at improving the relationship between the individual and their care partner. I will present an online reinforcement learning algorithm designed to personalize the delivery of these interventions. The algorithm's effectiveness is demonstrated through simulation studies conducted on a realistic test bed, which was constructed using data from a prior mobile health study. The proposed algorithm will be implemented in the ADAPTS HCT clinical trial, which seeks to improve medication adherence among adolescents undergoing allogeneic hematopoietic stem cell transplantation.

  • TILOS Seminar: The Dissimilarity Dimension: Sharper Bounds for Optimistic Algorithms

    TILOS Seminar Series

    Abstract: The principle of Optimism in the Face of Uncertainty (OFU) is one of the foundational algorithmic design choices in Reinforcement Learning and Bandits. Optimistic algorithms balance exploration and exploitation by deploying data collection strategies that maximize expected rewards in plausible models. This is the basis of celebrated algorithms like the Upper Confidence Bound (UCB) for multi-armed bandits. For nearly a decade, the analysis of optimistic algorithms, including Optimistic Least Squares, in the context of rich reward function classes has relied on the concept of eluder dimension, introduced by Russo and Van Roy in 2013. In this talk we shed light on the limitations of the eluder dimension in capturing the true behavior of optimistic strategies in the realm of function approximation. We remediate these by introducing a novel statistical measure, the “dissimilarity dimension”. We show it can be used to provide sharper sample analysis of algorithms like Optimistic Least Squares by establishing a link between regret and the dissimilarity dimension. To illustrate this, we will show that some function classes have arbitrarily large eluder dimension but constant dissimilarity. Our regret analysis draws inspiration from graph theory and may be of interest to the mathematically minded beyond the field of statistical learning theory. This talk sheds new light on the fundamental principle of optimism and its algorithms in the function approximation regime, advancing our understanding of these concepts.

  • Statistical insight for biomedical data science, with applications to single-cell RNA-sequencing data | Yiqun Chen

    Special Seminar Series

    My research centers around bringing statistical insights and understanding to the practice of modern data science, and I will cover two projects related to this research vision in this talk.

    The first part of the talk is motivated by the practice of testing data-driven hypotheses. In the biomedical sciences, it has become increasingly common to collect massive datasets without a pre-specified research question. In this setting, a data analyst might use the data both to generate a research question, and to test the associated null hypothesis. For example, in single-cell RNA-sequencing analyses, researchers often first cluster the cells, and then test for differences in the expected gene expression levels between the clusters to quantify up- or down-regulation of genes, annotate known cell types, and identify new cell types. However, this popular practice is invalid from a statistical perspective: once we have used the data to generate hypotheses, standard statistical inference tools are no longer valid. To tackle this problem, I developed a conditional selective approach to test for a difference in means between pairs of clusters obtained via k-means clustering.

    The proposed approach has appropriate statistical guarantees (e.g., selective Type 1 error control). In the second part of the talk, I will consider how to leverage large language models (LLMs) such as ChatGPT for biomedical discovery. While significant progress has been made in customizing large language models for biomedical data, these models often require extensive data curation and resource-intensive training. In the context of single-cell RNA-sequencing data, I will show that we can achieve surprisingly competitive results on many downstream tasks via a much simpler alternative: I input textual descriptions of genes into an off-the-shelf LLM, such as ChatGPT, to obtain low-dimensional representations of the genes, or “embeddings.” I then use these embeddings as features in downstream tasks. A similar approach enables LLM-derived embeddings of cells. This work highlights the potential of LLMs to provide meaningful and concise representations for biomedical data, and also raises a number of challenging statistical questions. Addressing these questions requires bringing principled statistical thinking to the practice of modern data science.

  • Algorithm Dynamics in Modern Statistical Learning: Universality and Implicit Regularization | Tianhao Wang

    Special Seminar Series

    Modern statistical learning is featured by the high-dimensional nature of data and over-parameterization of models. In this regime, analyzing the dynamics of the used algorithms is challenging but crucial for understanding the performance of learned models. This talk will present recent results on the dynamics of two pivotal algorithms: Approximate Message Passing (AMP) and Stochastic Gradient Descent (SGD). Specifically, AMP refers to a class of iterative algorithms for solving large-scale statistical problems, whose dynamics admit asymptotically a simple but exact description known as state evolution. We will demonstrate the universality of AMP's state evolution over large classes of random matrices, and provide illustrative examples of applications of our universality results. Secondly, for SGD, a workhorse for training deep neural networks, we will introduce a novel mathematical framework for analyzing its implicit regularization. This is essential for SGD's ability to find solutions with strong generalization performance, particularly in the case of over-parameterization. Our framework offers a general method to characterize the implicit regularization induced by gradient noise. Finally, in the context of underdetermined linear regression, we will show that both AMP and SGD can provably achieve sparse recovery, yet they do so from markedly different perspectives.