Contact Us

Give us a call or drop by anytime, we endeavor to answer all inquiries within 24 hours.

map

Find us

PO Box 16122 Collins Street West Victoria, Australia

Email us

info@domain.com / example@domain.com

Phone support

Phone: + (066) 0760 0260 / + (057) 0760 0560

Fireside Chat

Events Search and Views Navigation

Event Views Navigation

Today

Filters

Changing any of the form inputs will cause the list of events to refresh with the filtered results.

Latest Past Events

Flat minima and generalization in deep learning: a case study in low rank matrix recovery

Halıcıoğlu Data Science Institute Building, Room 123 3234 Matthews Lane, La Jolla

Abstract: Recent advances in machine learning and artificial intelligence have relied on fitting highly overparameterized models, notably deep neural networks, to observed data. In such settings, the number of parameters of the model is much greater than the number of data samples, thereby resulting in a continuum of models with near-zero training error. Understanding which of these models generalize well and which do not is the central open question in deep learning. Recent empirical evidence suggests one mechanism for generalization: the shape of the training loss around a local minimizer seems to strongly impact the model’s performance. In particular, flat minima -- those around which the loss grows slowly -- appear to generalize well. Clarifying this phenomenon can shed new light on generalization in deep learning, which still largely remains a mystery.

I will describe our recent work that takes a step towards this goal by focusing on the simplest class of overparameterized nonlinear models: those arising in low-rank matrix recovery. We analyze overparameterized matrix and bilinear sensing, robust PCA, covariance matrix estimation, and single hidden layer neural networks with quadratic activation functions. In all cases, we show that flat minima, measured by the trace of the Hessian, exactly recover the ground truth under standard statistical assumptions. These results suggest (i) a theoretical basis for favoring methods that bias iterates towards flat solutions and (ii) use of Hessian trace as a good regularizer for some learning tasks. We end by discussing the impact of depth on the generalization properties of flat solutions, which surprisingly is not always beneficial.

Detection and recovery of low-rank signals under heteroskedastic noise

Halıcıoğlu Data Science Institute Building, Room 123 3234 Matthews Lane, La Jolla

A fundamental task in data analysis is to detect and recover a low-rank signal in a noisy data matrix. Typically, this task is addressed by inspecting and manipulating the spectrum of the observed data, e.g., thresholding the singular values of the data matrix at a certain critical level. This approach is well-established in the case of homoskedastic noise, where the noise variance is identical across the entries. However, in numerous applications, such as single-cell RNA sequencing (scRNA-seq), the noise can be heteroskedastic, where the noise characteristics vary considerably across the rows and columns of the data. In such scenarios, the noise spectrum can differ significantly from the homoskedastic case, posing various challenges for signal detection and recovery. In this talk, I will present a procedure for standardizing the noise spectrum by judiciously scaling the rows and columns of the data. Importantly, this procedure can provably enforce the standard spectral behavior of homoskedastic noise -- the Marchenko-Pastur law. I will describe methods for estimating the required scaling factors directly from the observed data with suitable theoretical justification, and demonstrate the advantages of the proposed approach for signal detection and recovery in simulations and on real scRNA-seq data.

Improving Technical Communication with End Users about Differential Privacy

Halıcıoğlu Data Science Institute Building, Room 123 3234 Matthews Lane, La Jolla

Differential privacy (DP) is widely regarded as a gold standard for privacy-preserving computation over users’ data. A key challenge with DP is that its mathematical sophistication makes its privacy guarantees difficult to communicate to users, leaving them uncertain about how and whether they are protected. Despite recent widespread deployment of DP, relatively little is known about what users think of differential privacy and how to effectively communicate the practical privacy guarantees it offers.

This talk will cover a series of recent and ongoing user studies aimed at measuring and improving communication with non-technical end users about differential privacy. The first set explores users' privacy expectations related to differential privacy and measures the efficacy of existing methods for communicating the privacy guarantees of DP systems. We find that users care about the kinds of information leaks against which differential privacy protects and are more willing to share their private information when the risk of these leaks is reduced. Additionally, we find that the ways in which differential privacy is described in-the-wild set users' privacy expectations haphazardly, which can be misleading depending on the deployment. Motivated by these findings, the second set of user studies develops and evaluates prototype descriptions designed to help end users understand DP guarantees. These descriptions target two important technical details in DP deployments that are often poorly communicated to end users: the privacy parameter epsilon (which governs the level of privacy protections) and the distinctions between the local and central models of DP (which governs who can access exact user data). Based on joint works with Gabriel Kaptchuk, Priyanka Nanayakkara, Elissa Redmiles, Mary Anne Smart, including https://arxiv.org/abs/2110.06452 and https://arxiv.org/abs/2303.00738.