Shiva Kaul - Machine Learning Researcher

About me

I work on computational and statistical methods at the intersection of machine learning and healthcare (as described below). I graduated from the Ph.D. program at Carnegie Mellon's Computer Science Department. My thesis developed syntheses between classical and modern machine learning techniques which achieved "best of both worlds" results in terms of safety and accuracy. My advisor was the razor-sharp, incredibly-patient Geoff Gordon. Earlier, I earned an M.S. under Mahadev Satyanarayanan on the topic of human-in-the-loop machine learning. I was fortunate to spend time at Microsoft Research under Denny Zhou developing statistical learning theory for hierarchical classifiers.

Current work on meta-analysis

I am designing a system which answers questions about healthcare interventions in a rigorous, comprehensive manner. "Rigorous" requires meaningful, falsifiable guarantees about how well the causal effect is estimated. "Comprehensive" means using as much data as possible. Unfortunately, the two dominant paradigms for answering such questions — language models in artificial intelligence, and meta-analysis in medicine — don't have these properties. Language models don't enjoy correctness guarantees; meanwhile, to preserve their causal rigor, meta-analyses are restricted solely to randomized controlled trials. The solution is a careful fusion of these approaches which I call conformal meta-analysis.

This answers the question comprehensively by involving both randomized trials and observational data. Even if the latter are confounded, the resulting predictions are supported by rigorous correctness guarantees. See the paper for more details.

Multiple research groups and I are collaborating to achieve this vision.

An important baseline question to ask is: how well is meta-analysis currently working in evidence-based medicine? Along with Jimeng Sun and Zifeng Wang at University of Illinois, Urbana-Champaign, I am performing a large-scale empirical assessment of the predictive value of meta-analysis. Besides being interesting in its own right, this will establish a baseline for conformal meta-analysis to surpass.
Along with Rae Woong Park and Min-Gyu Kim at Ajou University, I am reconducting various meta-analyses with the aid of electronic health record data from South Korea's integrated hospital network. We believe conformal meta-analysis will deliver superior results in these applications.
In addition to these empirical projects, I am continuing to develop new statistical methodology as well. Soon, Maya Mathur (at Stanford University) and I will examine how meta-analytic prediction intervals must change to account for publication bias and related meta-scientific issues.

Selected papers

Meta-Analysis with Untrusted Data By Shiva Kaul and Geoffrey J. Gordon. ML4H 2024. Full arXiv version
Classical Improvements to Modern Machine Learning Ph.D. Thesis, CMU-CS-24-137
Linear Dynamical Systems as a Core Computational Primitive By Shiva Kaul. NeurIPS 2020. Selected for Spotlight Presentation. Code
Measuring the Sympathetic Response to Intense Exercise in a Practical Setting By Shiva Kaul, Anthony Falco, and Karianne Anthes. MLHC 2019
Margins and Opportunity By Shiva Kaul. AAAI/AIES 2018. Selected for Doctoral Consortium

Selected talks

Meta-Analysis with Untrusted Data Sunlab, University of Illinois, September 2024
Classical Improvements to Modern Machine Learning Ph.D. Thesis Defense, August 2024. Full PPT
Linear Dynamical Systems as a Core Computational Primitive NeurIPS Spotlight, December 2020. Poster and Video
Trusting Modern Machine Learning Allergan Scientific Series, January 2020
Symmetric Tensor Rank SELECT LAB, Carnegie Mellon, Spring 2013
Semidefinite Programming Hierarchies for Polynomial Programs 10-725 Optimization Lecture, Carnegie Mellon, Fall 2012
Conic Duality 10-725 Optimization Recitation, Carnegie Mellon, Fall 2012
Anticoncentration Regularizers for Stochastic Combinatorial Problems NIPS 2011 Workshop on Computational-Statistical Tradeoffs, December 2011
Electronic Structure Theory, Quickly SELECT LAB, Carnegie Mellon, January 2011

Open source

I am the creator and maintainer of the Rust CLI tool mky. Based on the same cryptographic infrastructure as passkeys, mky allows you to sign, encrypt, and verify messages using unphishable secret keys embedded on your mobile phone. (Like a Yubikey would, but you don't have to pay $50, carry it around, and worry about losing it). This means more secure ssh, scp, git commit signing, and age encryption — all in one easy-to-use bundle available on Linux, Mac, and Windows.

I make some effort to push changes and fixes upstream. For example, I'm working on some new deployment features for Bun and uWebSockets. I've merged small fixes to OpenSSH, agenix, various NixOS packages, torchdiffeq, torch, torch7, etc.

Teaching

I was a teaching assistant for following courses at Carnegie Mellon:

10-725 Optimization, a graduate-level course in optimization for machine learning researchers. Taught by Geoff Gordon and Ryan Tibshirani; assisted by myself, Wooyoung Lee, Aaditya Ramdas, and Kevin Waugh.
15-359 Probability and Computing, an undergraduate-level course in probability and stochastic processes. It was taught by Mor Harchol-Balter and Klaus Sutner.

I was the lead instructor for the following courses:

IDEA MATH: Playing Prediction Games, a volunteer course taught at a summer camp for talented high school students.
98-085 Ruby on Rails, a student-led, twice-offered undergraduate course on the (at the time) brand-new, highly-productive web application framework.

I created some concise measure-theoretic probability flashcards, while reading an assortment of books and taking a couple courses on the subject.

Personal

I have participated in distance running, CrossFit, and (these days) powerlifting / barbell training. I enjoy cooking and taking care of Arktos, my Samoyed dog.

Contact

Email is preferred.