 |
Harry Dong
Senior Researcher, Microsoft Research (MSR)
I am a Senior Researcher at MSR AI Frontiers in NYC. I received my Ph.D. in Electrical and Computer Engineering (ECE) from Carnegie Mellon University (CMU) where I explored interests in efficient machine learning algorithms with my advisor, Yuejie Chi while frequently collaborating with Beidi Chen and folks at the Air Force Research Laboratory (AFRL). During my graduate studies, I worked on inference efficiency, reasoning, and AI for materials science by balancing performance and computation, which made up my thesis. Additionally during this time, I have also spent summers at Meta (GenAI/MSL), Apple, and AFRL as an intern. Prior to CMU, I graduated with High Distinction from UC Berkeley with degrees in statistics and computer science in 2021. While there, I did research broadly in optimization and stochastic modeling. Before that, I grew up in the California Central Valley where instead of sandy beaches and cool tech, I was spoiled with rolling hills and amazing produce. Feel free to reach me via email: hdong920 [at] gmail [dot] com CV / Google Scholar / LinkedIn / GitHub / X (Twitter) |
Research Overview
My research roughly aims to make powerful deep learning models more practical for use. My main focus is on the algorithmic side of LLM inference efficiency/scaling/performance by leveraging inherent structures and patterns within the architecture, data, and/or pretrained weights. To help with this, I also like to uncover and understand the various subtle idiosyncrasies of these models. I have also devoted a significant amount of time investigating how LLMs and diffusion models can be applied to challenging science problems, particularly in materials science, where there may be physical constraints and low error tolerance. Previously, I have worked on various provable optimization methods for estimation, traffic routing, and neuroscience.
Areas
AI Efficiency: Efficient algorithms/architectures to reduce inference time and costs, with a focus on LLMs.
Reasoning: Understanding and improving inference scaling behavior of LLMs, with a focus on reasoning.
Machine Learning in Materials Science: Scalable methods for materials data that have underlying physics relationships.
Recent News
Aug 2026: I'll be co-teaching a tutorial with my Ph.D. advisor, Yuejie Chi, at UAI on LLM reasoning/CoT. Hope to see you in Amsterdam!
Jun 2026: Joined Microsoft Research AI Frontiers in NYC.
Apr 2026: Defended my PhD thesis, thank you to my committee and everyone who tuned in!
Jan 2026: Named a CPAL Rising Stars Awardee.
Jan 2026: 2 papers accepted to ICLR and 1 accepted to CPAL. Thanks to my collaborators!
Dec 2025: Gave a talk on our paper at the NeurIPS 2025 Workshop on Efficient Reasoning and received a best paper nomination on this work.
May 2025: Started my internship at Meta GenAI in NYC.
Awards
CPAL Rising Stars Award (2026)
Wei Shen and Xuehong Zhang Presidential Fellowship (2024)
Liang Ji-Dian Graduate Fellowship (2023)
Michel and Kathy Doreau Graduate Fellowship (2023)
NSF GRFP Honorable Mention (2023)
UC Berkeley High Distinction (2021)
Research Highlights
 |
Generalized Parallel Scaling with Interdependent Generations |
 |
Get More with LESS: Synthesizing Recurrence with KV Cache Compression for Efficient LLM Inference |
 |
Prompt-prompted Adaptive Structured Pruning for Efficient LLM Generation |
 |
A Lightweight Transformer for Faster and Robust EBSD Data Collection |
 |
Fast and Provable Tensor Robust Principal Component Analysis via Scaled Gradient Descent |