# Anurag Khandelwal

> Anurag Khandelwal is an Associate Professor of Computer Science at Yale University, where he leads the NOVA Lab. His research spans computer systems, networks, architecture, and security, including memory disaggregation, systems for AI and LLM serving, secure cloud systems, and serverless computing.

Canonical website: https://www.anuragkhandelwal.com/
Email: anurag.khandelwal@yale.edu

## Biography

Anurag Khandelwal is an Associate Professor of Computer Science at Yale University, where his group focuses broadly on problems in computer systems, networks and security. He received his Ph.D. from UC Berkeley, where he was advised by Ion Stoica. His work has been recognized by an NSF CAREER award, two NetApp faculty fellowships, a Roberts Innovation Award, paper and artifact distinctions at USENIX Security’20, ISCA’23, EuroSys’24, ISCA’25, ASPLOS'26, and MobiSys'26, and IEEE Micro Top Picks selections in 2023 and 2025.

## Research directions

### Memory Disaggregation

Scaling resources at server granularity wastes resources when compute and memory needs are mismatched, increasing both costs and carbon emissions. Memory disaggregation separates compute and memory into shared network-connected pools to improve resource efficiency, capacity, and elasticity. We are rethinking the cloud software and hardware stack to realize memory disaggregation:

- Scalable cache coherence for disaggregated shared memory pools: [MIND](https://www.anuragkhandelwal.com/papers/mind/) (SOSP'21), [CORD](https://www.anuragkhandelwal.com/papers/cord/) (ISCA'25), [Soul](https://www.anuragkhandelwal.com/papers/soul/) (OSDI'26)
- Fair sharing across disaggregated memory resources: [Karma](https://www.anuragkhandelwal.com/papers/karma/) (OSDI'23), [Spirit](https://www.anuragkhandelwal.com/papers/spirit/) (SOSP'25)
- Low-latency, high-throughput access to disaggregated memory: [PULSE](https://www.anuragkhandelwal.com/papers/pulse/) (ASPLOS'25), [Mage](https://www.anuragkhandelwal.com/papers/mage/) (SOSP'25)
- Understanding memory performance: [BulletTime](https://www.anuragkhandelwal.com/papers/bullettime/) (ISCA'26), [CounterPoint](https://www.anuragkhandelwal.com/papers/counterpoint/) (ASPLOS'26)

Our work on scalable cache coherence for disaggregated memory has been incorporated into [NVIDIA's Vera Rubin Architecture](https://developer.nvidia.com/blog/inside-nvidia-rubin-gpu-architecture-powering-the-era-of-agentic-ai/#accelerated_scale-up_communications).

### Secure cloud systems

As more privacy-sensitive applications move storage and computation to the cloud, encrypted data and secure enclaves can still leak sensitive information through access patterns. Existing defenses are often too expensive in bandwidth or storage to deploy widely. Our research studies these real-world access-pattern vulnerabilities and designs more efficient protections against them.

- Protecting against access pattern vulnerabilities: [Pancake](https://www.anuragkhandelwal.com/papers/pancake/) (USENIX Sec'20), [Shortstack](https://www.anuragkhandelwal.com/papers/shortstack/) (OSDI'22), [Weave](https://www.anuragkhandelwal.com/papers/weave/) (OSDI'25)
- Understanding access pattern vulnerabilities: [Length Leakage](https://www.anuragkhandelwal.com/papers/length-leakage/) (USENIX Sec'24), [Found in Translation](https://www.anuragkhandelwal.com/papers/fit/) (USENIX Sec'25)

### Systems for AI

Today’s AI serving systems waste substantial time and resources because they treat requests as independent and unpredictable, even though real workloads contain rich recurring structure in both arrival patterns and prompt content. We are building cloud AI serving platforms that treat workload structure as a first-class systems primitive:

- Scheduling for low-latency, high-throughput AI inference: [Shepherd](https://www.anuragkhandelwal.com/papers/shepherd/) (NSDI'23), [TimelyLLM](https://www.anuragkhandelwal.com/papers/timelyllm/) (MobiSys'26)
- Caching attention state across prompts for low-latency inference: [PromptCache](https://www.anuragkhandelwal.com/papers/promptcache/) (MLSys'24)

[PromptCache](https://www.anuragkhandelwal.com/papers/promptcache/) has been adopted in Gemini, OpenAI, and Anthropic for reusing attention states across LLM prompts.

### Storage and processing stacks for automated data

Emerging applications that rely on automated data sources --- ranging from smart vehicles to brain implants --- require processing, storing, and serving massive volumes of semantically rich data. We are developing systems for efficient ingestion of data without compromising query and processing performance by exploiting properties specific to machine-generated data:

- High-throughput compressed storage of high-dimensional data: [Trinity](https://www.anuragkhandelwal.com/papers/trinity/) (EuroSys'24)
- Distributed system for scalable Brain-Computer Interfacing (BCI): [SCALO](https://www.anuragkhandelwal.com/papers/scalo/) (ISCA'23), [MICRO Top Picks'23](https://www.anuragkhandelwal.com/papers/scalo-toppicks/)
- Distributed monitoring & diagnosis for high speed networks: [Confluo](https://www.anuragkhandelwal.com/papers/confluo/) (NSDI'19)

### Serverless Systems (past)

Serverless analytics workloads increasingly demand fine-grained, rapidly changing compute and memory resources, but existing cloud systems manage them too coarsely, forcing a tradeoff between performance and utilization under bursty, time-varying demand. We built a serverless analytics stack that treats elasticity and workload-aware multiplexing as first-class primitives:

- Position papers: [Berkeley View on Serverless](https://www.anuragkhandelwal.com/papers/berkeley-view-serverless/) (Tech Report), [Le Taureau](https://www.anuragkhandelwal.com/papers/sigmodserverless/) (SIGMOD'20), [What Serverless Should Become](https://www.anuragkhandelwal.com/papers/serverless-next/) (CACM'21)
- Enabling fast and cost-effective analytics over serverless functions: [Caerus](https://www.anuragkhandelwal.com/papers/caerus/) (NSDI'21), [Jiffy](https://www.anuragkhandelwal.com/papers/jiffy/) (EuroSys'22)

### Queries on compressed data (past)

As datasets grow beyond DRAM capacity, maintaining interactive query performance becomes difficult because spilling to slower secondary storage increases latency and lowers throughput. We developed systems that address this challenge using a fundamentally new approach: enabling rich query execution directly on compressed data, reducing the need to fully decompress or rely on large DRAM footprints.

- Enabling queries on compressed data: [Succinct](https://www.anuragkhandelwal.com/papers/succinct/) (NSDI'15), [ZipG](https://www.anuragkhandelwal.com/papers/zipg/) (SIGMOD'17), [Queries on Compressed Data](https://www.anuragkhandelwal.com/papers/phdthesis/) (PhD Thesis)
- Dynamic storage-performance tradeoff in data stores: [BlowFish](https://www.anuragkhandelwal.com/papers/blowfish/) (NSDI'16)

## Real-world impact

- 2025: [CORD](https://www.anuragkhandelwal.com/papers/cord/) incorporated in the design of [NVIDIA's Vera Rubin architecture](https://developer.nvidia.com/blog/inside-nvidia-rubin-gpu-architecture-powering-the-era-of-agentic-ai/#accelerated_scale-up_communications).
- 2024: [PromptCache](https://www.anuragkhandelwal.com/papers/promptcache/) adopted in [Gemini](https://ai.google.dev/gemini-api/docs/caching?lang=python), [OpenAI](https://platform.openai.com/docs/guides/prompt-caching), and [Anthropic](https://www.anthropic.com/news/prompt-caching) for reusing attention states across LLM prompts.
- 2023: The ideas underlying [SCALO](https://www.anuragkhandelwal.com/papers/scalo/) have been tested by Yale Neurosurgery in FPGA-based emulation frameworks for interfacing with the human brain in the operating room.
- 2021: [MIND](https://www.anuragkhandelwal.com/papers/mind/) is the foundational system and central pillar of the multi-institution $20-million NSF-funded AI Institute, [Athena](https://athena.duke.edu/what-we-do/edge-powered-ai-systems).
- 2019: [Confluo](https://www.anuragkhandelwal.com/papers/confluo/) employed in [Alibaba Cloud](https://medium.datadriveninvestor.com/high-speed-querying-with-confluo-from-riselab-445bed3f7f9a) for consuming and analyzing telemetry data.
- 2015: [Succinct](https://www.anuragkhandelwal.com/papers/succinct/) employed by [Databricks](https://www.databricks.com/blog/2015/11/10/succinct-spark-from-amplab-queries-on-compressed-rdds.html) [customers](https://spark-packages.org/package/amplab/succinct) to query compressed RDDs.

## Awards and honors

- Best Paper Award, ACM ASPLOS 2026: "CounterPoint: Using Hardware Event Counters to Refute & Refine Microarchitectural Assumptions"
- Best Paper Award Runner-Up, Best Artifact Award Runner-Up, ACM MobiSys 2026: "TimelyLLM: Time-sensitive LLM Serving System for Physical-I/O Limited Agents"
- IEEE Micro Top Picks in Computer Architecture, 2025: "CORD: Low-Latency, Bandwidth-Efficient and Scalable Release Consistency via Directory Ordering"
- Distinguished Artifact Award, ACM ISCA 2025: "CORD: Low-Latency, Bandwidth-Efficient and Scalable Release Consistency via Directory Ordering"
- NetApp Faculty Fellowship, 2024: To develop GPU-native vector storage and indexing for accelerating AI workloads.
- Best Student Paper Award, ACM EuroSys 2024: "TRINITY: A Fast Compressed Multi-attribute Data Store"
- IEEE Micro Top Picks in Computer Architecture, 2023: "SCALO: An Accelerator-Rich Distributed System for Scalable Brain-Computer Interfaces"
- Best Paper Award, ACM ISCA 2023: "SCALO: An Accelerator-Rich Distributed System for Scalable Brain-Computer Interfaces"
- Roberts Innovation Award, 2023: For innovations in data center resource disaggregation.
- NSF CAREER Award, 2021: For developing in-network memory management for disaggregated data centers.
- NetApp Faculty Fellowship, 2021: For developing elastic and compressed disaggregated memory for edge data centers.
- Distinguished Paper Award, USENIX Security 2020: "Pancake: Frequency Smoothing for Encrypted Data Stores"
- Madan Sundar Sahu Scholarship 2010, 2011, Indian Institute of Technology, Kharagpur: For outstanding academic performance during 2009-2010 and 2010-2011.

## Publications

Each paper has a landing page with its abstract, venue, DOI, PDF, and BibTeX.

- [Bringing Distributed Ordering to Heterogeneous Memory Consistency](https://www.anuragkhandelwal.com/papers/cord-toppicks/). IEEE Micro, 2026.
- [Efficient and Scalable Synchronization via Generalized Cache Coherence](https://www.anuragkhandelwal.com/papers/soul/). USENIX OSDI, 2026.
- [BulletTime: Time Dilation for High-Fidelity Tracing](https://www.anuragkhandelwal.com/papers/bullettime/). ACM ISCA, 2026.
- [TimelyLLM: Time-sensitive LLM Serving System for Physical-I/O Limited Agents](https://www.anuragkhandelwal.com/papers/timelyllm/). ACM MobiSys, 2026. Best Paper Award Runner-Up, Best Artifact Award Runner-Up.
- [CounterPoint: Using Hardware Event Counters to Refute & Refine Microarchitectural Assumptions](https://www.anuragkhandelwal.com/papers/counterpoint/). ACM ASPLOS, 2026. Best Paper Award.
- [Spirit: Fair Allocation of Interdependent Resources in Remote Memory Systems](https://www.anuragkhandelwal.com/papers/spirit/). ACM SOSP, 2025.
- [Scalable Far Memory: Balancing Faults and Evictions](https://www.anuragkhandelwal.com/papers/mage/). ACM SOSP, 2025.
- [Found in Translation: A Generative Language Modeling Approach to Memory Access Pattern Attacks](https://www.anuragkhandelwal.com/papers/fit/). USENIX Security, 2025.
- [Weave: Efficient and Expressive Oblivious Analytics at Scale](https://www.anuragkhandelwal.com/papers/weave/). USENIX OSDI, 2025.
- [CORD: Low-Latency, Bandwidth-Efficient and Scalable Release Consistency via Directory Ordering](https://www.anuragkhandelwal.com/papers/cord/). ACM ISCA, 2025. Distinguished Artifact Award, IEEE Micro’s Top Picks in Computer Architecture 2025.
- [PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory](https://www.anuragkhandelwal.com/papers/pulse/). ACM ASPLOS, 2025.
- [Exploring Intelligent Dynamic Resource Provisioning for Elastic Massive MIMO vRAN](https://www.anuragkhandelwal.com/papers/elasticmimo/). ACM HotMobile, 2025.
- [Length Leakage in Oblivious Data Access Mechanisms](https://www.anuragkhandelwal.com/papers/length-leakage/). USENIX Security, 2024.
- [Prompt Cache: Modular attention reuse for low-latency inference](https://www.anuragkhandelwal.com/papers/promptcache/). MLSys, 2024.
- [Distributed Brain-Computer Interfacing With a Networked Multi-accelerator Architecture](https://www.anuragkhandelwal.com/papers/scalo-toppicks/). IEEE Micro, 2024.
- [Trinity: A Fast Compressed Multi-attribute Data Store](https://www.anuragkhandelwal.com/papers/trinity/). ACM EuroSys, 2024. Best Student Paper Award.
- [Karma: Resource Allocation for Dynamic Demands](https://www.anuragkhandelwal.com/papers/karma/). USENIX OSDI, 2023.
- [SCALO: An Accelerator-Rich Distributed System for Scalable Brain-Computer Interfacing](https://www.anuragkhandelwal.com/papers/scalo/). ACM ISCA, 2023. Best Paper Award, IEEE Micro’s Top Picks in Computer Architecture 2023.
- [Shepherd: Serving DNNs in the Wild](https://www.anuragkhandelwal.com/papers/shepherd/). USENIX NSDI, 2023.
- [Prefetching Using Principles of Hippocampal-Neocortical Interaction](https://www.anuragkhandelwal.com/papers/cls/). ACM HotOS, 2023.
- [Shortstack: Distributed, Fault-tolerant, Oblivious Data Access](https://www.anuragkhandelwal.com/papers/shortstack/). USENIX OSDI, 2022.
- [Jiffy: Elastic Far-memory for Stateful Serverless Analytics](https://www.anuragkhandelwal.com/papers/jiffy/). ACM EuroSys, 2022.
- [Caerus: Nimble Task Scheduling for Serverless Analytics](https://www.anuragkhandelwal.com/papers/caerus/). USENIX NSDI, 2021.
- [What Serverless Computing Is and Should Become: The Next Phase of Cloud Computing](https://www.anuragkhandelwal.com/papers/serverless-next/). Communications of the ACM, 2021.
- [MIND: In-Network Memory Management for Disaggregated Data Centers](https://www.anuragkhandelwal.com/papers/mind/). ACM SOSP, 2021.
- [Pancake: Frequency Smoothing for Encrypted Data Stores](https://www.anuragkhandelwal.com/papers/pancake/). USENIX Security, 2020. Distinguished Paper Award.
- [Le Taureau: Deconstructing the Serverless Landscape & A Look Forward](https://www.anuragkhandelwal.com/papers/sigmodserverless/). ACM SIGMOD, 2020.
- [Queries on Compressed Data](https://www.anuragkhandelwal.com/papers/phdthesis/). University of California, Berkeley, 2019.
- [Cloud Programming Simplified: A Berkeley View on Serverless Computing](https://www.anuragkhandelwal.com/papers/berkeley-view-serverless/). UC Berkeley Technical Report, 2019.
- [Confluo: Distributed Monitoring and Diagnosis Stack for High-speed Networks](https://www.anuragkhandelwal.com/papers/confluo/). USENIX NSDI, 2019.
- [ZipG: A Memory-efficient Graph Store for Interactive Queries](https://www.anuragkhandelwal.com/papers/zipg/). ACM SIGMOD, 2017.
- [BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores](https://www.anuragkhandelwal.com/papers/blowfish/). USENIX NSDI, 2016.
- [Attacking Data Center Networks from the Inside](https://www.anuragkhandelwal.com/papers/attacking-dcn/). Microsoft Research Technical Report, 2015.
- [Succinct: Enabling Queries on Compressed Data](https://www.anuragkhandelwal.com/papers/succinct/). USENIX NSDI, 2015.

## Teaching

- CPSC 422/522: Operating Systems (Spring 2026, Spring 2025, Spring 2024)
- CPSC 433/533: Computer Networks (Spring 2022, Spring 2021, Spring 2020)
- CPSC 438/538: Big Data Systems (Fall 2025, Fall 2023, Fall 2021, Fall 2020)

## Prospective students

Anurag Khandelwal is looking for motivated graduate students and postdoctoral researchers. See https://www.anuragkhandelwal.com/group/ for current lab members.

## Primary pages

- Profile and news: https://www.anuragkhandelwal.com/
- Publications: https://www.anuragkhandelwal.com/publications/
- Research impact and awards: https://www.anuragkhandelwal.com/impact/
- NOVA Lab: https://www.anuragkhandelwal.com/group/
- Biography: https://www.anuragkhandelwal.com/bio/
- CV (PDF): https://www.anuragkhandelwal.com/cv/cv.pdf

## Authoritative identity links

- Yale Engineering: https://engineering.yale.edu/research-and-faculty/faculty-directory/anurag-khandelwal
- Google Scholar: https://scholar.google.com/citations?user=LlfHFqUAAAAJ&hl=en
- ORCID: https://orcid.org/0000-0002-2199-6391
- DBLP: https://dblp.org/pid/162/1936
- Semantic Scholar: https://www.semanticscholar.org/author/2712026
- GitHub: https://github.com/anuragkh
- Wikidata: https://www.wikidata.org/wiki/Q130845533

## Paper abstracts

### Bringing Distributed Ordering to Heterogeneous Memory Consistency

- Authors: Yanpeng Yu, Nicolai Oswald, Anurag Khandelwal
- Venue: IEEE Micro, 2026
- Page: https://www.anuragkhandelwal.com/papers/cord-toppicks/
- DOI: https://doi.org/10.1109/mm.2026.3701401

CORD is a novel cache-coherence protocol that brings distributed-systems techniques to tightly coupled heterogeneous processors, unlocking efficient and programmer-friendly shared memory for emerging heterogeneous workloads such as AI/ML. Cache-coherence protocols have historically enforced consistency at the source processor, inflating application latency and interconnect traffic. Central to CORD’s efficiency is its reframing of heterogeneous coherence as a distributed-systems problem: drawing on logical-timestamp algorithms for event ordering, CORD assigns consistency-tailored timestamps to write-through stores and orders them at the cache directory — near each store’s completion point at the LLC. In our evaluation, CORD approaches message-passing efficiency while incurring a fraction of the programming complexity, with negligible storage, area, and power overheads. The result is a concrete demonstration that long-standing architectural challenges in shared-memory designs can be tackled through the lens of distributed systems.

### Efficient and Scalable Synchronization via Generalized Cache Coherence

- Authors: Yanpeng Yu, Seung-seob Lee, Lin Zhong, Anurag Khandelwal
- Venue: USENIX OSDI, 2026
- Page: https://www.anuragkhandelwal.com/papers/soul/
- Code: https://github.com/yale-nova/Soul

Existing synchronization primitives scale poorly on disaggregated shared memory because they introduce redundant inter-cache communication over high-latency links. This work observes that synchronization generalizes cache coherence in time and space and minimally extends coherence protocols to support synchronization directly. Its Generalized cache-Coherence Protocol uses wait queues and variable-size cache lines for the temporal and spatial generalizations and is verified with model checking. Soul implements the protocol end to end and supports popular lock APIs without application changes. Against state-of-the-art locks, Soul improves unmodified real-world application performance at scale by one to two orders of magnitude with under 8 percent storage overhead.

### BulletTime: Time Dilation for High-Fidelity Tracing

- Authors: Michael Wu, Sibren Isaacman, Abhishek Bhattacharjee, Anurag Khandelwal
- Venue: ACM ISCA, 2026
- Page: https://www.anuragkhandelwal.com/papers/bullettime/
- DOI: https://doi.org/10.1109/ISCA66397.2026.00125
- Code: https://github.com/ysarch-lab/BulletTime

High-fidelity program tracing incurs frequent I/O that can delay application threads unevenly and reorder application and system operations relative to untraced execution, contaminating simulation and profiling results. This work formalizes the behavior under study and derives correctness requirements for traced execution in the presence of tracing delays. BulletTime implements a time-dilation approach atop Intel Pin that strategically slows application and system threads while meeting those requirements. In memory-contiguity and synchronization studies, existing tracing approaches change application behavior by as much as 20 times, while BulletTime keeps deviations below 10 percent even under extreme asymmetric delays.

### TimelyLLM: Time-sensitive LLM Serving System for Physical-I/O Limited Agents

- Authors: Neiwen Ling, Guojun Chen, Anurag Khandelwal, Lin Zhong
- Venue: ACM MobiSys, 2026
- Award: Best Paper Award Runner-Up, Best Artifact Award Runner-Up
- Page: https://www.anuragkhandelwal.com/papers/timelyllm/
- DOI: https://doi.org/10.1145/3745756.3809203
- Code: https://github.com/Neawhen/TimelyLLM

Large language models are increasingly integrated into physical-I/O-limited agents such as robots and voice assistants, yet existing serving systems optimize throughput without accounting for the gap between fast token generation and slow physical execution. TimelyLLM coordinates generation with agent behavior through segmented generation and scheduling, using the gap between plan generation and execution to reduce contention and response latency under multi-agent workloads. Built atop a widely used serving framework and evaluated with workloads from drones, robot arms, and quadrupeds, TimelyLLM improves time utility by up to 1.52 times and reduces overall waiting time by 84 percent.

### CounterPoint: Using Hardware Event Counters to Refute & Refine Microarchitectural Assumptions

- Authors: Nick Lindsay, Caroline Trippel, Anurag Khandelwal, Abhishek Bhattacharjee
- Venue: ACM ASPLOS, 2026
- Award: Best Paper Award
- Page: https://www.anuragkhandelwal.com/papers/counterpoint/
- DOI: https://doi.org/10.1145/3779212.3790145
- Code: https://github.com/NicholasLindsay/counterpoint-public

Hardware event counters can reveal detailed microarchitectural behavior, but vague specifications, opaque designs, and multiplexing noise make them hard to interpret. CounterPoint tests user-specified microarchitectural models, expressed as micro-path decision diagrams, for consistency with counter data. When observations disagree, it identifies plausible missing features and uses multidimensional confidence regions to handle multiplexing noise. Applied to the Haswell memory-management unit, CounterPoint uncovers multiple undocumented or underdocumented behaviors, including a load-store-queue-side TLB prefetcher, merging page-table walkers, and abortable page-table walks.

### Spirit: Fair Allocation of Interdependent Resources in Remote Memory Systems

- Authors: Seung-seob Lee, Jachym Putta, Ziming Mao, Anurag Khandelwal
- Venue: ACM SOSP, 2025
- Page: https://www.anuragkhandelwal.com/papers/spirit/
- DOI: https://doi.org/10.1145/3731569.3764805
- Code: https://github.com/yale-nova/spirit

Fair allocation in multi-user remote-memory systems is difficult because local cache and network bandwidth interact: more cache can reduce network traffic, while more bandwidth can reduce the need for cache. This dependency differs by application and is difficult to model offline. Spirit uses the microeconomics-inspired Symbiosis algorithm to learn these dependencies and trade cache and bandwidth across users at runtime. The paper shows theoretically and empirically that Symbiosis provides strong fairness properties. Compared with traditional allocation schemes, Spirit improves performance by up to 21.6 percent across dozens of applications with diverse resource needs.

### Scalable Far Memory: Balancing Faults and Evictions

- Authors: Yueyang Pan*, Yash Lala*, Musa Unal, Yujie Ren, Seung-seob Lee, Abhishek Bhattacharjee, Anurag Khandelwal, Sanidhya Kashyap
- Venue: ACM SOSP, 2025
- Page: https://www.anuragkhandelwal.com/papers/mage/
- DOI: https://doi.org/10.1145/3731569.3764842
- Code: https://github.com/rs3lab/mage-artifact

Page-based far-memory systems transparently expand application memory beyond one machine, but existing systems are designed for low thread counts and do not scale on modern multicore machines. The bottleneck is poor coordination between page fault-in and eviction operations, including TLB shootdowns, page accounting, and memory allocation. This paper develops three principles: always-asynchronous decoupling, cross-batch pipelined execution, and scalability prioritization. Implementations in the Linux kernel and a library OS improve batch-processing throughput by up to 4.2 times and reduce 99th-percentile latency for a latency-critical memcached workload by 94.5 percent.

### Found in Translation: A Generative Language Modeling Approach to Memory Access Pattern Attacks

- Authors: Grace Jia, Alex Wong, Anurag Khandelwal
- Venue: USENIX Security, 2025
- Page: https://www.anuragkhandelwal.com/papers/fit/
- Code: https://github.com/yale-nova/found-in-translation

Confidential-computing environments protect data and computation but leave paging under a cloud-controlled operating system, creating a page-access side channel. Earlier attacks largely assume simple mappings between application objects and pages and overlook correlations in access sequences. Found in Translation identifies parallels between page-access patterns and natural-language structure and uses a recurrent encoder-decoder to infer application-level object accesses from page-level sequences. Across AI model serving and semantic-search applications, FIT reconstructs object sequences with 71.7–99.9 percent average accuracy, substantially outperforming prior approaches.

### Weave: Efficient and Expressive Oblivious Analytics at Scale

- Authors: Mahdi Soleimani, Grace Jia, Anurag Khandelwal
- Venue: USENIX OSDI, 2025
- Page: https://www.anuragkhandelwal.com/papers/weave/
- Code: https://github.com/yale-nova/weave

Distributed cloud analytics can leak sensitive information through memory access patterns even when computation runs in trusted enclaves and all data and communication are encrypted. Prior protections often impose overheads logarithmic in dataset size and restrict the jobs that can run. Weave combines noise injection with hardware memory isolation to reduce network and compute overheads to a constant factor while preserving expressive analytics. It adds dataset- and workload-aware optimizations without weakening its guarantees and reduces end-to-end execution time by 4–10 times over prior state of the art on large real-world datasets.

### CORD: Low-Latency, Bandwidth-Efficient and Scalable Release Consistency via Directory Ordering

- Authors: Yanpeng Yu, Nicolai Oswald, Anurag Khandelwal
- Venue: ACM ISCA, 2025
- Award: Distinguished Artifact Award, IEEE Micro’s Top Picks in Computer Architecture 2025
- Page: https://www.anuragkhandelwal.com/papers/cord/
- DOI: https://doi.org/10.1145/3695053.3731074
- Code: https://github.com/yale-nova/CORD

Multi-processing-unit systems increasingly use cache-coherent shared memory and write-through accesses for producer-consumer communication. Existing release-consistency protocols order these writes at the source processor, causing unnecessary communication with the last-level-cache directory. CORD instead orders write-through accesses directly at the directory and uses new mechanisms to minimize ordering metadata across multiple directories. Gem5 evaluations show a 24 percent average performance improvement and 13 percent traffic reduction over source ordering with less than 1 percent storage, area, and power overhead. Compared with hand-optimized message passing, CORD incurs only about 3 percent performance overhead with a much simpler programming model.

### PULSE: Accelerating Distributed Pointer-Traversals on Disaggregated Memory

- Authors: Yupeng Tang, Seung-seob Lee, Abhishek Bhattacharjee, Anurag Khandelwal
- Venue: ACM ASPLOS, 2025
- Page: https://www.anuragkhandelwal.com/papers/pulse/
- DOI: https://doi.org/10.1145/3669940.3707253
- Code: https://github.com/yale-nova/pulse

CPU-side caches amortize network latency in disaggregated-memory systems but cannot help workloads that chase pointers through linked data structures. PULSE is a distributed pointer-traversal framework that moves these traversals closer to disaggregated memory while preserving expressiveness, energy efficiency, performance, and distributed execution. Evaluation shows low-latency, high-throughput, and energy-efficient execution for a broad range of pointer-traversal workloads that perform poorly with caching alone.

### Exploring Intelligent Dynamic Resource Provisioning for Elastic Massive MIMO vRAN

- Authors: Parthiban Annamalai, Minsung Kim, Anurag Khandelwal, Lin Zhong
- Venue: ACM HotMobile, 2025
- Page: https://www.anuragkhandelwal.com/papers/elasticmimo/
- DOI: https://doi.org/10.1145/3708468.3711876

Computational demand in virtualized radio-access networks changes rapidly, especially with massive MIMO, while current systems commonly provision CPU resources statically. This paper studies dynamic resource provisioning and integrates a reinforcement-learning controller into a real-time, fully software massive-MIMO vRAN prototype. Early experiments show no significant advantage over a simple steepest-gradient-descent controller, likely because current open-source vRAN platforms do not capture the full complexity of production systems. The paper identifies challenges and opportunities for more intelligent elastic provisioning.

### Length Leakage in Oblivious Data Access Mechanisms

- Authors: Grace Jia, Rachit Agarwal, Anurag Khandelwal
- Venue: USENIX Security, 2024
- Page: https://www.anuragkhandelwal.com/papers/length-leakage/

This paper studies how to prevent length leakage in oblivious-data-access mechanisms against passive, persistent adversaries. Protecting both object lengths and access patterns requires navigating a three-way tradeoff among storage footprint, bandwidth footprint, and information revealed to the adversary. The paper proves lower bounds on achievable storage and bandwidth for multiple leakage profiles and presents constructions that match or nearly match those bounds.

### Prompt Cache: Modular attention reuse for low-latency inference

- Authors: In Gim, Guojun Chen, Seung-seob Lee, Nikhil Sarda, Anurag Khandelwal, Lin Zhong
- Venue: MLSys, 2024
- Page: https://www.anuragkhandelwal.com/papers/promptcache/
- DOI: https://doi.org/10.48550/arXiv.2311.04934
- Code: https://github.com/yale-sys/prompt-cache

Prompt Cache accelerates large-language-model inference by reusing attention states across prompts. Many prompts share text such as system messages, templates, and context documents. By precomputing and storing attention states for these recurring segments, an inference server can reuse them when they appear in later prompts. Prompt Cache provides a schema for defining reusable prompt modules, ensuring positional accuracy and giving users an interface to access cached states. Across several models, the prototype reduces time-to-first-token latency by up to 8 times on GPU inference and 60 times on CPU inference while preserving output accuracy and requiring no model-parameter changes.

### Distributed Brain-Computer Interfacing With a Networked Multi-accelerator Architecture

- Authors: Raghavendra Pothukuchi, Karthik Sriram, Michał Gerasimiuk, Muhammed Ugur, Rajit Manohar, Anurag Khandelwal, Abhishek Bhattacharjee
- Venue: IEEE Micro, 2024
- Page: https://www.anuragkhandelwal.com/papers/scalo-toppicks/
- DOI: https://doi.org/10.1109/MM.2024.3411881

SCALO is a distributed brain-computer interface consisting of multiple wireless-networked implants on different brain regions. It opens new treatment options and avenues for studying brain-wide networks while preserving the tight power constraints of implanted devices. Its efficiency comes from a full-stack distributed design with accelerator-rich compute and cross-layer co-design of processing, networking, and storage, enabling fast distributed operation that had previously been impractical.

### Trinity: A Fast Compressed Multi-attribute Data Store

- Authors: Ziming Mao, Kiran Srinivasan, Anurag Khandelwal
- Venue: ACM EuroSys, 2024
- Award: Best Student Paper Award
- Page: https://www.anuragkhandelwal.com/papers/trinity/
- DOI: https://doi.org/10.1145/3627703.3650072
- Code: https://github.com/Trinity-data-store/Trinity

Attribute-rich machine-generated data drives real-time monitoring, diagnosis, and visualization tools that analyze multiple attributes at once. Trinity facilitates both query and storage efficiency across large volumes of multi-attribute records through MDTrie, a dynamic, succinct, multidimensional data structure combining a Morton-code generalization, a multi-attribute query algorithm, and a self-indexed trie. On real-world use cases, Trinity provides 7.2–59.6 times faster multi-attribute searches, storage comparable to OLAP column stores and 4.8–15.1 times smaller than NoSQL and OLTP systems, and point-query throughput comparable to NoSQL stores and up to 52.5 times higher than OLTP and OLAP systems.

### Karma: Resource Allocation for Dynamic Demands

- Authors: Midhul Vuppalapati, Giannis Fikioris, Rachit Agarwal, Asaf Cidon, Anurag Khandelwal,  Tardos
- Venue: USENIX OSDI, 2023
- Page: https://www.anuragkhandelwal.com/papers/karma/
- DOI: https://doi.org/10.48550/arXiv.2305.17222
- Code: https://github.com/resource-disaggregation/karma

Classical max-min fairness provides Pareto efficiency, strategy-proofness, and fairness when user demands are static, but loses one or more of these properties when demands vary over time. Karma is a resource-allocation mechanism for dynamic demands built around a credit system: users earn credits when others borrow their unused resources, and allocations are prioritized according to credit balances. The paper establishes guarantees for Pareto efficiency, strategy-proofness, and fairness, and evaluations on production workloads show that Karma minimizes performance disparity while maintaining Pareto-optimal system-wide performance.

### SCALO: An Accelerator-Rich Distributed System for Scalable Brain-Computer Interfacing

- Authors: Karthik Sriram, Raghavendra Pothukuchi, Michał Gerasimiuk, Muhammed Ugur, Oliver Ye, Rajit Manohar, Anurag Khandelwal, Abhishek Bhattacharjee
- Venue: ACM ISCA, 2023
- Award: Best Paper Award, IEEE Micro’s Top Picks in Computer Architecture 2023
- Page: https://www.anuragkhandelwal.com/papers/scalo/
- DOI: https://doi.org/10.1145/3579371.3589107
- Code: https://doi.org/10.5281/zenodo.7787128

SCALO is the first distributed brain-computer interface made of multiple wireless-networked implants placed on different brain regions. It enables new therapeutic and neuroscience applications while meeting the strict power and latency constraints that have historically limited interfaces to a single brain site. SCALO is designed as a full-stack distributed system with accelerator-rich compute and aggressive cross-layer hardware-software co-design across computation, networking, and storage. The result offers broader lessons for building energy-efficient networked systems with hardware accelerators from the ground up.

### Shepherd: Serving DNNs in the Wild

- Authors: Hong Zhang, Yupeng Tang, Anurag Khandelwal, Ion Stoica
- Venue: USENIX NSDI, 2023
- Page: https://www.anuragkhandelwal.com/papers/shepherd/

Model-serving systems must scale, guarantee high goodput, and maximize compute utilization despite tight latency constraints and unpredictable request streams. Shepherd decouples serving into planning and online-serving modules. The planner aggregates request streams into moderately sized groups whose demand is more predictable, enabling scalable utilization; the online algorithm uses preemption and model-specific batching to guarantee goodput under uncertainty. On production workloads, Shepherd delivers up to 18.1 times higher goodput and 1.8 times better utilization than prior systems while scaling to hundreds of workers.

### Prefetching Using Principles of Hippocampal-Neocortical Interaction

- Authors: Michael Wu, Ketaki Joshi, Andrew Sheinberg, Guilherme Cox, Anurag Khandelwal, Raghavendra Pradyumna Pothukuchi, Abhishek Bhattacharjee
- Venue: ACM HotOS, 2023
- Page: https://www.anuragkhandelwal.com/papers/cls/
- DOI: https://doi.org/10.1145/3593856.3595901

Memory prefetching improves performance across many systems layers, but achieving high accuracy with low overhead is increasingly difficult as memory hierarchies and application access patterns become more complicated. A prefetcher's ability to adapt to new access patterns is also becoming more important. Recent work has used deep learning to improve accuracy, albeit with impractical compute and storage costs. This paper proposes drawing on the learning mechanisms and memory architecture of the human brain, specifically the hippocampus and neocortex, to build resource-efficient, accurate, and adaptable prefetchers.

### Shortstack: Distributed, Fault-tolerant, Oblivious Data Access

- Authors: Midhul Vuppalapati*, Kushal Babel*, Anurag Khandelwal, Rachit Agarwal
- Venue: USENIX OSDI, 2022
- Page: https://www.anuragkhandelwal.com/papers/shortstack/
- DOI: https://doi.org/10.48550/arXiv.2205.14281
- Code: https://github.com/pancake-security/shortstack

Existing oblivious-data-access systems depend on a centralized, stateful trusted proxy; in failure-prone deployments this can break privacy guarantees or make the system unavailable. Shortstack introduces a distributed, fault-tolerant proxy architecture and a security model for studying such systems. It preserves the classical obliviousness guarantee even against a passive persistent adversary that can fail an arbitrary bounded subset of proxy servers at arbitrary times. Formal analysis establishes the guarantee, and evaluation shows that performance scales nearly linearly with the number of proxy servers.

### Jiffy: Elastic Far-memory for Stateful Serverless Analytics

- Authors: Anurag Khandelwal, Yupeng Tang, Rachit Agarwal, Aditya Akella, Ion Stoica
- Venue: ACM EuroSys, 2022
- Page: https://www.anuragkhandelwal.com/papers/jiffy/
- DOI: https://doi.org/10.1145/3492321.3527539
- Code: https://github.com/resource-disaggregation/jiffy

Stateful serverless analytics can use a remote-memory system for inter-task communication and intermediate data, but existing systems allocate memory at job granularity for the job's entire lifetime. This causes low utilization or degraded performance when intermediate-data sizes vary during execution. Jiffy is an elastic far-memory system that meets a job's instantaneous memory demand at seconds timescales. It efficiently multiplexes capacity across concurrent jobs, reducing slower persistent-storage accesses and improving execution time by 1.6–2.5 times on production workloads. The implementation runs on Amazon EC2, supports multiple distributed programming models, and natively supports a broad class of analytics applications on AWS Lambda.

### Caerus: Nimble Task Scheduling for Serverless Analytics

- Authors: Hong Zhang, Yupeng Tang, Anurag Khandelwal, Jingrong Chen, Ion Stoica
- Venue: USENIX NSDI, 2021
- Page: https://www.anuragkhandelwal.com/papers/caerus/

Serverless analytics requires optimizing job-completion time and execution cost rather than the utilization and isolation objectives of server-centric frameworks. Caerus addresses this new scheduling problem with the fine-grained NIMBLE algorithm, which pipelines tasks within a job and models pipelineable and non-pipelineable dependencies together with data generation, consumption, and processing rates. Across a broad range of analytics workloads, Caerus achieves both optimal cost and optimal job-completion time in practice.

### What Serverless Computing Is and Should Become: The Next Phase of Cloud Computing

- Authors: Johann Schleier-Smith, Vikram Sreekanti, Anurag Khandelwal, Joao Carreira, Neeraja J. Yadwadkar, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica, David A. Patterson
- Venue: Communications of the ACM, 2021
- Page: https://www.anuragkhandelwal.com/papers/serverless-next/
- DOI: https://doi.org/10.1145/3406011

Cloud computing's first phase simplified system administration by making infrastructure easier to configure and manage through virtual servers and networks. This article argues that the second phase should hide servers behind programming abstractions that simplify application development, shifting the cloud's focus from system administrators to programmers. Serverless computing asks cloud providers to assume more of the operational responsibility required to run applications well, while retaining remote servers as the invisible foundation. The article explains this transition, examines current limitations, and outlines what serverless computing should become.

### MIND: In-Network Memory Management for Disaggregated Data Centers

- Authors: Seung-seob Lee, Yanpeng Yu, Yupeng Tang, Anurag Khandelwal, Lin Zhong, Abhishek Bhattacharjee
- Venue: ACM SOSP, 2021
- Page: https://www.anuragkhandelwal.com/papers/mind/
- DOI: https://doi.org/10.1145/3477132.3483561
- Code: https://github.com/yale-nova/mind

Memory disaggregation promises transparent elasticity, high resource utilization, and hardware heterogeneity in data centers by physically separating memory and compute into network-attached resource blades. Existing designs, however, achieve performance at the cost of resource elasticity by restricting memory sharing to a single compute blade to avoid costly memory-coherence traffic. This work shows that emerging programmable network switches can enable an efficient shared-memory abstraction by placing memory-management logic in the network fabric. Centralizing memory management permits bandwidth- and latency-efficient in-network cache coherence while programmable switch ASICs support other management logic at line rate. MIND realizes these ideas as an in-network memory-management unit for rack-scale disaggregation, providing transparent resource elasticity while matching the performance of prior disaggregation proposals on real-world workloads.

### Pancake: Frequency Smoothing for Encrypted Data Stores

- Authors: Paul Grubbs*, Anurag Khandelwal*, Marie-Sarah Lacharité*, Lloyd Brown, Lucy Li, Rachit Agarwal, Thomas Ristenpart
- Venue: USENIX Security, 2020
- Award: Distinguished Paper Award
- Page: https://www.anuragkhandelwal.com/papers/pancake/
- Code: https://github.com/pancake-security/pancake

Pancake is the first system to protect key-value stores from access-pattern leakage attacks with small constant-factor bandwidth overhead. It uses frequency smoothing to transform plaintext accesses into uniformly distributed encrypted accesses and proves security against passive, persistent adversaries in a new formal model. Integrated with three production key-value stores, Pancake achieves 229 times the throughput of non-recursive Path ORAM and runs within 3–6 times of insecure baselines.

### Le Taureau: Deconstructing the Serverless Landscape & A Look Forward

- Authors: Anurag Khandelwal, Arun Kejariwal, Karthikeyan Ramasamy
- Venue: ACM SIGMOD, 2020
- Page: https://www.anuragkhandelwal.com/papers/sigmodserverless/
- DOI: https://doi.org/10.1145/3318464.3383130

Akin to the natural evolution of programming in assembly language to high-level languages, serverless computing represents the next frontier in the evolution of cloud computing: bare metal to virtual machines to containers to serverless. The genesis of serverless computing can be traced to the need to let programmers focus on application code while isolating system-management concerns such as instance selection, scaling, deployment, logging, monitoring, and fault tolerance. Although leading public-cloud vendors now support serverless computing, challenges remain, including dynamic optimization, stateful computation, efficient application bin-packing, and hardware heterogeneity. Inspired by Picasso's Le Taureau, this tutorial deconstructs the evolution of serverless to clarify the landscape and help advance both the paradigm and the applications built on it.

### Queries on Compressed Data

- Authors: Anurag Khandelwal
- Venue: University of California, Berkeley, 2019
- Page: https://www.anuragkhandelwal.com/papers/phdthesis/

Low-latency, high-throughput interactive query systems must serve massive datasets and increasingly rich operations. In-memory approaches are fast but falter once data exceeds DRAM capacity, while alternatives often support only limited queries. This dissertation develops algorithmic and data-structure solutions centered on querying compressed data. It presents Succinct for search, random access, range, and wildcard queries over compressed representations; BlowFish for dynamically trading storage against performance; ZipG for memory-efficient graph queries; and Sprint for regular expression queries through query rewriting.

### Cloud Programming Simplified: A Berkeley View on Serverless Computing

- Authors: Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, João Carreira, Karl Krauth, Neeraja J. Yadwadkar, Joseph E. Gonzalez, Raluca Ada Popa, Ion Stoica, David A. Patterson
- Venue: UC Berkeley Technical Report, 2019
- Page: https://www.anuragkhandelwal.com/papers/berkeley-view-serverless/
- DOI: https://doi.org/10.48550/arXiv.1902.03383

Serverless cloud computing handles nearly all system-administration work needed to make the cloud easier for programmers to use. It simplifies cloud programming in a way analogous to the shift from assembly to high-level languages. This paper reviews cloud computing's history, assesses the predictions of the 2009 Berkeley View, explains the motivation for serverless computing, describes applications that stretch current limits, and identifies obstacles and research opportunities. It argues that these problems are solvable and predicts that serverless computing will come to dominate cloud programming.

### Confluo: Distributed Monitoring and Diagnosis Stack for High-speed Networks

- Authors: Anurag Khandelwal, Rachit Agarwal, Ion Stoica
- Venue: USENIX NSDI, 2019
- Page: https://www.anuragkhandelwal.com/papers/confluo/
- Code: https://github.com/ucbrise/confluo

Confluo is an end-host stack that integrates with network-management tools to monitor and diagnose network-wide events from telemetry distributed across hosts, even on high-speed networks. Its Atomic MultiLog data structure supports highly concurrent reads and writes by exploiting the append-only, fixed-field-size nature of telemetry. For packet sizes of 128 bytes or larger, Confluo executes thousands of triggers and tens of filters at line rate for 10 Gbps links using a single core.

### ZipG: A Memory-efficient Graph Store for Interactive Queries

- Authors: Anurag Khandelwal, Zongheng Yang, Evan Ye, Rachit Agarwal, Ion Stoica
- Venue: ACM SIGMOD, 2017
- Page: https://www.anuragkhandelwal.com/papers/zipg/
- DOI: https://doi.org/10.1145/3035918.3064012
- Code: https://github.com/amplab/zipg

ZipG is a distributed, memory-efficient graph store for interactive queries. It stores graph data in compressed form and executes a broad range of graph queries directly on that representation, keeping a larger fraction of the workload in memory. Its minimal API can express functionalities from several industrial graph stores. On a 244 GB server, ZipG runs tens of thousands of queries over more than half a terabyte of raw graph data, achieving up to 23 times the throughput of Neo4j and Titan, with similar gains over Titan in distributed deployments.

### BlowFish: Dynamic Storage-Performance Tradeoff in Data Stores

- Authors: Anurag Khandelwal, Rachit Agarwal, Ion Stoica
- Venue: USENIX NSDI, 2016
- Page: https://www.anuragkhandelwal.com/papers/blowfish/

BlowFish is a distributed data store that provides a smooth tradeoff between storage and point-query performance and can move along that curve at fine time scales with little computational overhead. This capability supports practical uses including failure recovery and time-varying, spatially skewed workloads. On production-cluster scenarios, BlowFish uses 5.4 times less bandwidth and 2.5 times less repair time than state-of-the-art erasure codes while reducing the storage cost of replication from 3 times to 1.9 times. Dynamically navigating the tradeoff also achieves higher system-wide utility than selectively caching hot objects.

### Attacking Data Center Networks from the Inside

- Authors: Anurag Khandelwal, Navendu Jain, Seny Kamara
- Venue: Microsoft Research Technical Report, 2015
- Page: https://www.anuragkhandelwal.com/papers/attacking-dcn/

The shared nature of public-cloud data-center networks exposes co-hosted tenants to attacks launched from inside the network. This report evaluates security mechanisms offered by cloud providers and explores internal denial-of-service vulnerabilities. It describes overt attacks that are easy to mount but recognizable and covert attacks that are harder to detect and can have greater impact. It then discusses mitigations based on TCP pacing and network virtualization and proposes a source-based filtering scheme for regulating network communication.

### Succinct: Enabling Queries on Compressed Data

- Authors: Rachit Agarwal, Anurag Khandelwal, Ion Stoica
- Venue: USENIX NSDI, 2015
- Page: https://www.anuragkhandelwal.com/papers/succinct/
- Code: https://github.com/amplab/succinct

Succinct is a data store that enables efficient queries directly on a compressed representation of the input. Its compression technique permits random access as well as count, search, range, and wildcard queries. Unlike earlier techniques, Succinct supports these operations without storing indexes because the required information is embedded in the compressed representation. On real-world datasets, Succinct uses an order of magnitude less memory than systems with similar functionality, keeping more data in memory and delivering low query latency over a larger range of input sizes.