publications

2026

  1. Preprint
    An Approximate Queueing Model of LLM Inference Serving for SLO-Driven Autoscaling
    Vishakha Ramani and Asser N. Tantawi
    arXiv preprint arXiv:2609.20957, 2026
    Abs arXiv PDF

    We present a tractable approximate queueing model, built on Markovian assumptions, of how LLM inference servers interleave prefill and decode work. Each model-accelerator pair is described by three parameters: a fixed per-iteration overhead, a per-token compute cost, and a per-token KV-cache access cost. Mean-value analysis of per-iteration work is combined with a state-dependent Markov chain over batch occupancy to predict mean time to first token (TTFT) and inter-token latency (ITL), and the parameters can be fitted online from observed latencies. On an H100 GPU under light to moderate load, mean ITL error is about 5% for Llama-3.1-8B and 8% for Qwen2.5-14B, with TTFT errors of 14% and 16%. An autoscaling controller built on the model tracks a fourfold load ramp on an OpenShift H100 cluster, missing the SLO in 7 of 127 control cycles with in-loop median prediction errors of at most 5% for TTFT and 9% for ITL, compared with 27 of 128 cycles for a decode-throughput analyzer without a latency target.

  2. WVA: A Global Optimization Control Plane for llm-d
    Abhishek Malvankar, Vishakha Ramani, and others
    In IEEE International Conference on Cloud Computing (CLOUD), 2026

2025

  1. Bottlenecks of AI Inference: A Performance-Modeling Perspective
    Tarek Abdelzaher, Vishakha Ramani, and others
    Real-Time Systems, 2025

2024

  1. Optimal Scheduling for Multi-Step Status Update Processing
    Vishakha Ramani and Roy D. Yates
    In Asilomar Conference on Signals, Systems, and Computers, 2024
  2. Memory-Efficient Status Update Storage with Age Constraints
    Vishakha Ramani and Roy D. Yates
    In IEEE International Symposium on Information Theory (ISIT), 2024
  3. Read-Copy-Update for Status Updating Systems
    Vishakha Ramani and Roy D. Yates
    In IEEE INFOCOM Workshops, 2024
  4. Storing, Retrieving, and Processing Updates: A Timeliness Perspective
    Vishakha Ramani
    Rutgers University, 2024

2023

  1. Multi-Source Status Updating with Heterogeneous Freshness Requirements
    Vishakha Ramani and Roy D. Yates
    In International Symposium on Modeling and Optimization in Mobile, Ad hoc, and Wireless Networks (WiOpt), 2023
  2. Lock-Based or Lock-Less, Which is Fresh?
    Vishakha Ramani and Roy D. Yates
    In IEEE INFOCOM, 2023
  3. Mobile Edge Status Updating under Energy and Freshness Constraints
    Vishakha Ramani and Roy D. Yates
    In IEEE INFOCOM Workshops, 2023

2020

  1. I-MAC: An ICN Based Radio Access Network Architecture
    Vishakha Ramani
    Rutgers University, 2020

2017

  1. Cognitive Radio Spectrum Sensing under Imperfect Channel Conditions
    Vishakha Ramani and others
    China Communications, 2017