In a paper written with Asser Tantawi, now available on arXiv, we built that model. The claim that makes it useful is economy. An LLM server, for our purposes, is characterized by three parameters, each with units of time and each with a physical meaning. The first is the baseline overhead of one batched iteration, the kernel launches and synchronization and weight loading that happen regardless of what is in the batch. The second is the compute time per token in a forward pass, set by the accelerator’s arithmetic throughput. The third is the memory access time per token of KV cache, set by its memory bandwidth. Everything the paper predicts is built from these three numbers, the workload’s token lengths, and the arrival rate.

The modeling difficulty is that a serving engine like vLLM does not process requests one at a time. It runs continuous batching, where on every iteration boundary a finishing request departs and a waiting one is admitted, so a single forward pass mixes prefill chunks of one request with decode steps of many others. Our way through is a work-centric view. Averaged over its lifetime in the server, a request is equally likely to be found in any of its processing stages, so each active request contributes a calculable average amount of work per iteration. Iteration time then grows linearly in the number of active requests, with slope given by that per-request work and intercept given by the baseline overhead. That linearity is not an assumption of convenience. It shows up plainly in measurements, ours and others’.

On top of the work model sits a birth-death Markov chain over the number of requests in the system, with the batch size capped and the arithmetic tracking how the token budget forces long prompts to be split into chunks. Working through the chain yields the two latencies users actually experience, the time to first token and the gap between subsequent tokens, as functions of load. The same arithmetic yields the stability boundary, the arrival rate beyond which the queue grows without bound.

A model this compressed has to earn trust in two ways. The first is against data. On a vLLM server with an H100 GPU, across a grid of input and output lengths from 64 to 4096 tokens and a range of arrival rates, the model’s inter-token latency errors are about 5 percent for Llama-3.1-8B and 8 percent for Qwen2.5-14B, with time-to-first-token errors of 14 and 16 percent. The second is against physics. The fitted compute parameter puts the forward pass at roughly 80 percent of the H100’s peak arithmetic throughput, and the fitted memory parameter puts the attention kernel at about 71 percent of peak memory bandwidth. Between the two models, the fitted compute cost scales almost exactly with parameter count. The three numbers are not just degrees of freedom that happen to fit. They come out where physically meaningful quantities should, which is a reason to believe they mean what they claim.

The part of this work I am most attached to is that the model tunes itself. Nobody wants to rerun a calibration suite for every model and accelerator pair, and operating conditions drift as batch composition and cache pressure change. So the parameters are estimated from the latencies the service is already observing, first by an initial fit over a handful of observations, then refreshed continuously over a sliding window at every control cycle. A model that maintains its own calibration can live inside a controller indefinitely.

And that is where we put it. We embedded the model and its tuner in an autoscaling control loop and ran it against a real vLLM deployment of Llama-3.1-8B on H100 GPUs in an OpenShift cluster, with a 50 millisecond target on time to first token. Under a trapezoidal load ramp from 120 to 480 requests per minute, the controller scaled from one replica to a peak of four, settled at three under sustained peak, and scaled back symmetrically as load fell. The visible latency spikes during scale-up come from the roughly 90 seconds a new pod needs to become ready, a hardware and orchestration effect the model deliberately does not claim to predict. What the model does predict, the steady-state operating point, tracked observed latency within a few milliseconds through the baseline phases.

This work now ships as the queueing model analyzer and tuner inside the Workload Variant Autoscaler, the component of the open source llm-d project that scales inference deployments against their service level objectives. The larger argument, the one that runs through everything I work on, is that serving AI needs models with structure, not just measurements with trend lines. This paper is that argument in its smallest complete form.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

An object detector does not produce answers in one motion. A camera frame is pre-processed, features are extracted, and only then does classification yield something a decision can act on. Augmented reality and smart-city applications offload exactly this kind of work to nearby edge nodes, and the result is only useful if it is still current when it arrives. Multi-step processing is everywhere in real-time systems, yet the freshness literature had largely treated computation as a single act.

In Timely and Energy-Efficient Multi-Step Update Processing, written with Ivan Seskar and Roy Yates, we studied two-step processing as the fundamental building block. Two servers can be arranged in series, each owning one step of a pipeline, or in parallel, each running both steps independently. The judge of every arrangement is the age of information at the monitor receiving the finished updates. The constraint that makes the problem interesting is power.

The power model is where the numbers get vivid. Dynamic power in CMOS processors follows an alpha-power law, and with the technology exponent we use, power grows as the fifth power of processing speed. Running a step twice as fast costs roughly thirty two times the power. At that exchange rate, no one can afford to buy freshness with raw speed alone. The budget has to be spent where it changes what the monitor knows.

That framing exposes a phenomenon we call wasted power, which is work that never reduces the age at the monitor. It comes in several flavors. In parallel setups, one server can labor to finish an update while the other has already delivered a fresher one, making the first server’s effort worthless on arrival. In series setups, an update can be preempted or discarded mid-pipeline, taking the energy already spent on it along. Even idleness is a form of waste, capacity that could have been informing the monitor and was not. A pipeline can be fully busy and still be spending watts on information that will land too late to matter.

We built the series side as a family of four queueing disciplines, distinguished by what happens when the first server hands a fresh update to a second server that is still busy. In the fully preemptive design, the newcomer replaces the update in service and the displaced work is thrown away. A second design adds a waiting room holding exactly one update, where a newer arrival replaces whatever waits but never interrupts service. A third blocks instead, discarding fresh arrivals while the second server works. The fourth is synchronous, admitting a new update only when both servers stand idle, so a single update moves through the pipeline at a time. In every one of them the first server generates a fresh update the moment it finishes the previous one, so all the interesting congestion happens at the hand-off.

The parallel side is a spectrum of how much the two servers know about each other. At one end, each server runs both steps of its own update independently and delivers whenever it happens to finish. The coordinated alternating policy adds awareness, so both servers start on the same fresh update, and a server that falls behind abandons its work and restarts fresh once its twin carries a newer update into step two. The shared-intermediate policy goes furthest. Whichever server finishes step one first hands the intermediate result to both, they attack step two together, and the pair advances through the pipeline in lockstep. That last policy effectively fuses two processors into a single server that runs each step at twice the rate and never duplicates work.

Analytically, the series designs yield to the stochastic hybrid systems machinery that runs through the rest of my dissertation. The parallel designs forced an extension. When servers deliver independently, an arriving update only matters if it is fresher than what the monitor already holds, so the monitor’s age is a running minimum over competing deliveries. Standard age bookkeeping cannot express that, and the fix is to carry auxiliary state variables that track minima of ages across servers, a device we adapted from the age-of-gossip literature. Getting those reset maps right is the technical heart of the paper’s parallel results.

Then comes the optimization. Fix a power budget and ask how fast each step should run. Two structural facts make the problem tractable. In every model we analyzed, the average age at the monitor comes out as a function of the rate ratio \(\rho = \mu_1/\mu_2\) divided by the step two rate \(\mu_2\). And the power constraint caps \(\mu_2\) through a quantity we called the power-weighted processor activity, which also depends only on \(\rho\). A two-variable design problem therefore collapses to a one-dimensional search over the ratio. For the preemptive series design, the age is \(\tfrac{1}{\mu_2}(1 + \tfrac{1}{\rho})\), and pushing the budget through the constraint puts the optimum at \(\rho^* \approx 0.846\).

Two findings hold across every model we studied. The second step should always run at least as fast as the first, and no model wants the ratio above one. A slow exit stage is a bottleneck where nearly finished updates sit and age, which is the worst place to save power. And the optimal ratio does not depend on the size of the budget. The right split between the steps is structural, so extra power should scale both steps together rather than shift the balance.

The ranking of designs held surprises too. Among series disciplines, preemption in service wins, consistent with the freshness folklore that new should replace old, and the gap is not subtle. At the same service rates, the blocking design delivers exactly twice the average age of the preemptive one. Less expected, synchronous service beats the asynchronous designs, meaning one update moving through the system at a time outperforms designs that keep more in flight only to discard or delay them. Parallelism helps on top of that. Running the synchronous discipline on two independent servers, each on half the power budget, already beats its series counterpart, and the best design overall is the shared-intermediate policy. The common thread is that coordination is what eliminates wasted power. Servers that know what their peers have delivered do not burn watts on obsolete work.

One caveat belongs in the record. These models use exponential processing times, where preempting a job loses no accumulated progress in expectation. With service times whose completion becomes more certain as work accumulates, killing a nearly-finished update is genuinely wasteful, and the clean definition of wasted power blurs. Extending the framework there is open work.

This paper closes the arc of the series, and of this half of my dissertation. Wait before computing when variance is high. Read at the moment the threshold says to. And stop working when the work cannot make anyone better informed. I now spend my days on inference systems that run a two-phase workload, a compute-heavy prefill followed by a memory-bound decode, under latency targets and power budgets. The vocabulary has changed from monitors and updates to tokens and service level objectives, but the accounting question underneath is the one this paper asked. Which watts are actually buying information?

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

How often should a reader check shared memory for fresh data? Checking is not free. A read can cost bandwidth, processor time, or the latency of a query to a distant database. And a read that arrives before anything has changed buys nothing at all. The system pays full price to learn what it already knew, while a reader that checks too rarely lets its client drift out of date. Everyone who has refreshed a page that has not changed knows both failure modes personally.

In Efficient and Timely Memory Access, written with Ivan Seskar and Roy Yates, we made this trade-off exact. The model is deliberately spare. Time is slotted. In each slot a writer publishes a fresh update to memory with probability \(p\). A reader then chooses to idle or to sample the memory at a fixed cost \(c\), delivering whatever it reads to a client. The state is a pair of ages, the age of the update in memory and the age of the client’s copy, and the goal is to minimize the long-run average of client age plus sampling cost.

Posing this as a Markov decision process is natural, but there is a technical catch worth naming. Ages grow without bound, so the state space is countably infinite and the costs are unbounded, and in that regime a well-behaved optimal policy is not guaranteed to exist at all. Part of the paper’s work is verifying, through conditions from average-cost dynamic programming, that one does.

The answer has the simplest structure the problem allows. The optimal policy is a threshold. The reader should sample only when the memory holds an update it has not yet delivered and the client’s age has reached a critical value \(Y_0\). Everything else follows from that rule. If the memory refreshes and the reader judges the client still fresh enough to skip it, the right move is to keep idling until the memory changes again, because sampling in between would pay \(c\) to fetch what it already declined.

Better still, the threshold has a closed form. The optimal \(Y_0\) is the ceiling of \(\sqrt{2c + (1/p - 1/2)^2} - (1/p - 1/2)\), which grows like the square root of the sampling cost. The paper pairs this with a matching lower bound on the achievable average cost, so the formula is not just a heuristic that happens to work. Plotted against the threshold, the average cost traces the U shape this series keeps meeting. Sample too eagerly and the costs pile up faster than the age falls. Sample too rarely and the client’s age dominates everything.

The part I find most counterintuitive is how the threshold moves with the update rate. When updates arrive frequently, the optimal reader becomes choosier, not hungrier. Skipping a fresh update is cheap when an even fresher one is due shortly, so a high \(p\) raises the bar for acting. When updates are rare, the reader should take what it can get. Patience, it turns out, is something you can afford only when the world changes quickly.

This paper sharpened the story of the first post in a specific way. There, the reader was blind to the memory and lazy on a timer, and laziness helped mainly by absorbing variance in computation times. Here the reader can see the age of what is in memory, and with that information plus a price per read, the optimal behavior stops being a vague preference for patience and becomes a rule a system could implement in a few lines. Thresholds are the kind of structure practitioners can actually deploy, and proving one optimal rules out the entire zoo of more complicated policies.

One honest limitation points forward. The reader in this model is told when memory changes. A reader without that signal cannot wait for evidence of a fresh update, and the paper shows the age achieved with full knowledge is a lower bound on what such a reader can do, while leaving the optimal blind policy open. And in both papers so far, computing on the data was a single act. Real updates pass through several processing steps before they are useful, each step takes time, and running steps faster costs power at a brutal exchange rate. That is the final post in this series.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

Picture the compute node behind a smart intersection. Cameras and roadside sensors publish measurements continuously. An analytics process reads the latest data, spends some time computing a safety decision, and delivers the result to a monitor that acts on it. The monitor does not care how busy the pipeline was. It cares how old the information behind the current decision is by the time that decision arrives.

In Timely Processing of Updates from Multiple Sources, written with Ivan Seskar and Roy Yates, we modeled this loop as a publish-subscribe system. Two independent sources push time-stamped updates through a writer into shared memory, with Read-Copy-Update arbitrating between writer and reader, the same mechanism the first series dissected. A decision process subscribes by reading the latest pair of source updates, computes on it for a random time, and sends the derived decision update onward. Then it faces its only real choice. Read again immediately, or wait a while first?

Reading again immediately is the zero-wait policy, and it feels obviously right. The processor never idles, every cycle is spent turning data into decisions, and any deliberate delay looks like negligence. The paper shows that this instinct is wrong, and it shows exactly why.

The analysis tracks three coupled age processes. There is the age of the source updates sitting in memory, the age of the sample the decision process is computing on, and the age of the decision update at the monitor. The decision process controls none of the publishing. Sources update memory on their own schedule, so laziness cannot make the inputs fresher. What it can do is change which inputs get sampled and when the computation lands.

The cleanest result in the paper is a separation. For any waiting rule that depends only on the previous computation time, the average age at the monitor equals the average age at the reader’s input plus exactly the mean computation time. The output lags the input by \(\mathrm{E}[T]\), no matter how clever or lazy the reading policy is. That lag is a fixed tax, and it means the whole game is minimizing the input age. This reduces our problem to the classic update-or-wait problem, whose solution has a known shape. After a long computation, read again at once. After a short one, wait until a set amount of time has passed since the last read.

Whether waiting is worth much depends on the variance of the computation time. When computation times are exponential, the gains are modest. When they are log-normal with a heavy tail, which is much closer to how real compute behaves on a shared machine, zero-wait becomes clearly suboptimal. In our numerical evaluations at the heaviest tail we tested, the best lazy policy cut the average age at the monitor by roughly a third relative to reading at full tilt, and the gap widens as the tail grows. The intuition is that a long computation already delivered a stale result, and immediately grabbing the next sample tends to repeat the mistake. Waiting lets the memory refresh so the next computation starts from better inputs.

The memory side of the analysis produced two findings worth keeping. Because the writer in our model holds no queue, pushing updates at a higher total rate only helps freshness. And the max-age of a sampled pair is penalized by asymmetry between the sources. A pair is only as fresh as its stalest member, so one slow sensor quietly ages every decision computed from the pair.

The lesson I took from this paper is that utilization and timeliness are different objectives that only look aligned. A processor that never rests can serve its monitor worse than one that waits on purpose. But the reader in this model was blind. It never looked at the age of what was in memory, only at its own clock. Give the reader eyes and a price per read, and you can ask a sharper question. Exactly when should it read? That is the second post in this series.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

Consider a phone running visual SLAM, the technique that lets a device build a map of its surroundings while tracking its own position inside that map. Camera frames arrive continuously, and concurrent modules for tracking, mapping, and optimization all work against one shared global map in real time. Read-Copy-Update is an attractive way to coordinate them, because a module can pin the current version of the map and compute against it without ever blocking the writer. But every pinned version is a copy sitting in the memory of a device that does not have much to spare.

The first two posts in this series established that RCU often delivers fresher information than a lock, and that the advantage shows up on real hardware. Both also hinted at the price, because RCU buys its freshness with copies. A writer publishes a new version, readers still inside their critical sections keep the old one alive, and nothing can be reclaimed until the last of those readers lets go. In Age-Memory Trade-off in Read-Copy-Update, again with Jiachen Chen and Roy Yates, we asked whether that price is bounded. If updates keep arriving and old versions must survive their readers, can the number of retained copies grow without limit?

We made the writer as aggressive as the mechanism allows. In the model, it begins writing the next update the instant it publishes the previous one, with exponential write times, so fresh versions are published as a Poisson process at rate \(\alpha\). This is deliberately the worst case for memory. Read requests arrive as a Poisson process at rate \(\lambda\), and each holds its version for an exponentially distributed read time with mean \(1/\mu\), and a version is reclaimed only when its last reader finishes. One subtlety makes the analysis more than a textbook exercise. How long a version stays current depends on when the next update happens to be published, so the population of live versions is not the \(M/M/\infty\) queue it superficially resembles.

Two results summarize the paper. First, the average age of the current update in memory is \(2/\alpha\) for write rate \(\alpha\), so a faster writer keeps memory fresher, with no surprise there. Second, the average number of updates held in the system never exceeds \(1 + \lambda/\mu\), where \(\lambda\) is the read request rate and \(1/\mu\) is the mean read time, no matter how fast the writer publishes. Writing faster makes information fresher without blowing up the version count. The bound also has a clean interpretation in the limit of an infinitely fast writer. Each read request then effectively pins its own private version, the pinned versions do form an \(M/M/\infty\) queue, and their number is Poisson with mean \(\lambda/\mu\). Add the single current version and the bound is met exactly.

The satisfying part is where the bill lands. Memory consumption in RCU is governed by the read side, not the write side. Slow reads and frequent read requests are what hold versions in memory, so a system that needs to cap RCU’s footprint should shorten its read-side critical sections or consolidate its read requests, rather than throttle the writer and pay for the savings in staleness.

Taken together, the three papers form one argument. Synchronization primitives are not interchangeable implementation details, because they decide which version of the truth a reader observes. Their effects on freshness are measurable in a real forwarder, alongside timeliness penalties that throughput benchmarks never reveal. And the freshest mechanism carries a resource cost that can be quantified and belongs in the model next to the age it buys.

I now work on performance modeling and analysis for large language model inference at IBM Research, and the same accounting follows me there. An inference server holds key-value cache in GPU memory so that tokens keep streaming on time, and the question of how much memory a latency target costs is asked in gigabytes rather than table versions. The setting has changed, but the tradeoff has not.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

There is a practical problem at the heart of measuring information freshness, and it nearly sank this experiment before it started. The age of a received update is the gap between the receiver’s clock and the timestamp the sender stamped into the packet. On our testbed, NTP synchronizes clocks to about a millisecond, but the effects we needed to observe live at microsecond scale. Measured naively, the data would have been mostly clock error. Our workaround was to place the sender and the receiver on the same machine, so that every timestamp and every arrival referenced a single clock.

The experiment, described in Timely Mobile Routing: An Experimental Study with Jiachen Chen and Roy Yates, is the empirical companion to the analysis in part one. The stochastic models predicted that the choice between Read-Copy-Update and a Readers-Writer Lock changes the freshness of delivered information. We wanted to know whether that prediction survives contact with real hardware.

On the COSMOS testbed, we connected two machines over 25 Gbps links and built the forwarding application in DPDK, a kernel-bypass packet processing framework used in data centers and core routers. The source machine plays both the application server and 1000 mobile users at once. It emits two interleaved streams, data packets carrying timestamped application updates and control packets announcing user movements, with user popularity drawn from a Zipf distribution. The forwarder keeps its forwarding table in a hash table protected by either RCU or RWL. For each data packet it looks up the user, stamps the address it found, and sends the packet back to be scored as correctly delivered or misaddressed.

Before testing any synchronization, we ran a baseline with a single immobile user and no table lookups at all, just to understand the packet path. Even that produced a lesson. Average age traced a U shape against sending rate. Up to about 1 million packets per second, sending faster made information fresher, as expected. Beyond roughly 4 million packets per second, age began climbing again. The culprit was batching. DPDK fills transmission rings in bursts to maximize throughput, and once the offered rate is high enough, packets start sharing batches and queueing in the ring while their information grows old. A framework tuned entirely for throughput was quietly taxing timeliness. The scale of the effect grew with the workload too. Average ages of 1 to 10 microseconds in the one-user baseline became 5 to 10 milliseconds once the experiment averaged over 1000 users.

The synchronization results matched the analysis where it counted. With one control packet per hundred data packets, RCU’s cheap read side generally delivered fresher application updates than RWL at high sending rates, while at low rates the difference was negligible. When we raised the update ratio tenfold, both primitives struggled. The control ring backed up, the table lagged behind the users’ actual movements, and misaddressed packets multiplied, with RWL misdelivering more than RCU. Under RWL, average age even reproduced the classic curve from updating theory, first falling with sending rate and then rising as the system congested. Updating should be fast, but not too fast.

For me, the testbed’s job was to keep the theory honest, and it did more than that. It confirmed the effect the models predicted, and it exposed penalties no clean model contained, like the batch admission delay. A forwarder can look excellent in packets per second while quietly delivering addresses that are no longer true, which is why freshness has to be measured directly rather than inferred from throughput or latency.

The experiments also made RCU’s hidden cost tangible. Every RCU write creates another copy of the table entry, and stale copies linger until their readers finish, so the memory footprint grows and the implementation needs its own reclamation machinery. That raised a question the testbed could not answer. If a writer publishes updates as fast as it likes, does the pile of retained versions stay bounded? Answering that took another paper, and it is the subject of the final post in this series.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn

Somewhere in the network sits a forwarder whose job is to keep up with a moving target. An application server streams updates to a mobile user, the user changes their point of attachment as they move, and the forwarder maintains a table that records where the user currently is. Every application packet passing through consults that table for an address.

Two processes contend for the table. A writer records fresh locations as the user moves. A reader looks up addresses so application packets can be delivered. When a read returns a stale location, the packet is sent to the wrong place, and in our model a misaddressed packet is simply lost. Whether the user receives timely updates therefore depends on a piece of machinery most performance models never mention. It depends on how the table is locked.

In Lock-based or Lock-less: Which Is Fresh?, written with Jiachen Chen and Roy Yates during my PhD at Rutgers, we compared the two standard ways of protecting such a table. A Readers-Writer Lock enforces mutual exclusion, so a writer holding the lock blocks readers until it finishes. Read-Copy-Update takes the opposite view. Readers never block. The writer prepares a fresh copy of the data and atomically swaps in a pointer to it, while readers already in flight continue using the old version until they are done.

Benchmarked on throughput, this contest has a conventional winner. RCU is celebrated precisely because its read side is cheap, and operations per second favor it in read-mostly workloads. But our metric was not operations per second. We used the Age of Information, which measures how old a piece of information is at the moment it is delivered, and we tracked two quantities. One is the age of the location stored in the table. The other is the age of the application updates that actually reach the user.

The two ages are coupled through the table, so we modeled reader and writer together as a stochastic hybrid system. The discrete component is a five state Markov chain describing what the reader and writer are each doing, and the continuous component carries the age processes, which reset differently under RCU and RWL transitions. The payoff of the framework is a closed set of linear age balance equations whose solution gives the average ages without simulation.

The answer refused to pick a side. At low location update rates, RCU delivers fresher application updates, exactly as the throughput intuition suggests, because its light read side serves lookups quickly. At high location update rates, the ranking flips. The write lock in RWL, the very thing that makes readers wait, prevents a reader from walking away with an address that is in the middle of being replaced. RCU has no such restraint. Its reader cheerfully reads the old copy while the writer publishes the new one, and the packet goes out misaddressed. Blocking, usually cast as pure overhead, was acting as quality control.

The paper also redesigns both processes around preemption. A writer that receives a fresher location abandons the write in progress and starts over with the newer one. A reader lets the newest waiting application update claim the address returned by an in-flight read, rather than serving an older packet. In our numerical evaluations, preemption reduced the average application update age by close to 15 percent under RCU and 45 percent under RWL.

The result I keep returning to is not that one primitive won. It is that the synchronization primitive belongs inside the performance model rather than underneath it. The mechanism decides which update a reader actually observes, and no amount of downstream optimization can recover information that was already stale when it was read. That said, this conclusion rested on exponential service times and other modeling conveniences. The obvious challenge was to find out whether real hardware agrees, so we built a packet forwarder that measures its own freshness. That experiment is the subject of the second post in this series.

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn