The best time to read memory

Created on July 18, 2026

2026   ·   age-of-information   memory-access   performance-modeling   ·   research

Reading and computing on time · Part 2 of 3
Sometimes fresher means waiting · The best time to read memory · When a pipeline should stop working

How often should a reader check shared memory for fresh data? Checking is not free. A read can cost bandwidth, processor time, or the latency of a query to a distant database. And a read that arrives before anything has changed buys nothing at all. The system pays full price to learn what it already knew, while a reader that checks too rarely lets its client drift out of date. Everyone who has refreshed a page that has not changed knows both failure modes personally.

In Efficient and Timely Memory Access, written with Ivan Seskar and Roy Yates, we made this trade-off exact. The model is deliberately spare. Time is slotted. In each slot a writer publishes a fresh update to memory with probability \(p\). A reader then chooses to idle or to sample the memory at a fixed cost \(c\), delivering whatever it reads to a client. The state is a pair of ages, the age of the update in memory and the age of the client’s copy, and the goal is to minimize the long-run average of client age plus sampling cost.

Writer publish memory, age x sample Reader each slot, idle or pay c to read Client age y slotted time, a fresh update lands in a slot with probability p
The model. In each slot the writer may publish a fresh update, and the reader chooses between idling and paying c to bring the client up to date.

Posing this as a Markov decision process is natural, but there is a technical catch worth naming. Ages grow without bound, so the state space is countably infinite and the costs are unbounded, and in that regime a well-behaved optimal policy is not guaranteed to exist at all. Part of the paper’s work is verifying, through conditions from average-cost dynamic programming, that one does.

The answer has the simplest structure the problem allows. The optimal policy is a threshold. The reader should sample only when the memory holds an update it has not yet delivered and the client’s age has reached a critical value \(Y_0\). Everything else follows from that rule. If the memory refreshes and the reader judges the client still fresh enough to skip it, the right move is to keep idling until the memory changes again, because sampling in between would pay \(c\) to fetch what it already declined.

time in slots age at the client threshold Y₀ fresh update fresh update below Y₀, skip and keep idling first fresh update past the threshold, sample and reset the client
The optimal policy in action. The reader idles while the client’s age is below the threshold, then samples at the first fresh update past it. Illustrative.

Better still, the threshold has a closed form. The optimal \(Y_0\) is the ceiling of \(\sqrt{2c + (1/p - 1/2)^2} - (1/p - 1/2)\), which grows like the square root of the sampling cost. The paper pairs this with a matching lower bound on the achievable average cost, so the formula is not just a heuristic that happens to work. Plotted against the threshold, the average cost traces the U shape this series keeps meeting. Sample too eagerly and the costs pile up faster than the age falls. Sample too rarely and the client’s age dominates everything.

The part I find most counterintuitive is how the threshold moves with the update rate. When updates arrive frequently, the optimal reader becomes choosier, not hungrier. Skipping a fresh update is cheap when an even fresher one is due shortly, so a high \(p\) raises the bar for acting. When updates are rare, the reader should take what it can get. Patience, it turns out, is something you can afford only when the world changes quickly.

how often the memory refreshes optimal threshold Y₀ expensive reads cheap reads frequent updates make the reader choosier, a better opportunity is always close
An illustrative version of the paper’s threshold plot. The optimal threshold climbs with the update rate and with the sampling cost.
The optimal reader is a threshold rule with a closed form, and more frequent updates make it choosier rather than hungrier.

This paper sharpened the story of the first post in a specific way. There, the reader was blind to the memory and lazy on a timer, and laziness helped mainly by absorbing variance in computation times. Here the reader can see the age of what is in memory, and with that information plus a price per read, the optimal behavior stops being a vague preference for patience and becomes a rule a system could implement in a few lines. Thresholds are the kind of structure practitioners can actually deploy, and proving one optimal rules out the entire zoo of more complicated policies.

One honest limitation points forward. The reader in this model is told when memory changes. A reader without that signal cannot wait for evidence of a fresh update, and the paper shows the age achieved with full knowledge is a lower bound on what such a reader can do, while leaving the optimal blind policy open. And in both papers so far, computing on the data was a single act. Real updates pass through several processing steps before they are useful, each step takes time, and running steps faster costs power at a brutal exchange rate. That is the final post in this series.

← Sometimes fresher means waiting When a pipeline should stop working →

Vishakha Ramani is a Research Staff Member at IBM Research. She works on performance modeling and analysis of large-scale computing systems, most recently for large language model inference. She holds a PhD from Rutgers University, where she studied the timeliness of information in computing systems with Prof. Roy Yates. Google Scholar · LinkedIn