BenedictRESEARCH / NOTES
ASPR
Getting data ready before it is needed
If moving data is expensive, could I prepare it before the application asks for the next calculation?
A worker can be fast and still be waiting
A remote processor cannot calculate with data that has not arrived. If every request starts by sending the same inputs again, a fast worker may spend most of the interaction waiting on movement rather than doing useful arithmetic.
I wanted to know whether an application’s recent behaviour could give enough warning to prepare likely inputs. That might mean keeping an unchanged object nearby, arranging a transfer earlier or preparing a result locally.
What I tried to predict
I built and compared small models that looked at a history of requests and predicted future demand. One was a linear model with 524 parameters; another was a recurrent model, a GRU, with 1,616. I also explored associations between the operands used together in a calculation.
I tested them on generated execution traces with separate training, validation and test periods. These were controlled experiments, not recordings of a real CAD application. That let me test the method without pretending I already knew how a real application would behave.
Good-looking percentages hid a harder problem
The first linear model reached about 78% precision and 84% recall on its test trace. Precision asks how many predicted needs were real; recall asks how many real needs I managed to predict. Those figures looked much better than the next measure.
A calculation often needs several inputs together. Predicting most of them can still leave it waiting for the missing one. The per-object prediction head never assembled a complete bundle for the linear model in these tests. With a separate learned association table, about 40% of the bundles it actually forecast were exactly right. That conditional result is better, but it is not 40% accuracy over every future operation.
Preparation can become extra work
A wrong prediction uses bandwidth and memory for something the application never asks for. It can also evict data that would have been useful. I therefore measured model inference, memory use and interference with foreground work, not just prediction accuracy.
The cache and precomputation simulations gave conditional improvements in average cost. They did not improve the reported slow-tail figure: the 95th percentile stayed at about 121 ms in the precomputation comparison. A lower average was not a solution to every delay.
Knowing an input is likely is not knowing it is current
The runtime still has to know which version of an object the application needs. A prediction is not evidence that a buffer is unchanged, that a hidden dependency exists or that a result can be published early.
This led me to distinguish what I can see at a single intercepted call from what an application explicitly registers as immutable. More complete tracking is possible in principle, but it requires covering writers, aliases and lifetimes—not guessing them from a history of calls.
What I would test next
I would start with real application traces and measure whether preparation saves more than it costs. The complete input set and the cost of false predictions matter at least as much as model accuracy.
The work has not demonstrated an autonomous predictive-residency system accelerating a real remote application. What it has given me is a smaller, more useful question: when is there enough reliable warning to prepare the right data without disturbing the work already running?
Technical details and evidence
The original trace contains 7,200 ticks spaced 100 ms apart. Training, validation and test windows are separated with a 30-tick embargo. Exact precision/recall values and both models’ results remain in the archive.
There are two different bundle metrics. On the fresh trace, candidate_exact_bundle_accuracy is 0 for the linear per-object head and 0.008771929824561403 for the GRU. The separate learned association policy has exact_bundle_accuracy_when_forecast of 0.4029214850882532 over 1,643 emitted forecasts. They have different denominators and are not improvements of one metric from zero to 40%.
ASPR is the name used by the research documents; they do not define a full expansion. The prediction study, local CPU overhead, local reuse implementation and shadow-only planning are different stages of the work. Simulated placement is not a physical remote speedup.
The same forecasters on a fresh synthetic trace
Technical details and evidence
- Experiment identifier
- aspr-refined-forecast-fresh-trace
- Original title
- Frozen-threshold forecaster refinement on a fresh synthetic trace
- Measurement boundary
- Exploratory refinement evaluated on a fresh synthetic trace with thresholds frozen from the original validation split. The classification metrics below are measured; the workload latency attached to those forecasts elsewhere is simulated. Threshold sweeps over 0.3, 0.5, 0.7 and 0.9 are recorded separately in the source and are not the rows below.
- Hardware
- not recorded
- Software
- not recorded
- Input
- Fresh synthetic trace, separate from the original train and test split.
- Samples
- Linear fresh counts: 3284 true positives, 852 false positives, 706 false negatives. GRU fresh counts: 3213 true positives, 793 false positives, 777 false negatives.
- Statistical method
- precision, recall, joint class accuracy and exact-bundle accuracy at a frozen threshold
- Cold / warm state
- not recorded
- Network
- None; classification only.
- Serialization
- not recorded
- Correctness checks
- The frozen threshold, fresh counts and each fraction are recorded in the same file, so every fraction can be recomputed from its counts.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Linear refined precision | 0.7940038684719536 fraction | 4136 | tp / (tp + fp) on the fresh synthetic trace at a frozen threshold |
| Linear refined recall | 0.8230576441102757 fraction | 3990 | tp / (tp + fn) on the fresh synthetic trace at a frozen threshold |
| Linear refined joint class accuracy | 0.499163179916318 fraction | not recorded | joint class accuracy on the fresh synthetic trace; denominator not recorded |
| Linear refined exact-bundle accuracy | 0 fraction | not recorded | exact-bundle accuracy on the fresh synthetic trace; denominator not recorded |
| Linear refined discoverable bundle fraction | 0.7518796992481203 fraction | not recorded | share of bundles recorded as discoverable on the fresh trace |
| GRU refined precision | 0.8020469296055917 fraction | 4006 | tp / (tp + fp) on the fresh synthetic trace at a frozen threshold |
| GRU refined recall | 0.8052631578947368 fraction | 3990 | tp / (tp + fn) on the fresh synthetic trace at a frozen threshold |
| GRU refined joint class accuracy | 0.45244072524407253 fraction | not recorded | joint class accuracy on the fresh synthetic trace; denominator not recorded |
| GRU refined exact-bundle accuracy | 0.008771929824561403 fraction | not recorded | exact-bundle accuracy on the fresh synthetic trace; denominator not recorded |
| GRU refined discoverable bundle fraction | 0.7518796992481203 fraction | not recorded | share of bundles recorded as discoverable on the fresh trace |
Experimental limitations
- Exploratory refinement on a synthetic trace; not a new experiment family.
- Thresholds were frozen from the original validation split, not tuned on the fresh trace.
- Exact-bundle accuracy stays low even where precision and recall are stable.
- The fresh trace is synthetic; it is not real CAD.
On the fresh trace the linear model records precision 0.7940038684719536 and recall 0.8230576441102757 with exact-bundle accuracy 0.0; the GRU records precision 0.8020469296055917 and recall 0.8052631578947368 with exact-bundle accuracy 0.008771929824561403. The refinement stabilises the discriminative metrics but does not turn forecasting into complete-bundle reproduction.
Technical details and evidence
Modelling background precomputation and splitting work between two places
Technical details and evidence
- Experiment identifier
- aspr-local-precompute-simulation
- Original title
- Modeled learned local precomputation and dual placement at one-times and four-times CPU work
- Measurement boundary
- Arithmetic simulation only, assuming a fluid ten-percent CPU budget for background work. Two CPU work factors are modeled so that the sensitivity of the result to background cost is visible.
- Hardware
- not recorded
- Software
- not recorded
- Input
- Synthetic trace of 134 operations; background results, useful speculated results and pending work are recorded per mode.
- Samples
- Six simulated mode and work-factor combinations; 134 operations per combination.
- Statistical method
- modeled mean and 95th-percentile operation latency with background CPU work
- Cold / warm state
- Warm cache in every combination.
- Network
- Simulated only; no network was exercised.
- Serialization
- not recorded
- Correctness checks
- Pending work after the trace is zero and unknown-bundle rejections are not raised in the rows below.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Cache baseline, one-times CPU work, mean | 42.92724552238806 ms | 134 | modeled mean over the trace |
| Learned local predictive, one-times CPU work, mean | 30.49750298507462 ms | 134 | modeled mean over the trace |
| Dual placement, one-times CPU work, mean | 26.245946686567162 ms | 134 | modeled mean over the trace |
| Learned local predictive, four-times CPU work, mean | 38.22850298507463 ms | 134 | modeled mean over the trace |
| Dual placement, four-times CPU work, mean | 28.453419223880598 ms | 134 | modeled mean over the trace |
| All modes and work factors, 95th percentile | 120.6 ms | 134 | modeled 95th percentile, identical in all six combinations |
CALCULATED FROM THE MEASUREMENTSCache baseline, one-times CPU work, mean ÷ Dual placement, one-times CPU work, mean = 1.6355762×38.859467% lower time for Dual placement, one-times CPU work, mean.
CALCULATED FROM THE MEASUREMENTSCache baseline, one-times CPU work, mean ÷ Dual placement, four-times CPU work, mean = 1.5086850×33.717109% lower time for Dual placement, four-times CPU work, mean.
CALCULATED FROM THE MEASUREMENTSLearned local predictive, one-times CPU work, mean ÷ Dual placement, one-times CPU work, mean = 1.1619891×13.940670% lower time for Dual placement, one-times CPU work, mean.
Experimental limitations
- Arithmetic simulation with a ten-percent CPU budget assumption.
- The 95th percentile is 120.6 ms in all six combinations; no tail improvement is modeled.
- Background work is charged but not validated against a real scheduler.
- Four-times CPU work erodes the gain of predictive precomputation.
At one-times CPU work the cache baseline records 42.92724552238806 ms, learned local predictive precomputation 30.49750298507462 ms and dual placement 26.245946686567162 ms. At four-times CPU work the predictive mode degrades to 38.22850298507463 ms and dual placement to 28.453419223880598 ms. Every combination keeps a 120.6 ms 95th percentile, so precomputation is a mean-cost result only.
Technical details and evidence
How much it costs to ask a forecaster for an opinion
Technical details and evidence
- Experiment identifier
- aspr-inference-budget
- Original title
- Measured CPU inference and adaptation cost for both forecasters
- Measurement boundary
- Fresh-process CPU timing of feature construction plus one model call, repeated 5000 times per model. This is an inference budget, not workload latency and not a power measurement.
- Hardware
- not recorded
- Software
- not recorded
- Input
- The same feature construction and loaded weights used by the synthetic-trace forecasters.
- Samples
- 5000 timed calls per model; adaptation timed separately.
- Statistical method
- median, 95th percentile and CPU milliseconds per call
- Cold / warm state
- Warm process; the first call is included in the recorded samples.
- Network
- None; local CPU only.
- Serialization
- not recorded
- Correctness checks
- Not the subject of this measurement.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Linear, median call | 0.0446 ms | 5000 | median of 5000 calls |
| Linear, 95th-percentile call | 0.07280500000000002 ms | 5000 | 95th percentile of 5000 calls |
| Linear, CPU milliseconds per call | 0.053125 ms | 5000 | CPU time per call |
| GRU, median call | 0.0741 ms | 5000 | median of 5000 calls |
| GRU, 95th-percentile call | 0.126505 ms | 5000 | 95th percentile of 5000 calls |
| GRU, CPU milliseconds per call | 0.084375 ms | 5000 | CPU time per call |
Experimental limitations
- CPU inference only; no power telemetry and no GPU path.
- 5000 calls per model on one unrecorded host.
- The model and feature memory counters read zero in this file.
- Inference cost is not workload latency.
The linear model records a median of 0.0446 ms and a 95th percentile of 0.07280500000000002 ms at 0.053125 CPU milliseconds per call; the GRU records 0.0741 ms, 0.126505 ms and 0.084375 CPU milliseconds per call. Both budgets are well below the modeled operation latencies, so inference cost does not explain the simulated placement results.
Technical details and evidence
Whether background work slows the foreground
Technical details and evidence
- Experiment identifier
- aspr-interference-short-bursts
- Original title
- Short-burst response under no background work, unlimited background work and a ten-percent budget
- Measurement boundary
- Local CPU short-burst measurement of a fixed operation under three background-work conditions. This is not a battery-energy test and does not model sustained multi-tenant load.
- Hardware
- Local host with 8 threads; CPU model not recorded.
- Software
- not recorded
- Input
- Fixed burst operation repeated 30 times per condition.
- Samples
- 30 timed bursts per condition; 3 conditions.
- Statistical method
- median and 95th percentile of 30 bursts
- Cold / warm state
- Warm; bursts run back to back after the condition is established.
- Network
- None; local CPU only.
- Serialization
- not recorded
- Correctness checks
- Not the subject of this measurement.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| No background work, median | 9.663250000000001 ms | 30 | median of 30 bursts |
| No background work, 95th percentile | 11.99265 ms | 30 | 95th percentile of 30 bursts |
| Unlimited background work, median | 11.544599999999999 ms | 30 | median of 30 bursts |
| Unlimited background work, 95th percentile | 13.505559999999997 ms | 30 | 95th percentile of 30 bursts |
| Ten-percent background budget, median | 11.56305 ms | 30 | median of 30 bursts |
| Ten-percent background budget, 95th percentile | 13.307719999999998 ms | 30 | 95th percentile of 30 bursts |
Experimental limitations
- Short bursts; not a battery-energy test.
- Background work is recorded as CPU seconds and compressed bytes, but its source is not described in this file.
- One host and 8 threads; no multi-tenant or remote interference.
- All interference conditions are slower than the no-background condition.
The no-background median is 9.663250000000001 ms; unlimited background work gives 11.544599999999999 ms and the ten-percent budget gives 11.56305 ms. The 95th percentile rises from 11.99265 ms to 13.505559999999997 ms and 13.307719999999998 ms. Background work, whether unlimited or budgeted, does not reduce burst latency in this measurement.