BenedictRESEARCH / NOTES
OBP v2
Compute only what matters
Before I decide where a calculation should run, can I work out which parts actually need to happen?
A faster answer to the wrong question
It is easy to start a performance problem by asking for more hardware. I started asking an earlier question: does the application need everything it is about to calculate?
Imagine multiplying two large matrices, then multiplying that result by a vector. If the application only needs the final vector, the large intermediate matrix may never need to exist. Sending the original calculation to a faster machine would leave that unnecessary work intact.
Follow the result backwards
I explored working backwards from the result the application can observe. Which inputs and intermediate values contribute to it? Which outputs are never used? Could an equivalent sequence produce the same answer with less arithmetic or less temporary storage?
That became Observation-Boundary Placement: first identify the required work, then choose where the remainder runs. The second version also considers reusing an unchanged result and recomputing a value when that is cheaper than moving or keeping it.
Changing the calculation, not the computer
On one laptop, a matrix–matrix–vector example took about 210 ms when I built the intermediate matrix. Reordering the calculation reduced that to about 4.6 ms: roughly 46 times less time. A separate example that calculated only the requested slice fell from about 255 ms to 0.33 ms.
Those are useful results, but the stronger comparison changed the interpretation. NumPy’s multi_dot routine completed the first example in about 4.57 ms—essentially the same as my reordered path. I had demonstrated the cost of unnecessary materialization, not an advantage over the best available library.
The part I cannot optimise away
An application asking for the whole matrix cannot be given a slice instead. And two expressions that are equal on paper can differ in floating-point arithmetic, where rounding depends on the order of operations.
I used carefully constructed small-integer inputs represented in float64 for these examples and checked the required answers. I cannot turn that into permission to reorder arbitrary numerical code. The required output, side effects and numerical accuracy all constrain which shortcuts are legitimate.
Reusing an answer has a price too
I also compared computing again with returning an unchanged result from a cache. The interesting difference was not just whether a result existed, but how I knew its inputs had not changed.
When inputs were registered as immutable, checking their identity was much cheaper than scanning untracked buffers. That distinction matters to a runtime: a cache hit that requires a full scan and copy is not free.
What this changed about offloading
The result I take forward is not that every computation has a clever shortcut. It is that the remote machine should compete against the best legal local calculation, including the same shortcuts.
I have not shown a new physical remote-GPU advantage with OBP v2. Its scheduling simulations explore possibilities, while the local measurements establish something narrower and already useful: deciding what to compute can matter more than deciding where to compute it.
Technical details and evidence
OBP expands to Observation-Boundary Placement. The main CPU cohort used an Intel Core i7-8665U, Python 3.12.10, NumPy 2.2.6 and OpenBLAS 0.3.29; two warmups, five calibration trials and thirty held-out blocks. Laptop power and thermal state were uncontrolled.
At n=2048, the exact recorded medians are 209.94735 ms for left_f64, 4.59705 ms for right_certified_f64 and 4.57235 ms for multi_dot_f64. The full/direct slice pair is 254.98865/0.33210 ms. The paired comparison against multi_dot does not establish an advantage.
Scanning the inputs to establish the right to reassociate cost 90.6162 ms in the n=2048 example, against 4.59705 ms for the trusted-origin alternative. Checking a shortcut can be expensive. The assertion checks, cache experiments, numerical assumptions and simulated resource models are separate records; ordinary synchronous BLAS interception does not reveal an entire application graph.
Avoiding a large intermediate matrix
Technical details and evidence
- Experiment identifier
- obp-cpu-reassociation-n2048
- Original title
- Certified reassociation, slices and memo against native paths at n=2048
- Measurement boundary
- Eight timed operations on one fixed structured f64 fixture, measured on a single local CPU. Timings include the operation and demanded array construction; oracle checks run outside the timer. Comparison targets are NumPy paths in the same process, not remote or distributed implementations.
- Hardware
- Intel Core i7-8665U, 4 cores / 8 logical processors, Windows 11, shared laptop with uncontrolled power and thermal state.
- Software
- Python 3.12.10, NumPy 2.2.6, one-thread BLAS configuration per process; 2 warmups and 5 calibration trials before 30 held-out blocks.
- Input
- Structured integer-valued f64 fixture at n=2048, 67125248 recorded input bytes.
- Samples
- 30 held-out timing blocks per operation; 8 operations measured at this size.
- Statistical method
- median of 30 held-out blocks per operation
- Cold / warm state
- Warm within each held-out block; the memo row reuses a previously computed result.
- Network
- None; local process only.
- Serialization
- None; arrays stay in process memory.
- Correctness checks
- Each case records the reference sum, reference vector sum, vector digest and oracle time alongside the timed rows; the oracle runs outside the timer.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Native dense f32 matrix-vector product | 81.4829 ms | 30 | median of held-out timing blocks |
| Left-to-right f64 reassociation | 209.94735 ms | 30 | median of held-out timing blocks |
| Certified right-associated f64 boundary path | 4.59705 ms | 30 | median of held-out timing blocks |
| NumPy multi_dot f64 library path | 4.57235 ms | 30 | median of held-out timing blocks |
| Scanned right-associated f64 path | 90.61619999999999 ms | 30 | median of held-out timing blocks |
| Full demanded f64 slice | 254.98865 ms | 30 | median of held-out timing blocks |
| Direct demanded f64 slice | 0.3321 ms | 30 | median of held-out timing blocks |
| Warm memoised f32 result | 4.52055 ms | 30 | median of held-out timing blocks |
CALCULATED FROM THE MEASUREMENTSLeft-to-right f64 reassociation ÷ Certified right-associated f64 boundary path = 45.670017×97.810380% lower time for Certified right-associated f64 boundary path.
CALCULATED FROM THE MEASUREMENTSNumPy multi_dot f64 library path ÷ Certified right-associated f64 boundary path = 0.99462699×0.54020362% higher time for Certified right-associated f64 boundary path.
CALCULATED FROM THE MEASUREMENTSFull demanded f64 slice ÷ Direct demanded f64 slice = 767.80684×99.869759% lower time for Direct demanded f64 slice.
Experimental limitations
- One shared laptop with uncontrolled power and thermal state; results are not normalised to a machine class.
- Structured integer-valued f64 fixture; not an arbitrary floating-point matrix.
- 30 held-out blocks per operation; medians and paired intervals are local to this session.
- The multi_dot library path matches the certified path at n=2048, so no library advantage is claimed.
- Timings include operation and demanded array construction; oracle checks are excluded.
At n=2048 the certified right-associated path records 4.59705 ms against 209.94735 ms for the left-to-right reassociation; the unrounded quotient left/right is 45.67001664110679. The direct demanded slice records 0.3321 ms against 254.98865 ms for the full slice, quotient 767.8068352905751. NumPy multi_dot records 4.57235 ms and is the strongest observed vector method; the paired 95 % interval for right against multi_dot is [-0.18961868122873282, 0.18461703811543084], which includes zero, so no advantage over the library is claimed at this size. The paired 95 % savings interval for right against left is [0.9749434555072917, 0.9805127237538717] and for the direct slice against the full slice [0.9985821359777526, 0.9987835241950881].
Technical details and evidence
Two ways of keeping a result and not copying it
Technical details and evidence
- Experiment identifier
- obp-registered-vs-untracked-n1024
- Original title
- Registered immutable cache against untracked exact-scan cache at n=1024
- Measurement boundary
- Local CPU comparison of an optimised BLAS recomputation, a registered immutable cache hit and an untracked exact-scan cache hit on one synthetic immutable repeat. The registered path avoids re-copying the output; the untracked path is charged a full output copy.
- Hardware
- not recorded
- Software
- One-thread BLAS configuration recorded in the source; exact Python and NumPy versions not recorded.
- Input
- One synthetic immutable repeat at n=1024, with the full output published.
- Samples
- 30 held-out blocks per method; 3 methods measured.
- Statistical method
- median of 30 blocks per method
- Cold / warm state
- Warm cache hits for both cached rows; the optimised BLAS row recomputes.
- Network
- None; local process only.
- Serialization
- None; full output published in the source file.
- Correctness checks
- Both recorded checks pass: all warm results are byte and value exact with fresh ownership, and a changed input rejected the cached result.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Optimised BLAS recomputation | 12.912299999999998 ms | 30 | median of 30 blocks |
| Registered immutable cache hit | 1.24495 ms | 30 | median of 30 blocks |
| Untracked exact-scan cache hit with full copy | 8.19195 ms | 30 | median of 30 blocks |
CALCULATED FROM THE MEASUREMENTSOptimised BLAS recomputation ÷ Registered immutable cache hit = 10.371742×90.358418% lower time for Registered immutable cache hit.
CALCULATED FROM THE MEASUREMENTSUntracked exact-scan cache hit with full copy ÷ Registered immutable cache hit = 6.5801438×84.802764% lower time for Registered immutable cache hit.
Experimental limitations
- Synthetic immutable repeat, not a production workload.
- Only n=1024 measured; no other size in this file.
- The untracked exact-scan path is charged a full output copy while the registered path is not re-copied.
- Reported working and peak memory counters read zero in this file.
The registered cache row records 1.24495 ms against 12.912299999999998 ms for optimised BLAS, quotient 10.371741837021565, and 8.19195 ms for the untracked exact-scan cache, quotient 6.580143780874734 against the registered row. The registered path avoids both recomputation and the full output copy charged to the untracked scan; the 10.371741837021565 quotient over recomputation is a cache-reuse result, not a distributed-execution result.