BenedictRESEARCH / NOTES
Fusion Alpha
A fast GPU can still give a slow answer
Could I divide a workload between local and external compute without losing the benefit to startup and data movement?
Start with work I could describe explicitly
Before trying to intercept existing applications, I represented a workload as a graph of calculations. One stage produced data, another consumed it, and the system decided which machine should run each stage.
That made the question concrete. I could measure a local stage, an external GPU stage and the movement between them. It also forced me to account for something a kernel benchmark does not see: whether the GPU was actually ready to accept work.
The first correct chain did not give me a clean speedup
A complete local → GPU → local run returned the expected answer. The recorded total was about 288 seconds against 145 seconds locally. But the remote-run envelope also included a large local reference calculation used to check correctness, so I cannot present those totals as a clean production-latency comparison.
A separate isolated heavy stage recorded a modest remote advantage: about 134 seconds versus 147 seconds locally. It was one stage, not the whole chain, and it did not settle the larger question.
Ready and not ready are different machines
Cold connected-stage jobs took roughly 295 and 328 seconds, mostly waiting for acquisition and startup. Inside an already running job, repeated warm calculations were down to about 173 ms and 13 ms for the heavy and small examples.
Those numbers are not contradictory. They measure different circumstances. Reusing a process within one job also did not mean the managed service offered a reusable GPU session across jobs.
The resident-input experiment
I later used a persistent GPU process with the input already in device memory. At 4,096 values, local execution still won: about 666 ms against 805 ms remotely. At 262,144 values, remote wall time was about 931 ms against 2.99 seconds locally.
That larger case is a real, narrow win, with one observation per path. It is also conditional on the data already being there. A separate fresh transfer of about 5.3 MB took roughly 39 seconds in that setup—far more than the resident operation itself.
The planner had to compete with doing nothing clever
The repeated local-connection decision study recorded no wins for Fusion. In a small held-out comparison, always running locally also had a lower median than Fusion. Correct outputs did not make the placement decisions good.
A later live campaign ended before it measured any conditions. I count it as an incomplete experiment, not a missing positive result to be filled in by simulation.
What survived the early idea
The broad idea that external capacity would make the scheduler useful did not survive unchanged. What remained useful was the ability to explain what ran, what moved, which answer was checked and where the time went.
That pushed my later work down two paths: remove unnecessary computation before placing it, and test how an answer returns into a real application. Fusion was not the final architecture. It was how I learned which costs the next one could not ignore.
Technical details and evidence
The larger persistent RTX 5090 comparison records 2,993.3796 ms local wall and 931.2212 ms remote wall; GPU compute is a different interval at 70.17317399731837 ms. Both resident comparisons are single observations and exclude fresh ingress.
The early isolated positive profile identifies an NVIDIA L4 route; a later report labels it RTX 5090. The discrepancy remains attached to that experiment. The first full Fusion timer also includes roughly 155 seconds of local correctness-reference work.
Cold managed jobs, same-job warm repetitions, persistent-Pod requests, loopback policy tests and simulated retention policies remain separate experiments. The unfinished live crossover campaign measured zero of its twenty-four planned conditions.
The same idea at a much larger size, where the GPU did win
Technical details and evidence
- Experiment identifier
- alpha-isolated-positive
- Original title
- Cold managed GPU: isolated heavy Stage-B anchor
- Measurement boundary
- Isolated Stage B, not a complete local → GPU → local application workload.
- Hardware
- Receipt identifies an NVIDIA L4 route. A later report labels the route RTX 5090; the original profile identity is retained here.
- Software
- not recorded
- Input
- 250,000 input values, compute intensity 64.
- Samples
- 1 local and 1 remote observation.
- Statistical method
- Single recorded run unless stated otherwise.
- Cold / warm state
- Managed provider execution.
- Network
- not recorded
- Serialization
- not recorded
- Correctness checks
- CUDA verified and checksum artifact correct.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Local CPU | 146673.3167 ms | 1 | recorded value |
| Managed GPU elapsed | 134013 ms | 1 | recorded value |
CALCULATED FROM THE MEASUREMENTSLocal CPU ÷ Managed GPU elapsed = 1.0944708×8.6316427% lower time for Managed GPU elapsed.
Experimental limitations
- One observation per path.
- Provider compute sub-timing unavailable.
- Hardware labels differ between the original profile and later summary; no cross-GPU inference.
A narrow positive isolated-stage observation, not evidence of a general scheduler or application speedup.
Technical details and evidence
The first end-to-end run that moved work between a CPU and a GPU
Technical details and evidence
- Experiment identifier
- alpha-full-fusion
- Original title
- First correct heterogeneous Fusion: recorded full-run envelope
- Measurement boundary
- Recorded full-run totals. The Fusion envelope also contains separately executed local correctness-reference work.
- Hardware
- Remote NVIDIA GeForce RTX 5090. Local CPU model not recorded in the receipt.
- Software
- not recorded
- Input
- 250,000 input values, intensity 64; Stage-A object 5,032,313 bytes.
- Samples
- 1 recorded run per path.
- Statistical method
- Single recorded run unless stated otherwise.
- Cold / warm state
- Cold provider lifecycle and object staging.
- Network
- 17,249.517 ms recorded object-transfer interval; transport internals not recorded.
- Serialization
- not recorded
- Correctness checks
- correct=true; documented floating-point tolerance 0.01.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Local-only total | 145179.13710000002 ms | 1 | recorded value |
| Fusion recorded total | 288321.2843 ms | 1 | recorded value |
| Remote Stage B | 115571 ms | 1 | recorded value |
| Object transfer | 17249.517 ms | 1 | recorded value |
Experimental limitations
- Do not add remote sub-intervals to infer the reported total.
- Later report identifies roughly 155 s of correctness-reference work inside the total.
- Single historical run; not a production latency claim.
Functional local → GPU → local execution succeeded. The recorded Fusion total is slower, but its correctness-reference tail prevents treating this as an uncontaminated production latency comparison.
Technical details and evidence
A dedicated machine with its input already resident, at a small size
Technical details and evidence
- Experiment identifier
- alpha-persistent-small
- Original title
- Persistent GPU resident comparison: 4096 values
- Measurement boundary
- Resident-input wall-time comparison, with compute-only sub-intervals recorded separately.
- Hardware
- Persistent NVIDIA RTX 5090, 32,607 MiB VRAM; driver 580.178.04.
- Software
- CUDA 12.8; PyTorch 2.11.0+cu128 (campaign report).
- Input
- 4,096 resident values; intensity 4096 (campaign context).
- Samples
- 1 local and 1 remote wall observation.
- Statistical method
- Single recorded run unless stated otherwise.
- Cold / warm state
- Input already resident in GPU memory; 0 host-to-device uploads.
- Network
- Remote wall includes request/return. Fresh ingress excluded; separate campaign report records approximately 39 s for 5.3 MB.
- Serialization
- not recorded
- Correctness checks
- Campaign reports correct outputs and retained resident object identity.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Local wall | 666.2844 ms | 1 | recorded value |
| Resident remote wall | 805.4838000000001 ms | 1 | recorded value |
| Local compute | 666.2271 ms | 1 | recorded value |
| GPU compute | 53.95455099642277 ms | 1 | recorded value |
CALCULATED FROM THE MEASUREMENTSLocal wall ÷ Resident remote wall = 0.82718535×20.891889% higher time for Resident remote wall.
Experimental limitations
- Not a 20-block held-out campaign.
- Warm compute and wall time must not be interchanged.
- Input size comes from the accompanying original campaign report; the raw comparison pins input identity but does not repeat its size.
Residency is a condition of this test. The smaller resident case loses; the larger one wins at this boundary. Neither includes fresh-input WAN staging.
Technical details and evidence
Testing placement policies on data they had not seen
Technical details and evidence
- Experiment identifier
- alpha-heldout-policies
- Original title
- Counterbalanced held-out loopback policies
- Measurement boundary
- Held-out v3 policy study on the loopback decision corpus.
- Hardware
- not recorded
- Software
- not recorded
- Input
- not recorded
- Samples
- 3 counterbalanced repetitions per strategy.
- Statistical method
- Recorded policy medians.
- Cold / warm state
- not recorded
- Network
- Loopback.
- Serialization
- not recorded
- Correctness checks
- All outputs correct.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| All-local | 18.991 ms | 3 | median |
| Fusion | 19.833 ms | 3 | median |
| Fastest-resource policy | 22.433 ms | 3 | median |
| Remote-only | 122.027 ms | 3 | median |
Experimental limitations
- Original engineering report retained; raw held-out-v3 file is absent from this checkout.
- No broad planner generalisation from three repetitions.
The planner did not beat all-local on the held-out median. This is decision-quality evidence, not external GPU evidence.
Technical details and evidence
A study that never started
Technical details and evidence
- Experiment identifier
- alpha-study-b-termination
- Original title
- Study B v1: no conditions measured
- Measurement boundary
- Infrastructure failure before experiment bootstrap.
- Hardware
- RTX 5090 observed during preflight.
- Software
- CUDA 12.8; PyTorch 2.7.1+cu128.
- Input
- not recorded
- Samples
- 0 measured conditions.
- Statistical method
- Single recorded run unless stated otherwise.
- Cold / warm state
- not recorded
- Network
- not recorded
- Serialization
- not recorded
- Correctness checks
- No workload correctness measurement occurred.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| Measured conditions | 0 conditions | not recorded | recorded value |
| Planned conditions | 24 conditions | not recorded | recorded value |
| Transferred bundle | 42256285 bytes | not recorded | recorded value |
Experimental limitations
- Folder chronology and failed preflight are not benchmark completion.
- No replacement allocation or synthetic result substituted.
The bundle transfer and CUDA preflight succeeded, but the instance disappeared before measurements. This supplies no policy or performance result.