BenedictRESEARCH / NOTES
Splice
Moving a calculation without rewriting the application
Could I change where a calculation runs without asking the application to become a distributed program?
Applications already ask other code to do work
A native application often hands a calculation to a library: multiply these matrices, return a result here. That call is a useful boundary because the operation already has a defined purpose. I wanted to find out whether I could intercept selected calls and choose another execution path while leaving the application’s call unchanged.
Splice explores that boundary for numerical library calls. The attraction is not invisibility for its own sake. Requiring every existing application to adopt a new remote API would exclude much of the software I am interested in.
The function name tells only part of the story
A matrix multiplication call describes more than two rectangles of numbers. It can use different storage orders, transposed inputs, padding and an existing output that contributes to the answer. Input and output memory can overlap in ways a replacement must understand.
I therefore approached interception as a contract to preserve. Capturing the visible input, choosing where to execute, checking the answer and returning it must not quietly change the caller’s memory or error behaviour.
What I built and tested
I built interception around selected BLAS matrix-multiplication calls, beginning with Linux and then testing a separate native Windows route. BLAS is a standard interface for common numerical operations; it gave me a smaller, more testable problem than intercepting arbitrary application code.
The checks go beyond whether one multiply returns the right numbers. I tested inaccessible memory, overlapping buffers, changes made during callbacks, malformed results, repeated entry into the interceptor and failures while publishing output. Unsupported cases need to take the original path rather than produce a plausible-looking answer.
A successful hook is not an acceleration
It is tempting to stop when a remote answer arrives. But the application also paid for capture, transport, checking and return. A call that finishes quickly inside a worker can still make the caller wait much longer.
The later runtime measurements made that visible: the resident remote route in one repeated GPU comparison took about 12.9 seconds against 1.75 seconds locally. Interception gave me a place to make the decision; it did not make remote execution the right decision.
AutoCAD asked a different integration question
The AutoCAD work is related, but it is not the same mechanism as replacing a numerical-library call. I used the host’s geometry interfaces to return selected results into a live drawing session.
That experiment reinforced the same lesson. Returning a correct result to the host is an engineering milestone. Supporting more geometry, surviving failures and beating an equivalent local implementation each require their own work.
Where I would draw the line
I do not have a universal interceptor for arbitrary software. A useful implementation needs an operation whose inputs, effects and return behaviour can actually be understood.
The next step is not to intercept more functions indiscriminately. It is to find boundaries that are both safe to replace and expensive enough that a different execution path could pay for its overhead.
Technical details and evidence
The Linux implementation is the naught-splice package, with the naught_intercept library name, targeting Linux LP64 CBLAS. Selected entry points include cblas_sgemm and cblas_dgemm. The Windows path has separate native-provider tests; Linux results do not establish Windows compatibility.
The isolated Linux checks record 39 passing Rust tests in each of two builds and seven validation-script successes. Those are correctness/integration checks, not speed measurements. Experimental A40 resident-object tests also validate numerical vectors and rejected requests without establishing a GPU performance win.
Memory access, aliasing, exactly-once fallback and partial-publication checks remain available with the component and native-runtime records.
Checking each component on its own, then all four together
Technical details and evidence
- Experiment identifier
- rt-component-qualification-counts
- Original title
- Independent qualification counts for the four extracted components and their composed integration
- Measurement boundary
- The build, test and reproducibility counts recorded for each component when it was extracted and qualified in isolation, and for the composed integration of all four. The published counts are the qualification evidence for the component boundaries, not performance results.
- Hardware
- An isolated Linux CPU container image with a pinned digest, 1 CPU, 1,536 MB of memory and memory swap, a process limit of 128, a 768 MB target temporary store and a 64 MB temporary store. Compiler toolchain differed by component: 1.90.0 for the codec component and 1.99.0 for the interception and runtime components.
- Software
- Rust 2024 edition crates with exact dependency pins, vendored offline sources, locked builds, and an internal dependency resolved by exact lock checksum to the qualified artifact of the dependency.
- Input
- Not applicable. These are source and test qualification counts, not workload measurements.
- Samples
- Two independent clean locked builds per component, plus two builds of the composed integration.
- Statistical method
- Counts of passed and failed test cases, build outcomes and reproducibility flags.
- Cold / warm state
- Not applicable.
- Network
- No network use at build or test time; the container ran with no network and the dependency registry index was never consulted.
- Serialization
- Not applicable.
- Correctness checks
- Each component ran its retained test vectors unchanged from the frozen source, and the composed integration re-ran all four component suites plus a consumer probe and the ported interception scripts.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| contracts unit tests passed | 13 cases | 1 | 13 passed and 0 failed, with a compiler-resolved API comparison |
| codec component tests passed per build | 18 cases | 2 | 18 passed and 0 failed on each of two builds |
| interception component tests passed per build | 39 cases | 2 | 39 passed and 0 failed on each of two builds |
| frozen interception validation scripts passed | 7 cases | 1 | 7 passed and 0 failed, including malformed and truncated failure modes |
| runtime component tests passed, first build | 70 cases | 1 | 70 passed and 0 failed |
| runtime component tests passed, second build | 61 cases | 1 | 61 passed and 0 failed |
| composed integration tests passed per build | 145 cases | 2 | 13 contracts, 18 codec, 70 runtime, 39 interception and 5 consumer probe |
| composed integration interception scripts passed | 6 cases | 1 | 6 passed and 0 failed |
| packages in the composed lock | 27 cases | 1 | 27 of 27 verified acyclic by topological sort |
| release files staged and hash recorded | 22 cases | 1 | 22 staged files recorded with digests before promotion |
| codec component compiled reproducibility | 1 cases | 2 | 1 denotes byte-identical library artefacts across both builds |
| interception component compiled reproducibility | 1 cases | 2 | 1 denotes byte-identical library and shared-library artefacts across both builds |
| runtime component compiled reproducibility | 0 cases | 2 | 0 denotes not claimed for this component; both differing digests are retained |
| codec component crate byte reproducibility | 0 cases | 1 | 0 denotes crate bytes are not byte-deterministic across runs; the admission archive is bound by checksum |
CALCULATED FROM THE MEASUREMENTScodec component tests passed per build ÷ composed integration tests passed per build = 0.12413793×705.55556% higher value for composed integration tests passed per build.
CALCULATED FROM THE MEASUREMENTSruntime component tests passed, first build ÷ composed integration tests passed per build = 0.48275862×107.14286% higher value for composed integration tests passed per build.
Experimental limitations
- Scope is isolated Linux CPU only; no Windows, accelerator, desktop or publication qualification is inferred.
- The runtime component records compiled reproducibility as not claimed, because its two host-side builds produced different library digests; both digests are retained.
- The codec component records its crate bytes as non-reproducible, with the single admission archive bound by checksum in the run that produced it.
- A first attempt at the codec component failed on offline resolution of the internal dependency and that failure is retained.
- The contracts component had already been published, so its later additions are additive overlay files only.
- Independent review of the composed integration accepted the result but disclosed that one check reused a pre-composition receipt and that a large-message assertion was made on the compression path within the frame bound.
Each component was qualified in isolation and then qualified again inside the composed graph, which is the shape of the claim: the boundaries hold and the composition reproduces them. The published counts also show exactly where the evidence is thinner, which is the runtime component, whose compiled reproducibility is recorded as unclaimed rather than asserted.
The same workload through the interception path and the dispatcher
Technical details and evidence
- Experiment identifier
- rt-win-native-rust-runtime
- Original title
- Runtime path and interception path for the same native workload on a personal Windows machine
- Measurement boundary
- The same frozen workload executed through the runtime and through the compiled C interception path on a personal Windows machine, with the native provider loaded and admitted, plus the traffic accounting for the session. The native C baseline is a separate cohort.
- Hardware
- A personal Windows 11 Pro machine, build 10.0.26200, with an Intel i7-8665U, 4 cores and 8 logical processors, 8,325,652,480 bytes of memory and Intel UHD Graphics 620. Available memory was limited and builds and benchmarks ran sequentially with one compiler job; existing user processes were left running.
- Software
- The same pinned official OpenBLAS 0.3.34 x64 LP64 dynamic library of 51,379,251 bytes, loaded through a validating provider that checks the image, locks file and ancestor directory identities against write, deletion and rename, restricts dependency search to the system directory and validates imports and loaded function ownership.
- Input
- The frozen 2048 by 2048 f32 operands with recorded input digests, alpha 1, beta 0, producing a fully materialised 16,777,216-byte output.
- Samples
- Two calls per configuration in single fresh child processes. The offline test suite run reported 145 passed and 0 failed with no ignored case; the interception unit suite reported 17 passed and 0 failed with 1 ignored child helper that the parent executes as a fail-stop child; the compiled C to interception to runtime to native library path reported 52 cases passed, of which 48 were intercepted and 4 were exactly-once original fallbacks.
- Statistical method
- Single recorded values per interval; no repeated-sample distribution.
- Cold / warm state
- One cold child per configuration performs provider admission, input generation and hashing, then the two calls. The repeated call is the only warm observation.
- Network
- No remote worker is contacted. The two-operation session recorded zero outbound and zero inbound object-stream bytes over 1,722,169 microseconds of active observation.
- Serialization
- The two-operation session recorded zero outbound and zero inbound object-stream bytes, zero reservations and zero remote transfers. The traffic budget assessment records the policy target of 100,000,000 decimal bytes per active hour, that a full active hour was not observed, and that a remote raw comparison was not measured.
- Correctness checks
- Both calls produced a full 16,777,216-byte output with a SHA-256 matching the independent C fixture exactly, and the runtime additionally checked 64 independent f64 scalar dot products per call. Publication preflights output pages and, once any publication write is attempted, a failure reports uncertainty rather than recomputing the scaled output term.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| release direct provider executor, first call | 219.018 ms | 1 | single recorded interval |
| release direct provider executor, repeated call | 207.832 ms | 1 | single recorded interval |
| release runtime automatic local dispatcher, first call | 219.958 ms | 1 | single recorded interval |
| release runtime automatic local dispatcher, repeated call | 201.334 ms | 1 | single recorded interval |
| release paired direct, runtime and diagnostic comparison, first call | 635.56 ms | 1 | single recorded interval |
| release paired direct, runtime and diagnostic comparison, repeated call | 604.621 ms | 1 | single recorded interval |
| provider admission and load | 297.008 ms | 1 | single recorded interval |
| input generation and full input hashing | 362.825 ms | 1 | single recorded interval |
| diagnostic whole workload, two-call paired comparison | 2382.333 ms | 1 | single recorded inner interval, excluding final publication and library drop |
| launcher wall | 2640 ms | 1 | approximate figure reported as about 2.640 seconds, including process, serialisation and unload overhead |
| runtime offline test suite, passed | 145 cases | 1 | 145 passed with 0 failed and 0 ignored |
| runtime offline test suite, failed | 0 cases | 1 | 0 failed |
| interception native unit suite, passed | 17 cases | 1 | 17 passed with 0 failed |
| interception native unit suite, ignored | 1 cases | 1 | one ignored child helper, executed by the parent as a fail-stop child |
| compiled C to interception to runtime to native library cases, passed | 52 cases | 1 | 48 intercepted plus 4 exactly-once original fallbacks |
| object stream bytes observed over the two-operation session | 0 bytes | 1 | zero outbound and zero inbound bytes |
| active observation window for the traffic accounting | 1722.169 ms | 1 | a full active hour was not observed |
| traffic budget policy target | 100000000 bytes | 1 | decimal bytes per active hour |
| provider archive downloaded during setup, outside the operation counters | 40860061 bytes | 1 | engineering setup download |
CALCULATED FROM THE MEASUREMENTSrelease runtime automatic local dispatcher, repeated call ÷ release paired direct, runtime and diagnostic comparison, repeated call = 0.33299207×200.30745% higher time for release paired direct, runtime and diagnostic comparison, repeated call.
CALCULATED FROM THE MEASUREMENTSrelease runtime automatic local dispatcher, first call ÷ release runtime automatic local dispatcher, repeated call = 1.0925030×8.4670710% lower time for release runtime automatic local dispatcher, repeated call.
Experimental limitations
- Two calls per configuration with no statistical qualification and no 30-block campaign.
- The traffic accounting covers eligible local calls only and explicitly does not cover arbitrary application usage.
- The installation traffic of a 40,860,061-byte provider archive was not captured and is outside the per-operation counters, so no claim of zero total session traffic is made.
- The provider archive is unsigned and the library loader does not sandbox library initialisers.
- A four-thread native configuration failed to allocate under a 512 MiB cap, so the observed shape of the threading cost is incomplete.
- No remote worker, remote traffic or accelerator execution is represented.
Through the full runtime dispatcher the frozen workload takes about 202 to 220 ms per call, essentially the same as the independent native C baseline of about 214 to 218 ms, and the whole call including the paired direct and runtime comparison plus diagnostics takes about 605 to 636 ms. The overhead of routing through the runtime on this machine is therefore small in absolute terms. The session moved no network bytes at all, because eligible local execution opens no transport. None of this is an acceleration claim and none of it is a remote comparison.
Checking the resident-object interface against a real accelerator
Technical details and evidence
- Experiment identifier
- rt-a40-resident-conformance
- Original title
- Resident-object conformance on a real CUDA accelerator
- Measurement boundary
- A conformance run of the experimental resident-object interface against a real accelerator: 128 configuration vectors and 22 admission rejections, with device counters and a cost envelope. The run qualifies the interface shape on that hardware and nothing beyond it.
- Hardware
- One NVIDIA A40 with 46,068 MiB of memory, compute capability 8.6 and driver 570.195.03, on a kernel 6.8.0-60-generic host running Ubuntu 24.04 with glibc 2.39. The CUDA runtime library actually loaded by the conformance process was version 12.8.90, with the driver library, the BLAS library and the BLAS-lt library recorded by size. No compiler toolkit was present in the image, which the runtime design does not require.
- Software
- A runtime build compiled with the experimental resident feature explicitly enabled and default features off, using a pinned toolchain and 33 vendored dependency archives, with the lockfile byte-identical before and after the build.
- Input
- 128 configuration vectors spanning f32 and f64, row-major and column-major layouts, no-transpose and transpose combinations on both operands, four alpha and beta pairs, and padded and unpadded tails. A separate set of 22 admission-rejection cases exercised the negative admission path.
- Samples
- 128 conformance vectors and 22 admission rejections in one qualifying run. The earlier failed orchestration attempts are retained separately and are not counted.
- Statistical method
- Counts of passed vectors, passed rejections and device counter totals. No timing distribution is published.
- Cold / warm state
- Each vector was executed at three object versions, giving 384 executions and 1,152 references. No cold-start or steady-state split is published.
- Network
- Not recorded as a separate figure in this qualification.
- Serialization
- Not recorded as a separate figure in this qualification.
- Correctness checks
- The run reported all 128 vectors passing with indices exactly 1 to 128 and all 22 admission rejections passing, with the harness result and the admission result both reported. Requested-owned allocation returned to zero at clean close.
| Original metric | Exact value | Samples | Statistic |
|---|---|---|---|
| conformance vectors passed | 128 cases | 128 | indices exactly 1 to 128 across dtypes, layouts, transposes, scalar pairs and padding |
| admission rejections passed | 22 cases | 22 | negative admission cases |
| requested-owned allocation high-water | 4195024 bytes | 128 | requested-owned device allocation high-water |
| requested-owned allocation at clean close | 0 bytes | 1 | requested-owned bytes at close |
| upload bytes credited | 32640 bytes | 128 | store-owned host-to-device counter after successful validated operations |
| device-to-device clone bytes credited | 24192 bytes | 128 | store-owned device-to-device counter |
| materialisation bytes credited | 56832 bytes | 128 | store-owned device-to-host counter |
| executions | 384 calls | 128 | 128 vectors at three object versions |
| references | 1152 calls | 128 | references across the run |
| evictions | 0 calls | 128 | evictions recorded during the run |
| device total memory | 48305799168 bytes | 1 | 46,068 MiB reported by the device, expressed in bytes |
| envelope cost estimate | 0.091 usd | 1 | estimate from a price readback, not a billing receipt |
| envelope hard cap | 1 usd | 1 | the declared hard cap for the qualification envelope |
| elapsed across the short-lived instances | 1134000 ms | 6 | 18.9 minutes total across 6 instances against a 90 minute allowance |
CALCULATED FROM THE MEASUREMENTSenvelope cost estimate ÷ envelope hard cap = 0.091000000×998.90110% higher value for envelope hard cap.
Experimental limitations
- The resident feature is default-off and no worker advertises it.
- This is a single experimental trusted in-process conformance, not a full acceptance of the accelerator gate.
- There is no real device fault injection, lease or quota enforcement, pinned eviction test, physical copy tracing or private and global memory enforcement.
- Requested-owned counters are interface counters and are not device memory enforcement.
- The cost figure is an estimate from a price readback and is not a billing receipt; provider timer enforcement was not proved.
- No general accelerator readiness is claimed.
The resident-object interface behaves as specified on real hardware across 128 configurations and 22 negative admissions, with allocation returning to zero on close and a requested-owned high-water of about 4 MB. The interface is qualified in shape. It is not qualified in fault behaviour, memory enforcement, performance or product readiness, and the source constant that records non-validation is carried unchanged.