Get more performance from your real-time intelligence stack, with an agent that actually understands your target edge compute and precisely explains remaining headroom
Orin & Thor
Supported platforms today. More coming soon.
The steps below let RunLocal optimize your proprietary stack while minimizing IP sharing. For open-source models, just share the repository and target HW/SW versions.
Your coding agent uses the RunLocal Skill to locally prepare a Request Package from your internal model stack. You review it, then your agent uploads it to the RunLocal Platform.
No weights or training data are uploaded; full source code is not required. See the docs for details.
Our agentic ML compiler autonomously optimizes your model stack for your target compute platform (running on RunLocal’s infrastructure).
It experiments with inference optimizations, including custom kernels, quantization, operator fusion, and scheduling. It doesn’t perform distillation or broader model architecture changes (currently).
Optimized code, results, and insights are uploaded to the RunLocal Platform as a Deliverable Package. Your coding agent downloads it and integrates the changes locally. You then validate the results locally.
The problem RunLocal solves
The challenge for an agent is deciding what to try, what to measure to understand bottleneck root causes, and when to stop optimizing. While device access and profiling tools help an agent understand an existing implementation, they don't help predict how much potential optimizations might help because of interacting performance effects.
RunLocal combines a custom MLIR compiler, a causal and predictive hardware model calibrated on real benchmarks, and device knowledge that's reusable across workloads. Skip to our secret sauce
An agent can identify a hotspot and suggest plausible optimizations, without a reliable basis for ranking which changes will help most.
Choosing between them requires predicting how compiler choices affect interacting hardware mechanisms across the workload. A change can relieve one bottleneck while making another more important, and testing every possibility on-device takes too long.
More device runs can leave the root cause of a slowdown unclear and the next optimization little better informed.
Different bottlenecks can produce similar timings. Changing several things at once can hide which caused an improvement, so the agent needs experiments that isolate the relevant causal effects within a limited device budget.
An agent can stop with performance headroom left, or keep searching where little remains, because recent results have plateaued.
A plateau describes the results found so far. Estimating the remaining opportunity requires performance bounds grounded in the workload and device, with assumptions and uncertainty made clear.
The cost to your real-time intelligence stack
You're leaving performance on the table, without a clear sense of how much is remaining, and taking longer to hit your performance targets.
Improve performance, reduce costs, and plan your next move with more confidence.
Run your stack faster on the same compute
Fit larger models on the same compute
Do more without hiring more rare ML compiler talent
Meet performance targets with cheaper compute
Better estimate whether models will fit your compute
Better estimate performance headroom before committing
When standard compilers and libraries don’t deliver the performance you need
Workloads whose shapes or execution paths vary at runtime, such as sparse LiDAR perception with changing occupied-voxel counts.
Multiple models sharing hardware or feeding into one another, where coordinating execution and data movement can reduce overall latency.
Custom and emerging model operations that need more than standard library kernels.
Our secret sauce
RunLocal combines a custom MLIR compiler, a causal and predictive hardware model calibrated on real benchmarks, and device knowledge that's reusable across workloads.
RunLocalOur Hardware Reasoning Engine models instructions, memory access, resource contention and dependencies together. Calibrated on benchmarks we collect from real devices, it identifies what limits execution and predicts the effect of changing it. The causal model accounts for changes across the whole workload, including the bottleneck that follows.
Our dialects represent computation, data movement, dependencies and scheduling. Compiler transformations preserve this information through to generated code. Our reasoning engine uses it to predict the combined hardware effects of each optimization candidate, compare expected runtimes and identify remaining constraints before device testing. Each prediction is connected to a specific implementation.
Our reasoning engine derives a lower runtime bound from the computation, data movement and dependencies a workload requires. The gap to current performance bounds the remaining opportunity. It also simulates removing individual constraints to predict the next bottleneck and show which changes bring execution closer to that limit.
When benchmarks leave device behavior uncertain, our Experiment Selection Engine predicts how different explanations would respond to additional tests. It selects measurements that distinguish those explanations and resolve the missing evidence. This focuses device time on improving the model’s predictions, so the agent can compare optimization candidates with better information.
Our Hardware Knowledge Base retains benchmarks across instruction types, memory patterns and concurrency, together with calibrated device behavior. We validate predictions on benchmarks excluded from calibration. New workloads combine this knowledge with their own instructions and dependencies, reusing the work of measuring and explaining the device across optimization projects.
Help customers get more from your silicon. Bring their models onto your platform faster, broaden your model zoo, and improve the software behind it.
Book A Call (opens in a new tab)Port and optimize customer models faster without hiring more support engineers.
Add and maintain support for more models without hiring a large internal optimization team.
Use insights from RunLocal to improve your compiler, libraries, and developer tools.
See how you use RunLocal, and read the docs for more detail.
The challenge is helping the agent decide which implementation changes to try, understand their combined effects, and know how much performance remains. RunLocal adds compiler control, calibrated hardware models, reusable experiment knowledge and infrastructure for running and validating the search. See our secret sauce.
Building it means maintaining compiler integrations, hardware calibrations, reliable device experiments and correctness checks across models, devices and SDK versions. RunLocal provides that system and the hardware knowledge already collected, so your team can focus on its autonomy/perception stack.
We’re also building reusable compiler rules and experiment knowledge across projects, extending the coverage an individual team would otherwise need to develop itself.
For supported workloads, our own MLIR-based compiler provides an alternative path when the standard vendor compilation path restricts useful changes. On Qualcomm, it can produce QNN graphs directly while retaining Qualcomm’s backend and runtime. On NVIDIA, we can generate custom kernels for selected parts of a workload and retain TensorRT where it is useful. The choice depends on the workload and the performance opportunity.
Compiler-level changes include custom kernels, operator fusion, memory layouts, data movement, precision choices and execution scheduling. We consider the full inference workload, including interactions between operations and models. Distillation and broader model architecture changes are outside the current scope.
No. The Request Package describes your stack without requiring weights, training data, or full source code. See the docs.
A Deliverable Package on the RunLocal platform: optimized code or a binary, a performance report, and integration instructions. Your coding agent downloads and integrates it locally. You validate correctness and performance on your target compute.
A subscription gives your team access to RunLocal for a fixed recurring fee. Optimization requests are priced separately. For each request, we agree on performance targets, provide a fixed quote, and estimate the timeline. You only pay for that optimization if the agreed targets are met.
Start by booking a call to discuss your stack and target compute. We can then align on a 1-week pilot to prove RunLocal's optimization capability before you commit to something longer-term.
Tell us about your stack and target hardware. We’ll discuss where RunLocal could help and whether a 1-week pilot is a good fit.
BACKED BY
