EdgeKernelBench
Unified LLM-Driven Kernel Generation
for Edge Backends
News
The EdgeKernelBench project website is now available.
Our paper is now available on arXiv.
The research
Abstract
Recent works show that LLMs can generate kernels with strong performance on server GPUs. Edge deployment presents distinct challenges because the hardware is heterogeneous, the software stacks are fragmented, and the resource budgets are tight. LLM kernel generation across diverse edge backends has yet to be studied systematically.
We introduce EdgeKernelBench, which combines a unified IR deployment framework with a hierarchical benchmark for edge workloads. The unified IR captures reusable computation, scheduling, and memory decisions and connects to each target backend through a compiler interface. The benchmark spans progressively larger program scopes, from individual operators to compact model graphs, across representative edge application domains. We instantiate the design with TVM as a reference compiler and validate it across heterogeneous edge backends.
With a single generation attempt, the best model achieves 64.5% Correct@1 but only 25.5% Fast@1.0. This gap shows that functional correctness does not consistently translate into performance gains. We therefore introduce the Layered Edge Kernel Agent (LEKA), which organizes optimization through a layered IR. On the agent evaluation subset, LEKA achieves 43.5% Fast@1.0, compared with 13.0% for direct generation and 26.1% for Claude Code. LEKA and Claude Code use the same base model and optimization budget. Together, EdgeKernelBench and LEKA support unified evaluation and layered optimization of kernels generated by LLMs across edge backends.
The benchmark
A unified benchmark
for edge workloads.
EdgeKernelBench evaluates LLM-generated kernels for heterogeneous edge devices, from individual operators to compact model graphs. A unified intermediate representation connects generation to compilation and evaluation on real hardware through TVM.
Devices evaluated
in the paper
CPU / LLVM
Snapdragon 8 Gen 3
Mobile intelligence
GPU / CUDA
Jetson AGX Orin
Embodied robotics
GPU / OpenCL
RK3588 · Mali-G610
Smart AIoT
Task coverage
The 331 canonical tasks yield 650 backend-specific instances: 204 on CPU/LLVM, 218 on GPU/CUDA, and 228 on GPU/OpenCL. Input shapes are adapted to each deployment setting.
Measured on real devices
Results
One-shot generation
The strongest model, GPT-5.6-sol, achieves 64.5% correctness and 25.5% Fast@1.0 across all 650 task instances.
650 task instances per model
| Model |
|---|
Fast@1.0: correct and faster than TVM with MetaSchedule.
Evaluation details
Compile@1 measures compilation success. Correct@1 also requires all output checks to pass. Fast@1.0 additionally requires valid timings and speedup above 1×. All scores are percentages.
MNN & TensorRT
comparison
One-shot kernels are compared with MNN 3.3.0 and TensorRT 10.3.0 on the same devices. Gains are concentrated in individual operators.
Comparison details
Each curve shows the share of tasks with correct outputs, valid timings, and speedup above p. At p = 0, it measures correctness with valid timings. Each engine uses its own fixed supported subset, so these percentages have different denominators from the TVM results above.
The optimization framework
Layered optimization with LEKA.
LEKA coordinates agents across Relax, TIR, and target code, using hardware feedback and a shared candidate tree.
Relax
Graph structure and layout
TIR
Scheduling and memory reuse
Target code
Hardware-specific refinement
Agent comparison
69 instances · 23 per backend · GLM-5.3-Flash
One-shot: 42.0%
Relative to Claude Code
Comparison details
Fast@1.0 measures correct programs that outperform TVM with MetaSchedule. LEKA and Claude Code use the same base model, task contracts, evaluator, and per-task time budgets. Geometric mean speedup includes slower results. The paper rounds the overall 3.75× value to 3.8×.
Optimization in practice
Compact SqueezeNet1.1 on RK3588
LEKA combines graph rewrites, shared-memory tiling, and register blocking to optimize a compact SqueezeNet1.1 benchmark task. Each stage preserves useful improvements from the preceding layers.
Speedup relative to the initial candidate.
Register blocking improves matrix multiplications while preserving TIR-optimized convolutions.
Case analysis
The Relax agent adopts NHWC layout and rewrites 1×1 convolutions as matrix multiplications, creating opportunities for lower-level optimization. Building on this graph, the TIR agent uses shared-memory tiling to reuse convolution inputs.
The OpenCL agent then applies register blocking to the matrix multiplications while preserving the TIR-optimized convolutions. Reducing repeated input loads and accumulating results in registers brings the speedup to 3.47× over the initial implementation. The improvements build on one another as each agent refines its part of the computation.