Backboard Inference
More inference. Same hardware.
License Backboard’s inference engine to run AI models faster and serve more demand on your existing GPUs.
Built for cloud providers, AI platforms, and enterprises running their own inference.
Measured against vLLM.
Our latest internal benchmarks at 128 concurrent requests.
Against vLLM
BF16 vs. BF16
FP8 vs. FP8
Median generation speed
2.49×
2.51×
Total token throughput
+21%
+22%
End-to-end latency, p90
38% lower
40% lower
BF16 results use the configuration with no reported quality flags. FP8 results also reported no quality flags.
Generation speed, to scale.
Median generation speed relative to vLLM at 128 concurrent requests. Ten blocks is vLLM’s speed.
BF16 vs. BF16
FP8 vs. FP8
Research that earns its place in production.
Backboard develops inference software and GPU kernels to get more useful work from existing compute.
Cloud providers
Increase serving capacity across your GPU fleet.
AI platforms
Deliver faster generation as demand grows.
Enterprises
Optimize inference on infrastructure you control.
Your model. Your hardware. Your results.
Start with a benchmark against your current serving stack. Measure speed, capacity, and quality on the workloads that matter to you.