Rob Imbeault
·
Gemma 4 Revisited: Up to 2.38× Faster Than vLLM
What if rapid iteration led to huge leaps, rather than small steps?
We went back to our original Gemma 4 12B benchmark to see how much more performance we could get out of the same GPU. Our latest inference engine not only outdid our previous results, but also widened the lead over vLLM.
Backboard’s time to first token (TTFT) at p90 is up to 2.38× faster than vLLM. We were already clearly ahead with a peak of 2.18×, but we’ve extended our lead with our latest version.
The improvements were also pronounced with output throughput.
Backboard now produces up to 68% more tokens per second than vLLM where we previously delivered ‘just’ 34% more.
These weren’t isolated gains, either.
Across all eight tested combinations of prompt length and concurrency, the latest Backboard results delivered more throughput and shorter first-token waits than vLLM compared to our previous Gemma 4 run.
Quicker starts and more capacity are here, now.
How does Backboard compare against itself?
You want to know that your engine provider is constantly improving, not just besting the competition.
The latest Backboard engine reduced p90 TTFT by up to 2.17×.
There were large improvements at different prompt sizes and user loads, too.
Workload | Previous p90 TTFT | Backboard best p90 TTFT | Shorter wait |
1K, 1 request | 85.6 ms | 70.6 ms | 1.21× |
1K, 4 requests | 174.7 ms | 80.4 ms | 2.17× |
1K, 16 requests | 273.6 ms | 169.3 ms | 1.62× |
1K, 64 requests | 509.3 ms | 367.5 ms | 1.39× |
8K, 1 request | 547.0 ms | 501.7 ms | 1.09× |
8K, 4 requests | 818.6 ms | 565.8 ms | 1.45× |
8K, 16 requests | 1,480.8 ms | 1,040.0 ms | 1.42× |
8K, 64 requests | 3,365.6 ms | 2,040.3 ms | 1.65× |
TTFT multiples are the previous p90 latency divided by the latest p90 latency. Lower latency is better.
More output from the same workload
You also get more capacity once the engine begins working.
Backboard now provides up to 54% more output throughput than before, with very real upgrades across other concurrency and prompt size levels.
Workload | Previous output tok/s | Backboard best output tok/s | Throughput multiple |
1K, 1 request | 203.6 | 314.1 | 1.54× |
1K, 4 requests | 779.4 | 968.7 | 1.24× |
1K, 16 requests | 2,082.0 | 2,636.7 | 1.27× |
1K, 64 requests | 3,822.0 | 4,246.7 | 1.11× |
8K, 1 request | 179.1 | 264.4 | 1.48× |
8K, 4 requests | 529.6 | 616.0 | 1.16× |
8K, 16 requests | 961.8 | 1,138.5 | 1.18× |
8K, 64 requests | 1,254.0 | 1,333.8 | 1.06× |
Throughput multiples are Backboard's best output tokens per second divided by the previous result. Higher throughput is better.
These are performance boosts you’re going to notice, even with heavier demand.
How this translates to the real world
This isn’t just a theoretical exercise.
Higher output throughput can wring out more workload capacity from the hardware you have.
Shorter first-token waits can make your agents and other interactive experiences feel more responsive, even when a glut of users risks bogging it down.
You’re getting more from the same engine for a wide range of scenarios.
Methodology
Our methodology remains the same as in the previous benchmark. We used SemiAnalysis InferenceX to evaluate google/gemma-4-12B-it at BF16 precision on a single NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB of memory.
The workloads used input lengths of 1,024 and 8,192 tokens, with 1,024 output tokens, at concurrency levels of 1, 4, 16, and 64 requests. The original evaluation used prompt seed 0 at every point; the 8K, four-request configuration also used seeds 1 and 2, with the published result averaged across those three seeds.
Backboard used one inference configuration across benchmark points. Its draft head shortlists speculative candidates in 4-bit before BF16 rescoring; the target model remains BF16.
