Backboard homeBlog

Rob Imbeault

·

Gemma 4 Revisited: Up to 2.38× Faster Than vLLM

What if rapid iteration led to huge leaps, rather than small steps?


We went back to our original Gemma 4 12B benchmark to see how much more performance we could get out of the same GPU. Our latest inference engine not only outdid our previous results, but also widened the lead over vLLM. 


Backboard’s time to first token (TTFT) at p90 is up to 2.38× faster than vLLM. We were already clearly ahead with a peak of 2.18×, but we’ve extended our lead with our latest version.


The improvements were also pronounced with output throughput.


Backboard now produces up to 68% more tokens per second than vLLM where we previously delivered ‘just’ 34% more. 


These weren’t isolated gains, either.


Across all eight tested combinations of prompt length and concurrency, the latest Backboard results delivered more throughput and shorter first-token waits than vLLM compared to our previous Gemma 4 run.


Quicker starts and more capacity are here, now.


How does Backboard compare against itself?


You want to know that your engine provider is constantly improving, not just besting the competition.


The latest Backboard engine reduced p90 TTFT by up to 2.17×.


There were large improvements at different prompt sizes and user loads, too.


Workload

Previous p90 TTFT

Backboard best p90 TTFT

Shorter wait

1K, 1 request

85.6 ms

70.6 ms

1.21×

1K, 4 requests

174.7 ms

80.4 ms

2.17×

1K, 16 requests

273.6 ms

169.3 ms

1.62×

1K, 64 requests

509.3 ms

367.5 ms

1.39×

8K, 1 request

547.0 ms

501.7 ms

1.09×

8K, 4 requests

818.6 ms

565.8 ms

1.45×

8K, 16 requests

1,480.8 ms

1,040.0 ms

1.42×

8K, 64 requests

3,365.6 ms

2,040.3 ms

1.65×

TTFT multiples are the previous p90 latency divided by the latest p90 latency. Lower latency is better.


More output from the same workload


You also get more capacity once the engine begins working.


Backboard now provides up to 54% more output throughput than before, with very real upgrades across other concurrency and prompt size levels.


Workload

Previous output tok/s

Backboard best output tok/s

Throughput multiple

1K, 1 request

203.6

314.1

1.54×

1K, 4 requests

779.4

968.7

1.24×

1K, 16 requests

2,082.0

2,636.7

1.27×

1K, 64 requests

3,822.0

4,246.7

1.11×

8K, 1 request

179.1

264.4

1.48×

8K, 4 requests

529.6

616.0

1.16×

8K, 16 requests

961.8

1,138.5

1.18×

8K, 64 requests

1,254.0

1,333.8

1.06×

Throughput multiples are Backboard's best output tokens per second divided by the previous result. Higher throughput is better.


These are performance boosts you’re going to notice, even with heavier demand.


How this translates to the real world


This isn’t just a theoretical exercise.


Higher output throughput can wring out more workload capacity from the hardware you have.


Shorter first-token waits can make your agents and other interactive experiences feel more responsive, even when a glut of users risks bogging it down.


You’re getting more from the same engine for a wide range of scenarios.


Methodology


Our methodology remains the same as in the previous benchmark. We used SemiAnalysis InferenceX to evaluate google/gemma-4-12B-it at BF16 precision on a single NVIDIA RTX PRO 6000 Blackwell GPU with 96 GB of memory.


The workloads used input lengths of 1,024 and 8,192 tokens, with 1,024 output tokens, at concurrency levels of 1, 4, 16, and 64 requests. The original evaluation used prompt seed 0 at every point; the 8K, four-request configuration also used seeds 1 and 2, with the published result averaged across those three seeds.


Backboard used one inference configuration across benchmark points. Its draft head shortlists speculative candidates in 4-bit before BF16 rescoring; the target model remains BF16.

Backboard

Ottawa, Canada
Burlingame, California

Backboard © 2026