Same GPU.4.7 timesthe tokens.
leface is an LLM inference engine for teams who can't buy an H100. It brings its own GPU kernels, so a single RTX 4090 serves Qwen3.8-27B at 244 tok/s to a user, 4.7× vLLM on the same card.
Qwen3.8-27B, one RTX 4090, both engines at their measured speed. Every block is a token, a filled wall is one answer leface vLLM
Ask it something. Watch the meter.
Your prompt runs on the same RTX 4090 the benchmarks were measured on. Nothing is cached or replayed: the timings below are what your browser received.
The answer streams here, token by token.Same GPU, every load. Median of three runs.
Output tokens per second, greedy decoding, a different coding prompt per user, 128 tokens each. vLLM 0.32 runs its fastest configuration on every row.
RedHat INT4 weights in both engines; vLLM in the mode that fits 24 GB. vLLM 0.32 in its fastest configuration per row; median of 3 runs.
What you get. Nothing you have to change.
The same card, more tokens
On the RTX 4090 you already have, Qwen3.8-27B answers at 244 tok/s. Nothing to buy.
Long context in 24 GB
A full 128k-token context for a 27B model fits on one consumer card, and a 20k-token follow-up starts in 0.2 s.
Identical output
Speed costs nothing in quality: the tokens are the ones plain decoding would produce, and HumanEval+ says 94.5 for the 27B.
Drop-in API
OpenAI-compatible chat completions with streaming. Point your client at it and keep your code.
Faster, not looser. Same answers.
The speed costs no quality: the output is identical, token for token, to plain decoding. What 4-bit weights cost, measured on HumanEval+ (164 tasks, greedy, thinking off):
| model | leface 4-bit | vLLM fp8 |
|---|---|---|
| Qwen3.5-4B | 81.7 / 74.4 | 86.0 / 79.9 |
| Ornith-1.5-9B | 79.3 / 73.2 | 80.5 / 73.8 |
| Qwen3-14B | 87.8 / 82.3 | 90.9 / 86.0 |
| Qwen3.8-27B | 94.5 / 90.2 | INT4: does not fit in 24 GB |
base / plus pass@1, %. The small models lose 1–6 points to fp8 in 4-bit; the 27B keeps 94.5.
Straight answers.
One rented RTX 4090 (24 GB), Oct 2026. Greedy decoding, 128 output tokens per user, a different short coding prompt per user, thinking off. Every row is the median of three runs (spread under 2 %). vLLM 0.32 runs its best mode on every row, including its own speculative modes where they are faster.
No. For a given prompt, leface produces the same tokens as plain decoding of the same weights; the test suite checks that token for token.
From about 64 concurrent users up the GPU is compute-bound and the lead narrows to roughly 1.7–2.4×. Long-prompt prefill is bound by the card's compute too: a 64k-token prompt still takes ~21 s on a 4090.
Qwen3.5 / 3.6 / 3.8 hybrids, Qwen3 and Qwen3-MoE, Qwen2.5, Llama 3 (including its RoPE scaling), Mistral, Phi-3, Mixtral and DeepSeek-V2 on CUDA; AMD MI355X is next.
Soon. It serves an OpenAI-compatible API, so switching is a base URL. Ask for early access and you get the first release.