0:00 / 0:00
llama.cpp llama-server · Nemotron 3.5 Bolt 74B-A9B · RTX 3090 alone · 5.7K-token prompt
hf-packed-345x8.1b · 76.8 tok/s decode · 1 stream · IQ2_XXS
decode
76.8tok/s
prefill
1917tok/s
ttft p50
2969ms
| model | NVIDIA-Nemotron-3.5-Bolt-74B-A9B-24GB.gguf |
| model id | hf-packed-345x8.1b (from the gguf) |
| quantisation | IQ2_XXS · 2.06 bit |
| size | 74B params · 72B active · 22.0 GB · MoE 496 |
| engine | llama-server b11668 |
| host | linux |
| gpus | NVIDIA GeForce RTX 3090 / NVIDIA RTX A6000 |
| memory | ? |
| streams | 1 |
| workload | 5680 in / 361 out · cache 0% |
| context | 16384 window · 1 slot |
| config | fa on · b 4096 · ub 4096 · ngl 99 · offload full |
| machine | 269.9 of 550 W |
| prompt set | not in a comparison set |
| recorded | 2026-10-17T22:11:16Z |
| toktape | v0.6.0 |
Details
The transcript, the card as text, and why each caveat fired — read out of the record in your browser, the same way Replay draws it.
! 2 caveats —
bandwidth_over_ceiling, recorded. The card that comes with this record spells each one out.Page views are counted with Cloudflare Web Analytics: no cookies, no fingerprinting. The toktape binary itself sends nothing anywhere.
