BenchZgxFury

The best values observed among the selected measurements. Higher throughput (tok/s) is better; a lower latency (TTFT = delay before the first token, TPOT = delay between two tokens) is better.

A summary automatically written from the measures displayed (it follows the filters): what to remember, family by family.

An LLM server serves several users at the same time: the concurrency is the number of simultaneous requests. The aggregate throughput (tokens generated per second, all requests combined) increases with concurrency, while latency per user deteriorates. The TTFT is the waiting time before the first word, the TPOT the time between two words (p50 = median value, p99 = unfavorable case experienced by 1% of requests). Each point is the median of several repetitions. The gray dotted lines are published figures (reference) for the same model and the same profile.

The goodput is the quantity of useful work served while respecting a quality of service objective (SLO): here TTFT p99 ≤ ms and inter-token latency p99 ≤ ms. The bar indicates how many simultaneous users the machine supports without exceeding these thresholds; the curve shows the share of compliant requests (“attainment”) as the load increases.

llama.cpp micro-benchmark, no server: pp = prompt processing (prefill, tokens/s), tg = text generation (tokens/s). The suffix _d4096 means that the context already contains 4096 tokens, which slows down both phases.

Score (%) on standard tasks: mathematical reasoning (GSM8K), knowledge and reasoning (MMLU-Pro), science, coding, image comprehension. A quantified model (NVFP4, FP8, GGUF, etc.) is faster and more compact but can lose precision: when its BF16 version is also evaluated, the difference (“delta”) beyond points is reported as an alert.

The RTFx (real-time factor) indicates how many seconds of audio are transcribed per second: 1000× means that an hour of audio is processed in 3.6 s. The WER (word error rate, %) measures precision: lower = better.

Embedding models transform text into a vector for semantic search (RAG). We measure the number of requests per second according to the length of the text (256, 512 or 2048 tokens) and concurrency, p99 latency, and search quality (average MTEB score, higher = better).

Same indicators as for LLMs, but the requests contain an image (512 px or 1024 px) or a video. Image processing especially lengthens the TTFT.

Number of 1024x1024 frames (30 steps) produced per second based on batch size, latency per frame, GPU memory at peak, and duration per clip for video generation.

Latency before the first audio response according to the number of simultaneous conversations (workers). If the server could not start (expected on aarch64 architecture for certain dependencies), the go / no-go table indicates the blocking step.

Matrix-Game 4.0 (Skywork) replaces the graphics rendering of a real game with one video diffusion model : the native game runs in low-poly, and the GPU “repaints” each frame in real time from this view and the player's controls. Two scenes are measured: Doom (ancient egypt, first person shooting) and kart (racing, race). Three frame rates per second: fps model = images produced by the model alone, FPS online = images actually served to the client during the session (what the player sees), FPS encoded = images output from the H.264 video encoder. We also note the delay before the first image, server startup, latency per image block (chunk) and VRAM at peak. THE clip shows dual view low-poly | rendered to judge by eye. A software OpenGL rendering (without NVIDIA EGL) is reported: its FPS are not representative.

Each model receives the same instructions and must produce, at temperature 0, seven complete HTML5 games in a single file (snake, breakout, flappy, invaders, tetris, doom, kart), according to several modes : skills (a single answer, with identical best practice sheets for all models), agent (the pi agent writes the file, tests it itself and corrects it, with the same skills) and possibly oneshot (deposit only). The captions read “model · fashion.” The generated file is loaded into a Chromium without screen for 15 s with injected keys: we check that it loads, starts and runs without crashing, we count rendered images and console errors. A local judge model then rates the code out of 10 (completeness in relation to the instructions, playability, quality of the code); the overall score is the average. The mention “self-assessment” indicates that the judge is the model evaluated, which can bias the score. Click a thumbnail to play to the generated game and compare the models by hand.

Diarization divides a recording into speaking turns allocated to each speaker. The DER (diarization error rate, %) is the portion of time misattributed: lower = better. It breaks down into words missed, false alarm (speech wrongly detected) and confusion between speakers. The RTFx indicates how many seconds of audio are processed per second; in fashion streaming, we also measure the processing latency of each audio chunk.

Power, temperature and GPU clock readings every second during a measurement (nvidia-smi dmon), then, when it was read, the activity of the processor and the memory of the Station every 2 s (total CPU, busy cores, server and client containers, memory used). The point cloud relates average power and throughput to estimate energy efficiency (tokens per second and per watt).

Each line is a JSON record. Click a header to sort, tap to filter, export to CSV. Failed measurements display their error message on hover.