Practical engineering

Eight GPUs are not automatically eight times faster

Compare time to completion, communication behaviour and total GPU-hours before choosing a larger training configuration.

2026-09-11 · Arvica Cloud · Analysis & buying guide

Calculate speed-up and cost separately

Consider an explicitly hypothetical job that takes ten hours on one GPU and two hours on eight GPUs, with identical work and comparable results. Speed-up is 5×, while scaling efficiency is 5 ÷ 8 = 62.5%. Consumption rises from ten to sixteen GPU-hours. With identical per-GPU hourly pricing and no other charges, the faster run costs 60% more. This example is arithmetic, not a performance claim or quote. A deadline may justify that trade-off; a fixed budget may not.

Ask what the eight GPUs actually share

Ask for the exact GPU variant, host layout and GPU-to-GPU topology; distinguish communication within a server from communication between servers. NVIDIA NCCL Tests can measure collective communication correctness and performance. [1] Treat such results as a diagnostic layer. They do not establish training throughput on your model. A quoted network line rate alone is insufficient evidence of useful application scaling.

Keep the comparison honest

Agree whether the experiment keeps the total batch size fixed or grows the workload with the GPU count. Record model revision, sequence length, precision, optimiser, dataset and stopping criterion. If the model cannot fit on one GPU, use the smallest valid configuration as the baseline and state that explicitly. Compare two, four and eight GPUs only when each is technically valid; include validation quality and recovery behaviour, not just step time.

Write acceptance around the actual job

Propose a bounded trial using a representative training segment and a checkpoint restart. Agree the maximum elapsed time and total charge before the trial. Separate setup delays from steady-state results and retain logs of any failed steps. For a long reservation, request the tested topology and software assumptions in the configuration summary. A successful test on a different machine is useful context, but not acceptance of the machine being rented.

Sources

Reviewed: 2026-09-11

NVIDIA — NCCL Tests ↗

Apply this to your workload

Explore the next step →Request a compute quote →