Local LLM workstation: how much VRAM a model needs.
Running a large language model locally is a memory problem first. The model has to fit in GPU memory, and the size of the model you can load is set by total VRAM. This guide covers how to estimate it and which of our three AI workstation builds provides each capacity.
- 32GBone RTX 5090 or RTX PRO 4500
- 48–96GBone RTX PRO 5000 or 6000
- 144–192GBtwo RTX PRO GPUs
- 288–384GBfour RTX PRO GPUs
Estimating VRAM for a model
Model weights occupy roughly parameters × bits per weight ÷ 8. A 70-billion-parameter model at 4-bit quantization is about 35GB of weights; with context and runtime overhead it requires approximately 48GB of VRAM. Context length and batch size add to every figure below.
| Model and precision | VRAM | Configuration |
|---|---|---|
| 30B parameters, 4-bit | Fits in 32GB | JNCS AI 5090, or one RTX PRO 4500 |
| 70B parameters, 4-bit | Approximately 48GB | One RTX PRO 5000 (48GB) |
| 70B parameters, 8-bit | Fits in 96GB | One RTX PRO 6000 (96GB), or two RTX PRO 5000 (48GB) |
| Larger models, longer context, several users | 144GB to 384GB | Two or four RTX PRO 5000 or 6000 GPUs |
These are reference figures for the weights plus working overhead. Longer context windows, larger batch sizes, and serving several users at once all raise the requirement, so leave headroom.
Which build provides which capacity
All three share the same assembly, burn-in, and quote process. They differ in total VRAM and in the platform sized to the GPU count.
JNCS AI 5090: 32GB
One GeForce RTX 5090 on Ryzen 9000 and X870E, with up to 128GB of DDR5. Runs on a standard 15A, 120V circuit.
RTX 5090 AI workstation →JNCS AI PRO: 32GB to 192GB
One or two RTX PRO Blackwell GPUs on Threadripper 9000 and TRX50, with 64GB to 512GB of quad-channel registered ECC DDR5.
RTX PRO Blackwell workstation →JNCS AI MAX: 32GB to 384GB
One, two, or four RTX PRO Blackwell GPUs on Threadripper PRO 9000 and WRX90, with 128GB to 1TB of eight-channel registered ECC DDR5.
Multi-GPU AI workstation →The rest of the system
VRAM decides what loads. These decide how well the machine keeps the GPUs supplied.
System memory
A common guideline is system memory at least equal to total VRAM. The configurator shows total VRAM for the selected GPUs and flags a memory selection below it.
Storage
Model files and datasets are large. Each build boots from a Samsung 9100 PRO PCIe 5.0 NVMe drive of 1TB to 8TB, with additional NVMe slots and optional enterprise hard drives for bulk storage.
PCIe lanes
Multi-GPU builds use Threadripper platforms so every GPU runs at a full PCIe 5.0 x16 link: 80 lanes on Threadripper 9000, 128 on Threadripper PRO 9000.
Power
A single-GPU system runs on a standard 15A circuit. Two RTX PRO 6000 cards, or any four-GPU configuration, need a dedicated 20A circuit, and four RTX PRO 6000 cards need a 30A, 220–240V circuit.
Assembly, burn-in, and delivery
Each system is assembled by hand at our shop in Fairport, NY, by a shop that has built computers for more than thirty years.
Fit and power validated
The configurator checks physical fit, memory, PCIe slots, and power on every selection. The power supply is sized automatically to the GPU count and estimated draw, with a 12V-2x6 cable for every GPU.
Burn-in under load
Every system is stress-tested under sustained full load and firmware-updated before it leaves the bench, so the first long job is not the first test.
Windows 11 Pro installed
Windows 11 Pro is installed and updated before shipment.
Warranty and support
Every build carries a 3-year warranty and lifetime technical support from the shop that built it.
Quoting and payment
The configurator shows a price for every part and an estimated total, including assembly, burn-in, and testing, as the build is configured. GPU and memory pricing changes frequently, so a technician confirms GPU availability and final pricing by email, with lead time, typically within 1-2 business days. There is no cost and no obligation to request a quote.
Payment for AI workstations is by ACH bank transfer; credit cards are not accepted for these systems. An invoice with bank details is issued once the quote is approved. Systems ship across the United States from Fairport, NY, with local pickup available.
Local LLM hardware questions
How much VRAM do I need to run an LLM locally?
Can a 70B model run on a single GPU?
Is an RTX 5090 enough for local LLMs?
Does system RAM matter if the model runs on the GPU?
What is the largest configuration you build as a workstation?
How is an AI workstation ordered and paid for?
Related AI workstation builds
RTX 5090 AI workstation
One GeForce RTX 5090, 32GB VRAM, Ryzen 9000, standard 15A circuit.
View →RTX PRO 6000 Blackwell workstation
One or two RTX PRO Blackwell GPUs, up to 192GB VRAM, Threadripper 9000.
View →Multi-GPU AI workstation
Up to four RTX PRO Blackwell GPUs, up to 384GB VRAM, Threadripper PRO 9000.
View →Size the build to the model.
Open the configurator, select a build and GPU count, and it shows total VRAM, estimated power draw, and the circuit the system needs as you go.
Have an exact parts list or a department spec sheet? Send the whole list. For more than four GPUs or rackmount systems, see AI servers and larger systems.