Best GPU for a home AI server
Last updated
The best GPU for a home AI server holds your models at a power level a home can live with. A used RTX 3090 (24 GB) is the value pick. Olares One's RTX 5090 Mobile (24 GB) is capped at 175 W.
Olares One → Home AI server → Ollama · Open WebUI · Immich → Family AI that stays at home
How much GPU memory does a home server need?
For most homes, 24 GB. Memory decides which models run at full speed, so start there. A 14B model needs about 9 GB, a 32B model about 19-20 GB and a 70B model about 40 GB (approximate rule of thumb at 4-bit). A 24 GB card holds 32B models with room left for context, which covers most family use.
When a model does not fit, part of it runs from system memory and answers slow down sharply.
Which is the best GPU for a home AI server?
We pick 24 GB at a moderate power limit. A used RTX 3090 is the value choice if you build. NVIDIA's desktop RTX 5090 has more memory, 32 GB, but NVIDIA rates it at 575 W total graphics power. The RTX 5090 Mobile in Olares One has 24 GB with a 175 W max TGP, per Olares' product page.
| GPU | Memory | Power figure | How you buy it (prices read 12-13 Nov 2026) |
|---|---|---|---|
| RTX 3090, used | 24 GB GDDR6X, 936 GB/s | Tower about 515 W at sustained full load (SpecPicks estimate) | In a DIY tower, about 3,800 USD (SpecPicks estimate) |
| RTX 5090 desktop | 32 GB GDDR7 | 575 W total graphics power; NVIDIA lists 1000 W system power | Card 1,999 USD at NVIDIA, plus a PC |
| RTX 5090 Mobile | 24 GB GDDR7 | 175 W max TGP | Inside Olares One, 4,999 USD |
| Apple M5 Ultra GPU | 96 GB unified (base), 1.2 TB/s | Mac Studio rated 480 W maximum continuous | Mac Studio, 5,499 USD at Apple |
| AMD Radeon 8060S | 128 GB shared with the CPU | Boxes with 230 W to 400 W power supplies | Ryzen AI Max+ 395 mini PCs |
Why do power and heat matter more at home?
A home server sits near people. A GPU that turns hundreds of watts into heat needs fast fans and warms the room during long jobs. Lower power limits make quiet cooling easier. That is the main argument for a mobile-class GPU in a home server, even though it holds less than the desktop card with the same name.
Olares says Olares One measured 38.8 dB in its lab with the GPU at 175 W and the CPU at 55 W. We have no matching noise figures for the other options, so read reviews of any specific build. The quiet box guide and the power guide go further.
Do you need CUDA at home?
You need CUDA if the family wants image or video generation, or tools built for NVIDIA first. For plain chat you don't, because Ollama runs on NVIDIA, Apple and AMD GPUs. Our pick changes if 70B models matter more than CUDA. A Mac Studio's unified memory holds them, and no 24 GB card does.
In Olares' own benchmark (October 2025), Olares One produced a FLUX.1 dev image with Nunchaku FP4 in 15.51 s the first time and 8.32 s after that. The build or buy guide compares the box with a used RTX 3090 tower.
Questions & answers
Is 24 GB of VRAM enough for a home AI server?
For most households, yes. It holds models up to about 32B at 4-bit (approximate), which covers chat, summaries and coding help. It is not enough to run 70B models at full speed.
Should I buy an RTX 5090 desktop card for home AI?
It gives 32 GB for 1,999 USD at NVIDIA's store (read 12 November 2026), but it needs a full PC and NVIDIA lists 1000 W of system power. Plan for the heat and noise if the PC sits in a living space.
Sources for this entry
- Used RTX 3090 tower build: maker's page (Maker-published)
- NVIDIA GeForce RTX 5090 (desktop card): maker's page (Maker-published)
- Olares One: maker's page (Maker-published)
- Olares One product page (Company-published)
- Apple Mac Studio (M5 Ultra): maker's page (Maker-published)
- GMKtec EVO-X2 (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- Framework Desktop (Ryzen AI Max+ 395, 128GB): maker's page (Maker-published)
- Olares One FAQ (Olares docs) (Company-published)
- Olares blog: local AI hardware benchmarking (Oct 2025) (Company-published)
Continue this reading path
How the related websites connect
Cite this entry
Best Home AI Server. “Best GPU for a home AI server.” Content reviewed 13 November 2026.
https://besthomeaiserver.com/best-gpu-for-home-ai-server/