Release
Lawyer’s $9,450 private AI server of 12 used Nvidia V100s now runs every DeepSeek V4.1 Flash expert at 4-bit NVFP4
Tom's Hardware · By Shane Downing ·
The Brief
- A lawyer upgraded a private AI server from eight to twelve used Nvidia V100 GPUs.
- The server now runs every DeepSeek V4.1 Flash expert at 4-bit NVFP4, per Tom's Hardware.
- Reported performance reaches up to 184 tokens per second for code; long documents start answering in about half the time.
Why it matters
It shows that large open-weight mixture-of-experts models can be run on cheap, used datacenter GPUs in a private setup, which matters for developers weighing local inference against hosted APIs.
Who’s affected
- Developers running local LLM inference on older Nvidia GPUs
- Teams considering private hosting of DeepSeek models
- DeepSeek V4.1 Flash
- NVFP4
- #local-llm
- #gpu
- #deepseek
- #quantization
- #self-hosting
A lawyer has upped the ante by upgrading his private AI server from eight to 12 used Nvidia V100s, which can run every DeepSeek V4.1 Flash expert at 4-bit NVFP4. Performance can reach up to 184 tokens per second for code, and long documents start answering in about half the time…