Release2 sources
Inkling-2-Mini in 5.4 GB Memory
Hacker News: Front Page ·
The Brief · written from 2 sources
- LookAtMyQuant compresses the text weights of Inkling-2-Mini, a Thinking Machines model, from 18.89 GB.
- Two compressed versions are offered, at about 5.38 GB and 8.87 GB.
- The model has 26.4 billion physical parameters, of which 5.4 billion are active.
Why it matters
A model of this size shrinking to roughly 5.4 GB makes it more practical to run locally on machines with limited memory.
Who’s affected
- Developers running large language models locally
- Teams interested in quantized Inkling-2-Mini deployments
- #quantization
- #local-llm
- #model-compression
- #inkling-2-mini
- #memory-optimization
Timeline
Full coverage · 2 sources
- 2h ago ↗
DEV Community: python
LookAtMyQuant comprime Inkling-2-Mini para 5,38 GB - 4h ago ↗
Hacker News: Front Page
Inkling-2-Mini in 5.4 GB Memory