International
The Morning Dev

Only the news worth knowing

‹
 

Release2 sources

Inkling-2-Mini in 5.4 GB Memory

Hacker News: Front Page ·

The Brief · written from 2 sources

  • LookAtMyQuant compresses the text weights of Inkling-2-Mini, a Thinking Machines model, from 18.89 GB.
  • Two compressed versions are offered, at about 5.38 GB and 8.87 GB.
  • The model has 26.4 billion physical parameters, of which 5.4 billion are active.

Why it matters

A model of this size shrinking to roughly 5.4 GB makes it more practical to run locally on machines with limited memory.

Who’s affected

  • Developers running large language models locally
  • Teams interested in quantized Inkling-2-Mini deployments
  • #quantization
  • #local-llm
  • #model-compression
  • #inkling-2-mini
  • #memory-optimization

Timeline

  1. Hacker News: Front Page

    Inkling-2-Mini in 5.4 GB Memory

  2. DEV Community: python

    LookAtMyQuant comprime Inkling-2-Mini para 5,38 GB

Full coverage · 2 sources

How do you feel about this?

0 Comments