- 125 tokens per second on an M4 Pro MacBook Pro
- 512k tokens context window
- 40B total, 2.5B active parameters mixture-of-experts
Pretty cool release!
nvtop 3 days ago
- 125 tokens per second on an M4 Pro MacBook Pro
- 512k tokens context window
- 40B total, 2.5B active parameters mixture-of-experts
Pretty cool release!
dofm 3 days ago
I really like Muse Glimmer, and on low reasoning level it's both helpful and quite quick.
But I won't turn down a model that is faster.
Nemotron Bolt appears to be really quite good but unfortunately it is quite slow on macOS/Metal in llama.cpp at the moment because of lack of support for a tensor type (or something… not sure I understand).
kamranjon 4 days ago
aistripes 3 days ago
trehcrob 4 days ago
nittanymount 2 days ago