e0qdk@reddthat.com to LocalLLaMA@sh.itjust.worksEnglish · 1 month agoQwen3.8-Flash-Next Weights Released (125B-A6B)huggingface.coexternal-linkmessage-square12linkfedilinkarrow-up10arrow-down10
arrow-up10arrow-down1external-linkQwen3.8-Flash-Next Weights Released (125B-A6B)huggingface.coe0qdk@reddthat.com to LocalLLaMA@sh.itjust.worksEnglish · 1 month agomessage-square12linkfedilink
minus-squareMultiplexer@discuss.tchncs.delinkfedilinkEnglisharrow-up0·1 month agoWouldn’t the whole 125B non-n-gram parameters still have to fit into VRAM, though?
minus-squareBeefAndPoultry@lemmus.orglinkfedilinkEnglisharrow-up0·1 month agoIt’s MoE so you can use --n-cpu-moe https://lemmus.org/post/24235317 Low number of active parameters (6B) means you don’t need much VRAM to get decent speeds
Wouldn’t the whole 125B non-n-gram parameters still have to fit into VRAM, though?
It’s MoE so you can use --n-cpu-moe
https://lemmus.org/post/24235317
Low number of active parameters (6B) means you don’t need much VRAM to get decent speeds