• 0 Posts
  • 4 Comments
Joined vor 3 Jahren
cake
Cake day: 21. Juni 2023

help-circle


  • This is exactly why I am so sad that I didn’t buy more DDR4 back when it was reasonable. I run Unsloth’s Qwen 3.5 122B A10B UD_Q4_k_XL and while it works great, I really wish I had enough ram for Q6 or even Q8. The speed difference won’t be wildly worse, but the quality of output is noticeable. I’m just glad that it works as well as it does in Q4. It’s mostly limited by my main ram bandwidth, the GPU helps but I only barely hit 15t/s decode with MTP hitting >80%.