• troed@fedia.io
    link
    fedilink
    arrow-up
    0
    ·
    1 month ago

    Having issues with it (Unsloth’s gguf).

    0.27.136.404 I srv  proxy_reques: proxying request to model unsloth/Qwen3.8-27B-GGUF:Q4_K_M on port 55741
    [55741] 0.14.424.772 W srv    operator(): got exception: {"error":{"code":500,"message":"\n------------\nWhile executing CallExpression at line 106, column 32 in source:\n...first %}↵            {{- raise_exception('System message must be at the beginnin...\n                                           ^\nError: Jinja Exception: System message must be at the beginning.","type":"server_error"}}
    
    
    • troed@fedia.io
      link
      fedilink
      arrow-up
      0
      ·
      1 month ago

      Got an answer - there were indeed issue with some quants

      Actually edit - UD-Q4_K_XL and all UD-* has our corrected chat teaplate

      We need to update the non UD-* ones with our chat template - stay tuned