Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. Some say that this improves the quality, others don't notice any difference. If you use these models PLEASE COMMENT with your findings. I would like feedback that these are actually used and useful so I don't keep uploading quants no one is using. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
**Model details:** - Parameter count: 321B (source checkpoint) - Input support: text, image (with mmproj file) - details - Speculative decoding: yes (MTP) - details - imatrix: yes - details **Don't know which to choose?** Grab Q4_K_M (200.83GB) - usually a good mix of size and performance. Download instructions available here The files marked `true` in the Split column above are stored as multiple parts in a folder. To download all the parts to a local folder, run:
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (Goekdeniz-Guelmez_Josiefied-Qwen3-14B-abliterated-v3-Q8_0) or download them all in place (./)
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (Goekdeniz-Guelmez_Josiefied-Qwen3-8B-abliterated-v1-Q8_0) or download them all in place (./)
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
All quants made using imatrix option with dataset from here Run them directly with llama.cpp, or any other llama.cpp based project This model has vision capabilities, more details here: https://github.com/ggml-org/llama.cpp/pull/12344
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트 · 이미지→ 출력 텍스트
All quants made using imatrix option with dataset from here Run them directly with llama.cpp, or any other llama.cpp based project Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
This model has vision capabilities, more details here: https://github.com/ggml-org/llama.cpp/pull/12344 Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
*ONLY* Q4_0 is expected to be better, but figured while I'm at it I might as well make others to see what happens? gemma-3-27b-it-qat-Q4_0.gguf | Q4_0 | 15.62GB | Should be improved due to QAT, offers online repacking for ARM and AVX CPU inference. Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
All quants made using imatrix option with dataset from here Run them directly with llama.cpp, or any other llama.cpp based project This model has vision capabilities, more details here: https://github.com/ggml-org/llama.cpp/pull/12344
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
*ONLY* Q4_0 is expected to be better, but figured while I'm at it I might as well make others to see what happens? gemma-3-4b-it-qat-Q4_0.gguf | Q4_0 | 2.37GB | false | Should be improved due to QAT, offers online repacking for ARM and AVX CPU inference. Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
No chat template specified so default is used. This may be incorrect, check original model card for details. Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
No chat template specified so default is used. This may be incorrect, check original model card for details. Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
All quants made using imatrix option with dataset from here - llama.cpp - ramalama - LM Studio - koboldcpp - Jan AI - Text Generation Web UI - LoLLMs Note: if it's a newly supported model, you may need to wait for an update from the developers.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
All quants made using imatrix option with dataset from here - llama.cpp - ramalama - LM Studio - koboldcpp - Jan AI - Text Generation Web UI - LoLLMs Note: if it's a newly supported model, you may need to wait for an update from the developers.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
All quants made using imatrix option with dataset from here - llama.cpp - ramalama - LM Studio - koboldcpp - Jan AI - Text Generation Web UI - LoLLMs Note: if it's a newly supported model, you may need to wait for an update from the developers.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
All quants made using imatrix option with dataset from here - llama.cpp - ramalama - LM Studio - koboldcpp - Jan AI - Text Generation Web UI - LoLLMs Note: if it's a newly supported model, you may need to wait for an update from the developers.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (google_medgemma-27b-it-Q8_0) or download them all in place (./)
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (google_medgemma-4b-it-Q8_0) or download them all in place (./)
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
입력 텍스트→ 출력 텍스트
