**Model details:** - Parameter count: 12B (source checkpoint) - Input support: text - Speculative decoding: no - imatrix: yes - details **Don't know which to choose?** Grab Q4_K_M (8.82GB) - usually a good mix of size and performance. Download instructions available here These quants were made with llama.cpp release b11490 - if this model's architecture is newly supported, you'll need that release or newer to run them.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (Mellum2-12B-A2.5B-Instruct-Q8_0) or download them all in place (./)
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗All quants made using imatrix option with dataset from here Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run:
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (Meta-Llama-3.1-8B-Instruct-abliterated-Q8_0) or download them all in place (./) The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗All quants made using imatrix option with dataset from here - 30-07-2024: Updated chat template to fix small bug with tool usage being undefined, if you don't use the built-in chat template it shouldn't change anything Thank you kalomaze and Dampf for assistance in creating the imatrix calibration dataset
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (Meta-Llama-3-70B-Instruct-Q8_0) or download them all in place (./) The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗All quants made using imatrix option with dataset provided by Kalomaze here A great write up with charts showing various performances is provided by Artefact2 here The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗Using llama.cpp release b5074 with my PR changes from here for quantization. All quants made using imatrix option with dataset from here Run them directly with llama.cpp, or any other llama.cpp based project
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗