If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have. If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗All quants made using imatrix option with dataset from here If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: You can either specify a new local-dir (L3-8B-Stheno-v3.2-Q8_0) or download them all in place (./)
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗Some of these quants (Q3_K_XL, Q4_K_L etc) are the standard quantization method with the embeddings and output weights quantized to Q8_0 instead of what they would normally default to. If the model is bigger than 50GB, it will have been split into multiple files. In order to download them all to a local folder, run: Previously, you would download Q4_0_4_4/4_8/8_8, and these would have their weights interleaved in memory in order to improve performance on ARM and AVX machines by loading up more data in one pass.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text · Image→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Category pending
Input Not verified→ Output Not verified
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text · Image→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text · Image→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗**Model details:** - Parameter count: 9B - Input support: text, image (with mmproj file) - details - Speculative decoding: no - imatrix: yes - details **Don't know which to choose?** Grab Q4_K_M (5.84GB) - usually a good mix of size and performance. Download instructions available here These quants were made with llama.cpp release b11081 - if this model's architecture is newly supported, you'll need that release or newer to run them.
Input Text · Image→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗The first thing to figure out is how big a model you can run. To do this, you'll need to figure out how much RAM and/or VRAM you have. If you want your model running as FAST as possible, you'll want to fit the whole thing on your GPU's VRAM. Aim for a quant with a file size 1-2GB smaller than your GPU's total VRAM. If you want the absolute maximum quality, add both your system RAM and your GPU's VRAM together, then similarly grab a quant with a file size 1-2GB Smaller than that total.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗**Model details:** - Parameter count: 24B (source checkpoint) - Input support: text - Speculative decoding: no - imatrix: yes - details **Don't know which to choose?** Grab Q4_K_M (15.35GB) - usually a good mix of size and performance. Download instructions available here These quants were made with llama.cpp release b11606 - if this model's architecture is newly supported, you'll need that release or newer to run them.
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
Basic profile indexed · More details to follow
Input Text→ Output Text
Price not verifiedCheck multimodal and channel pricing separately
View details ↗