This is a Vicuna-like model with only 68M parameters, which is fine-tuned from LLaMA-68m on ShareGPT data. The model is mainly developed as a base Small Speculative Model in the MCSD paper. As a comparison, it can be better aligned to the Vicuna models than LLaMA-68m with little loss of alignment to the LLaMA models.
GGUF quantizations of dphn/Dolphin-Mistral-24B-Venice-Edition, **built from version 1.2**, with vision adapters and an importance matrix. All the Dolphin-Mistral-24B-Venice-Edition GGUFs I could find on Hugging Face were converted from an earlier release, so I built these from version 1.2, introduced in `337ce042026e`: *"Updated to version 1.2 - vision + 131k context + improved tool calling"*. Every quant was measured by KL-divergence against the BF16 weights on held-out wikitext-2 (128 chunks). `BF16` (43.92 GB) is the reference and is included for anyone wanting to re-quantize without re-downloading the safetensors.
GGUF quantizations of Gemma-4-31B-JANG_4M-CRACK for use with llama.cpp, LM Studio, Ollama, and other GGUF-compatible inference engines. The original model uses **JANG v2 mixed-precision MLX quantization** (attention 8-bit + MLP 4-bit), which is only compatible with vMLX. Standard tools (llama.cpp, LM Studio, oMLX, mlx-lm) cannot load this format due to mixed per-layer bit widths. **Note:** Since the original was already quantized (avg 5.1 bits), the dequantized f16 intermediate is an approximation. Re-quantizing to GGUF introduces minimal additional quality loss since the attention layers were preserved at 8-bit in the original.
This Dolphin is *really good* at coding, I trained with a lot of coding data. It is *very* obedient but it is not DPO tuned - so you still might need to encourage it in the system prompt as I show in the below examples. New in 2.6 - Fixed a training configuration issue that improved the quality a lot - Due to popular demand, added back samantha-based empathy data - Replaced synthia and pure-dove with Capybara This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
My appreciation for the sponsors of Dolphin 2.8: - Crusoe Cloud - provided excellent on-demand 10xL40S node - Winston Sou - Along with a generous anonymous sponsor, donated a massive personally owned compute resource! - Abacus AI - my employer and partner in many things. This model is based on Mistral-7b-v0.2 a new base model released by MistralAI on March 23, 2024 but they have not yet published on HuggingFace. Thanks to @alpindale for converting / publishing. Dolphin is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
Adjust provider, category or input/output filters. New models are added continuously.
COST EXPLORER
Great models. Clear costs.
Cost analysis
Adjust monthly token usage to compare estimated API costs.
1 M = 1 million tokens · USD Compares base text-token rates within the same channel
Uses collected, comparable USD-per-million-token rates. Snapshots exclude cache discounts, tiers, extra reasoning output and taxes. Check current source prices.
MODEL HANDBOOK
Understand before you choose
Selection guide & FAQ
How should API prices be compared?+
Separate input, output and cache rates, then estimate the same workload. Image, audio and video rates may use items, duration or resolution rather than tokens.
Do open weights mean free commercial use?+
Not necessarily. Check each version’s license, commercial restrictions and policies. Self-hosting also incurs compute and maintenance costs.
Is the top-ranked model the best for me?+
Rankings depend on tasks, versions and methods. Compare accuracy, latency, reliability and costs on your own workload.