This is a Vicuna-like model with only 68M parameters, which is fine-tuned from LLaMA-68m on ShareGPT data. The model is mainly developed as a base Small Speculative Model in the MCSD paper. As a comparison, it can be better aligned to the Vicuna models than LLaMA-68m with little loss of alignment to the LLaMA models.
GGUF quantizations of dphn/Dolphin-Mistral-24B-Venice-Edition, **built from version 1.2**, with vision adapters and an importance matrix. All the Dolphin-Mistral-24B-Venice-Edition GGUFs I could find on Hugging Face were converted from an earlier release, so I built these from version 1.2, introduced in `337ce042026e`: *"Updated to version 1.2 - vision + 131k context + improved tool calling"*. Every quant was measured by KL-divergence against the BF16 weights on held-out wikitext-2 (128 chunks). `BF16` (43.92 GB) is the reference and is included for anyone wanting to re-quantize without re-downloading the safetensors.
GGUF quantizations of Gemma-4-31B-JANG_4M-CRACK for use with llama.cpp, LM Studio, Ollama, and other GGUF-compatible inference engines. The original model uses **JANG v2 mixed-precision MLX quantization** (attention 8-bit + MLP 4-bit), which is only compatible with vMLX. Standard tools (llama.cpp, LM Studio, oMLX, mlx-lm) cannot load this format due to mixed per-layer bit widths. **Note:** Since the original was already quantized (avg 5.1 bits), the dequantized f16 intermediate is an approximation. Re-quantizing to GGUF introduces minimal additional quality loss since the attention layers were preserved at 8-bit in the original.
This Dolphin is *really good* at coding, I trained with a lot of coding data. It is *very* obedient but it is not DPO tuned - so you still might need to encourage it in the system prompt as I show in the below examples. New in 2.6 - Fixed a training configuration issue that improved the quality a lot - Due to popular demand, added back samantha-based empathy data - Replaced synthia and pure-dove with Capybara This model is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
My appreciation for the sponsors of Dolphin 2.8: - Crusoe Cloud - provided excellent on-demand 10xL40S node - Winston Sou - Along with a generous anonymous sponsor, donated a massive personally owned compute resource! - Abacus AI - my employer and partner in many things. This model is based on Mistral-7b-v0.2 a new base model released by MistralAI on March 23, 2024 but they have not yet published on HuggingFace. Thanks to @alpindale for converting / publishing. Dolphin is uncensored. I have filtered the dataset to remove alignment and bias. This makes the model more compliant. You are advised to implement your own alignment layer before exposing the model as a service. It will be highly compliant to any requests, even unethical ones. Please read my blog post about uncensored models. https://erichartford.com/uncensored-models You are responsible for any content you create using this model. Enjoy responsibly.
Curated and trained by Eric Hartford, Lucas Atkins, and Fernando Fernandes, and Cognitive Computations We have retrained our LLama-3-70b fine tune to address behavioral issues in the initial 2.9 dataset. Specifically, Systemchat was causing the model to be *too* reliant on the system prompt. Additionally, it had an occasional quirk that would cause the model to overly reference the system prompt. We also found generation length was at times not sufficient for any given task. We identified the culprit as Ultrachat. Accounting for these concerns, we removed systemchat and ultrachat from the dataset. It is otherwise identical to dolphin-2.9. Our appreciation for the sponsors of Dolphin 2.9.1: - Crusoe Cloud - provided excellent on-demand 8xL40S node - OnDemand - provided inference sponsorship
Curated and trained by Eric Hartford, Lucas Atkins, and Fernando Fernandes, and Cognitive Computations We have retrained our LLama-3-8b fine tune to address behavioral issues in the initial 2.9 dataset. Specifically, Systemchat was causing the model to be *too* reliant on the system prompt. Additionally, it had an occasional quirk that would cause the model to overly reference the system prompt. We also found generation length was at times not sufficient for any given task. We identified the culprit as Ultrachat. Accounting for these concerns, we removed systemchat and ultrachat from the dataset. It is otherwise identical to dolphin-2.9. Our appreciation for the sponsors of Dolphin 2.9.1: - Crusoe Cloud - provided excellent on-demand 8xL40S node