> **The near-lossless Apple-Silicon build** of `AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16`. Every language-decoder linear is `mxfp8` (8-bit, group 32); the tied soft-capped head and the vision/audio projectors are kept at **bf16**. Built and validated on a **MacBook Pro M4 Pro (48 GB)**. > > **Target hardware: Apple Silicon (M-series), best on 24 GB+ unified memory.** Full multimodal (text + image + audio) via `mlx-vlm`. > > Want the smallest build for a 16 GB Mac? See the compact FP4 sibling: `…-MLXFP4`. This is the **maximum-fidelity** member of the MLX quant grid: measured **top-1 token agreement 0.924 and median KL ≈ 0.002 nats** against the BF16 source on the model's own greedy output — below the perceptual/sampling-noise floor on the typical token. **Near-lossless**, and the recommended build when quality is paramount. **0 → running on a fresh Mac** (no Python, no tools needed) — `uv` installs a correct Python + the deps for you:
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
> **The compact, fast, 16 GB-friendly Apple-Silicon build** of `AEON-7/Gemma-4-12B-it-AEON-Abliterated-K4-BF16`. True 4-bit `mxfp4` on the bulk of the decoder, **8-bit `mxfp8` on the two residual-writers that carry the abliteration edit**, and **bf16** on the soft-capped head and the vision/audio projectors. Built and validated on a **MacBook Pro M4 Pro (48 GB)**. > > **Target hardware: Apple Silicon (M-series), runs on 16 GB+ unified memory** (peaks ~10.1 GB). Full multimodal (text + image + audio) via `mlx-vlm`. > > Want maximum fidelity? See the near-lossless 8-bit sibling: `…-MLX-8bit`. This is the **smallest** member of the MLX quant grid (9.3 GB) and the **fastest single-stream** — a high-quality compact build for Apple Silicon on a tight unified-memory budget. It stays fully coherent, keeps the abliteration intact (**0/8 refusals on the harmful probe**), and preserves the full vision/audio path. For the tightest possible match to BF16, the near-lossless **MLX-8bit** sibling is one click away. **0 → running on a fresh Mac** (no Python, no tools needed) — `uv` installs a correct Python + the deps for you:
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
Complete copy-paste recipe — pull the container, pull the model (fresh), pull the DFlash drafter (fresh), then serve with the vetted DGX Spark recipe (DFlash n=10, drafter `flash_attn`, body `triton_attn`). The recipe notes below cover tag-specific drafter backends and the production GPU-util / context-tier profile; see also Container Image Details. **Recipe notes — the drafter backend is tied to the image tag:** - **On `:latest` / `:2026-07-01-v0.24.0` (current; same behavior as the `:2026-06-18-v0.23.0-dflashfix` fix lineage):** the drafter **must** use `attention_backend: flash_attn` (enabled for multimodal Gemma targets by PR #41703's `use_mm_prefix=False`). `flex_attention` **crashes at the first request** on this image (non-contiguous KV view in the new KV-sharing path). `--enable-prefix-caching` is safe and recommended — soak-validated with DFlash (see the fixes section below). - **On the rollback tag `:2026-06-04-pr44389` (pre-fix):** the reverse — only `flex_attention` loads (`flash_attn` fails with `partial multimodal token full attention not supported`), the drafter silently runs without sliding-window support (~0% acceptance beyond 2k-token contexts), and `--enable-prefix-caching` triggers a slow acceptance collapse (see below). Not recommended. - Either way the body stays on `triton_attn` (Gemma's heterogeneous head dims), and do **not** set `--kv-cache-dtype fp8` — DFlash's non-causal drafter requires BF16 KV. `--reasoning-parser gemma4` cleanly splits `enable_thinking` output into `reasoning_content`. `num_speculative_tokens=10` is the value our production fleet runs; the drafter's trained block size is 16 and z-lab ships 15 — an n=10-vs-15 re-bench on the fixed drafter is pending (the earlier "10 beats 15" result was measured on the pre-fix drafter and shouldn't be treated as settled). The model's `config.json` already ships the vision ignore-list fix (`re:.*embed_vision.*`, `re:.*vision_tower.*`) so vLLM keeps the vision tower BF16. This default profile suits agentic gateways — one large full-context chat plus many short-lived subagents under the `--max-num-seqs 64` cap. For short-context throughput benchmarking, use `--max-model-len 32768 --max
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). > **Deployment, operations & benchmarks → github.com/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash** > > The GitHub repo is the source of truth for the production deployment guide, hardware-tuned docker-compose configs (DGX Spark NVFP4, A100/H100 BF16), full configuration reference, measured throughput benchmarks, and `AGENTS.md` — an operator's manual that pre-empts common stale-documentation traps for AI coding agents working on this stack. > **🆕 2026-05-01 — MTP head grafted in.** This repo now ships with the original `mtp.*` head (15 tensors, ~0.85 GB) restored from the `Qwen/Qwen3.6-27B` base. vLLM's `--speculative-config '{"method":"qwen3_5_mtp","num_speculative_tokens":3}'` works directly on the BF16 checkpoint with no extra steps. Measured on DGX Spark: **mean accepted length 3.3/3, P0 ≈ 90% acceptance, avg draft acceptance 78%** — comparable to the base model, confirming abliteration doesn't damage the model's top-K distribution that MTP relies on. Credit to `@tcclaviger` for the empirical finding (discussion #6) that MTP doesn't need retraining post-abliteration. **No retraining was performed.** The MTP head is the unmodified base — only the residual-stream-writing weights of the LM were orthogonalized in the original v8 abliteration pass; the MTP head is independent of that pathway.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
분류 준비 중
입력 미확인→ 출력 미확인
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). > **The flagship, max-fidelity Apple-Silicon build** of `AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16`. Affine **8-bit** (group-64) on the compressible bulk of the hybrid decoder, and **BF16** on the parts that don't tolerate it: the Gated-DeltaNet / Mamba state dynamics, the full vision tower, and the in-weights MTP head. Built and validated on a **MacBook Pro M4 Pro (48 GB)**. > > **Target hardware: Apple Silicon (M-series), 36 GB+ unified memory** (peaks 29.85 GB). Full multimodal (text + image) via `mlx-vlm`. > > Want a smaller, faster build? See the compact sibling: `…-Multimodal-MLX-FP4` (16 GB, 1.85× faster decode). This is the **fidelity** member of the MLX quant grid (29.5 GB on disk, 8.634 bpw) — the tightest match to the BF16 source you can run natively on Apple Silicon. Affine 8-bit is near-lossless on the residual-writers, and the recipe keeps the numerically fragile **Gated-DeltaNet** recurrence, the entire **vision tower**, and the **MTP head** in BF16 so the hybrid SSM, multimodal path, and self-speculation all stay intact. For a tight unified-memory budget (24 GB+ Macs) at ~3.2× the single-stream throughput, the compact **MLX-FP4** sibling is one click away.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). > **The compact, fast, MTP-accelerated Apple-Silicon build** of `AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-BF16`. True 4-bit `mxfp4` on the bulk of the hybrid decoder, **8-bit affine islands on the sensitive GQA k/v + embedding + head**, and **bf16** on the Gated-DeltaNet state dynamics, the vision tower, and the MTP head. Built and validated on a **MacBook Pro M4 Pro (48 GB)**. > > **Target hardware: Apple Silicon (M1+), runs on 24 GB unified memory** — 16 GB on disk, ~17 GB peak. **15.2 tok/s single-stream, 26.5 tok/s with MTP self-speculation (1.78× lossless).** Full multimodal (text + image) via `mlx-vlm`. > > Want maximum fidelity? See the near-lossless 8-bit sibling: `…-MLX-8bit`. This is the **smallest** member of the MLX quant grid (16 GB) and the **fastest single-stream** — a high-quality compact build for Apple Silicon that fits a **24 GB** Mac and, with its native MTP drafter, decodes at **26.5 tok/s** — roughly **3.2× the 8-bit's 8.2 tok/s**. It stays fully coherent, keeps the abliteration intact, and preserves the full vision path. For the tightest possible match to BF16, the near-lossless **MLX-8bit** sibling is one click away.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). > **Deployment, operations & benchmarks → github.com/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash** > > The GitHub repo is the source of truth for the production deployment guide, hardware-tuned docker-compose configs, full configuration reference, measured benchmarks, and `AGENTS.md` — an operator's manual that pre-empts common stale-documentation traps. > **🙏 Reference recipe credit:** The modelopt + MTP graft pipeline used to build this variant is based on **sakamakismile**'s validated Qwen3.6-27B-NVFP4-MTP series (22K+ downloads). They worked out the modelopt config, the per-projection quantization choices, and the MTP-head graft technique on the un-abliterated base; we adapted the same recipe to AEON-Ultimate's abliterated weights. The reference benchmark numbers cited below are theirs. Full credit for the recipe → sakamakismile.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). > 💎 **Want higher quality?** This is the **XS (smallest-footprint)** body. If you have the VRAM — it's only *slightly* larger — the full **Multimodal-NVFP4-MTP** body runs at **parity speed** with materially **higher quality-eval scores**, and is the recommended DGX Spark daily-driver. Stay on **XS** when footprint / VRAM is tight. > **Deployment, operations & benchmarks → github.com/AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored-DFlash** > > The GitHub repo is the source of truth for the production deployment guide, hardware-tuned docker-compose configs, full configuration reference, measured benchmarks, and `AGENTS.md` — an operator's manual that pre-empts common stale-documentation traps.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
> [!IMPORTANT] > **Superseded by Qwen3.8 — use this instead:** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-NVFP4-MIXED` > > **Operator recipes (Spark Dynamic DFlash lattice, TP=2, RTX):** `AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED` > > This Qwen3.6-27B card stays online for history and existing downloads. For new work, move to the Qwen3.8 Ultimate Uncensored NVFP4-MIXED release. That line is far superior in capability and reaches substantially more of what the architecture can do, with a more refined mixed quantization methodology (NVFP4 + FP8 lattice) than these older Qwen3.6 exports. > > Serve with the unified container: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (Spark / GB10) or `ghcr.io/aeon-7/aeon-vllm-ultimate-rtx:latest` (RTX discrete). NVFP4 hardware-quantized release of `AEON-7/Qwen3.6-27B-AEON-Ultimate-Uncensored`. Same model, same 0/50 refusal rate, same preserved capabilities — now compressed from **51 GB BF16** to **26 GB NVFP4** for native FP4 hardware throughput on DGX Spark / GB10 / Blackwell sm_121a. This release is **multimodal-preserved** (vision tower stays BF16 — text + image inference fully functional) and **hybrid-attention-preserved** (the 48 linear-attention / GatedDeltaNet layers stay BF16; FP4 only applies to the 16 full-attention layers' output projections + all MLPs).
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
> ✅ **Validated on the unified AEON vLLM Ultimate image `ghcr.io/aeon-7/aeon-vllm-ultimate:latest`** (= `:2026-06-18-v0.23.0-dflashfix`; rollback `:2026-06-11-pr41703`) — vLLM 0.23.0 built from source for GB10 / sm_121a + the AEON DFlash stack. Loads + serves cleanly (compressed-tensors MoE) — **including working multimodal image understanding** (a vision tensor-prefix bug was fixed 2026-06-18 — see the **Vision fix** section below) — and with DFlash `num_speculative_tokens: 11` now scales to **c=64** with **up to ~740 tok/s aggregate** and **no concurrency crash**. This single image supersedes the older `vllm-spark-omni-q36:v1.2` container. **Recommended container base.** > **🚀 PRODUCTION DEPLOYMENT GUIDE: github.com/AEON-7/Qwen3.6-35B-A3B-heretic-NVFP4-DFlash** > > The GitHub repo is the **definitive turn-key setup** for DGX Spark — end-to-end > deployment guide, validated OpenClaw config, and a concurrency-sweep benchmark harness. > > - Image: `ghcr.io/aeon-7/aeon-vllm-ultimate:latest` (vLLM 0.23.0 source-built for sm_121a + the AEON speculative-decoding stack — see What we fixed for the DGX Spark) > - Pairs with `z-lab/Qwen3.6-35B-A3B-DFlash` drafter (must be **post 2026-04-19** revision) > - **Production-stable** under sustained chat load and concurrent agent fleets up to c=64 on DGX Spark v1 of this checkpoint had `model.language_model.layers.X.*` keys remapped to `model.layers.X.*` so vLLM's text-only `Qwen3_5MoeForCausalLM` loader would pick them up. That layout was unstable in production — intermittent NaN/crash in the prefix-strip codepath during real chat sessions.
번역 대기 중 · 원문 표시
입력 텍스트 · 이미지→ 출력 텍스트
**Early Access Draft.** The uncensored BF16 of Qwen/Qwen3.8-27B. Abliterated for coherence and better answers, not for a vanity KL of zero. Vision tower and native MTP head are the unmodified base. This is still the full-precision master a later NVFP4 sibling will bake from. It is not the finished cut. Do not dequant-edit-requant an NVFP4 lattice. > **New: AEON DFlash2 drafter for AEON ULTIMATE Qwen3.8-27B.** A speculative-decoding drafter custom-trained for this model family (it pairs with the NVFP4-MIXED build). Measured on one DGX Spark with the previous training stage (lk3): +21% faster than stock DFlash2 at T=0.6, +35% at T=1.0, and 3.2× faster than no drafter. Lossless: the 27B still verifies every token. **Initial gated access for Patreon supporters:** early-access post · membership.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
**Announcement.** This Hub card is the release notice for the MIXED deploy cut. Deeper release notes and expanded serve recipes land on Patreon as more battle testing and raw data comes in - not all at once on day one. I've spent a lot of months learning what abliteration actually costs. Turns out the trap most people fall into is chasing a vanity KL of zero and calling every judge-R a leftover refusal. That is how you lobotomize a 27B. I wanted the opposite: **willfully compliant**, coherent, useful - and still a real model when the answer gets long. So this is the ~**23G** ModelOpt **mixed** deploy cut of AEON-7/Qwen3.8-27B-AEON-ULTIMATE-UNCENSORED-BF16. Not a blunt NVFP4 dump. Not a dequant-edit-requant circus. A lattice designed so the last layers can still *write code that passes unit tests*, while the bulk of the MLP rides NVFP4 where Blackwell is actually hungry for it.
번역 대기 중 · 원문 표시
입력 텍스트→ 출력 텍스트
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
입력 텍스트→ 출력 텍스트
This model was converted to GGUF format from `PartAI/Dorna2-Llama3.1-8B-Instruct` using llama.cpp via the ggml.ai's GGUF-my-repo space. Refer to the original model card for more details on the model. Note: You can also use this checkpoint directly through the usage steps listed in the Llama.cpp repo as well. Step 2: Move into the llama.cpp folder and build it with `LLAMA_CURL=1` flag along with other hardware-specific flags (for ex: LLAMA_CUDA=1 for Nvidia GPUs on Linux).
번역 대기 중 · 원문 표시
분류 준비 중
입력 미확인→ 출력 미확인
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
분류 준비 중
입력 미확인→ 출력 미확인
