A **W4A16 (INT4 weight, FP16 activation) quantization** of `Qwen/Qwen3.6-27B`, produced with Intel's AutoRound. - **Base**: Qwen3.6-27B (27B dense VLM, Apr 21 2026) - **Quant**: INT4 W4A16, group_size 128, symmetric - **Tool**: `auto-round` (default recipe, 200 iters, torch.compile) - **Size**: 18 GB (down from ~54 GB BF16) — **3x reduction** - **MTP preserved**: The native Multi-Token Prediction head is kept in BF16, enabling **native speculative decoding** in vLLM (≈90% draft acceptance in our tests, ~2x throughput) - **Accuracy**: Default AutoRound recipe preserves quality well; layer-norm weights, router layers, RMSNorm, `linear_attn.in_proj_a/b`, and MTP's fusion `fc` are kept unquantized (they're small and benefit from full precision) Requires vLLM that supports Qwen3_5 MTP (most recent nightlies — tested with `eugr/spark-vllm-docker` fork `0.19.1rc1.dev39+g7055d32a7`):

译文排队中 · 当前显示来源原文

视觉问答

输入文本 · 图像→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Qwen3.6-35B-A3B-abliterated-ROCmFP4

Lord-H4D3ZS

基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。

基础资料已收录 · 缺失资料后续补充

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Qwen3.8-Distill-35B-A3B-Coder-Abliterated

Lord-H4D3ZS

A 2-bit **ROCmFPX** GGUF of a distilled Qwen3.6-35B-A3B (MoE, 256 experts / ~3B active), sized to run on a **16GB consumer GPU**. Ships with the MTP (`nextn`) head and build instructions for the matching runtime. > **Honest status:** this is a proof-of-concept. On the internal 10-task smoke eval the distilled > model **tied its base** (6/10 vs 6/10) — no regression, no measurable gain yet — and it is now > quantized to 2-bit, which trades quality for fit. Publishing it as a reproducible artifact of the > pipeline (distill → graft MTP → ROCmFPX 2-bit GGUF), **not** as a benchmark-winning coder. > The quality fix is a larger, tool-calling-heavy corpus — a separate follow-up run. 16GB card: context and concurrency share one KV pool — pick single-stream long context (`-c 32768 -np 1`) **or** many short sessions (`-c 8192 -np 8`).

译文排队中 · 当前显示来源原文

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Agents-A1-GGUF

LordNeel

基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Ornith-1.0-35B-GGUF-llamacpp-tp1

LordNeel

Single-GPU llama.cpp GGUF package for `deepreinforce-ai/Ornith-1.0-35B`. The supported serving policy is `tp=1` — one model copy per GPU. Multi-GPU tensor-parallel serving is intentionally out of scope for this version. This release ships six body quants (Q3_K_M → Q8_0) plus an **integrated IQ4_XS-MTP graft** that adds a native multi-token-prediction (MTP) draft head for low-concurrency speculative decode. Quant quality is measured against the upstream BF16 GGUF with a native llama.cpp next-token top-64 KL-divergence probe over 32 coding prompts. > **MTP graft note:** active MTP is byte-for-byte identical to target-only at the > next-token (API-visible) level, and gives ~1.3× single-stream decode. It is > **not** bit-exact to target-only over long deterministic sequences — see > MTP. Use target-only for strict reproduction, > MTP for throughput on open-ended work.

译文排队中 · 当前显示来源原文

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗