mlx-serve pack of Qwen/Qwen3.8-Flash-Next, the Qwen4 preview architecture (`model_type: qwen4_exp`), quantized to 3.3 bits per weight with imatrix calibration so it fits a 64 GB Mac with room for 64k of context. 52 GB resident. Includes the MTP head and the vision tower (image and video input). Measured against the bf16 checkpoint on 1200 held-out next-token positions (agent, code, prose, math; documents the calibration never saw). Top-1 is how often the pack's greedy token equals bf16's; mass is the bf16 probability of the pack's pick, with bf16's own pick as the ceiling. Same positions, same server, cold prefills. The 3.3 bpw pack keeps 99% of the 4-8 pack's probability mass at 21 GB less.

译文排队中 · 当前显示来源原文

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw

ddalcu

mlx-serve pack of Qwen/Qwen3.8-Flash-Next, the Qwen4 preview architecture (`model_type: qwen4_exp`). Runs on a 128 GB Mac with about 75 GB resident. Includes the MTP head and the vision tower (image and video input). This is the calibrated twin of Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit: the same layout and the same size, byte for byte (every expert 4-bit group 64, the rest 8-bit), but every expert is quantized with an importance matrix collected on the bf16 model, so the scales follow the activations the weights actually see. Same speed, same memory, closer to bf16. The importance matrix was collected on the bf16 checkpoint over 1.35 M tokens of agent transcripts, code (including SWE-bench Lite issues and patches), prose and math, every expert of every layer hit. Quality is scored against the bf16 model's own logits on 290 held-out positions (top-1: the pack's greedy token equals bf16's; KLD: KL(bf16 || pack) over bf16's top-1024 tokens; the KLD mean is dominated by a few flat prose positions, so the median is the steadier number):

译文排队中 · 当前显示来源原文

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗

Qwen3.8-Flash-Next-MLX-Serve-mixed-4-8bit

ddalcu

> **Use Qwen3.8-Flash-Next-MLX-Serve-iQ-MLX-4.7bpw instead.** > It is this pack re-quantized with an importance matrix collected on the bf16 > model: same layout, same 75 GB, same bytes read per token, closer to bf16 > (KLD median 0.0052 vs 0.0066, top-1 88.6% vs 87.9% against bf16's own > logits). This pack stays up for existing installs but gets no more updates. > > Switching from this pack? `ngram_table.bin` (32 GB) is identical in both. > Move it into the new pack's folder before downloading and the download > skips it: > > mlx-serve pack of Qwen/Qwen3.8-Flash-Next, the Qwen4 preview architecture (`model_type: qwen4_exp`). Runs on a 128 GB Mac with about 75 GB resident. Includes the MTP head and the vision tower (image and video input). Measured on an M4 Max 128 GB with mlx-serve 26.8.11: ~75 GB resident, decode ~60 tok/s serial and 78 tok/s with MTP (`--mtp`; +41% on code, a few percent slower on prose), prefill ~730 tok/s, a needle at 24.8k tokens recovered with sparse attention engaged. Prefix cache on, images included.

译文排队中 · 当前显示来源原文

文本生成

输入文本→ 输出文本

价格待核验多模态与渠道费用须分别核对

查看详情 ↗

加入对比来源 / 文档 ↗