This model was obtained by quantizing the weights of Qwen3-4B to INT4 data type. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%. Only the weights of the linear operators within transformers blocks are quantized. Weights are quantized using a asymmetric per-group scheme, with group size 64. The GPTQ algorithm is applied for quantization, as implemented in the llm-compressor library. The model was evaluated on the OpenLLM leaderboard tasks (versions 1 and 2), using lm-evaluation-harness, and on reasoning tasks using lighteval. vLLM was used for all evaluations.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
- **Model Architecture:** Qwen3NextForCausalLM - **Input:** Text - **Output:** Text - **Model Optimizations:** - **Weight quantization:** FP4 - **Activation quantization:** FP4 - **Release Date:** - **Version:** 1.0 - **Model Developers:**: Red Hat Quantized version of Qwen/Qwen3.5-122B-A10B. This model was obtained by quantizing the weights and activations of Qwen/Qwen3.5-122B-A10B to FP4 data type. This optimization reduces the number of bits per parameter from 16 to 4, reducing the disk size and GPU memory requirements by approximately 75%. Only the weights and activations of the linear operators within transformers blocks of the language model are quantized.
译文排队中 · 当前显示来源原文
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
This model is a quantized version of Qwen/Qwen3.5-4B. Evaluation results and reproduction steps are provided below. This model was obtained by quantizing the weights and activations of Qwen/Qwen3.5-4B to FP8 data type, ready for inference with vLLM. This optimization reduces the model weights from 9.3 GB to 8.0 GB on disk (~14% reduction). Activations are quantized dynamically at inference time using per-tensor scaling, requiring no calibration data.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
This model is a quantized version of Qwen/Qwen3.5-4B. Evaluation results and reproduction steps are provided below. This model was obtained by quantizing the weights of Qwen/Qwen3.5-4B to INT4 data type while keeping activations in original precision, ready for inference with vLLM. This optimization reduces the model weights from 8.8 GB to 5.2 GB on disk (~41% reduction). The reduction is less than the theoretical 75% because the vision encoder, token embeddings, and linear attention layers remain in BF16.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本 · 图像→ 输出 文本
- **Model Architecture:** Qwen/Qwen3.5-9B - **Input:** Text / Image - **Output:** Text - **Model Optimizations:** - **Weight quantization:** FP8 - **Activation quantization:** FP8 - **Model size:** 14.0 GB (reduced from 19.3 GB in BF16) - **Release Date:** 2026-05-11 - **Version:** 1.0 - **Model Developers:** RedHatAI This model is a quantized version of Qwen/Qwen3.5-9B. Evaluation results and reproduction steps are provided below. This model was obtained by quantizing the weights and activations of Qwen/Qwen3.5-9B to FP8 data type, ready for inference with vLLM.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
- **Model Architecture:** Qwen/Qwen3.5-9B - **Input:** Text / Image - **Output:** Text - **Model Optimizations:** - **Weight quantization:** INT4 - **Activation quantization:** None - **Model size:** 11 GB (reduced from 19.3 GB in BF16) - **Release Date:** 2026-04-24 - **Version:** 1.0 - **Model Developers:** RedHatAI This model is a quantized version of Qwen/Qwen3.5-9B. Evaluation results and reproduction steps are provided below. This model was obtained by quantizing the weights of Qwen/Qwen3.5-9B to INT4 data type while keeping activations in original precision, ready for inference with vLLM.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本 · 图像→ 输出 文本
> [!Note] > This repository contains FP8-quantized model weights and configuration files for the post-trained model in the Hugging Face Transformers format. > > These artifacts are compatible with Hugging Face Transformers, vLLM, SGLang, KTransformers, etc. > > The quantization method is fine-grained fp8 quantization with block size of 128, and its performance metrics are nearly identical to those of the original model. Following the February release of the Qwen3.5 series, we're pleased to share the first open-weight variant of Qwen3.6. Built on direct feedback from the community, Qwen3.6 prioritizes stability and real-world utility, offering developers a more intuitive, responsive, and genuinely productive coding experience. If you have hardware with more compute than memory bandwidth, you may prefer this MoE variant for performance reasons.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本 · 图像→ 输出 文本
This is a preliminary version (and subject to change) of NVFP4 quantized Qwen/Qwen3.6-35B-A3B model. The model has both weights and activations quantized to NVFP4 format with vllm-project/llm-compressor. It is compatible and tested against vllm main. Deploy it with: `vllm serve RedHatAI/Qwen3.6-35B-A3B-NVFP4 --reasoning-parser qwen3 --moe_backend flashinfer_cutlass`. If you have hardware with more compute than memory bandwidth, you may prefer this MoE variant for performance reasons.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
分类待补充
输入 待核验→ 输出 待核验
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本 · 图像→ 输出 文本
- **Model Architecture:** Qwen3_5ForConditionalGeneration - **Input:** Text / Image - **Output:** Text - **Model Optimizations:** - **Weight quantization:** INT4 - **Activation quantization:** None - **Release Date:** 2026-08-17 - **Version:** 1.0 - **Model Developers:** RedHatAI This model is a quantized version of Qwen/Qwen3.8-27B. It was evaluated on several tasks to assess its quality in comparison to the unquantized model. This model was obtained by quantizing the weights of Qwen/Qwen3.8-27B to INT4 data type while keeping activations in original precision, with FP8 KV cache, ready for inference with vLLM.
译文排队中 · 当前显示来源原文
输入 文本 · 图像→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本 · 图像→ 输出 文本
