We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B | 3 Trillion |16 | 2048 | 16 | 2048 | | OLMo 7B | 2.5 Trillion | 32 | 4096 | 32 | 2048 | | OLMo 7B Twin 2T | 2 Trillion | 32 | 4096 | 32 | 2048 | We are releasing many checkpoints for these models, for every 1000 traing steps. The naming convention is `step1000-tokens4B`. In particular, we focus on four revisions of the 7B models:
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs, and details involved in training these models. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B July 2024 | 3.05 Trillion | 16 | 2048 | 16 | 4096 | | OLMo 7B July 2024 | 2.75 Trillion | 32 | 4096 | 32 | 4096 | All revisions/branches are listed in the file `revisions.txt`. Or, you can access all the revisions for the models via the following code snippet:
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. This model has been converted from allenai/OLMo-7B for the Hugging Face Transformers format. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B | 3 Trillion |16 | 2048 | 16 | 2048 | | OLMo 7B | 2.5 Trillion | 32 | 4096 | 32 | 2048 | | OLMo 7B Twin 2T | 2 Trillion | 32 | 4096 | 32 | 2048 | We are releasing many checkpoints for these models, for every 1000 training steps. These have not yet been converted into Hugging Face Transformers format, but are available in allenai/OLMo-7B.
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
We expand on our Olmo model series by introducing Olmo Hybrid, a new 7B hybrid RNN model in the Olmo family. Olmo Hybrid dramatically outperforms Olmo 3 in final performance, consistently showing roughly 2x data efficiency on core evals over the course of our pretraining run. We also show gains in performance on long-context benchmarks, as well as improved inference efficiency (throughput and memory) on long-context lengths by a factor of 75%. The training of our hybrid model makes use of Olmo 3 7B, except that we change the learning rate schedule to be a standard cosine schedule rather than the piecewise schedule used by Olmo 3. Additionally, we use the improved data mix of Olmo 3 32B instead of the Olmo 3 7B mix. The table below highlights the architecture differences in our hybrid model. Our overall layer matches the transformer architecture of Olmo 3 7B, except that 75% of layers use gated DeltaNet heads instead of attention heads. The layers alternate so that 3 contain DeltaNet sublayers followed by 1 with a multihead attention sublayer. In particular, each head uses gated DeltaNet heads, extended with negative eigenvaluesWe reduced the number of heads from 32 to 30 while keeping the head dimension fixed at 128 (effectively reducing dmodel from 4096 to 3840). Lastly, head dimension is doubled, which is the default behavior for DeltaNet.
翻訳待ち・原文を表示
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
入力 テキスト→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト→ 出力 テキスト
Full BF16 version of olmOCR-2-7B-1025-FP8. We recommend using the FP8 version for all practical purposes except further fine tuning. This is a release of the olmOCR model that's fine tuned from Qwen2.5-VL-7B-Instruct using the olmOCR-mix-1025 dataset. It has been additionally fine tuned using GRPO RL training to boost its performance at math equations, tables, and other tricky OCR cases. Quick links: - 📃 Paper - 🤗 SFT Dataset - 🤗 RL Dataset - 🛠️ Code - 🎮 Demo
翻訳待ち・原文を表示
入力 テキスト · 画像→ 出力 テキスト
Quantized to FP8 Version of olmOCR-2-7B-1025, using llmcompressor. This is a release of the olmOCR model that's fine tuned from Qwen2.5-VL-7B-Instruct using the olmOCR-mix-1025 dataset. It has been additionally fine tuned using GRPO RL training to boost its performance at math equations, tables, and other tricky OCR cases. Quick links: - 📃 Paper - 🤗 SFT Dataset - 🤗 RL Dataset - 🛠️ Code - 🎮 Demo
翻訳待ち・原文を表示
入力 テキスト · 画像→ 出力 テキスト
This is a preview release of the olmOCR model that's fine tuned from Qwen2-VL-7B-Instruct using the olmOCR-mix-0225 dataset. Quick links: - 📃 Paper - 🤗 Dataset - 🛠️ Code - 🎮 Demo The best way to use this model is via the olmOCR toolkit. The toolkit comes with an efficient inference setup via sglang that can handle millions of documents at scale.
翻訳待ち・原文を表示
入力 テキスト · 画像→ 出力 テキスト
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基本情報を収録・詳細は順次追加
入力 テキスト · 画像→ 出力 テキスト
