Olmo-3-7B-RL-Zero-General
allenai基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
언어부터 이미지, 오픈 웨이트부터 API까지. 기능과 실제 비용을 비교하세요.
다음 아이디어에 맞는 AI를 찾아보세요.
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
번역 대기 중 · 원문 표시
We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
번역 대기 중 · 원문 표시
We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding. Olmo is a series of **O**pen **l**anguage **mo**dels designed to enable the science of language models. These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolci datasets. We are releasing all code, checkpoints, logs (coming soon), and associated training details. The quantized model is more sensitive to data types and CUDA operations. To avoid potential issues, it's recommended to pass the inputs directly to CUDA using:
번역 대기 중 · 원문 표시
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B | 3 Trillion |16 | 2048 | 16 | 2048 | | OLMo 7B | 2.5 Trillion | 32 | 4096 | 32 | 2048 | | OLMo 7B Twin 2T | 2 Trillion | 32 | 4096 | 32 | 2048 | We are releasing many checkpoints for these models, for every 1000 traing steps. The naming convention is `step1000-tokens4B`. In particular, we focus on four revisions of the 7B models:
번역 대기 중 · 원문 표시
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs, and details involved in training these models. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B July 2024 | 3.05 Trillion | 16 | 2048 | 16 | 4096 | | OLMo 7B July 2024 | 2.75 Trillion | 32 | 4096 | 32 | 4096 | All revisions/branches are listed in the file `revisions.txt`. Or, you can access all the revisions for the models via the following code snippet:
번역 대기 중 · 원문 표시
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
OLMo is a series of **O**pen **L**anguage **Mo**dels designed to enable the science of language models. The OLMo models are trained on the Dolma dataset. We release all code, checkpoints, logs (coming soon), and details involved in training these models. This model has been converted from allenai/OLMo-7B for the Hugging Face Transformers format. The core models released in this batch are the following: | Size | Training Tokens | Layers | Hidden Size | Attention Heads | Context Length | |------|--------|---------|-------------|-----------------|----------------| | OLMo 1B | 3 Trillion |16 | 2048 | 16 | 2048 | | OLMo 7B | 2.5 Trillion | 32 | 4096 | 32 | 2048 | | OLMo 7B Twin 2T | 2 Trillion | 32 | 4096 | 32 | 2048 | We are releasing many checkpoints for these models, for every 1000 training steps. These have not yet been converted into Hugging Face Transformers format, but are available in allenai/OLMo-7B.
번역 대기 중 · 원문 표시
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
We expand on our Olmo model series by introducing Olmo Hybrid, a new 7B hybrid RNN model in the Olmo family. Olmo Hybrid dramatically outperforms Olmo 3 in final performance, consistently showing roughly 2x data efficiency on core evals over the course of our pretraining run. We also show gains in performance on long-context benchmarks, as well as improved inference efficiency (throughput and memory) on long-context lengths by a factor of 75%. The training of our hybrid model makes use of Olmo 3 7B, except that we change the learning rate schedule to be a standard cosine schedule rather than the piecewise schedule used by Olmo 3. Additionally, we use the improved data mix of Olmo 3 32B instead of the Olmo 3 7B mix. The table below highlights the architecture differences in our hybrid model. Our overall layer matches the transformer architecture of Olmo 3 7B, except that 75% of layers use gated DeltaNet heads instead of attention heads. The layers alternate so that 3 contain DeltaNet sublayers followed by 1 with a multihead attention sublayer. In particular, each head uses gated DeltaNet heads, extended with negative eigenvaluesWe reduced the number of heads from 32 to 30 while keeping the head dimension fixed at 128 (effectively reducing dmodel from 4096 to 3840). Lastly, head dimension is doubled, which is the default behavior for DeltaNet.
번역 대기 중 · 원문 표시
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
基础档案已收录,详细介绍后续补充。可先查看厂商、分类与来源。
기본 정보 등록 · 상세 정보 보완 예정
제공사, 분류, 입출력 조건을 조정하세요. 새 모델이 계속 등록됩니다.
월간 토큰 사용량으로 예상 API 비용을 비교하세요.
수집된 백만 토큰당 USD 요금입니다. 캐시 할인, 구간 요금, 추가 추론 출력, 세금은 제외됩니다. 최신 요금은 출처에서 확인하세요.
입력, 출력, 캐시 요금을 구분하고 동일 작업량으로 비교하세요. 이미지, 음성, 영상은 개수, 시간, 해상도로 과금될 수 있습니다.
반드시 그렇지는 않습니다. 버전별 라이선스, 상업 이용 제한, 정책을 확인하세요. 자체 호스팅에는 연산 및 유지 비용도 듭니다.
순위는 작업, 버전, 평가 방식에 따라 다릅니다. 실제 업무로 정확도, 지연, 안정성, 비용을 비교하세요.