This is the same model as SantaCoder but it can be loaded with transformers >=4.28.1 to use the GPTBigCode architecture. We refer the reader to the SantaCoder model page for full documentation about this model There are two versions (branches) of the model: * `main`: Uses the `gpt_bigcode` model. Requires the bigcode fork of transformers. * `main_custom`: Packaged with its modeling code. Requires `transformers>=4.27`. Alternatively, it can run on older versions by setting the configuration parameter `activation_function = "gelu_pytorch_tanh"`. The model was trained on GitHub code. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. You should phrase commands like they occur in source code such as comments (e.g. `# the following function computes the sqrt`) or write a function signature and docstring and let the model complete the function body.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
The SantaCoder models are a series of 1.1B parameter models trained on the Python, Java, and JavaScript subset of The Stack (v1.1) (which excluded opt-out requests). The main model uses Multi Query Attention, a context window of 2048 tokens, and was trained using near-deduplication and comment-to-code ratio as filtering criteria and using the Fill-in-the-Middle objective. In addition there are several models that were trained on datasets with different filter parameters and with architecture and objective variations. The `final` model is the best performing model and was trained twice as long (236B tokens) as the others. This checkpoint is the default model and available on the `main` branch. All other checkpoints are on separate branches with according names. The model was trained on GitHub code. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. You should phrase commands like they occur in source code such as comments (e.g. `# the following function computes the sqrt`) or write a function signature and docstring and let the model complete the function body.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
StarCoder2-15B model is a 15B parameter model trained on 600+ programming languages from The Stack v2, with opt-out requests excluded. The model uses Grouped Query Attention, a context window of 16,384 tokens with a sliding window attention of 4,096 tokens, and was trained using the Fill-in-the-Middle objective on 4+ trillion tokens. The model was trained with NVIDIA NeMo™ Framework using the NVIDIA Eos Supercomputer built with NVIDIA DGX H100 systems. The model was trained on GitHub code as well as additional selected data sources such as Arxiv and Wikipedia. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. Here are some examples to get started with the model. You can find a script for fine-tuning in StarCoder2's GitHub repository.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
StarCoder2-3B model is a 3B parameter model trained on 17 programming languages from The Stack v2, with opt-out requests excluded. The model uses Grouped Query Attention, a context window of 16,384 tokens with a sliding window attention of 4,096 tokens, and was trained using the Fill-in-the-Middle objective on 3+ trillion tokens. The model was trained on GitHub code as well as additional selected data sources such as Arxiv and Wikipedia. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. Here are some examples to get started with the model. You can find a script for fine-tuning in StarCoder2's GitHub repository.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
StarCoder2-7B model is a 7B parameter model trained on 17 programming languages from The Stack v2, with opt-out requests excluded. The model uses Grouped Query Attention, a context window of 16,384 tokens with a sliding window attention of 4,096 tokens, and was trained using the Fill-in-the-Middle objective on 3.5+ trillion tokens. The model was trained on GitHub code as well as additional selected data sources such as Arxiv and Wikipedia. As such it is _not_ an instruction model and commands like "Write a function that computes the square root." do not work well. Here are some examples to get started with the model. You can find a script for fine-tuning in StarCoder2's GitHub repository.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
This is a 164M parameters model with the same architecture as StarCoder (8k context length, MQA & FIM). It was trained on the Python data from StarCoderData for ~6 epochs which amounts to 100B tokens. The model was trained on GitHub code, to assist with some tasks like Assisted Generation. For pure code completion, we advise using our 15B models StarCoder or StarCoderBase. Fill-in-the-middle uses special tokens to identify the prefix/middle/suffix part of the input and output:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
