AgroNT is a DNA language model trained on primarily edible plant genomes. More specifically, AgroNT uses the transformer architecture with self-attention and a masked language modeling objective to leverage highly available genotype data from 48 different plant speices to learn general representations of nucleotide sequences. AgroNT contains 1 billion parameters and has a context window of 1024 tokens. AgroNt uses a non-overlapping 6-mer tokenizer to convert genomic nucletoide sequences to tokens. As a result the 1024 tokens correspond to approximately 6144 base pairs. Our pre-training dataset was built from (mostly) edible plants reference genomes contained in the Ensembl Plants database. The dataset consists of approximately 10.5 million genomic sequences across 48 different species. All reference genomes for each specie were assembled into a single fasta file. In this fasta file, all nucleotides other than A, T, C, G were replaced by N. A tokenizer was used to convert strings of letters into sequences of tokens. The tokenizer's alphabet consisted of the 4 6 = 4096 possible 6-mer combinations obtained by combining A, T, C, G, as well as five additional tokens representing standalone A, T, C, G, and N. It also included three special tokens: the pad [PAD], mask [MASK], and class [CLS] tokens. This resulted in a vocabulary of 4104 tokens. To tokenize an input sequence, the tokenizer started with a class token and then converted the sequence from left to right, matching 6-mer tokens when possible, or using the standalone tokens when necessary (for instance, when the letter N was present or if the sequence length was not a multiple of 6).
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 向量
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
The Nucleotide Transformers are a collection of foundational language models that were pre-trained on DNA sequences from whole-genomes. Compared to other approaches, our models do not only integrate information from single reference genomes, but leverage DNA sequences from over 3,200 diverse human genomes, as well as 850 genomes from a wide range of species, including model and non-model organisms. Through robust and extensive evaluation, we show that these large models provide extremely accurate molecular phenotype prediction compared to existing methods Part of this collection is the **nucleotide-transformer-500m-human-ref**, a 500M parameters transformer pre-trained on the human reference genome. The model is made available both in Tensorflow and Pytorch. - **Repository:** Nucleotide Transformer - **Paper:** The Nucleotide Transformer: Building and Evaluating Robust Foundation Models for Human Genomics
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
The Nucleotide Transformers are a collection of foundational language models that were pre-trained on DNA sequences from whole-genomes. Compared to other approaches, our models do not only integrate information from single reference genomes, but leverage DNA sequences from over 3,200 diverse human genomes, as well as 850 genomes from a wide range of species, including model and non-model organisms. Through robust and extensive evaluation, we show that these large models provide extremely accurate molecular phenotype prediction compared to existing methods Part of this collection is the **nucleotide-transformer-v2-500m-multi-species**, a 500m parameters transformer pre-trained on a collection of 850 genomes from a wide range of species, including model and non-model organisms. Until its next release, the `transformers` library needs to be installed from source with the following command in order to use the models:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
The Nucleotide Transformers are a collection of foundational language models that were pre-trained on DNA sequences from whole-genomes. Compared to other approaches, our models do not only integrate information from single reference genomes, but leverage DNA sequences from over 3,200 diverse human genomes, as well as 850 genomes from a wide range of species, including model and non-model organisms. Through robust and extensive evaluation, we show that these large models provide extremely accurate molecular phenotype prediction compared to existing methods Part of this collection is the **nucleotide-transformer-v2-50m-multi-species**, a 50M parameters transformer pre-trained on a collection of 850 genomes from a wide range of species, including model and non-model organisms. Until its next release, the `transformers` library needs to be installed from source with the following command in order to use the models:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
分类待补充
输入 待核验→ 输出 待核验
