Peptide-trained Chemical Language Model using 10.8M peptides and 12.6M small molecules for MLM pretraining. Loading the tokenizer is not possible with transformers. A custom tokenizer must be loaded from the 'tokenizer' directory found at at https://github.com/AaronFeller/PeptideCLM An example script for this can be found in the repository. A short example is below (note, the tokenizer directory must be downloaded):
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
The model architecture design, pretraining, and evaluation are documented in our preprint: **A Family of Pretrained Transformer Language Models for Russian**. The model is pretrained by the SberDevices team. * Task: `mask filling` * Type: `encoder` * Tokenizer: `BPE` * Dict size: `120 138` * Num Parameters: `178 M` * Training Data Volume `30 GB`
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
The model architecture design, pretraining, and evaluation are documented in our preprint: **A Family of Pretrained Transformer Language Models for Russian**. The model is pretrained by the SberDevices team. * Task: `mask filling` * Type: `encoder` * Tokenizer: `BBPE` * Dict size: `50 257` * Num Parameters: `355 M` * Training Data Volume `250 GB`
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
A multilingual language model trained on IndicCorp v2 and evaluated on IndicXTREME benchmark. The model has 278M parameters and is available in 23 Indic languages and English. The models are trained with various objectives and datasets. The list of models are as follows: - IndicBERT-MLM [Model] - A vanilla BERT style model trained on IndicCorp v2 with the MLM objective - +Samanantar [Model] - TLM as an additional objective with Samanantar Parallel Corpus [Paper] | [Dataset] - +Back-Translation [Model] - TLM as an additional objective by translating the Indic parts of IndicCorp v2 dataset into English w/ IndicTrans model [Model] - IndicBERT-SS [Model] - To encourage better lexical sharing among languages we convert the scripts from Indic languages to Devanagari and train a BERT style model with the MLM objective Fine-tuning scripts are based on transformers library. Create a new conda environment and set it up as follows:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
Pretrained RoBERTa BASE model on assorted Thai texts (78.5 GB). The script and documentation can be found at this repository. The architecture of the pretrained model is based on RoBERTa [[Liu et al., 2019]](https://arxiv.org/abs/1907.11692). You can use the pretrained model for masked language modeling (i.e. predicting a mask token in the input text). In addition, we also provide finetuned models for multiclass/multilabel text classification and token classification task.
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
基础档案已收录，详细介绍后续补充。可先查看厂商、分类与来源。
基础资料已收录 · 缺失资料后续补充
输入 文本→ 输出 文本
Differences between GENA-LM (`gena-lm-bigbird-base-t2t`) and DNABERT: - BPE tokenization instead of k-mers; - input sequence size is about 36000 nucleotides (4096 BPE tokens) compared to 512 nucleotides of DNABERT; - pre-training on T2T vs. GRCh38.p13 human genome assembly. This repository also contains models that are finetuned on downstream tasks and models that are used in our GENA-Web web tool for genomic sequence annotation: - splice sites prediction (branch gena_web_spliceai) GENA-LM (`gena-lm-bigbird-base-t2t`) model is trained in a masked language model (MLM) fashion, following the methods proposed in the BigBird paper by masking 15% of tokens. Model config for `gena-lm-bigbird-base-t2t` is similar to the `google/bigbird-roberta-base`:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
Pretrained model on English language using a masked language modeling (MLM) objective. It was introduced in this paper and first released in this repository. This model, as all ALBERT models, is uncased: it does not make a difference between english and English. Disclaimer: The team releasing ALBERT did not write a model card for this model so this model card has been written by the Hugging Face team. ALBERT is a transformers model pretrained on a large corpus of English data in a self-supervised fashion. This means it was pretrained on the raw texts only, with no humans labelling them in any way (which is why it can use lots of publicly available data) with an automatic process to generate inputs and labels from those texts. More precisely, it was pretrained with two objectives:
译文排队中 · 当前显示来源原文
输入 文本→ 输出 文本
