**HerBERT** is a BERT-based Language Model trained on Polish corpora using Masked Language Modelling (MLM) and Sentence Structural Objective (SSO) with dynamic masking of whole words. For more details, please refer to: HerBERT: Efficiently Pretrained Transformer-based Language Model for Polish. Model training and experiments were conducted with transformers in version 2.9. The training dataset was tokenized into subwords using a character level byte-pair encoding (``CharBPETokenizer``) with a vocabulary size of 50k tokens. The tokenizer itself was trained with a tokenizers library.
SPECTER is a pre-trained language model to generate document-level embedding of documents. It is pre-trained on a powerful signal of document-level relatedness: the citation graph. Unlike existing pretrained language models, SPECTER can be easily applied to downstream applications without task-specific fine-tuning. If you're coming here because you want to embed papers, SPECTER has now been superceded by SPECTER2. Use that instead.
CamemBERTav2 is a French language model pretrained on a large corpus of 275B tokens of French text. It is the second version of the CamemBERTa model, which is based on the DebertaV2 architecture. CamemBERTav2 is trained using the Replaced Token Detection (RTD) objective with 20% mask rate on 275B tokens on 32 H100 GPUs. The dataset used for training is a combination of French OSCAR dumps from the CulturaX Project, French scientific documents from HALvest, and the French Wikipedia. The model is a drop-in replacement for the original CamemBERTa model. Note that the new tokenizer is different from the original CamemBERTa tokenizer, so you will need to use Fast Tokenizers to use the model. It will work with `DebertaV2TokenizerFast` from `transformers` library even if the original `DebertaV2TokenizerFast` was sentencepiece-based. The new update includes: - Much larger pretraining dataset: 275B unique tokens (previously ~32B) - A newly built tokenizer based on WordPiece with 32,768 tokens, addition of the newline and tab characters, support emojis, and better handling of numbers (numbers are split into two digits tokens) - Extended context window of 1024 tokens
This model is a **70.91% smaller** version of intfloat/multilingual-e5-small optimized for **French** language via vocabulary size reduction using the trimming method. This trimmed model should perform similarly to the original model with only 32,768 tokens and a much smaller memory footprint. However, it may not perform well for other languages as tokens not commonly used in the selected languages were removed from the vocabulary. - **Number of texts used for mining**: 200,000 texts - **Dataset**: lbourdois/fineweb-2-trimming
Adjust provider, category or input/output filters. New models are added continuously.
COST EXPLORER
Great models. Clear costs.
Cost analysis
Adjust monthly token usage to compare estimated API costs.
1 M = 1 million tokens · USD Compares base text-token rates within the same channel
Uses collected, comparable USD-per-million-token rates. Snapshots exclude cache discounts, tiers, extra reasoning output and taxes. Check current source prices.
MODEL HANDBOOK
Understand before you choose
Selection guide & FAQ
How should API prices be compared?+
Separate input, output and cache rates, then estimate the same workload. Image, audio and video rates may use items, duration or resolution rather than tokens.
Do open weights mean free commercial use?+
Not necessarily. Check each version’s license, commercial restrictions and policies. Self-hosting also incurs compute and maintenance costs.
Is the top-ranked model the best for me?+
Rankings depend on tasks, versions and methods. Compare accuracy, latency, reliability and costs on your own workload.
