MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers". Please find the information about preprocessing, training and full details of the MiniLM in the original MiniLM repository. Please note: This checkpoint can be an inplace substitution for BERT and it needs to be fine-tuned before use!
MiniLM is a distilled model from the paper "MiniLM: Deep Self-Attention Distillation for Task-Agnostic Compression of Pre-Trained Transformers". Please find the information about preprocessing, training and full details of the MiniLM in the original MiniLM repository. Please note: This checkpoint uses `BertModel` with `XLMRobertaTokenizer` so `AutoTokenizer` won't work with this checkpoint!
The language model Phi-1 is a Transformer with 1.3 billion parameters, specialized for basic Python coding. Its training involved a variety of data sources, including subsets of Python codes from The Stack v1.2, Q&A content from StackOverflow, competition code from code_contests, and synthetic Python textbooks and exercises generated by gpt-3.5-turbo-0301. Even though the model and the datasets are relatively small compared to contemporary Large Language Models (LLMs), Phi-1 has demonstrated an impressive accuracy rate exceeding 50% on the simple Python coding benchmark, HumanEval. Phi-1 has been integrated in the `transformers` version 4.37.0, please ensure that you are using a version equal or higher than it. Given the nature of the training data, Phi-1 is best suited for prompts using the code format:
The language model Phi-1.5 is a Transformer with **1.3 billion** parameters. It was trained using the same data sources as phi-1, augmented with a new data source that consists of various NLP synthetic texts. When assessed against benchmarks testing common sense, language understanding, and logical reasoning, Phi-1.5 demonstrates a nearly state-of-the-art performance among models with less than 10 billion parameters. We **did not** fine-tune Phi-1.5 either for **instruction following or through reinforcement learning from human feedback**. The intention behind crafting this open-source model is to provide the research community with a non-restricted small model to explore vital safety challenges, such as reducing toxicity, understanding societal biases, enhancing controllability, and more. For a safer model release, we exclude generic web-crawl data sources such as common-crawl from the training. This strategy prevents direct exposure to potentially harmful online content, enhancing the model's safety without RLHF. However, the model is still vulnerable to generating harmful content. We hope the model can help the research community to further study the safety of language models.
Adjust provider, category or input/output filters. New models are added continuously.
COST EXPLORER
Great models. Clear costs.
Cost analysis
Adjust monthly token usage to compare estimated API costs.
1 M = 1 million tokens · USD Compares base text-token rates within the same channel
Uses collected, comparable USD-per-million-token rates. Snapshots exclude cache discounts, tiers, extra reasoning output and taxes. Check current source prices.
MODEL HANDBOOK
Understand before you choose
Selection guide & FAQ
How should API prices be compared?+
Separate input, output and cache rates, then estimate the same workload. Image, audio and video rates may use items, duration or resolution rather than tokens.
Do open weights mean free commercial use?+
Not necessarily. Check each version’s license, commercial restrictions and policies. Self-hosting also incurs compute and maintenance costs.
Is the top-ranked model the best for me?+
Rankings depend on tasks, versions and methods. Compare accuracy, latency, reliability and costs on your own workload.
