This is the cointegrated/rubert-tiny2 model fine-tuned for classification of emotions in Russian sentences. The task is multilabel classification, because one sentence can contain multiple emotions. The model on the CEDR dataset described in the paper "Data-Driven Model for Emotion Detection in Russian Texts" by Sboev et al. The model has been trained with Adam optimizer for 40 epochs with learning rate `1e-5` and batch size 64 in this notebook.
This is a port of the multilingual SONAR text encoder (https://huggingface.co/facebook/SONAR) to the `transformers` format from `fairseq2`. Its embeddings are expected be equal to those the official implementation (https://github.com/facebookresearch/SONAR), but the latter stays the source of truth. The encoder supports the same 202 languages as NLLB-200 (see also the source model card and FLORES-200 lang code mapping).
Figure 1: ColBERT's late interaction, efficiently scoring the fine-grained similarity between a queries and a passage. As Figure 1 illustrates, ColBERT relies on fine-grained **contextual late interaction**: it encodes each passage into a **matrix** of token-level embeddings (shown above in blue). Then at search time, it embeds every query into another matrix (shown in green) and efficiently finds passages that contextually match the query using scalable vector-similarity (`MaxSim`) operators. These rich interactions allow ColBERT to surpass the quality of _single-vector_ representation models, while scaling efficiently to large corpora. You can read more in our papers:
**RVN** is a double-refined abliterated variant of **Qwen3.8-27B**, built on top of `trohrbaugh/Qwen3.8-27B-heretic-ara` (an ARA abliteration by **Tim Rohrbaugh**) and further refined with **two additional full-weight ARA passes** targeting residual refusals. It retains very low behavioral damage (KL ≈ 0.0085) while reducing harmful-prompt refusals from 3/100 (source) to **0–1/100** in independent measurements. > **Note on this repository's history.** This repo previously hosted the original > `Qwen3.8-27B-Heretic-Q4_K_M.gguf` (single-quant release from the earlier > `trohrbaugh/Qwen3.8-27B-heretic` source). That file is kept as **legacy** for > download-count continuity and backward compatibility — it is the *older* abliteration > variant and is superseded by the RVN files below. Prefer the RVN quants for new > deployments. > **Not for all audiences.** This model has reduced safety guardrails by design. It is > intended for adult audiences (18+) doing research, creative writing, roleplay, and > uncensored generation. Certain guardrails are intentionally left in place; use > responsibly and in accordance with your local laws.
The model is first pre-trained for 8 epochs on a cleaned version of the BabyLM Challenge strict track dataset. After pre-training, the model is instruction-tuned on two additional datasets (8.7M words total) for 10 epochs : The model has been submitted to the 2025 BabyLM Challenge – Interaction Track: https://huggingface.co/spaces/BabyLM-community/babylm-leaderboard-2025-all-tasks This model was introduced in the paper: **“CLASS-IT: Conversational and Lecture-Aligned Small-Scale Instruction Tuning for BabyLMs”** *(Capone, Bondielli & Lenci, BabyLM Challange 2025)* 📄 ArXiv: 2510.25364