This model is a fine-tuned version of menhior/wav2vec2-large-xls-r-300m-turkish-colab-full on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.5709 - Wer: 0.2893 The following hyperparameters were used during training: - learning_rate: 8e-05 - train_batch_size: 16 - eval_batch_size: 8 - seed: 42 - gradient_accumulation_steps: 2 - total_train_batch_size: 32 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - lr_scheduler_warmup_steps: 500 - num_epochs: 40
**MERaLiON-3-ASR** is the speech-recognition line in the MERaLiON-3 generation of Speech-Text Large Language Models, developed by I2R, A\*STAR, Singapore. It is purpose-built for Singapore- and Southeast-Asia-centric ASR, with broad coverage across regional languages, dialects, and natural conversational code-switching. - **Languages:** English (Global and Singapore), Mandarin, Malay, Tamil, Indonesian, Thai, Vietnamese - **Chinese dialects:** Cantonese, Hokkien - **Code-switching:** natural conversational English ↔ {Mandarin, Malay, Tamil, Vietnamese, Cantonese, Hokkien} mixtures, including Singlish **MERaLiON-3-ASR-API** is the hosted production endpoint, tuned for real-time and enterprise transcription workloads.
🎉**Phi-4**: [mini-reasoning | reasoning] | [multimodal-instruct | onnx]; [mini-instruct | onnx] Phi-4-multimodal-instruct is a lightweight open multimodal foundation model that leverages the language, vision, and speech research and datasets used for Phi-3.5 and 4.0 models. The model processes text, image, and audio inputs, generating text outputs, and comes with 128K token context length. The model underwent an enhancement process, incorporating both supervised fine-tuning, direct preference optimization and RLHF (Reinforcement Learning from Human Feedback) to support precise instruction adherence and safety measures. The languages that each modal supports are the following: - Text: Arabic, Chinese, Czech, Danish, Dutch, English, Finnish, French, German, Hebrew, Hungarian, Italian, Japanese, Korean, Norwegian, Polish, Portuguese, Russian, Spanish, Swedish, Thai, Turkish, Ukrainian - Vision: English - Audio: English, Chinese, German, French, Italian, Japanese, Spanish, Portuguese 📰 Phi-4-multimodal Microsoft Blog 📖 Phi-4-multimodal Technical Report 🏡 Phi Portal 👩🍳 Phi Cookbook 🖥️ Try It on Azure, GitHub, Nvidia, Huggingface playgrounds 📱Huggingface Spaces Thoughts Organizer, Stories Come Alive, Phine Speech Translator
This model was introduced in SpeechT5: Unified-Modal Encoder-Decoder Pre-Training for Spoken Language Processing by Junyi Ao, Rui Wang, Long Zhou, Chengyi Wang, Shuo Ren, Yu Wu, Shujie Liu, Tom Ko, Qing Li, Yu Zhang, Zhihua Wei, Yao Qian, Jinyu Li, Furu Wei. Disclaimer: The team releasing SpeechT5 did not write a model card for this model so this model card has been written by the Hugging Face team. Motivated by the success of T5 (Text-To-Text Transfer Transformer) in pre-trained natural language processing models, we propose a unified-modal SpeechT5 framework that explores the encoder-decoder pre-training for self-supervised speech/text representation learning. The SpeechT5 framework consists of a shared encoder-decoder network and six modal-specific (speech/text) pre/post-nets. After preprocessing the input speech/text through the pre-nets, the shared encoder-decoder network models the sequence-to-sequence transformation, and then the post-nets generate the output in the speech/text modality based on the output of the decoder.
A unispeech-sat-base model that was fine-tuned on 100h hours of Librispeech on 16kHz sampled speech audio. When using the model make sure that your speech input is also sampled at 16Khz. Authors: Sanyuan Chen, Yu Wu, Chengyi Wang, Zhengyang Chen, Zhuo Chen, Shujie Liu, Jian Wu, Yao Qian, Furu Wei, Jinyu Li, Xiangzhan Yu **Abstract** *Self-supervised learning (SSL) is a long-standing goal for speech processing, since it utilizes large-scale unlabeled data and avoids extensive human labeling. Recent years witness great successes in applying self-supervised learning in speech recognition, while limited exploration was attempted in applying SSL for modeling speaker characteristics. In this paper, we aim to improve the existing SSL framework for speaker representation learning. Two methods are introduced for enhancing the unsupervised speaker information extraction. First, we apply the multi-task learning to the current SSL framework, where we integrate the utterance-wise contrastive loss with the SSL objective function. Second, for better speaker discrimination, we propose an utterance mixing strategy for data augmentation, where additional overlapped utterances are created unsupervisely and incorporate during training. We integrate the proposed methods into the HuBERT framework. Experiment results on SUPERB benchmark show that the proposed system achieves state-of-the-art performance in universal representation learning, especially for speaker identification oriented tasks. An ablation study is performed verifying the efficacy of each proposed method. Finally, we scale up training dataset to 94 thousand hours public audio data and achieve further performance improvement in all SUPERB tasks..*
**VibeVoice-ASR-BitNet** is a compressed variant of VibeVoice-ASR optimized for **real-time inference on edge CPUs** — no GPU required. Through heterogeneous quantization, the model is compressed from 4.62 GB to **1.58 GB** while achieving **1.6–2.3× faster** inference than Whisper.cpp with real-time capability (RTF ➡️ **Report:** VibeVoice-ASR-BitNet Technical Report ➡️ **Base Model:** microsoft/VibeVoice-ASR This project was conducted by members of Microsoft Research. If you have suggestions, questions, or observe unexpected behavior, please contact us at VibeVoice@microsoft.com.
**VibeVoice-ASR** is a unified speech-to-text model designed to handle **60-minute long-form audio** in a single pass, generating structured transcriptions containing **Who (Speaker), When (Timestamps), and What (Content)**, with support for **Customized Hotwords** and over **50 languages**. ➡️ **Demo:** VibeVoice-ASR-Demo ➡️ **Report:** VibeVoice-ASR Technical Report - **🕒 60-minute Single-Pass Processing**: Unlike conventional ASR models that slice audio into short chunks (often losing global context), VibeVoice ASR accepts up to **60 minutes** of continuous audio input within 64K token length. This ensures consistent speaker tracking and semantic coherence across the entire hour.
