🤗 SheetSage2 Music audio to editable scores Melody · Chords · Beats · Key · Structure 🎵 YuE2 project · 💻 Code · 🚀 Quick start · 📊 Benchmarks · 📄 SheetSage2 arXiv · 📄 YuE2 arXiv · YuE2 PDF · 📚 Citation **SheetSage2 turns music recordings into lead sheets and timed musical annotations.** Transcribe a complete song, edit its ABC or MIDI, render a piano preview, or extract embeddings and token predictions for your own tools.
🎧 Demo · 🗳️ Music Arena · 🚀 Quick start · 🎙️ Cover · 🤖 Edit · ⚡ Speed · 📊 Benchmarks · 📄 arXiv · PDF · 📄 SheetSage2 arXiv · 📚 Citation **YuE2 is an open music generation model with frontier song quality competitive with Suno v5/v6.** Turn lyrics and a style prompt into a complete song with vocals and accompaniment, then shape its melody and chords through an editable score. > **🎧 YuE2 needs your ears** > > Help evaluate YuE2 alongside leading proprietary music models in our blind listening arena. Listen to anonymous music clips, choose your preference, and contribute to our listener preference study. > > **Listen & vote →** · No account needed.
🎧 Demo · 🚀 Quick start · ⚡ Speed · 📊 Benchmarks · 📄 arXiv · PDF · 📚 Citation **YuE2-Vae turns acoustic latents into 48 kHz stereo music.** It is the default decoder for song generation and listening, with full-song decoding, an audio encoder, and Hugging Face loading. Use it with **🤗 YuE2-3B** for full-song generation, or load it on its own to encode and decode audio. 🤗 Alternative decoder: YuE2-Vae-legacy.
Multilingual-CLIP extends OpenAI's English text encoders to multiple other languages. This model *only* contains the multilingual text encoder. The corresponding image model `Vit-B-16Plus` can be retrieved via instructions found on `mlfoundations` open_clip repository on Github. We provide a usage example below. To use both the multilingual text encoder and corresponding image encoder, we need to install the packages `multilingual-clip` and `open_clip_torch`. None of the M-CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS-COCO dataset, we see the following **R@10** results:
Multilingual-CLIP extends OpenAI's English text encoders to multiple other languages. This model *only* contains the multilingual text encoder. The corresponding image model `ViT-B-32` can be retrieved via instructions found on OpenAI's CLIP repository on Github. We provide a usage example below. To use both the multilingual text encoder and corresponding image encoder, we need to install the packages `multilingual-clip` and `clip`. None of the M-CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS-COCO dataset, we see the following **R@10** results:
Multilingual-CLIP extends OpenAI's English text encoders to multiple other languages. This model *only* contains the multilingual text encoder. The corresponding image model `ViT-L-14` can be retrieved via instructions found on OpenAI's CLIP repository on Github. We provide a usage example below. To use both the multilingual text encoder and corresponding image encoder, we need to install the packages `multilingual-clip` and `clip`. None of the M-CLIP models have been extensivly evaluated, but testing them on Txt2Img retrieval on the humanly translated MS-COCO dataset, we see the following **R@10** results:
This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations. Carbon emissions can be estimated using the Machine Learning Impact calculator presented in Lacoste et al. (2019).
AI-assisted model selection · Quantized for consumer hardware · A suggested starting point This repository is a **suggested toolkit** of open-source AI models assembled to run entirely locally on consumer-grade hardware — specifically an **NVIDIA RTX 3060 (12GB VRAM) with 32GB system RAM**. Each model was chosen to fit within these constraints while offering strong output quality for the VRAM budget. The selection process was **AI-assisted** — I consulted several AI assistants to weigh benchmarks, quantization trade-offs, and model capabilities, then settled on the common recommendations you see here. This is **not a definitively optimal combination**: better models may already exist, and stronger ones are released regularly. Treat it as a well-reasoned starting point rather than a final answer — your own testing on your own hardware is the real benchmark.
Fine-tuned facebook/wav2vec2-large-xlsr-53 in Lithuanian using Common Voice. When using this model, make sure that your speech input is sampled at 16kHz. You can see the training states here The script used for training can be found here
