Meta Releases Open AI Music Models
Meta has unveiled a new batch of open AI models focused on multi-modal processing, including music generation. The release signals Big Tech's push into AI applications that blend text, vision, and audio, creating new opportunities for projects at the intersection of engineering and creative arts.
Meta's latest open AI release is a family of models named AudioCraft, which includes MusicGen for creating music from text, AudioGen for generating sound effects from text, and an improved EnCodec decoder for higher-quality output with fewer distortions. This move is part of Meta's broader strategy to foster an open-source ecosystem around its AI tools, similar to its approach with the LLaMA series of language models. MusicGen was trained on 20,000 hours of Meta-owned or licensed music, ensuring a cleared dataset for training. It is available in multiple sizes, ranging from 300 million to 3.3 billion parameters, allowing for scalability depending on the user's hardware, with a GPU with at least 16GB of RAM recommended for local operation. A newer model, JASCO (Joint Audio and Symbolic Conditioning for Temporally Controlled Text-to-Music Generation), offers more granular control over the generated music. Unlike models that rely solely on text prompts, JASCO can incorporate inputs like specific chords or beats, giving creators more precise control over the musical output. The decision to open-source these models, with code available on platforms like Hugging Face, is a strategic move by Meta. By allowing developers to build upon their technology, Meta aims to accelerate innovation in AI-generated audio and potentially establish its models as industry standards, a strategy that has proven successful with past open-source releases like PyTorch and React. While these tools offer new creative possibilities, Meta acknowledges the training data for MusicGen has a larger proportion of Western-style music. By open-sourcing the models, Meta hopes the community will help address and mitigate such biases. The models are released under licenses that generally permit non-commercial use and modification, allowing for broad experimentation and research.
