The release includes companion apps and DAW plugins
AK#33
Google just released Magenta RealTime 2 on Hugging Face
The only open-weights model for real-time continuous music generation on device.
Steer it with text, audio, or MIDI at ~200ms latency.
Positive users praise Magenta RealTime 2 for enabling low-latency live music generation on devices, while negative users criticize its sound quality as poor or generic and raise artist compensation concerns.
Introducing Magenta RealTime 2, a new open model musicians can play as an instrument!
Run low-latency, live music synthesis natively on your MacBook using MIDI, text, and audio. 🎶
We love seeing Google’s open model ecosystem grow!
Introducing Magenta RealTime 2 🎺
- Open model for live music generation - Just 2.4B parameters, perfect for on-device - Low latency control - Control with audio, MIDI, and text
We're releasing it with a series of apps to experiment directly in Mac!
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
That’s so cool! I love the creativity of those guys. An open model for live music generation only 2.4B parameters. If you are bored on long flights you can now start creating bangers
Introducing Magenta RealTime 2 🎺
- Open model for live music generation - Just 2.4B parameters, perfect for on-device - Low latency control - Control with audio, MIDI, and text
We're releasing it with a series of apps to experiment directly in Mac!
This one feels special. Finally we have real time on device generation w/ instrument-like (200ms) latency. Run it in your DAW or on an airplane! Proud to have been a part of this
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
I'm so proud of the team for this work. Live, Local, Open, MRT2 is just a lot of fun, and is an AI Instrument puts the person back at the center of making music.
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
This year marks the 10 year anniversary of Magenta! We’re releasing Magenta RealTime 2, which draws from so many of the ideas that the group has been pursuing over the past 10 years: interactive, genre-bending, open-source tools by and for musicians
Finally, you can truly *play* the model as an instrument, letting you decide how much control you want and how much you want it to riff along with you.
AND open weights AND runs on your machine for ultra-low latency.
Hope you enjoy it!
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
Ready to start playing? Download the apps directly to your MacBook https://magenta.withgoogle.com/mrt2
Latent modulation has so much untapped potential. LFO 🤝 AI
Really proud to share something we’ve been working on for a while: Magenta RealTime 2 (MTR2), a live music model that is highly interactive (MIDI, audio, text, lots of parameters) and low-latency (~200ms end-to-end), and runs locally on a MacBook!
Blog: https://magenta.withgoogle.com/magenta-realtime-2 Weights: https://huggingface.co/google/magenta-realtime-2
Download them here: http://g.co/magenta/mrt2
It's been a joy working with Yotam over these years. From NSynth and AI Duet, to Magenta RealTIme 2, there's a thread of building actual instruments with this technology that is close to both our hearts.
This year marks the 10 year anniversary of Magenta! We’re releasing Magenta RealTime 2, which draws from so many of the ideas that the group has been pursuing over the past 10 years: interactive, genre-bending, open-source tools by and for musicians
Not just a prompt box, more than just a synthesizer, MRT2 opens up a ton of new possibilities. Here are some our favorites 🤘
MIDI Steering: Hold down a note or chord and the model generates an entire ensemble that follows your notes.
Text-to-Synth: Type a description (e.g., “string ensemble”, “disco funk”) to generate a new MIDI instrument.
Look out for some quick followups we have planned, such as finetuning on your own data, and new performance tools (like at the top of the this post @eardrummer_man).
Build your own: https://github.com/magenta/magenta-realtime
Super cool work from some of my Deepmind colleagues
Introducing Magenta RealTime 2 (MRT2): the live music model you can play as an instrument.
MRT2 offers MIDI and prompt controls, and runs natively on a MacBook with <200ms latency.
Open weights. Open source inference engine. Suite of apps and plugins.
Hear what it can do and try it out for yourself below 🧵
Audio Cloning: Drop in a brief audio snippet to emulate that sound.
We’re releasing the weights together with an OSS library (JAX/MLX), a C++ inference engine for efficient streaming generation, and a few app examples, including a plugin to run the model directly in your DAW: https://magenta.withgoogle.com/mrt2
