Repaint

Repaint operations allow you to edit sections of your clips using AI. Did you Generate an amazing song, except one part of it turned out awful? Repaint lets you regenerate just that region while the rest remains untouched.

How to

The remix button (shown as a paintbrush) is disabled by default, but will enable as soon as you make a valid selection. Simply drag your mouse across the section of your clip that you want to edit, and then click the button. A new dialog window will open up where you can tell the AI what you want to change. Click the generate button, wait for the new audio to load, and the selected region will be updated.

One way to optimize your repaints is to make sure that your lyrics in the meta data menu are updated in accordance with what you are trying to achieve. For example if you want a piano solo, consider adding something like [piano solo] in the actual lyrics text as well.

Understanding repainting

Repaint is a difussion based editing technique that makes localized modifications by leveraging surrounding context. What that means is that MelonSound will understand the context of the selection that you want to change by looking at the song in its entirety and how the selected region fits into it.

The key takeaway is that you are not simply creating a black hole inside of a song that can be filled with "anything". You are telling MelonSound to try and fit something into the existing song without disrupting it.

Limitations

The quality of a repaint operation will degrade the longer the repaint selection is. This is partly a technical limitation of the underlying AI models, but also a conceptual limitation. MelonSound enforces a limit of 3s-30s per repaint, and if you make a selection that is too long or too short then the repaint button will disable itself.

Furthermore, when you execute a repaint the AI model needs to understand the full context outside of the repaint to be able to make the new version fit in. As the repaint region grows, this context conversely shrinks, causing a decline in quality. MelonSound enforces no limitation in this sense, but it is something worth keeping in mind!

Editing Lyrics

You can change the lyrics of a song using repainting.

To optimize your result for editing lyrics, first make sure that you edit your song's lyrics in the right side timeline lyrics tab. These lyrics are used by the audio engine when executing AI operations and needs to be kept up to date. Secondly, write specifically in the repaint prompt that the lyrics should be changed (sometimes it can help to include what they should be changed to as well!).

Editing lyrics using repainting always works best if the new lyrics sound similar to the old lyrics, and, perhaps most importantly, if there are already lyrics in the repainted section to begin with. In cases where the selected region is instrumental and you want to add lyrics the model tends to struggle a bit more.

Sometimes you may need to try a few times to get the desired effect, so be patient.

What should repainting not be used for?

Sometimes you may want to, for example, add or remove instruments from a section of a song or make very significant changes. This is generally not a use case for repaint. Repainting is fundamentally made to change a part of a song while still making it fit naturally inside of its context. Adding brand new elements that stand out is counterintuitive to this purpose. Think of repainting as a way of adjusting something in a context; you can change the section but not the context itself.