🏛 Watch the Community Town Hall here
The Music Demixing (MDX) Challenge is an opportunity for researchers and machine learning enthusiasts to test their skills by creating a system able to perform audio source separation.
Such a system, given an audio signal as input (referred to as “mixture”), will decompose it in its individual parts.
Audio source separation has different declinations, depending on the signal the system is working on. Music source separation systems take a song as input and output one track for each of the instruments. Speech enhancement systems take noisy speech as input and separate the speech content from the noise.
Such a technology can be employed in many different areas, ranging from entertainment to hearing aids. For example, the original master of old movies contains all the material (dialogue, music and sound effects) mixed in mono or stereo: thanks to source separation we can retrieve the individual components and allow for up-mixing to surround systems. Sony already restored two movies with this technology in their Columbia Classics collection. Karaoke systems can benefit from the audio source separation technology as users can sing over any original song, where the vocals have been suppressed, instead of picking from a set of “cover” songs specifically produced for karaoke.
The Music Demixing Challenge (MDX) will focus on music source separation and it follows the long tradition of the SiSEC MUS challenges (results of the 2018 competition: SiSEC MUS 2018). Participants will submit systems that separate a song into four instruments: vocals, bass, drums, and other (the instrument “other” contains signals of all instruments other than the first three, e.g., guitar or piano).
Participants are allowed to train their system exclusively on the training set of MUSDB18-HQ dataset or they can use their choice of data. According to the dataset used, participant will be eligible either for Leaderboard A or Leaderboard B respectively.
Hidden Test Data
The test set of the MDX challenge will be closed: participants will not have access to it, not even outside the challenge itself; this allows a fair comparison of all submissions. The set was created by Sony Music Entertainment (Japan) Inc. (SMEJ) with the specific intent to use it for the evaluation of the MDX challenge. It is therefore confidential and will not be shared with anyone outside the organization of the MDX challenge.
The MDX challenge will feature two leaderboards.
Participants in Leaderboard A will be allowed to train their system exclusively on the training part of MUSDB18-HQ dataset. This dataset has become the standard in literature as it is free to use and gives anyone the possibility to start training source separation models.
Participants that use the compressed version of the dataset (MUSDB18) are still eligible for leaderboard A.
Participants in Leaderboard B, instead, will not be constrained in the choice of data for training and any available material can be used by the participants.
The total prize pool is 10,000 CHF, which will be divided equally among the two leaderboards.
🥇 1st: 3500 CHF
🥈 2nd: 1000 CHF
🥉 3rd: 500 CHF
🥇 1st: 3500 CHF
🥈 2nd: 1000 CHF
🥉 3rd: 500 CHF
You are eligible for prizes in both the leaderboards.
💪 Getting Started
The starter kit of the competition is available at https://github.com/AIcrowd/music-demixing-challenge-starter-kit.
🚀 Baseline System
The MDX challenge will feature two baselines:
🖊 Evaluation Metric
As an evaluation metric, we are using the signal-to-distortion ratio (SDR), which is defined as,
where S𝑖𝑛𝑠𝑡𝑟(n) is the waveform of the ground truth and Ŝ𝑖𝑛𝑠𝑡𝑟(𝑛) denotes the waveform of the estimate. The higher the SDR score, the better the output of the system is.
In order to rank systems, we will use the average SDR computed by
for each song. Finally, the overall score is obtained by averaging SDRsong over all songs in the hidden test set.
The following Python code shows how they are computed:
Please note that the organizers (Sony and INRIA) will not get access to the submitted entries - everything is handled by AIcrowd and AIcrowd guarantees for the security of your submissions. However, the organizers plan to write an academic paper and for this will get access to the output (i.e., the separations) of the top-10 entries for each leaderboard. For more information, please see the challenge rules.
🚟 ISMIR Workshop
We will host a satellite workshop for ISMIR 2021 which will give all participants the opportunity to come together and share their experience during this challenge.
📅 Competition Timeline
The MDX challenge will take place in 2 Rounds which differ in the evaluation dataset that is used for ranking the systems.
For this, we splitted the hidden dataset into 3 (roughly) equally-sized parts. During the 1st Round, participants can see the scores of their submission on the first-third of the hidden dataset. During the 2nd Round, participants can see their scores on the first- and second-third of the hidden dataset.
The ranking of the Final leaderboard will be based on the scores on all songs of the hidden test set.
Here is a summary of the competition timeline:
📅 Round 1: May 3rd - June 13th, 12 PM UTC
📅 Round 2: June 14th - July 31st, 12 PM UTC
🥶 Team Freeze deadline: 23rd July, 12 PM UTC
Beginning of August
- End of the challenge, Final leaderboard is made public with scores on all songs from the hidden test set.
- Distribution of prizes based on this final leaderboard.
💪 Challenge Page: https://www.aicrowd.com/challenges/music-demixing-challenge-ismir-2021
🗣️ Discussion Forum: https://www.aicrowd.com/challenges/music-demixing-challenge-ismir-2021/discussion
📱 Challenge Organizers
Yuki Mitsufuji, Sony Group Corporation, R&D Center, Japan
Giorgio Fabbro, Sony Group Corporation, R&D Center, Germany
Stefan Uhlich, Sony Group Corporation, R&D Center, Germany
Fabian-Robert Stöter, INRIA, France
🚀 Make your submissions from inside Google Colab