This is my journey on how to consolidate or merge all my different sources of music without having duplicates.

Primary Tool to accomplish this is Lexicon DJ software (currently $20 or $400 for liftetime license). If I was young or had a lot of years left of DJing I would easily pay for the Lifetime License since its sooooo powerful but at $20/month I have around two years to reach the $320 with the 20% discount they are offering currently so stickign with monthly for now.

Expectation: The tool is working with some large databases and thousands of files so patience is needed to allow the software to be accurate and reliable. There is no stop or cancel once you start doing a task so work in small batches.

Another Tool I use to ChronoSync to make sure my source is solid. So I sync from multiple places to one harddrive to make sure I have everything in one place.

Now my current scenario is....

I use Apple Music when I'm not DJing so I like to have all my music in Apple Music just for the sync to my iPhone/iPad functionality so when using CarPlay or out and about I have my music and listen to my playlists. So I have the standard Music/Music/Media folder with songs I have in Apple Music

I use to have Serato DJ Pro so I do have a Serato folder and a Crates folder with lots of music.

Finally I have Virtual DJ that I use for DJing since it works on all mixers (not locked to a Serator Mixer or Rekord Box Mixer).

My Plan

1. Import everything from Apple Music to Lexicon using XML file

2. Import everything from Serato

3. Import everything from Virtual DJ

4. Search for Duplicates using Lexicon DJ and move Duplicates to another Harddrive (since using the software is new and until I'm 100% confident I don't fully trust it yet) -- NOTE: This took hours to get through but I have 200,000 plus songs

5. Fix Titles (Fix all Captial Letters, all lowercase letters)

6. Remove Comments

7. Export to Apple Music XML file

8. Import new XML file in Apple Music -- NOTE: This took hours to get through but I have 200,000 plus songs. Just leave it alone and let Apple Music do its thing. Once it gets done, Exit out of Apple Music so it saves the database. Then reOpen Apple Music and let it do its analysis, download apple art, etc.. but you can go back to Lexicon and work on making the Library better for Virtual DJ with options Apple Music doesn't use like cue points, Key, etc..

   1. Question is.. when you import the Playlist (XML) that you exported with Lexicon DJ that has all these other sources from Serato.. and you have consolidate songs turned on, does it copy the songs to the Music/Music/Media folder? IF not, you might have to move them manually and after Apple Music processes/imports all files you may have to repeat the entire process.

9. ...

Here's how to export your Lexicon library to Apple Music/iTunes while ensuring your files are properly consolidated:

1. Export from Lexicon:

   • Go to File → Export → Apple Music/iTunes

   • Choose "Full Sync" (this ensures all tracks and playlists are included)

   • Disable "Copy files to iTunes Media folder" (important for consolidation later)

   • Click Export and save the .xml file

2. Prepare Apple Music:

   • Open Apple Music/iTunes

   • Go to File → Library → Organize Library

   • Check "Consolidate files" (this will copy all referenced files to Apple's Media folder)

   • Click OK (this may take a while for large libraries)

3. Import to Apple Music:

   • Go to File → Library → Import Playlist

   • Select the .xml file you exported from Lexicon

   • Wait for the import to complete

4. Verify Consolidation:

   • Check that all files now exist in:~/Music/Music/Media (macOS) orC:\Users\YourUsername\Music\iTunes\iTunes Media (Windows)

Important Notes:

• The consolidation step (Step 2) is what actually copies your files to Apple's media folder

• If you skip consolidation, Apple Music will reference files in their current locations

• For future updates, you'll need to repeat this full process as Apple Music doesn't support incremental updates to playlists

I wanted to import the crates into Apple Music so I created a folder called import and added all the folders and files into it.

Then to get these songs into Apple Music I ran:

find "/Users/djzah/Music/Music/import" -type f -exec mv -f {} "/Users/djzah/Music/Music/Media.localized/Automatically Add to Music.localized" \;

So why do I use mv (move) instead of cp (copy) is to save space. also I use -f to force move to save space on drive.

When the move is completed I delete the import folder I used since everything that can be moved over is moved to the Add Music Automatically folder.

I open Apple Music and it should injest all new songs into the Apple Music Database.

NOW create an export XML file and in Lexicon DJ import that Apple Music file but make sure you are doing a merge (since you are adding to existing).

Finally you may want to do one last Duplicates check to make sure you are clean before you SYNC/EXPORT to Apple Music with all songs.

From Lexicon DJ make sure you delete the musiclibrary and then open up Apple Music by holding the Option key and click on Music and select create library.. Change from Music 1 to just Music and when clicking Ok select yes replace. Then close Apple Music.

In Lexicon DJ click Sync to Itunes which works on creating you an XML file

You may learn that with Apple Music (old iTunes) that it will time out (become Not Responding) when trying to import the XML file created by Lexicon DJ. Nothing wrong with Lexicon DJ and more of an issue with large database but no worries, there is a way to fix it.

Create file: split_playlist.py

Copy the following and paste into that file you just created

import plistlib
import copy
import math

# Load your file
with open("library.xml", "rb") as f:
    data = plistlib.load(f)

tracks = data["Tracks"]
playlists = data["Playlists"]

CHUNK_SIZE = 5000  # adjust — if 5k is too many change to 200 tracks per file

track_ids = list(tracks.keys())
num_chunks = math.ceil(len(track_ids) / CHUNK_SIZE)

for i in range(num_chunks):
    chunk_ids = set(track_ids[i * CHUNK_SIZE:(i + 1) * CHUNK_SIZE])
    chunk_tracks = {k: v for k, v in tracks.items() if k in chunk_ids}

    # Only keep playlist items that reference tracks in this chunk
    chunk_playlists = []
    for pl in playlists:
        new_pl = copy.deepcopy(pl)
        items = pl.get("Playlist Items", [])
        new_pl["Playlist Items"] = [
            item for item in items
            if str(item.get("Track ID", "")) in chunk_ids
        ]
        if new_pl["Playlist Items"]:
            chunk_playlists.append(new_pl)

    chunk_data = copy.deepcopy(data)
    chunk_data["Tracks"] = chunk_tracks
    chunk_data["Playlists"] = chunk_playlists

    out_file = f"playlist_chunk_{i+1}.xml"
    with open(out_file, "wb") as f:
        plistlib.dump(chunk_data, f, fmt=plistlib.FMT_XML)
    print(f"Wrote {out_file} with {len(chunk_tracks)} tracks")

You will need to replace library.xml in the above code with the name of your playlist that you are trying to import (should still end in xml).

then just run:

python3 split_playlist.py

Now you will have several smaller, more acceptable, xml files that Apple Music should be able to process without any problems.

Still trying to learn how to make this Lexicon DJ software work for me.

I decided I'm going to use the "Find Unused Files" first so everything is accounted for

Next utilize "Find Duplicates" thinking maybe a duplicate could be pointing to a lost file

Once those two are completed now use the "Find Lost or Broken Tracks"