A fast way to remove duplicate songs?

[McDougal]

Hey all,

My friend has a huge music library, and it's a complete mess. Their music was in various places on the hard drive....along with messy tags and file names.

I organized all the music into the iTunes folder by using the Consolidate feature in iTunes, and it worked well. Except for one problem, there are now multiple copies of many songs/albums.

Is there a quick way to delete duplicate songs, instead of going through file explorer and deleting them manually? I know iTunes has a feature to display duplicate songs....but you still have to go through every one.

Thanks guys for the help !

[CrowdAds]

[musichound]

This might be a paid app or shareware, but I found this:

http://www.easyduplicatefinder.com/f...&crtag=adwords

Seems this is what you're looking for.

[McDougal]

Originally Posted by musichound

This might be a paid app or shareware, but I found this:

http://www.easyduplicatefinder.com/f...&crtag=adwords

Seems this is what you're looking for.

Thanks for the link. I'll give it a try.

[skip252]

The fastest software for finding duplicates I know is Duplicate Music Files Finder. It's very fast but not very sophisticated. It does a decent rough sort but it misses things like "Part 1", "Part 2", "Remix". You can get a lot of false positives that way if you have multiple versions of the same song that are on different albums.

Mediamonkey also has a decent way to find duplicates as part of it's Files To Edit Function. Add the files to the MM library and it identifies the duplicates a couple of different way to make it easier to eliminate any duplicates.

If your tags are good a tag based move will positively identify any dupes. Your tags have to be really well done for it to work properly. Once you're sure your tags are tight you can have foobar2000 or MediaMonkey move the files. Neither of those will overwrite an existing files so whatever doesn't make it to the new location was a duplicate. It's one of those things it's best to try on a small sample of your library first. You won't lose any files but you can wind up with files in places you don't expect and then it can be hard to tell what's a duplicate.

Since I have a library that has both lossless and lossy files I like to sort them by codec. With fb2k I use a pattern of

[%album artist%/][%album%/][%codec%/][%tracknumber%] - [%title%]

That works out to

<Album Artist>\<Album>\<Extension>\<Track#:2> - <Title>

in MM.

[McDougal]

Thanks skip252 for the suggestion. I will give Duplicate Music Files Finder a try.

I also forgot to mention that because this isn't my computer I'd prefer not to install any media players, and it may be a day or two before I can try this software.

[skip252]

A portable installation of fooby2k doesn't need to be installed. It runs just fine from a external drive. The copies I run from my DAPs and USB keys work just as well as the one i have installed on my PC.

There's no file associations and it doesn't write to the registry so using it on another system doesn't affect that system in any way. You can even set up the library to read the files and nothing will change anything as long as you don't move any files or write tags.

[McDougal]

Interesting, I wasn't aware that foobar can be used as a portable app. I will look into this as well.

However, I'm not certain that all tags are correct, and the library is already organized as follows:

<Artist>\<Album>\<track#> <title>

He only has lossy mp3/aac music files so lossless isn't an issue.

I will report back later this week when I get a chance to look at the library.

[skip252]

The use of [%codec%/] will also separate lossy files. The mp3 files would wind up in a folder named MP3 and AAC files get placed in a ACC folder.

There wouldn't be a need to create those. The only folder that must exist is the first folder. The rest are created by the use of the slash "\" character. If you create the first folder and name it something like "Sorted Files" the rest are created automatically.

You can even use the options to move all the files in the existing folders and delete any empties. That way any files used for album art that aren't embedded come along for the ride and wind up in the same folders as the music files.

The option to delete the empty folders is dead safe and you won't have a bunch of empty folders to deal with later. fooby2k will only delete folders that are truly empty. If there's any system or hidden files, the folder wont be deleted.

Any folders that are left after the move have something in them. It may be hidden system folders, jpgs or duplicate music files. I inspect those manually. It can be a fair amount of work but it's a lot less after the move than before.

I've also used the option to move the entire file contents or not to clean up collections that were part of a WMP library. If you chose to only move the music files all the crappy little .jpgs that they add to the folders don't come along. It also leaves behind all the .ini files that aren't usually easy to find in a standard search. Getting rid of all that garbage has cleared up the problem of AA showing up that doesn't seem to exist for a couple people I've helped.

[McDougal]

I wasn't aware that the code would separate lossy files as well. Maybe I will do this for my own library .

And another thing....if I don't use the Album Artist field, will it still be sorted out into Artist? Or should I change the code to artist only?

[skip252]

What I put there is what works for me in just about every situation as long as the files are tagged. The Album Artist part defaults to Artist if that value isn't present for me.

You can always check to see if a particular title formatting will work for you using the file Operations Preview pane. What you see there is what foobar2000 is going to try to do if you OK what you see. If you see files with the same name or a formatting you don't want, you can make adjustments using the "..." box at the end of the file pattern name input box.

As you make adjustments the changes to the expected pattern are shown as quickly as fb2k can redraw them. If you use a small set of test files to see the pattern you see the changes as fast as you can type. Once you have a pattern you like you can save it as a preset so you don't have to remember the pattern. When you want to do the same file operation you just pick the preset.

I found saving presets to be the way to go. I once typed an extra space into a move pattern and created several hundred copies I didn't realize what had happened until I ran out of space on a drive that was supposed to last me for a few more years.

After I fixed the file pattern I did the same thing I described here. I tried to move the files to another directory using the repaired title formatting pattern and the extras errored out. I renamed those in place and copy/pasted back to the original drive. Since the folder structure and names were exactly the same they over wrote the copies without having to delete gigs of files.

[McDougal]

Great, thanks for the help ! I now have my music library organized in AAC and MP3 folders along with the usual filename.

[McDougal]

I downloaded a portable install of foobar2000 and put it on my flash drive. From there, I managed to clean up the music library .

I used the filename pattern you posted except changed the codec folder to the beginning, so it's like this:

[%codec%/][%album artist%/][%album%/][%tracknumber%] - [%title%]

Turns out he had some WMA files around so I will find out what to do about those later. I saved a couple GBs of duplicate songs, lol.

A couple of albums were not tagged, and those were put in the root of My Music which is OK for now.

Thanks again skip252 for all the help, the music library is no longer an unorganized mess !

[skip252]

WMA??!! Ewwwww!!! Here's hoping you enlightened them about the virtues of LAME. After all, friends don't let friends use crappy, outdated codecs.

Glad it worked out. Somehow fb2k has gained a reputation for being an incredibly difficult to setup, geeks only software. It's true that a lot of geeks like it but it just isn't hard to have a good, basic setup. I think part of the rep it has for being a toughie comes from when people try to tweak every part of it and get lost in all of the possible options.

It's definitely one of those thing that gets easier the more you use it and become accustomed to it's internal conventions. Then it's much easier it is to extend it's capabilities. That's if you want more out of it. It does most things really well out the box with minimal setup.

[McDougal]

Originally Posted by skip252

WMA??!! Ewwwww!!! Here's hoping you enlightened them about the virtues of LAME. After all, friends don't let friends use crappy, outdated codecs.

Glad it worked out. Somehow fb2k has gained a reputation for being an incredibly difficult to setup, geeks only software. It's true that a lot of geeks like it but it just isn't hard to have a good, basic setup. I think part of the rep it has for being a toughie comes from when people try to tweak every part of it and get lost in all of the possible options.

It's definitely one of those thing that gets easier the more you use it and become accustomed to it's internal conventions. Then it's much easier it is to extend it's capabilities. That's if you want more out of it. It does most things really well out the box with minimal setup.

Lol yes, I have told them about LAME MP3. But the problem is I don't know if they have the original CDs for the music, and I want to see if they can find the discs before I do lossy transcoding. Hopefully the discs will be found and then I will introduce LAME as well .

If, they can't find the discs, is there a better way to do lossy transcoding to minimize quality loss? Or should I just use foobar and LAME?

[skip252]

Trancoding is always best done by going by what you hear. As you know, it's best not to do it at all. If you find it's the only way I'd probably try a few relatively high quality settings and see if there's a difference you can ABX. I'd use fooby for the entire process. Which software you use won't change the inevitable quality loss.

There's always a chance that if the original bitrate was high enough there won't be many objectionable artifacts. That's pretty hit or miss in my experience.

I'd definitely keep the originals if the discs aren't available. There may be issues you run into later that re-encoding at different setting may help. Those originals are the best those files can ever sound. I'd keep them just in case you find there's just no settings that don't cause audible degradation and decide to just live with them as they are.

[McDougal]

Yeah, that's what I thought.

Still don't know if he's got the original CDs, but the files are lower-quality, in the 192k range. So maybe transcoding to -V2 would be ok.

I just came up with an idea - I'm going to check and see if the songs are on Amazon's MP3 store, and if they are, I will try uploading them to the cloud player. If matched correctly (which sometimes doesn't happen), the songs will be matched to Amazon's 256kbps MP3's.

[skip252]

I don't think that 192 kbps would be considered low quality. Not the best but pretty decent. I have some older wma files at that setting and tbh I can't tell them from decent quality setting LAME files except for the lack of gapless playback. YMMV of course.

I didn't think the Amazon Cloud accepted WMA and checking it, it doesn't. I was just looking and they have to be converted to mp3 or m4a. That shouldn't matter though. It all comes back as ~256 kbps mp3. Pretty sweet deal as long as what you have gets matched.

[McDougal]

Didn't realize Amazon won't accept WMA files. As you say though, doesn't matter. I haven't listened to the files yet so I can't say about their sound quality. But true, there is a lot worse than 192k.

I'll give it a try later this week when I have the chance.