android
  #1  
Old 06-24-2012, 06:02 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default Way to eliminate duplicates in long Playlist file?

Re: Sana e280 v1 / Rockboxed

Hello,

I have some very long playlist files which I keep adding files to.
I noticed that there are some duplicates (tracks which I didn't realize I had already added, and added again). Has anyone found a way to locate an then remove duplicates in long playlist files (other than manually doing so? Since the entries are not sorted it would be hard to see where dupes are).

Is there any program (like Sansa Playlister) which can create playlists from my computer (in MSC mode)? Playlister was great but it creates *.PLA files which do not run in rockbox. I loved that program! I am liking rockbox a lot, but miss the functionality of Sansa Playlister.

Thanks for your help!
Reply With Quote

Advertisement [Remove Advertisement]

  #2  
Old 06-25-2012, 07:17 AM
JSP62's Avatar
JSP62 JSP62 is offline
Member
 
Join Date: Aug 2006
Location: Boston, MA, USA
Posts: 232
Default

Best thing I can recommend is the good old "Find & Replace" command in word or notepad.
__________________
HOW TO ASK A QUESTION THE RIGHT WAY
POSTING AND YOU
------------------------------------------
With knowledge, comes great power.
With power, comes great responsibility.
With responsibility, comes great wisdom.
Reply With Quote

  #3  
Old 06-25-2012, 07:37 AM
kp-j3's Avatar
kp-j3 kp-j3 is offline
Member
 
Join Date: Nov 2011
Posts: 295
Default

It's probably something you'd have to write yourself that essentially iterates through every entry in the playlist, searches for that string, and then captures those entries that occur more than once.

A fairly simple prospect using *nix tools, but I'm sure it's doable in powershell and with some histrionics in dos.
__________________
Daily drivers - Cowon J3 32GB/64GB microSD, Sennheiser hd202s
Others - Cowon D2+ 8GB, Iaudio 7 8 & 16GB, Sandisk m260, Samsung yp-r0 8GB, SGP 5.0
Reply With Quote

  #4  
Old 06-25-2012, 07:54 AM
JSP62's Avatar
JSP62 JSP62 is offline
Member
 
Join Date: Aug 2006
Location: Boston, MA, USA
Posts: 232
Default

Quote:
Originally Posted by kp-j3 View Post
It's probably something you'd have to write yourself that essentially iterates through every entry in the playlist, searches for that string, and then captures those entries that occur more than once.

A fairly simple prospect using *nix tools, but I'm sure it's doable in powershell and with some histrionics in dos.

Agreed. Like a batch file type dos command, right?
__________________
HOW TO ASK A QUESTION THE RIGHT WAY
POSTING AND YOU
------------------------------------------
With knowledge, comes great power.
With power, comes great responsibility.
With responsibility, comes great wisdom.
Reply With Quote

  #5  
Old 06-25-2012, 09:24 AM
kp-j3's Avatar
kp-j3 kp-j3 is offline
Member
 
Join Date: Nov 2011
Posts: 295
Default

Quote:
Originally Posted by JSP62 View Post
Agreed. Like a batch file type dos command, right?
Right, but string manipulation is kind of painful in dos. That's why I use bash, perl, etc. I actually had written something similar to check duplicate folders on internal storage vs. sd card for my j3, but I can't seem to find it now.
__________________
Daily drivers - Cowon J3 32GB/64GB microSD, Sennheiser hd202s
Others - Cowon D2+ 8GB, Iaudio 7 8 & 16GB, Sandisk m260, Samsung yp-r0 8GB, SGP 5.0
Reply With Quote

  #6  
Old 06-25-2012, 11:23 AM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Thanks for all your replies, to my first question.

My second question was:

Is there any program (like Sansa Playlister) which can create playlists from my computer (in MSC mode)? Playlister was great but it creates *.PLA files which do not run in rockbox. I loved that program! I am liking rockbox a lot, but miss the functionality of Sansa Playlister.

Does anyone know of a similar program to Sansa Playlister that works with rockbox?

Thanks!
Reply With Quote

  #7  
Old 06-25-2012, 11:28 AM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Re: finding duplicate entries in a rockbox playlist

I was able to open the playlist in notepad, copy ALL, and then paste the list into Excel.
Thankfully, each line of the playlist is entered into a separate cell down the column of the spreadsheet. If you then do DATA | SORT, the column is put into alpha order and you can at least quickly see where you have duplicates. You still have to go back to notepad and manually remove one of each duplicate, but at least it helps locate duplicates quickly.
Reply With Quote

  #8  
Old 06-25-2012, 11:50 AM
Tapeworm's Avatar
Tapeworm Tapeworm is offline
Member
 
Join Date: Jun 2008
Location: California, the Granola State
Posts: 481
Default

Quote:
Originally Posted by dma44 View Post
Is there any program (like Sansa Playlister) which can create playlists from my computer (in MSC mode)? Playlister was great but it creates *.PLA files which do not run in rockbox. I loved that program!
While not a dedicated playlisting program, Winamp does a bang-up job of creating .m3u format playlists.
Reply With Quote

  #9  
Old 06-25-2012, 11:52 AM
skip252 skip252 is offline
Administrator
 
Join Date: Dec 2007
Location: Chicago
Posts: 5,204
Default

Any software that creates .m3u or .m3u8 playlists can be used to create playlists for a Rockboxed player. I use foobar2000 after setting it the way marc2003 outlines here. The setup seems complex but if you follow the steps properly it only needs to done once. After that you just place the files in a playlist and save it. The paths will be right for the playlists to always work.

You can edit your existing list that has duplicates easily using foobar2000 if all the files are on a single drive. Just open the playlist and use the Remove Duplicates function and save it again. If the files are on both the internal and external memories you'll need to correct the file paths for foobar2000 to see the files from both memories.

Rockbox saves the path to the external memory as /<micoSD1>/. If you open the playlist in a text editor and do a Find and Replace to change that to \:your drive letter\ foobar2000 should see all the files and then be able to remove any duplicates.

One thing, foobar2000 sees duplicates as the same file in the same location. If the duplicate entries are in different locations or have different extensions they aren't seen as duplicates and won't be removed.
Reply With Quote

  #10  
Old 06-25-2012, 12:41 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

skip252,
Thanks for letting me know about foobar2000.
I hadn't heard of it.
I will study it and give it a try.
I have a question about what your wrote:
Rockbox saves the path to the external memory as /<micoSD1>/. If you open the playlist in a text editor and do a Find and Replace to change that to \:your drive letter\ foobar2000 should see all the files and then be able to remove any duplicates.
Once you find and replace /<micoSD1>/ to become \:U\
[where drives are T: and U:]
is that just a temporary edit, just while running foobar2000?
I.E. If you saved that, would the playlist still run in rockbox?

Also I noticed that you changed front slashes (/) to backslashes (\).
Do you have to do that for the other slashes in each track's listing as well?
Otherwise you'd have both front and backslashes in same track entry.

(i.e. \:U\ArtistName1/AlbumName1/Trackname1.mp3)

Thanks for the help.
Reply With Quote

  #11  
Old 06-25-2012, 12:43 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Tapeworm,

Thanks for your suggestion of winamp. But I'm assuming you'd have to import all the music into a winamp library. I'm hoping to find a program that will not require this so you can create playlists on the computer with the e280 plugged into usb -- working directly with the tracks on the player.
Reply With Quote

  #12  
Old 06-25-2012, 02:26 PM
Tapeworm's Avatar
Tapeworm Tapeworm is offline
Member
 
Join Date: Jun 2008
Location: California, the Granola State
Posts: 481
Default

Quote:
Originally Posted by dma44 View Post
Tapeworm,

Thanks for your suggestion of winamp. But I'm assuming you'd have to import all the music into a winamp library. I'm hoping to find a program that will not require this so you can create playlists on the computer with the e280 plugged into usb -- working directly with the tracks on the player.
Not really. Just set up Winamp to "watch" your Player and/or card drives as the default. Piece of cake as they say.
Reply With Quote

  #13  
Old 06-25-2012, 02:50 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Tapeworm,

And does winamp let you choose tracks from both locations (internal & external) into the same playlist?

Also, with Rockbox, do you need to shut down before inserting or removing an external micro sdhc card?

Thanks.
Reply With Quote

  #14  
Old 06-25-2012, 03:17 PM
skip252 skip252 is offline
Administrator
 
Join Date: Dec 2007
Location: Chicago
Posts: 5,204
Default

The only path corrections I've found necessary for foobar2000 to read the playlist is the one I listed. After that fb2k was able to read the rest of the playlist.

When I saved it back to the playlist folder I use the marc20003 setup I have as my standard and the playlists work properly. You may want to experiment with saving the playlists in different places if you don't want to bother with that setup. I use that setup because it suits my needs, your may quite possibly be different.

Rockbox will strip away drive assignments to different levels depending on the save location of the playlist. Playlist saved directly to the root of the external card as .m3u or .m3u8 using foobar2000, Winamp, XMPlay, MusicBee or even (ugh!) Windows Media player should work just fine with no additional editing.

That's one of the reasons I never found a use for a specialized program like the one you mention to make playlists for Rockbox. Any playlist maker that can create a .m3u playlist works. If you really want to you could make your playlists with a text editor.

BTW, there's no need to import anything into the Winamp library to use it as a playlist maker. Just drag the files into the Playlist Editor, arrange them the way you want, then save the playlist in the root of the external card. The OF won't play it but Rockbox will.

I strongly suggest experimenting with various playlist makers. foobar2000 suits my needs. Before that I used XMPlay and did a simple find and replace in a text editor to fix file paths. I mentioned Windows Media Player because I have a friend that manages a Rockboxed Fuze with that and loves it. You couldn't beat me with a stick and make me use WMP but it works for them.

For me a big part of the attraction of rockbox is that it frees me to manage my music in a way that works for me. I think a with a bit of experimentation it shouldn't be tough for you to find what works best for you.

Edit:

I insert and remove cards on my Rockboxed players on the fly. I make sure it's not being read as part of a playlist or attempt it as I'm accessing the files on the player but other than that I exchange them without shutting down. When I exchange a card I get a brief message it's reading the card and it's ready to go.

Last edited by skip252; 06-25-2012 at 03:25 PM.
Reply With Quote

  #15  
Old 06-25-2012, 03:53 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Thank you Skip252.
I appreciate your detailed explanation.
I installed foobar2000 and followed the instructions and it has worked as you said.

In terms of inserting/removing micro cards while rockbox is running you said:
I insert and remove cards on my Rockboxed players on the fly. I make sure it's not being read as part of a playlist or attempt it as I'm accessing the files on the player but other than that I exchange them without shutting down. When I exchange a card I get a brief message it's reading the card and it's ready to go.
Could you please explain how I make sure it's not being read as part of a playlist. I am usually playing some playlist. Are you saying that if the loaded playlist contains any files from the card, then don't remove it?
Would I have to load a different playlist that did not have any tracks from the external card before removing it?

Thanks so much. You have helped immensely.
Reply With Quote

  #16  
Old 06-25-2012, 05:05 PM
dma44 dma44 is offline
Junior Member
 
Join Date: Jul 2008
Posts: 10
Default

Skip252,

Actually, I just tried to open a playlist in Foobar2000 that had one track from internal and one from external.

You had written:
Rockbox saves the path to the external memory as /<micoSD1>/. If you open the playlist in a text editor and do a Find and Replace to change that to \:your drive letter\ foobar2000 should see all the files and then be able to remove any duplicates.
But the only way I could get foobar to "see" the external track was to have replaced the /<microSD1>/ with X:\
(not \:your drive letter\)

But it does work. And the "remove duplicates" function is awesome.
Reply With Quote

  #17  
Old 06-25-2012, 05:58 PM
saratoga saratoga is offline
Rockbox Developer / Moderator
 
Join Date: Apr 2007
Posts: 3,604
Default

Quote:
Originally Posted by dma44 View Post
Skip252,

Actually, I just tried to open a playlist in Foobar2000 that had one track from internal and one from external.

You had written:
Rockbox saves the path to the external memory as /<micoSD1>/. If you open the playlist in a text editor and do a Find and Replace to change that to \:your drive letter\ foobar2000 should see all the files and then be able to remove any duplicates.
But the only way I could get foobar to "see" the external track was to have replaced the /<microSD1>/ with X:\
(not \:your drive letter\)

But it does work. And the "remove duplicates" function is awesome.
<microsd1> is a rockbox thing, windows uses drive letters instead.

A few days ago I changed the playlist code a bit so that if your files are all on the SD card, and you put the playlist there, you can skip the <microSD1> part and just use a windows drive letter and rockbox will figure out what you mean. If you want to have files on both the internal and external memory, you'll need the <microSD1>, though hopefully this can be fixed in the future.
__________________
Interested in Google's Summer of Code ? PM me.
Reply With Quote

  #18  
Old 06-25-2012, 06:25 PM
skip252 skip252 is offline
Administrator
 
Join Date: Dec 2007
Location: Chicago
Posts: 5,204
Default

Seems my save paths are different. I have the the leading "/" that your playlist didn't. No matter, you have it working and that's the most important part.

As far as changing the card I use the context menu that comes up with a long Select and check the playing song's properties. If the playing file isn't from the card I'll change the card and start playback of a new playlist.

I've found that if I let the playlist continue it may play normally skipping any missing files or it may throw an error message about not being able to access the playlist control. If I start a new list to playing, playback continues without interruption.
Reply With Quote

Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump



All times are GMT -5. The time now is 09:43 PM.