Fix for slow startup time (and possibly for firmware upgrade issues)

[bwprius]

Hi,

I want to format my 8 GB SDHC card. I have started using the program but now I am unsure what it exactly does?

I have chosen the SD card drive but reading the thread here I have doubts whether it could also erase the flash memory of my D 2.

So, my question is:

What does the Panasonic Card Format Utility exactly do? What does it erase and format?

bwprius

[CrowdAds]

[The DarkSide]

Got my 8gb D2 back w/2.54 installed - Luvin' It!!!

[mtycer]

Originally Posted by Apex_

Since the D2's internal memory shows up just like a flash drive would, you can format it with the Panasonic tool just the same. Fresh out of the box the D2 is fine, just make sure you never format it with Windows if something happens to it! If it ever refuses to turn on (i.e. gets "bricked"), never EVER let Windows try to format it. So basically don't worry about it when you first get it, since from the factory they're formatted properly.

The same goes for any SD cards you use. Format them with the Panasonic tool if you want a nice decrease in startup time on the D2.

Though I didn't have any major issues with bugs or bricking, my startup was very slow with a brand new D2 right out of the box with no Windows format, so I still think its worth using the Panasonic tool on a freshy.

[The DarkSide]

Hm, now I see what everyone means about the format tool taking ages - 8gb card will be awhile. I just want my tunes back on and the whole erase is SLOW!!! If my player is slow I may do the quick format on my D2 and see what happens as it seems it's prob recommended.

[The DarkSide]

If I reformat the internal memory w/the panasonic flash tool (all music & etc backed up) will I have to re-install the firmware or just the music and vids? This, as I know, will include all the settings & pics also (custom wallpaper, etc).

EDIT: Just did the format of the internal memory and now with the 8gb sd card + full int it boots in 3-4 circles instead of 8 (the 8 was after formatting the sd w/the Panasonic tool FIRST - also full now). Like I said - THANKS for saving me grief and giving me a reason to get my fave player back.

[petsin]

I have read carefully all what was written here and did some research both on Wikipedia and www.sdcard.org and have now some comments:

It seems to me thate there is no magic in this problem of the speed of SD card reading/writting, and that it must be related to the type of formatting. In the SDA site you can find the "official" SD formatting tool for windows (http://www.sdcard.org/about/downloads/) and, surprise!, it is the same file offered in Panasonic site.
Looking at the user's manual of the tool, you can see it will format the SD card in FAT32 and 32K cluster size providing no options to change this. However, the "default" cluster size in Win XP is 4k, which is 8 times smaller.
Well, I don't want to bore you with technical bla-bla, so I will just tell you that a cluster size of 4k is optimized for small files in average and for efficient usage of a Hard disk space, whereas a 32K cluster is much better (faster) for big size files (like audio, images and video ones). The small cluster means the read and write software needs to do a lot more operations (8x on 4K clusters) to read a large file than when using a 32K cluster.
So, as soon as I get my D2 + 16GB Transcend card next week , I will try to prove this theory by using any formatter that allows to specify the cluster size to 32K. I may even try a 64K if its available. Even the windows format.exe accepts the cluster option (/A) on the command line version.

This could be the explanation why the Panasonic/SDA formater improves also the speed on the D2 internal memory which I am sure it is NOT an embedded SD card. It is just the optimized cluster size that does the trick.

Note that Win XP can also format the card using FAT (FAT16), but this will not work in cards bigger than 2Gb a believe.

I will post my results next week.

[mtycer]

Petsin, that sounds encouraging. I hope you can get a 64k format that works. It would be nice if I could make it even faster! Will the D2 have any problem being able to read 64k clusters?

[dfkt]

You can use Windows' diskmgmt.msc to specify cluster sizes for formatting as well.

For most people it's probably easier to just use the Panasonic formatter, since it's a one-click affair, more or less.

I might test 64k clusters vs. 32k - but since there's one recommended setting for SD cards, maybe it's best to stick with that.

[petsin]

I would not assume that the 32K is the "recommended" size for SD cards. As I explained, it all depends on the average file size you intend to store in the file system. For small files, if you have a large cluster size, you will waste a lot of space because the minimum allocation unit is the cluster. Since flash cards are mostly used for BIG files, I guess 32K was chosen for being suited for the typical applications. But for a card used in a PDA, 4K cluster may be more effcient.

However, I am not sure if there is a particular problem trying 64K with the D2 firmware. In the end, the cluster size is related to the higher level software and the particular file system used (FAT12, FAT16, FAT32, NTFS, etc.) and not asociated to any particular hardware. How the hardware arranges to read/write data in the card is a complete different story. I think it is worth trying the 64K option... though I would test first formatting at 32k and compare this with the Panasonic/SDA formatting to be sure there is nothing else affecting the read/write speed.

[dfkt]

Here's some results cause I wanted to know...

Test card: 2GB SanDisk MicroSD, transferring files via a Hama 1000&1 card reader.

diskmgmt.msc: Clusters smaller than 32KB don't work when formatting to FAT16, clusters larger than 16KB don't work when formatting to FAT32.

Configuration			Small files	Large files (W/R)
-----------------------------------------------------------------
FAT "standard" cluster		58s	2s	19s	10s
FAT 32 KB cluster		58s	2s	19s	10s
FAT 64 KB cluster		60s	2s	19s	10s

FAT32 "standard" cluster	52s	2s	20s	10s
FAT32 512 B cluster 		83s	4s	22s	10s
FAT32 2048 B cluster		54s	2s	20s	10s
FAT32 8192 B cluster 		48s	2s	20s	10s
FAT32 16 KB cluster 		54s	2s	19s	10s

SDFormatter FAT 32 KB		55s	2s	19s	10s

Small files = 7.4MB, 229 files, 21 folders
Large files = 95.3MB, 3 files, 1 folder

Even if different cluster sizes don't seem to bring a lot of read/write speed improvement on Windows (although the FAT32/512B setting seriously messes up the performance) - the D2 definitely starts a lot faster after I used the Panasonic formatter instead of the Windows built-in one.

[petsin]

Interesting comparison, but using a PC with a powerful CPU and Windows XP overheads, may not show so clearly the speed differences.

However, the problem with FAT32 is that it needs a minimum of 65,526 clusters, so 32,768 * 65,526 = 2,147,155,968 which is larger than the 2,000,000,000 bytes of the card you used. Therefore, to use 32K clusters in FAT32 you need a 4Gb (>2Gb) card and for 64K clusters you need an 8Gb (>4Gb) card.

Anyway, I suspect you will see a lot more difference in access speed on the tiny processor and firmware of a player than on a desktop PC.

I wonder what the Panasonic/SDA formatter does for smaller cards, may be it just uses the largest cluster size available for the particular card (16K for 2Gb, 32K for 4Gb and 64K for 8Gb?) The example in the manual only shows a 4Gb card. I will be able to test all this when I get my large 16Gb card... which should be able to format with 64K clusters.

[Xenodius]

Dang, DFKT... respect! I wish I had enough time to do stuff like that.

Interesting results-- I think when I update the firmware on my D2, I will try the Panasonic app again. I used it once, but I only got it down to 10-11 seconds with only 8/8 + 8/16gb (SDHC) =16gb total data on there.

If 64k clusters make a significant start time difference, then I will definitely make sure my card is formatted prop :-)

[dfkt]

I just wanted to know if one could speed up SD cards with different formatting options, seems that's not the case. Windows "standard" clusters seem to be fine. I think we can assume diskmgmt.msc uses 32KB clusters for FAT16 SD cards anyways, same as the Panasonic app.

Anyways, for the D2 it's pretty simple: any standard Windows format option takes 3-4 times longer to start the player up than using the SD formatter app. Seems there's more to it than cluster sizes... ?

Sorry for the slight off-topic.

[petsin]

Originally Posted by dfkt

... the D2 definitely starts a lot faster after I used the Panasonic formatter instead of the Windows built-in one.

dfkt, did you notice any startup speed difference on the D2 when using the 2Gb card with 16K clusters compared to the "standard" 4K clusters?

And do your remember what the SD formatter said after finishing formatting your 2Gb card? it should display the file system and cluster size used according to the manual. I wonder what was the cluster size used in that case... 16Kb?

Then may be there is really something else in the way the card is formatted by the SD formatter compared to what windows does, but I can't imagine what could that be...

[dfkt]

As I wrote above, Windows won't let me format the card with FAT16 clusters smaller than 32KB (and FAT32 larger than 16KB). The Panasonic formatter uses 32KB clusters, presumably the same as Windows' "standard" setting - since I can only choose between 32K and 64K for FAT16. And Windows XP complains when choosing 64K that it will not be compatible with older versions of Windows. Maybe in old Windows versions you could choose 4K clusters, and they fixed it in the meantime?

[dfkt]

Ok, some real-life on-topic results. Card formatted in the D2's SD slot, and the D2 used as card reader for transferring files. I put 127 MP3s (560MB) on the card. That's not so many, but it still shows that there's a HUGE difference between the Panasonic formatter and the Windows formatter. Something that wasn't obvious with the high-speed Hama reader, but very obvious with the lame D2 card slot.

Format		Write	1st Startup	Normal Startup
------------------------------------------------------
Pana FAT 32K  	3:03	8s		5s
Win FAT 32K	5:00	11s		7s
Win FAT32 8K	5:37	11s		7s

Transfer speed is incredibly fast with the proper formatter, almost twice as fast as with the same settings on the Windows formatter. Seems the D2's card slot is quite picky compared to my Hama reader. 1st and normal startup times aren't that different, but it sure would show better on a full 8/16GB card. (No, I really don't want to wait hours to format and fill up my 8GB card... ) Just stating the obvious, as we all know... but seems there is more than just the cluster size to it.

FWIW, my full 8GB Kingston card takes 30 seconds for the 1st startup and 5 seconds for a normal one as well. IIRC, it took well over a minute before I formatted it with the Panasonic app.

[petsin]

dfkt, I agree with you that there must be something more than the cluster size. The fact that both formatters using the same type and cluster (FAT16 and 32K clusters) give different behaviour on the D2 is a good proof. I did some tests on a 2GB SD card I have, and though I could see some performance improvement when using large clusters (32K vs 4K), the difference was not that much on the PC as you also found.

I tried the SD formatter to see what would be the default formating for a 2GB card, and indeed it switches to FAT16 instead of FAT32, but keeps the cluster size to 32K. And on a 128Mb card, the SD formatter would use a 16K cluster, indicating that it will try to use the largest cluster possible (Win XP would use a 2K cluster for this card size). It seems that FAT32 is only used on SDHC cards (i.e. 4Gb and up).

However, in both ocassions that I used the SD formatter, windows would not recognize the card after formatting it until I removed and reinserted it, showing that there must be something else different as this did not happen when I formatted the card with windows, using the same FAT and cluster parameters. I will search for more info. in the web and will post any findings.

As for your previous post describing the allowed cluster sizes for FAT16 and FAT32, the values you found are particular for a 2Gb card. FAT16 uses only 16 bits for counting clusters, so the card (disk) can not have more than 65,536 clusters, so that means 30K would be the smallest cluster size for your card under FAT16. On the other hand, FAT32 uses 32 bits to count clusters, so it can have up to 4,294,967,296 clusters (but it is limited to 4,177,918 for other reasons), so there is no limit on the minimum size of the clusters for your card, but the specification of FAT32 says that the minimum number of clusters must be 65,526 which means you can not use a cluster size bigger than 8K in a 2Gb card using FAT32. Old Win 95 did not recognize clusters of 64K, and thus the message you got. The current cluster size supported by FAT16 and FAT32 is 512, 1024, 2048, 4096, 8092, 16K, 32K and 64K. Larger size like 128K and 256K can be used when sector size is bigger than 512 bytes, which is the default for most storage media.

[petsin]

Yesterday, searching the web I found something that could be related to this mistery: The FAT system (16/32) includes a safety feature which creates a duplicate or backup FAT (file allocation table) on any FAT16 or FAT32 file system. However, this affects the performance as the OS needs to update both FATs on every access. However, the standard allows to have only one FAT if desired producing a gain in performance over the FAT with backup. Now, I need to find a tool that allows to control the creation of the backup FAT (Windows format.exe does not allow to disable the backup FAT creation) and compare performances against the SDFormatter and see whether this is the trick it uses...

For more info. here are two articles that discuss how to optimize flash cards and bring up the issue of the backup FAT table:
http://www.pocketpcmag.com/cms/blog/...d-tests-a-must
http://pocketpcmag.com/forum/topic.asp?TOPIC_ID=17921

[dougisfunny]

I haven't tested it yet, but I was looking into what you were describing.

mkdosfs is a linux tool that formats stuff, lots of options. The one option in particular you might be interested is the -f option

-f number-of-FATs
--------------
Specify the number of file allocation tables in the file system.
The default is 2. Currently the Linux MS-DOS file system does
not support more than 2 FATs.

Also, there is a windows port of it, you can find here.

Of course there are also other options for selecting the cluster size.

http://www1.mager.org/mkdosfs/

[dougisfunny]

So I tested it on a Hard Drive partition, as I don't have a flash card of any size yet. The utility is able to do either 1 or 2 FATs, also it allowed me to do 64k clusters.

When I get my D2 I'll actually test on a SDHC card and try the various things.

I'll also look up what differences the SD formatter makes.