Fixing sluggish write performance of USB flash (thumb) drives

Nov 13 2012 Published by under linux, tech

    This has been noted in various places around the web but in practice what I did seems to be a combination of various writings so I have documented my own experiences here.

    Background

    I recently acquired a (yet another) USB flash drive, this a 16 GB “Dolphin” brand. The actual device reports as “048d:1165 Integrated Technology Express, Inc.” when interrogated using lsusb. I am using it to transfer transcoded Kaffeine PVR recordings from my PC to the set top box in the lounge for more comfortable watching.

    On first use, however, it took what seemed like forever to transfer a 250MB AVI file, over USB2, and looking at the GKrellM chart the write data rate appeared to be a very poor 350 kB/sec. So it seemed yet again, I needed to optimise a USB disk before it was adequate for use.

    In theory, to simplify things to one sentence, flash disk (and in particular, modern SSD) should be faster than spinning disks, as access is a true physical random access operation, without having to wait for the heads to be in the right spot. However this is invalidated due the blocky nature of flash disk writes. The actual reason for the poor write speed is that the default partition starts at the 63rd sector (byte 32256) on the disk, and USB flash drives, SD cards, etc. are designed to write data in chunks of say 128kB at a time. Even if you only write one sector, the entire 128kB (or 256 sectors) must be (re-read first and) written. So when a partition is not aligned on a 128kB boundary, more writes than otherwise necessary are required, slowing performance. USB flash drives generally employ FAT32 so they are usable on the widest variety of devices (including set top boxes) and the general experience of FAT32 is that write performance is severely affected if the partition alignment does not match the flash write size, for both the partition and the FAT master table itself.

    Procedure

    The procedure I follow for doing fixing misaligned flash drives is:

    1. Find a Linux computer, or reboot using a live Linux distribution such as SysRescueCD
    2. Destroy the existing partition.
    3. Recreate a single partition, ensuring it starts at the 256th sector (byte 131072, or 128kB)
    4. Format the partition to FAT32. with the following non default options:
      • override the default sectors per cluster to ensure clusters are aligned. This comes at some expense of apparent usable space, but the performance gain for writing large files such as video files is more than worth it.
      • Adjust the “reserved” sectors so that the FAT table itself is aligned to 128kB.

    Detailed Steps

    The following command sequence will accomplish this under Linux. This assumes your drive is at /dev/sdd, this will vary depending on what other disks you have.

    1. Run GNU fdisk with units in sector mode not cylinder mode. Then print the existing partition table (enter p when prompted. Below you can see the start sector of the existing partition is at sector 63. Note this is also a primary partition. This is typical of USB flash disks you might purchase at the local supermarket…

    2. Delete the partition:
    3. Recreate the partition, aligned at sector 256 (131072 bytes), and set the type back to FAT32 LBA (in this case matching what previously existed) (type ‘c’, or 0x0c, i.e. FAT32 LBA). Use of FAT32 LBA allows use to start the filesystem on an arbitrary sector bearing no relationship to legacy cylinders, etc. The final sector depends on the disk size.
    4. Save changes:
    5. Format the partition, setting the number of reserved sectors so that the FAT table remains aligned at a 128kB boundary. Assuming sectors per cluster, s=128 (65536 bytes), and our partition length of 31567469 sectors, we want the first fat to start at the 256th sector within the partition (which is OK as the partition itself is aligned.) For some sizes of flash disk, this can be an iterative process, but generally setting the number of reserved sectors to 256 will achieve what we want.

    6. This is the most important step – verify that the chosen number of reserved sectors has resulted in an aligned FAT table and aligned data area.


      The important figure here, is the data area sector – it must be an integer multiple of 256, and 256 x 17 == 4362 in this example.
    7. Test the result. I copied a 256 MB file onto the drive, and GKrellM is now reporting ~2.5MB/sec. More importantly, it finished in approx. one eighth of the time compared to before reformatting.

    The improved write performance should be just as noticeable from Windows.

    Be Sociable, Share!

No responses yet