Using iTunes backup and restore can cause thousands of your music tracks to be lost.
The procedure for transferring your iTunes music library from your old Windows PC to your brand new iMac is very simple:
- On the PC, use the Backup command in iTunes to write your 5,016 songs onto 6 DVDs.
- On the iMac, insert each DVD in turn and let iTunes restore all the songs.
- Enjoy your songs, all 3,679 of them.
If you compare step 3 with step 1, you will see one snag: only 73% of your songs have survived the process. More than a quarter of your music library has been silently lost.
Step by step: what happened
- When I import my CDs into iTunes, iTunes automatically retrieves the track names from the Internet. In some cases these contain semi-colons: for instance, one artist name in my collection is “Eugene Ormandy; Philadelphia Orchestra”.
- When iTunes backs up my music to DVD, it creates folder names from the artist and album names: “22 Eugene Ormandy; Philadelphia Orc”.
- MacOS Leopard is unable to open any folder on DVD whose name contains a semi-colon. It appears in Finder as a white document rather than a blue folder, and double-clicking it results in a message saying “The operation could not be completed. An unexpected error occurred (error code -43)”.
- When you use iTunes on MacOS to restore music from a backup, and it is unable to read a folder from the DVD, it stops reading from that DVD and asks you to insert the next one. It does not report any error, and it does not attempt to read any further folders from the DVD, even though there may be hundreds of readable files still left to go.
The net result is massive data loss in a backup-and-restore procedure that uses Apple software throughout. There is thus no argument about who is responsible.
This bug is serious because it silently destroys data.
Working round the bug
You need to make sure that none of your artist, album or track names have semicolons in them. To do this, search for a semicolon in iTunes and manually edit every name. In my case there are only 215 tracks that I have to edit.
Then throw away your old backup DVDs and make a new backup.
If you do not have access to your original iTunes library then you will have to get hold of a Windows PC (a Mac, at least a Leopard Mac, will not work). Copy the entire backup DVD to the hard disk. Rename all files and folders that have semicolons in their names. Burn a new DVD. You should also edit the “contents.xml” file by hand if you want your playlists to continue working.
What the bugs are and how to correct them
MacOS Leopard cannot read valid DVDs
If a semicolon is allowable as a character in folder names on a DVD then the MacOS software must be corrected so that such folders can be opened. Note that there is no problem with folders on your hard disk containing semicolons: the problem only arises with DVDs.
Note also the necessity of conforming to usage as well as to standards: if Windows has no problem reading a DVD folder that has a semicolon in its name, mere standards compliance is not a very good excuse for MacOS to fail to do so.
iTunes Backup is not writing valid DVDs
If, on the other hand, you insist that semicolons aren’t allowed in folder names on DVDs, why on earth is iTunes Backup writing them? A backup program that writes unreadable backups is worse than no backup program at all.
iTunes Restore silently discards readable music tracks
If you are restoring from a list of folders on a DVD, the logical thing to do if you can’t read one folder is to go on to the next one. iTunes Restore does not do this. Once it has failed to read “22 Eugene Ormandy; Philadelphia Orc”, it ignores everything else on the DVD, from “23 Falla” to “37 Gesualdo”. Of the 1,337 music tracks that were lost by the backup-and-restore process, 1,122 were lost for this reason.
iTunes Restore does not report unreadable music tracks
If you are restoring from a DVD and you fail to read a folder, you should tell the user. But with iTunes Restore there is no indication that anything at all is wrong. If I hadn’t happened to try playing a particular track soon after the restore I might never have known about the problem until I had got rid of my PC’s iTunes library: and then it would have been too late.
This is quite simply bad software engineering. Even if the MacOS DVD driver is changed so that semicolons are allowed, it is vital that iTunes should report any errors that may arise in the future. Nothing is worse than a Restore function that pretends to have restored everything when it hasn’t.
Finder doesn’t report errors
Take your DVD and use Finder to copy it to your desktop. Finder will take a while over the process and you’ll think that you have done it successfully; but look at the desktop copy of the DVD (which you know is identical to the DVD, because you just copied from it and there were no errors reported) and you’ll see that all semicolon-containing folders were silently ignored. This is another piece of bad engineering. If you destroy or lose data you must always let the user know.
Is Apple strangling itself?
Almost all software has bugs. A vital part of any software developer’s nervous system is the proprioceptive flow of information from the periphery back to the centre. People who can’t feel pain end up losing limbs because of it: small injuries, abscesses, infections aren’t noticed until it’s too late and the body rots. Software producers who can’t feel their users’ pain are deprived of vital information about bugs and problems, so they fester unnoticed until the software products rot away.
The trouble with facilitating the flow of information from the periphery to the centre is that so much of it is nonsense. The challenge is to get the real bugs to the software developers while filtering out the “But where is the ‘any’ key?” queries early on.
With Cardbox we charge subscriptions for technical support and we answer every question: it costs us time but we know we’ll never miss anything. But then we’re small and so we can afford to do this. When you have millions of users the economics are very different.
Apple, like Microsoft, provides no way for the public to report bugs. This saves a good deal of work but ultimately – as the heading suggests – it leads to starvation.
What are the alternatives? You can post to a discussion forum and hope that a bored engineer happens to notice that (hidden in the verbiage or the misunderstanding or whatever) there is something that seriously needs attention.
You can give your telephone technical support people the ability to report bugs: they will have filtered out the silly questions, so the quality of the information coming into the centre will be better and the quantity will be manageable. The charming Irishman I talked to on the phone this morning promised to submit a bug report.
You can rely on the blogosphere to flag up the issues it finds interesting. If enough people riot then you do something; if not, not. This harnesses the wisdom of crowds (and all the rest) but it wrecks your reputation as a responsible supplier.
There is no easy answer to the question of information flow. But all the same, I hope that Apple manages to correct this bug. That iMac is so beautiful, it’s not fair that it should be so destructive.