[slim] Backing up ripped music

[max.spicer]

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.

This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

[Mark Bennett]

The standard recommendation for this is to buy another disk,
clone your disk onto it, take it out and store it somewhere
safe. Repeat periodically.

RAID is good at keeping systems running during disk failures,
but it's not a backup solution.

The problem with RAID solutions, is that if something goes
wrong with the OS/FS or with user error it'll probably
be replicated on both drives, and bang goes (some of) your
backup.

Post by max.spicer

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.
This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

[Mike Hanson]

The cheapest and easiest method is with extra hard drives. You have two
basic options for hosting the drive:

- External USB2 enclosure for an 3.5" IDE drive. You can find these
for $20-50US.
- Removable trays, which you can get for around $15US (bay+tray), and
additional trays cost around $10US. I use products from
www.kingwin.com (see "Mobile Rack" and "Mobile Tray"). You can use
IDE or SATA drives.


-=> Mike Hanson <=-

[Neil Davidson]

Best way (in my opinion) is to get a second drive in an external
Firewire/USB case and just copy all your data across to it then disconnect
and store the backup in a cupboard somewhere.

-----Original Message-----
From: discuss-bounces-***@public.gmane.org
[mailto:discuss-bounces-***@public.gmane.org] On Behalf Of max.spicer
Sent: 10 April 2005 11:14
To: discuss-***@public.gmane.org
Subject: [slim] Backing up ripped music

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.

This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of data.
How do you go about backing this sort of thing up? The only really feasible
thing I can think of is to buy a second hard disk and use RAID 1 (mirrored
pair of disks). After that, I just have to hope that something doesn't
happen that takes out the whole computer. Has anyone else got any workable
solutions?


--
max.spicer

The wild things roared their terrible roars and gnashed their terrible teeth
and rolled their terrible eyes and showed their terrible claws but Max
stepped into his private boat and waved good-bye

[Manuel Rathmann]

Well Max - this is what I am exactly trying to work out. The idea I had is
to buy a Buffalo Linkstation. It's an external harddrive which you can link
to your PC either via a wireless network or hardwired. Besides having an
external drive which can be in a physical different location than your PC
(in case your whole PC drowns/burns etc.) you can also run slimserver on
it. This means you do not have to have your PC running in order to use your
Squeezebox. You simply mirror whatever you have on your PC onto the
Linkstation and you're done. So you've got a backup solution AND a way
of not having to run your PC all the time.

The only thing is and that's what I am trying to find out is - the
Linkstation is not supposed to run any software on it but some clever
people found a way around this. The only thing I don't know is if this
workaround does still work with the latest Linkstations...

Anyone any idea?

Cheers,
Manuel

-----Original Message-----
From: discuss-bounces-***@public.gmane.org
[mailto:discuss-bounces-***@public.gmane.org] On Behalf Of max.spicer
Sent: Sonntag, 10. April 2005 11:14
To: discuss-***@public.gmane.org
Subject: [slim] Backing up ripped music

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.

This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

[max.spicer]

Post by Manuel Rathmann
Well Max - this is what I am exactly trying to work out. The idea I had is
to buy a Buffalo Linkstation. [...]

Oh sorry, I've hijacked your thread in that case. I hadn't understood
what a linkstation was - I assumed it was just another piece of network
kit. Maybe I should have actually _read_ your post... ;-)

[Kevin O. Lepard]

So you've got a backup solution AND a way of not having to run your
PC all the time.

Of course, the Buffalo Linkstation is just a PC, too. Running
slimserver on it is a cool hack, but if reliability is really your
goal, do you really want to hack the very device you're using to
store the data?

Kevin

[Robert Wallace]

I'm probably a bit old-fashioned. I tend to think of hard disks as
production media not suitable to off-line long term storage. I'm sure
that's no longer the case...

Any way, when I ripped my CD collection for the 4th (and final time) --
to FLAC -- I backed up the FLAC files onto DVD. Now, as I rip new
purchases, I hold a copy of the FLACs in a directory until I have enough
to fill another DVD and I burn those, too. Of course, the ulitmate
off-line backup of a CD, is the CD itself. Mine go into a box in the
basement. (belt and suspenders)

Robert

Post by max.spicer

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.
This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

[max.spicer]

How many FLAC albums do you get on a DVD on average? I keep meaning to
compile some statistics on the average size of my compressed albums,
but a quick scan shows they tend to be between 200-400MB. Nice and
accurate! ;-)

[DrRobert]

I have been trying to deal with the backup issues as well. My problems
is scale; I currently have about 24000 ripped flac tracks on hard disks
which is about half of my cds. This is currently taking up a little
over 1.0 TB on my linux box. Duping the drives would be painfully
expensive. A raid setup would be almost as expensive and you would not
be protected from OS problems (but it is Linux, so that's not likely).
I ultimately decided to back them up to DVDs. IF any single drive dies
I will have to spend an hour copying back in a bunch of DVDs, but that
is small compared to the time it took cdparanoia to rip them and for me
to get the tags correct. All total I will use less than 500 DVDs and
those will only cost about $250-$350. Since a single 400G internal
drive costs about $250 this price seems reasonable. I still have the
problem that DVD-Rs,while more stable than CD-Rs, will probably not
last all that long, particularly if you label them (which I did; 500
unlabel DVDs is the stuff of nightmares). Hopefully they will last long
enough for the new vertical storage high capacity hard drives to become
cheap. I think DVDs are the best compromise between time, cost, and
security but there is no perfect answer. I have been averaging about
15cds to a dvd when flacced.

[Robert Wallace]

Oh, around 10 to 20 - Nice and accurate ;=)

Post by max.spicer
How many FLAC albums do you get on a DVD on average? I keep meaning to
compile some statistics on the average size of my compressed albums,
but a quick scan shows they tend to be between 200-400MB. Nice and
accurate! ;-)

[radish]

Post by Robert Wallace
I'm probably a bit old-fashioned. I tend to think of hard disks as
production media not suitable to off-line long term storage. I'm sure
that's no longer the case...
Any way, when I ripped my CD collection for the 4th (and final time) --
to FLAC -- I backed up the FLAC files onto DVD. Now, as I rip new
purchases, I hold a copy of the FLACs in a directory until I have enough
to fill another DVD and I burn those, too. Of course, the ulitmate
off-line backup of a CD, is the CD itself. Mine go into a box in the
basement. (belt and suspenders)
Robert
From what I've read of recent tests, you'll probably run into problems

with the DVD media degrading before the spare HDD gives up. My personal
approach to this problem (I'm currently at 150GB and growing fast) is to
keep two complete copies (one on my main desktop PC, one on my HTPC in
the living room). Every time I rip a new CD I run a script to sync the
two up. If one of the drives was to die, the chances are I'd have time
to buy a replacement before the other one did - seeing as my current
stats are 2 dead drives in 15 years of computing :)

[Kevin O. Lepard]

Of course, the ulitmate off-line backup of a CD, is the CD itself.
Mine go into a box in the basement.

*grin* Mine are in folders behind the bed.

Kevin

[Mark Bennett]

Post by Robert Wallace
I'm probably a bit old-fashioned. I tend to think of hard disks as
production media not suitable to off-line long term storage. I'm sure
that's no longer the case...

But for me it's not really long term. Since I rerun the backup
roughly every month to catch any updates, I only need it to
last that long. If it doesn't do that, then I'm really worried
about the rest of my system.

[max.spicer]

What flac options are you using? I think I'm using -6.

[jan van mourik]

I used this in EAC (compression options -> External Compression ->
Additional command line options):
-6 --replay-gain -V -T "artist=%a" -T "title=%t" -T "album=%g" -T
"date=%y" -T "tracknumber=%n" -T "genre=%m" -T "comment=EAC
0.95prebeta5 / FLAC 1.1.0" %s

And as an example these are the sizes for Steely Dan (everybody should
have these :-)
Can't Buy A Thrill (1972) 242Mb
Countdown To Ecstasy (1973) 238Mb
Pretzel Logic (1974) 199Mb
Katy Lied (1975) 206Mb
The Royal Scam (1976) 245Mb
Aja (1977) 236Mb
Gaucho (1980) 216Mb
Two Against Nature (2000) 331Mb
Everything Must Go (2003) 257Mb ==> as WAV it takes 466Mb

jan

[radish]

I keep meaning to compile some statistics on the average size of my
compressed albums, but a quick scan shows they tend to be between
200-400MB

Wow. Mine average 500, I must have some kind of broken FLAC codec ;)
More seriously, I think it must have to do with the type of music. I
always see people saying that you get decent compression with FLAC, and
that albums come in at 300MB or so, but the majority of mine seem to be
500 or higher. I have a number around 600 - seems like very little
compression going on there. It's my personal theory (backed up by very
little evidence) that it depends on the level of dynamics within the
music.

[Phillip Kerman]

DVDs are not exactly archival. Naturally, hard drives have the issue that
your OS may not support the HD in the future. I often feel like a dung
beetle copying all my stuff from computer to computer. Anyway, I just use
HDs.



Thanks,
Phillip

[Kevin O. Lepard]

Post by Phillip Kerman
DVDs are not exactly archival

There are lots of arguments on that. Frankly, I wish MO had caught
on; that's probably the most "archival" there is. Even chiseling
something into a rock won't last forever.

Unfortunately, migrating data between old and new media isn't going
to go away, though it would be nice. I figure my DVD-Rs, properly
stored, will last long enough to be transferred to whatever replaces
them (Blu-Ray? DVD-HD? Punchcards?). Shoot, most of my 5.25" disks
that had sat in a box for 10 years worked the last time I went
through them before their ultimate disposal.

Kevin

[Phillip Kerman]

Shoot, most of my 5.25" disks that had sat in

a box for 10 years worked the last time I went through them
before their ultimate disposal.

I think that magnetic media--although it's sensitive to fields--is notably
better for long-term archiving. I believe CDs are estimated at about 10
years where magnetic is like 20. CD media is metal that will rust where
rust (on disk) won't rust. I think it's a pretty interesting topic
actually.

Thanks,
Phillip

[Jeff Coffler]

Post by radish

I keep meaning to compile some statistics on the average size of my
compressed albums, but a quick scan shows they tend to be between
200-400MB

Wow. Mine average 500, I must have some kind of broken FLAC codec ;)

Mine are much shorter as well. Here's a short sample:

225356 Billy Joel/The Stranger
297292 Billy Joel/Storm Front
621032 Billy Joel/Greatest Hits
1143716 Billy Joel/
268332 Bruce Springsteen/Darkness Of The Edge Of Town
229276 Bruce Springsteen/Greetings from Asbury Park, N.J
304836 Bruce Springsteen/Born In The U.S.A
506368 Bruce Springsteen/The River
286272 Bruce Springsteen/The Wild, The Innocent & The E Street Shuffle
1367888 Bruce Springsteen/Live 1975-85
215872 Bruce Springsteen/Nebraska
282456 Bruce Springsteen/Tunnel Of Love
231556 Bruce Springsteen/Born to Run
3692948 Bruce Springsteen/

I use maximum compression when I rip the albums.

Note that Billy Joel/Greatest Hits is 2 disks, Bruce Springsteen/The River
is 2 disks, and Bruce Springsteen/Live is 3 disks.

So, on average, I seem to do much better than your average. Check your
compression settings.

-- Jeff

[Robert Wallace]

I have a lot of CDs that were originally released on vinyl. That limits
them, in most cases, to 25-40 minutes in length. Considering the
rule-of-thumb of a bit over 10 MB / minute of audio, uncompressed and
approx 50% compression with FLAC, I generally get 200 - 250 MB (in
FLAC) for a 30 minute album, er... CD.

Newer releases *generally* have longer running times and will, of
course, take more MBs. So I'd guess. radish, that most of your CDs fall
in the "newer release" category. Of course, you can trade processing
time and get a bit better compression, if you set up your FLAC program
that way, but even the highest compression settings aren't going to get
you much over 50% (consistently) with FLAC.

Robert

Post by radish

I keep meaning to compile some statistics on the average size of my
compressed albums, but a quick scan shows they tend to be between
200-400MB

Wow. Mine average 500, I must have some kind of broken FLAC codec ;)
More seriously, I think it must have to do with the type of music. I
always see people saying that you get decent compression with FLAC, and
that albums come in at 300MB or so, but the majority of mine seem to be
500 or higher. I have a number around 600 - seems like very little
compression going on there. It's my personal theory (backed up by very
little evidence) that it depends on the level of dynamics within the
music.

[Mark Bennett]

Well for single albums, mine seem to vary between pretty small
and very large:

164M /media2/Flac/Enya/Watermark
to
689M /media2/Flac/Fish/Mixed Company

(This CD is so long that I can't rip the last track, and I've
tried 4 different DVD drives on Linux and Windows... It's also
a live recorded CD, so is probably not as "clean" as a studio
CD, I guess this could affect the compression ratio.)

Overall the largest "album" is:

1.5G /media2/Flac/Yes/Keys to Ascension

(which is a 4-disc set - I rip and tag multi-disc albums
as if they were one long album).

About 380 of 509 albums are less than 400MB, and on a first
analysis they seem to be fairly evenly distributed through
the range.

Post by radish

I keep meaning to compile some statistics on the average size of my
compressed albums, but a quick scan shows they tend to be between
200-400MB

Wow. Mine average 500, I must have some kind of broken FLAC codec ;)
More seriously, I think it must have to do with the type of music. I
always see people saying that you get decent compression with FLAC, and
that albums come in at 300MB or so, but the majority of mine seem to be
500 or higher. I have a number around 600 - seems like very little
compression going on there. It's my personal theory (backed up by very
little evidence) that it depends on the level of dynamics within the
music.

[radish]

Well, yes, most of my albums are "full", i.e. 72 mins, which will make
them larger than 40-45 mins pop albums. But, I'm still not seeing the
ratios. For example, I took a random selection of 4 tracks from an
album, and compared them at different settings:

4 tracks, total play time 16:06

WAV: 162mb
FLAC (0): 129mb (1.26:1)
FLAC (4): 122mb (1.32:1)
FLAC (8): 117mb (1.38:1)

[FYI - I use 4 as my default setting)

The best I could get for these settings is a ratio of approx 1.4:1, no
where near the touted 2:1 (50%). Looking over my collection, this is
fairly typical. I'm not suggesting other people are lying, and there's
nothing wrong with my compression settings, so that's why I'm assuming
that it's the style of music.

[Mark Bennett]

If I look at the tracks in my shortest and longest albums
I get:
h:mm Min Max
164M 0:40 Enya/Watermark 28.4% 52.8%
561M 1:18 The Stone Roses/The Very Best Of 63.6% 76.1%

(My previously reported largest single album turned out
to be a double...)

I've also got an audio book (simple speech) compressed,
and this is typically ~20%, but down as low as 16.5% in
places.

The bottom line is that the compression ratio you get is
down to the complexity of the music.

Looking at my entire collection (509 "albums", 6350 tracks)
I have 20 days, 15 hrs, 18 mins and 33 secs of music. This
works out at 299,970MB of uncompressed data.

The flac data takes up 185,871MB, which shows an average of
62%, which isn't bad.

Post by radish
Well, yes, most of my albums are "full", i.e. 72 mins, which will make
them larger than 40-45 mins pop albums. But, I'm still not seeing the
ratios. For example, I took a random selection of 4 tracks from an
4 tracks, total play time 16:06
WAV: 162mb
FLAC (0): 129mb (1.26:1)
FLAC (4): 122mb (1.32:1)
FLAC (8): 117mb (1.38:1)
[FYI - I use 4 as my default setting)
The best I could get for these settings is a ratio of approx 1.4:1, no
where near the touted 2:1 (50%). Looking over my collection, this is
fairly typical. I'm not suggesting other people are lying, and there's
nothing wrong with my compression settings, so that's why I'm assuming
that it's the style of music.

[Ken Gilmore]

Robert Wallace <rawallace-

Post by Robert Wallace
I tend to think of hard disks as
production media not suitable to off-line long term storage.

If you have an external drive, you can mirror your music
collection, then take the backup offline. Your offline drive
might only have a dozen hours on it - it should last 1000x
longer than the drive serving your music 24/7.

I do this, and also make DVD-R backups of everything.

[John Stimson]

My collection is currently 600 hours, and takes up 200 GB. It's all
FLAC except for 4 songs in AAC.

At 10MB/min (an approximation) for raw 16bit/44.1kSps audio, the FLAC
is 55.4% as big as uncompressed WAV data would be. That's about what
the FLAC website claims is typical...

[Kevin O. Lepard]

Post by max.spicer
How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

Well, my music collection is fairly static. I add a bit from time to
time but it doesn't change radically on a day to day basis.

I back up two copies to DVD-R. I like the Verbatim MediDiscs as
opposed to generics. I've had trouble with generic DVD-R. The
Verbatim MediDiscs are DICOM compliant and acceptable for use in
HIPPA applications so at the very least should tend to be among the
higher quality DVD-R media. They do cost a little more, though.

I keep one set off-site and another set in a media safe in the house.

If you're less picky and have a dual-layer driver, you'd need 10-12
disks for your collection twice that for a single layer disk. Not
too onerous by my standards, YMMV.

I also keep 3 different copies of the ripped music data on 3
different hard drives in 2 different computers in the house.

A RAID would be nice, too, but it's not really a backup. RAID vs.
backup has been discussed here before.

HTH

Kevin

[healy]

Post by max.spicer

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.
This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

My personal setup has been like so:

Linux Box with:
30gb boot hard drive
3ware raid controller with two 250gb drives in a raid 1 mirror
a single 250gb drive on a controller card

All my mp3's (currently 25,845 of them) are on the raid mirror, and
every hour a rsync script runs to backup the raid mirror to the single
250gb drive.

Recently I just had the boot drive seize on me so I've had to rebuild
the box. I took the time to move the guts into a rack mount server
case. However, I lost one of the drive bays due to the case size.
I've since moved the single 250gb drive to a firewire enclosure and
attached it to my mac. I still do the rsync backup of the mirror but
it's now going to another computer instead of self contained. A touch
safer since it's not all in the same box. However, it is in the same
room, in the same city, in the same state, in the same country...

It depends on how paranoid you really want to get about disaster
recovery.

-Healy

[Dane Jackson]

Post by max.spicer

[..] my harddrive just went up in smoke - most probaply because I

left it running for too long due to my Squeezebox.
This sort of thing scares me a lot. I reckon I'll have spent 36 hours
ripping all my music when I'm done, and there will be around 100GB of
data. How do you go about backing this sort of thing up? The only
really feasible thing I can think of is to buy a second hard disk and
use RAID 1 (mirrored pair of disks). After that, I just have to hope
that something doesn't happen that takes out the whole computer. Has
anyone else got any workable solutions?

I just arranged with one of my friends to backup my music to his computer
nightly using rsync. I'm already backing up my mail and a large portion
of my home directories. I'm thinking about sending him a spare 60 gig
drive I have lying about to mount it in, but that will probably be later.
This machine has plenty of spare capacity.

Since I'm in Seattle Washington and the machine is in Dublin Ohio, short
of thermo-nuclear warfare, my music should be safe. In the event of
thermo-nuclear warfare, I will have bigger fish to fry.