External Raid Setup and video?

[macunkie]

I'm upgrading my "old" 533 Mhz G4 Dp with a FireWire 800 PCI card:
http://www.mac-pro.com/s.nl/sc.2/cat...it.A/id.463/.f

and an External Raid system, http://eshop.macsales.com/shop/firewire/1394/USB/EliteAL/RAID/, for system backup as well as storing video content, (mostly .avi files) to serve over my net work to my EyeHome system.

Can any one recommend a basic "format" for the raid system to meet my needs of both backing up content from my 533 Mhz system as well as serving the video content?

All the "striping", and "mirroring" stuff is a bit confusing?

Any bottom line setup advice?

Thanks,

 

[Danksi]

I'm upgrading my "old" 533 Mhz G4 Dp with a FireWire 800 PCI card:
http://www.mac-pro.com/s.nl/sc.2/cat...it.A/id.463/.f

and an External Raid system, http://eshop.macsales.com/shop/firewire/1394/USB/EliteAL/RAID/, for system backup as well as storing video content, (mostly .avi files) to serve over my net work to my EyeHome system.

Can any one recommend a basic "format" for the raid system to meet my needs of both backing up content from my 533 Mhz system as well as serving the video content?

All the "striping", and "mirroring" stuff is a bit confusing?

Any bottom line setup advice?

Thanks,

For this particular unit:
Striped = RAID0, two drives are configured as one 'faster' drive. If one drive fails, ALL data is lost.

Mirrored = RAID1, two drives are configured to mirror or copy each other. If one drive fails, in theory you still have your data on the remaining drive. I'm not entirely sure whether this 'slows' the drive down any.

I'm considering the same enclosure for backup purposes. I understand that this unit comes in 'Striped' format as standard and that you have to run a configuration program to convert it into two separate drives again, then use the OSX disk-utility RAID configuration tool to create a RAID1/Mirrored setup.

In terms of streaming media across your network, I can't really comment, as I've not really been successful in making a Firewire drive available to other machines (a Windows PC) across my home network.

 

[macunkie]

Thanks Danksi!

For this particular unit:
Striped = RAID0, two drives are configured as one 'faster' drive. If one drive fails, ALL data is lost.

Mirrored = RAID1, two drives are configured to mirror or copy each other. If one drive fails, in theory you still have your data on the remaining drive. I'm not entirely sure whether this 'slows' the drive down any.

I'm considering the same enclosure for backup purposes. I understand that this unit comes in 'Striped' format as standard and that you have to run a configuration program to convert it into two separate drives again, then use the OSX disk-utility RAID configuration tool to create a RAID1/Mirrored setup.

In terms of streaming media across your network, I can't really comment, as I've not really been successful in making a Firewire drive available to other machines (a Windows PC) across my home network.

My 500gb raid drive is on it's way, when I get it hooked up I'll post to my "trial and error" attempts at my setup.

Thanks again,

 

[sushi]

My 500gb raid drive is on it's way, when I get it hooked up I'll post to my "trial and error" attempts at my setup.

I believe that this means you have two 250GB HDs in RAID0 (Striped) setup.

If you decide to use RAID1 (Mirrored) your capacity will drop to around 250GB. This sounds good for data redundancy. However, I have had Mirror drives get out of sync and loose data. So there is always a risk.

Regardless of which RAID configuration that you use, backup your RAID array at least once per week to another Hard Drive. More often if you have critical data. Of course this solution requires you to purchase at least two HDs for backup which is a bit expensive.

 

[Danksi]

However, I have had Mirror drives get out of sync and loose data. So there is always a risk.

Was this using software (i.e. OSX Disk Utility) or a hardware based RAID controller?

This is my concern, that even a RAID1 would get messed up, requiring another backup drive anyway.

I wanted to remove a load of old files from my iMac's main drive, but being a pack-rat, not risk loosing them. There's too much for a DVD burn really so hard drive was my preferred choice.

Perhaps the better option for me would be to use two separate drives and configure a copy of SilverKeeper to run scheduled daily/weekly backups, to alternating external drives. That way I still have two drives worth of 'critical' data and a backup of the latest 24-hours worth of work. One drive could be hidden somewhere, just in case some thieving rat-bag pays a visit.

 

[whocares]

FWIW:

RAID != Backup


RAID are designed to be used as rapid, redundant (or both) *live* disks. If you screw with your files on a RAID 0 or 1 (or any other type of RAID for that matter) you loose your data... A true backup is on a non erasable medium with long life expectancy, stored in a different location than the original. To this day, CDs seem to be the best backup solution. RAID systems are used for editing video (high speed drives) or typically on servers for on-the-fly "backups" (i.e. redundancy).

My suggestion to you is: make a true backup of all your files, and then serve them from a standard hard drive - no RAID required.

 

[sushi]

Was this using software (i.e. OSX Disk Utility) or a hardware based RAID controller?

This is my concern, that even a RAID1 would get messed up, requiring another backup drive anyway.

I wanted to remove a load of old files from my iMac's main drive, but being a pack-rat, not risk loosing them. There's too much for a DVD burn really so hard drive was my preferred choice.

Perhaps the better option for me would be to use two separate drives and configure a copy of SilverKeeper to run scheduled daily/weekly backups, to alternating external drives. That way I still have two drives worth of 'critical' data and a backup of the latest 24-hours worth of work. One drive could be hidden somewhere, just in case some thieving rat-bag pays a visit.

I was using the Mac OS X Drive Utility. Early X.1 and X.2. Had too many problems and gave up. I might try X.4 and see if it is any better.

I am currently looking at a few cards. One is from Sonnet Tech.

A couple of my friends used dedicated RAID devices and had some problems as well.

I think that the key is to make backups of your data. That is the problem with large amounts of data. How do you routinely back it up?

Personally, I back up to external FW drives. I alternate between two. Works well.

I would hate to use DVDs let alone CDs to make my backups each week/day depending on my needs. A 250GB backup on CDs would take a couple of huge stacks. At least with DVDs it would take a few less -- about one sixth.

 

[macunkie]

I was using the Mac OS X Drive Utility. Early X.1 and X.2. Had too many problems and gave up. I might try X.4 and see if it is any better.

I am currently looking at a few cards. One is from Sonnet Tech.

A couple of my friends used dedicated RAID devices and had some problems as well.

I think that the key is to make backups of your data. That is the problem with large amounts of data. How do you routinely back it up?

Personally, I back up to external FW drives. I alternate between two. Works well.

I would hate to use DVDs let alone CDs to make my backups each week/day depending on my needs. A 250GB backup on CDs would take a couple of huge stacks. At least with DVDs it would take a few less -- about one sixth.

There are times when backing up to CDs or DVDs may be benifical like storing "critical data" at a location separate from one's main data, but I agree, daily backup needs are not for this medium.

My 533 Mhz G4 is a dedicated "media" and "web" server. In terms of a "media" server. I've been listenting to my iTunes collection, (close to 18 gigs of tunes), via this server and wireless to my stereo for sometime. I've since discovered EyEHome and have been slowly building a collection of Xvid (avi) movies to be served via EyeHome and wireless as well. The Raid solution is going to be dedicated to the "movie" storage end of my server needs. I already have a 80GB Firewire 400 external drive that I will dedicate to backing up my other routine, "more critcal" data.

So again, my initial inquiry as to how to set up a Raid drive is relative to that drive being dedicated to serving video content.

Thanks for all the great input!

 

[whocares]

So again, my initial inquiry as to how to set up a Raid drive is relative to that drive being dedicated to serving video content.

Thanks for all the great input!

A RAID 0 (ie stripped) strips the data between two (or more drives). This has 2 advantages:
1. No loss of HD space
2. Faster read and write speeds (writing a 200 MB file is more a less like writing a 100 MB file)
And 1 huge drawback:
1. If either drive crashes you loose everything (ie a RAID 1 is 2x more liable to crashing than a normal drive)
Primary use: video editing.

A RAID 1 (mirrored) mirrors the data between 2 drives, ie both drives have the exact same data content. This has a 1 huge advantage:
1. Redundancy. If one drive crashes, the other is there to restore the data.
And 1 big disadvantage:
1. You loose half your storage space.
Primary use: servers.

You can also build a stripped+mirrored RAID using 4 drives which combines the advantages of RAID 0 and RAID 1. Other more exotic RAIDs also exists, but are more difficult to set up and require dedicated hardware solution.


I suppose in you case, RAID 1 (mirrored) would be the solution as the extra speed won't be needed. Again, if you have backups of your data, don't bother with RAID, just use a standard HD.

 

[macunkie]

A RAID 0 (ie stripped) strips the data between two (or more drives). This has 2 advantages:
1. No loss of HD space
2. Faster read and write speeds (writing a 200 MB file is more a less like writing a 100 MB file)
And 1 huge drawback:
1. If either drive crashes you loose everything (ie a RAID 1 is 2x more liable to crashing than a normal drive)
Primary use: video editing.

A RAID 1 (mirrored) mirrors the data between 2 drives, ie both drives have the exact same data content. This has a 1 huge advantage:
1. Redundancy. If one drive crashes, the other is there to restore the data.
And 1 big disadvantage:
1. You loose half your storage space.
Primary use: servers.

You can also build a stripped+mirrored RAID using 4 drives which combines the advantages of RAID 0 and RAID 1. Other more exotic RAIDs also exists, but are more difficult to set up and require dedicated hardware solution.


I suppose in you case, RAID 1 (mirrored) would be the solution as the extra speed won't be needed. Again, if you have backups of your data, don't bother with RAID, just use a standard HD.

Since I've already upgraded my system with a FireWire 800 Pci card and am awaiting shipment of the 500gb Raid drive, based on what you have described, I should go with the RAID 1. So I've cut my 500GB's to 250GB, but my collection of Xvid's are small for now and I can add more storage later. It takes some work "converting/backing up" my Dvd's to a single compressed avi file and the idea of having it lost with the RAID0 configuration doesn't sound good. I'm not worried about the speed too much. I'm guessing that even in the RAID 1 config. my system will be accessing files a bit faster than the current FireWire 400 external drive I'm currently testing with.

Thanks again!

 

[sushi]

There are times when backing up to CDs or DVDs may be benifical like storing "critical data" at a location separate from one's main data, but I agree, daily backup needs are not for this medium.

Understand your point.

However, I just back up to an external HD and store it in a different place. I rotate between external HDs.

If your data changes often, trying to keep a current backup via burning CD/DVDs is hard and it is easy to loose interest in maintaining your backups. With external HDs having large capacity and being small and easy to carry, I find that a better solution.

I have one caviat to the above. When uploading pictures of an event, I burn to a CD/DVD each time. That way my original pics are backed up.

 

[macunkie]

However, I just back up to an external HD and store it in a different place. I rotate between external HDs.

By rotating do you mean saving the same data on different external HDs for duplicate copies of data?

If your data changes often, trying to keep a current backup via burning CD/DVDs is hard and it is easy to loose interest in maintaining your backups. With external HDs having large capacity and being small and easy to carry, I find that a better solution.

I agree with loosing interest in maintaining backups via CD/DVDs. I'm just not willing to keep a "box" of data in a tangible format. Same reason I have ripped all of my audio cd's to serve or burn at will.

I have one caviat to the above. When uploading pictures of an event, I burn to a CD/DVD each time. That way my original pics are backed up.

I do like the concept of backing up original photos. I wish I had done this years ago. One's need for editing a particular image may change due to relative needs.

 

[sushi]

By rotating do you mean saving the same data on different external HDs for duplicate copies of data?

Yep.

I do my backups on Sunday. So the week ending 5 March I backup to Hard Drive A. Then the week ending 12 March I backup to Hard Drive B. Then the week ending 19 March I back up to Hard Drive A.

What I have found with CD/DVD backup copies, is that I tend not to back up as often so my data becomes stale because of all the changes.

On a side note, almost daily I back up critical files to one Hard Drive. It takes about 12 minutes via FW. This has saved me more times than I can count.

One important point with backups, is that you need to store them at alternate locations. Otherwise if your location is compromised by an earthquake, tsunami, etc., you might loose all your backup data.

 

[RedTomato]

I am in the process of building my own RAID array -

G3 Powermac 300MHZ 1GB ram £50 second hand
RocketRaid 2220 8port SATA II PCI-X £150 from ebay (£250 new)
3x 300GB SATA Maxtor diamondmax £80 each.

It'll run at RAID5 - means the equivilant of one disk is used for parity data so I will get 600GB of space out of my 3x300GB. Any one disk can fail and the RAID will still survive.

The RocketRaid 2220 was expensive, but will keep going for the next 10 years, when I can swap it into a £50 G5 powermac

It can hot-migrate too, meaning I can add extra 300GB drives to the array till all 8 ports are full. (don't think I'll get that far, but if neccessary, I can add a second 2220 card which will co-operate for a 16 drive array - have to store the second lot of 8 drives outside the powermac tho! )

I'm quite happy with this little setup, and it's far cheaper and bigger than buying one of the NAT or external RAID cases on the market, which seem outrageously overpriced to me.

Comments?

.. RedTomato ..

 

[Danksi]

RocketRaid 2220 8port SATA II PCI-X £150 from ebay (£250 new)

The RocketRaid 2220 was expensive, but will keep going for the next 10 years, when I can swap it into a £50 G5 powermac

Second Generation PowerMac G5 I guess, since the latest supports PCI-Express, which is not compatible with PCI-X.

 

[RedTomato]

Second Generation PowerMac G5 I guess, since the latest supports PCI-Express, which is not compatible with PCI-X.

Flash-flash! Wrong! Most of the earlier G5's are PCI, all the way up to the single 2.7 Ghz.

http://www.lowendmac.com/ppc/g5-27.html

No idea what the intel Powermac will be on. I ain't bothered - a high end G4 would be good enough for another 5 years.

The G3 I have the array on is a 300Mhz - even my mobile phone has a 200Mhz processor and is 1/100th the space (and also cost about £50) ...

..RedTomato..

Edit: changed an audist word.

 

[macunkie]

View attachment 43410

A RAID 0 (ie stripped) strips the data between two (or more drives). This has 2 advantages:
1. No loss of HD space
2. Faster read and write speeds (writing a 200 MB file is more a less like writing a 100 MB file)
And 1 huge drawback:
1. If either drive crashes you loose everything (ie a RAID 1 is 2x more liable to crashing than a normal drive)
Primary use: video editing.

A RAID 1 (mirrored) mirrors the data between 2 drives, ie both drives have the exact same data content. This has a 1 huge advantage:
1. Redundancy. If one drive crashes, the other is there to restore the data.
And 1 big disadvantage:
1. You loose half your storage space.
Primary use: servers.

You can also build a stripped+mirrored RAID using 4 drives which combines the advantages of RAID 0 and RAID 1. Other more exotic RAIDs also exists, but are more difficult to set up and require dedicated hardware solution.


I suppose in you case, RAID 1 (mirrored) would be the solution as the extra speed won't be needed. Again, if you have backups of your data, don't bother with RAID, just use a standard HD.

Well, I successfully configured my 500GB OWC Mercury Elite-AL PRO 800 RAID Dual Drive 7200RPM. I have it set up as a "mirror" of two approx 232 gig drives. I'm satisfied with the loss of storage due to the fact that this drive is dedicated to serving my video (avi) files through my network to my EyeHome system. The video playback actually has improved quite a bit and this was my one of my goals of upgrading my G4 with Firewire 800 and an external Firewire 800 drive.

What I don't understand is in the "event" that "one" of the mirror drives crashes does the "other mirror" automatically kick and and serve? In the event of a "crash" will I be able to "attempt" a repair on just one mirror and if so, will the data on the "non crashed" mirror be able to be copied to the repaired mirror?

My next project is to reformatt my 80gig Firewire 400 external drive to eventually include a bootable drive as a backup of my OS as well as a partion for "daily" backups...

 

[Danksi]

View attachment 43410

Well, I successfully configured my 500GB OWC Mercury Elite-AL PRO 800 RAID Dual Drive 7200RPM. I have it set up as a "mirror" of two approx 232 gig drives.

What I don't understand is in the "event" that "one" of the mirror drives crashes does the "other mirror" automatically kick and and serve? In the event of a "crash" will I be able to "attempt" a repair on just one mirror and if so, will the data on the "non crashed" mirror be able to be copied to the repaired mirror?

Thanks for the update. I was interested in hearing about this setup.

From what I understand for this setup, if one of the mirrored drives dies, the data should still be available. Disk Utility may also provide you with a warning.

Since your hardware isn't hot-swapable, you'll have to power down the unit and replace the faulty drive. When you power it back on you can use Disk-Utility to 'rebuild' the new drive, once again mirroring the one with your data on it. More detail in the Disk-Utility help pages under RAID.

 

[macunkie]

Thanks for the update. I was interested in hearing about this setup.

From what I understand for this setup, if one of the mirrored drives dies, the data should still be available. Disk Utility may also provide you with a warning.

Since your hardware isn't hot-swapable, you'll have to power down the unit and replace the faulty drive. When you power it back on you can use Disk-Utility to 'rebuild' the new drive, once again mirroring the one with your data on it. More detail in the Disk-Utility help pages under RAID.

Ya, the setup has really improved my video playback quality. I thought it was souly dependent on the transfer rate of my network via Airport Extreme and AirPort Express config, but Firewire 800 has really improved the server read time of my files.

I guess I'm going to have to cross the "crash bridge" when it comes. Hopefully no time soon!

 

[hsilver]

If you'd like faster throughput and more dependabilty consider the eSata Raid enclosure also from macsales.com. Holds 2 Sata drives of your choice. You'll need the enclosure, $80, 2 drives - 300gb Seagate SATAs were going for $100 a couple weeks ago at Outpost.com or $150 at macsales and a SATA Controller card instead of the FW card $100. Also the cheaper Firmtek card is SATA so you need special eSata to Sata cables (2) for $24 each. Comes out to a few more dollars but faster and more dependable.

 

[macunkie]

If you'd like faster throughput and more dependabilty consider the eSata Raid enclosure also from macsales.com. Holds 2 Sata drives of your choice. You'll need the enclosure, $80, 2 drives - 300gb Seagate SATAs were going for $100 a couple weeks ago at Outpost.com or $150 at macsales and a SATA Controller card instead of the FW card $100. Also the cheaper Firmtek card is SATA so you need special eSata to Sata cables (2) for $24 each. Comes out to a few more dollars but faster and more dependable.

This sounds great! I'll have to give it a go the next "storage" purchase I make!

 

[Danksi]

This sounds great! I'll have to give it a go the next "storage" purchase I make!

eSATA seems to be the best way forward at the moment, can be cheaper than FW800 and there's more and more drive enclosures.

I picked up the G-Drive-Q because it has FW400/800 & eSATA ports. My thinking was I'd eventually upgrade to a PowerMac and it'll either have an eSATA port already or I could easily add one and it's a faster connection than FW800. For the moment is does well with my FW400 connected iMac.

 

[macunkie]

eSATA seems to be the best way forward at the moment, can be cheaper than FW800 and there's more and more drive enclosures.

I picked up the G-Drive-Q because it has FW400/800 & eSATA ports. My thinking was I'd eventually upgrade to a PowerMac and it'll either have an eSATA port already or I could easily add one and it's a faster connection than FW800. For the moment is does well with my FW400 connected iMac.

Nice Drive!

Question: I'm going to order/install a new IDE hardrive in my G4 PowerMac.
I'm in the process of installing "Tiger" (finally) on my G4 as well. Before I ran the installation I ran disk utility and it showed the original Maxtor HD (60gigs) as failing. I"m just usining it as a slave and storage, but need to replace it. I want to replace it with a 200 to 250 gig drive. Can you check the link below and give me your reccomendations if any?

Thanks!

http://eshop.macsales.com/shop/hard-drives/3.5-IDE-ATA/

 

[Danksi]

Nice Drive!

Question: I'm going to order/install a new IDE hardrive in my G4 PowerMac.
I'm in the process of installing "Tiger" (finally) on my G4 as well. Before I ran the installation I ran disk utility and it showed the original Maxtor HD (60gigs) as failing. I"m just usining it as a slave and storage, but need to replace it. I want to replace it with a 200 to 250 gig drive. Can you check the link below and give me your reccomendations if any?

Thanks!

http://eshop.macsales.com/shop/hard-drives/3.5-IDE-ATA/

I use a 300GB Maxtor DiamondMax 10 in my USB2.0 enclosure for backups. I was using it for my media scratch disk before I got the G-DriveQ. The Maxtor has been reliable and fairly quiet whilst accessing data.

Feedback I've seen suggests Hitachi are very good (they use them in the G-Drives), not so good for the others. Mixed reviews for Maxtor's, but I've not had any trouble myself.

Hope that helps.

 

[macunkie]

I use a 300GB Maxtor DiamondMax 10 in my USB2.0 enclosure for backups. I was using it for my media scratch disk before I got the G-DriveQ. The Maxtor has been reliable and fairly quiet whilst accessing data.

Feedback I've seen suggests Hitachi are very good (they use them in the G-Drives), not so good for the others. Mixed reviews for Maxtor's, but I've not had any trouble myself.

Hope that helps.

Thanks Danksi! I think I'll stick with Maxtor. This is what I'm using with my current master IDE drive. I think the original 60gig that came with my 533 G4 has just got too many "read/write miles" on it. It's over 5 years old now.

Thanks again for your input!