Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

tobio

macrumors regular
Sep 5, 2004
146
0
London
cluster it?

This may be a glaringly obvious question, but I was of the impression that jobs such as video encoding don't distribute well, therefore I thought that an xgrid cluster would only be helpful in a situation where I multiple different clips to encode at once. So would having an Xgrid cluster speed up the encoding of H.264 video so that it wasn't such a monster?

In compressor there are a couple of HD H.264 presets, and it is easy to make your own. Because I only have the one mac, does anyone else know how clustering/xgrid works with encoding jobs? I can render a 2 hour movie in about 30 hours (two pass), but I would happily justify throwing a dedicated mac in the corner to share the load if it would make a difference to the time.
 

Rod Rod

macrumors 68020
Sep 21, 2003
2,180
6
Las Vegas, NV
tobio said:
This may be a glaringly obvious question, but I was of the impression that jobs such as video encoding don't distribute well, therefore I thought that an xgrid cluster would only be helpful in a situation where I multiple different clips to encode at once. So would having an Xgrid cluster speed up the encoding of H.264 video so that it wasn't such a monster?

In compressor there are a couple of HD H.264 presets, and it is easy to make your own. Because I only have the one mac, does anyone else know how clustering/xgrid works with encoding jobs? I can render a 2 hour movie in about 30 hours (two pass), but I would happily justify throwing a dedicated mac in the corner to share the load if it would make a difference to the time.
You can distribute video encoding jobs using Apple Qmaster. First install Qmaster on your machines and then create your cluster. From Compressor you can send the encoding job to your cluster and it will first copy what files it needs to the other machines and then all machines with cluster services will share the processing load.
 

tobio

macrumors regular
Sep 5, 2004
146
0
London
Rod Rod said:
You can distribute video encoding jobs using Apple Qmaster. First install Qmaster on your machines and then create your cluster. From Compressor you can send the encoding job to your cluster and it will first copy what files it needs to the other machines and then all machines with cluster services will share the processing load.

Perhaps this could be the perfect justification for having a little stack of mac minis in the corner...

I wonder which would be quicker, four 1.25Ghz mac minis at £1440, or one Dual 2Ghz Powermac at £1350

:rolleyes:
 

ammon

macrumors regular
Sep 24, 2005
231
40
Colorado
Video encoding is very processor hungry. Speed is everything! If you have all the minis connected with gigabit, they should be significantly faster than the G5!
 

tobio

macrumors regular
Sep 5, 2004
146
0
London
But... minis don't have gigabit ethernet.

Its a shame I dont have the resources to battle them out, but i suspect the G5 would win.

I do have the resources to do such a test with PCs tho... but first I would have to learn how to set up some linux and openmosix and all that stuff. the results from that should be indicative of minis vs a powermac, but I dont know when or if I will ever get the time to do this.

I wouldn't expect four minis to be able to encode a video four times quicker than one on its own, but it depends on how the cluster and the encoder talk to each other. If it was smart enough to split the movie into four equal parts, and send each bit to one of the minis to process, then put the file back together when they are all finished then that would be four times as fast. However high quality video encoding needs two passes, so you wouldn't be able to split the job up like that until after the first pass has happened, because the four minis wouldn't yet know what to do.

If you were doing CBR encoding then it may be workable, but the resulting file wouldn't be as good as the 2-pass one therefore it wouldn't be worth doing?

has anyone actually got some real data on this kind of exercise?
 

Rod Rod

macrumors 68020
Sep 21, 2003
2,180
6
Las Vegas, NV
jelloshotsrule said:
hey rod rod, what cam did you shoot that sample thing with?
Hi jelloshotsrule, I shot it with a JVC HD10. I'm happy with some shots in there more than others. The lady wearing orange turned out great, for example, while the white guy with glasses was underexposed. I threw that sequence together super fast; it was a real rush job.
 

punkmac

macrumors regular
Jan 27, 2004
231
0
H.264 as DvD backup

mymemory said:
I am here in Final Cut Pro. I wouldn't care about the codec in question but everytime I am exporting something FCP use it as a default what actually suck because the render time in insanely slow what makes the work useless. I rather use DV Pro of course and Sorenson for internet if that is the case.


H.264 Rocks! I am currently backing up my DVD collection to H.264.

I am able to shrink a dvd to 1/4 the size with no loss in quality. Sure this is a slow process but what the hell, the computer is just sitting there idle all night.

Bring on Downloadable HD movies!


I.
 

Chundles

macrumors G5
Jul 4, 2005
12,037
493
Sdashiki said:
But what are you doing with the h.264 files? Only a fast new computer can play them, not a set top DVD player, yet.

Only a new computer can play HD H.264 files, my iBook handles H.264 rips from DVD very nicely. I can get a movie at almost 3/4 the quality of a DVD onto a CD. It takes a long time (very very long time, like all night) to rip a DVD to H.264 but the quality is outstanding. Much better than .avi
 

jelloshotsrule

macrumors G3
Feb 7, 2002
9,596
4
serendipity
Rod Rod said:
Hi jelloshotsrule, I shot it with a JVC HD10. I'm happy with some shots in there more than others. The lady wearing orange turned out great, for example, while the white guy with glasses was underexposed. I threw that sequence together super fast; it was a real rush job.

hah. in all honesty i haven't gotten to see the clip yet (every time i look here i'm on my dumb pc at work...), but i wanted to know what hdv cam you had used since you seemed to think the result was good. i'll be posting a new thread about why, and look forward to your advice in it.....

but, so looking at b&h i see a ~$5500 jvc gy-hd10 (ou?) and then a ~$2500 jvc jy-hd10 (o)... which did you use? or another one altogether? thanks!
 

Sdashiki

macrumors 68040
Aug 11, 2005
3,529
11
Behind the lens
Like I said, kinda pointless and time consuming for no reason other than some geeky cool factor?

You take a DVD, watchable on your LARGE and nice television set in the living room.

Rip it to your iBook at h.264.

How do you watch this newly compressed video? On your iBook? WOW that must be such fun to watch.
 

saunders45

macrumors 6502a
Jul 29, 2004
513
0
It may not be the same as watching on a tv, however, when traveling alot, it sure gets a better battery life than running the dvd drive constantly.
 

punkmac

macrumors regular
Jan 27, 2004
231
0
watching dvd

Sdashiki said:
Like I said, kinda pointless and time consuming for no reason other than some geeky cool factor?

You take a DVD, watchable on your LARGE and nice television set in the living room.

Rip it to your iBook at h.264.

How do you watch this newly compressed video? On your iBook? WOW that must be such fun to watch.


I don't own a TV. All the video we watch is on our 17" iMac or 19" Crt.
 

Rod Rod

macrumors 68020
Sep 21, 2003
2,180
6
Las Vegas, NV
jelloshotsrule said:
but, so looking at b&h i see a ~$5500 jvc gy-hd10 (ou?) and then a ~$2500 jvc jy-hd10 (o)... which did you use? or another one altogether? thanks!
I used the JY-HD10U. The GY-HD100 is the higher end, $5500 camera that just came out this month. The JY-HD10U came out last year or the year before. You can find used JY-HD10Us and GY-HD1Us on eBay for around $1400 or less. Check out the HDV Acquisition Equipment section of HDV Info Net in case you haven't yet.
 

bbyrdhouse

macrumors 6502
Oct 2, 2002
300
0
Elm Grove, LA
G4 Faster than G5

Believe it or not but:

Last night I began encoding Season 1 disc 4 of The Andy Griffith Show for IPod 5G.
Here are the conditions to my experiment.

I ripped the same disc with Mac the Ripper.
I then used Handbrake to encode.
I encoded the entire dvd (from my hard drive mind you) with the follwing variables being the same on both machines:
  1. DVD Time of original content = 3 hours 24 minutes
  2. File format MP4
  3. Codec H.264
  4. Average bitrate 800
  5. Audio Sample rate 44,000KHZ
  6. Audio bitrate 112 KBS
  7. Screen size output 320x240
  8. 2 pass encoding

I knew that it was going to take a while because of my options, but I was not ready for what I found after 6 hours of encoding.

I found that my 15" Powerbook with 1 Gig of Ram, 1.67 Ghz proc was on it's second pass at 20% done
While my Dual 2.7 G5, with 1.5 Gig of Ram was only at 80% of the first pass.

I could not believe it I would have thought it be the other way around.

I started them exactly the same time.

The Powerbook encodes at 18-19 frames per second and the G5 Tower encodes at 10-11 frames per second.

What do you make of that?
 

FireArse

macrumors 6502a
Oct 29, 2004
900
110
bbyrdhouse said:
I then used Handbrake to encode.
I encoded the entire dvd (from my hard drive mind you)
What do you make of that?

I think thats where your problem lies. If you're encoding with Handbrake the x264 codec is optimized more (i think) for the G4 than for the G5.

If you can get your DVD rip to be read from within QuickTime 7 Pro, then encode from there. You can bet your life that the Offical H.264 Encoder within QT Pro has been optimized to the balls for the Dual G5.

It is faster than that, I'm sure.

F
 

bbyrdhouse

macrumors 6502
Oct 2, 2002
300
0
Elm Grove, LA
FireArse said:
...If you're encoding with Handbrake the x264 codec is optimized more (i think) for the G4 than for the G5.

If you can get your DVD rip to be read from within QuickTime 7 Pro, then encode from there. You can bet your life that the Offical H.264 Encoder within QT Pro has been optimized to the balls for the Dual G5.

It is faster than that, I'm sure.

F

Well, that will be my next experiment. I certainly did not know that, if I had I would have done so in the beginning.
Thanks for the advice.
 

FireArse

macrumors 6502a
Oct 29, 2004
900
110
bbyrdhouse said:
Well, that will be my next experiment. I certainly did not know that, if I had I would have done so in the beginning.
Thanks for the advice.

Would you mind coming back online after you've done the comparison? I'd love to know how the QT H.264 fares with x.264 (open source version) accross the differing Hardware you have. You'll still use the same video?

Cheers,

FireArse
 

bbyrdhouse

macrumors 6502
Oct 2, 2002
300
0
Elm Grove, LA
FireArse said:
Would you mind coming back online after you've done the comparison? I'd love to know how the QT H.264 fares with x.264 (open source version) accross the differing Hardware you have. You'll still use the same video?

Cheers,

FireArse

I havn't yet found an easy way to get the vob (DVD Rip) into Quicktime Pro.

However I was finally able to Re-encode the same file (3 hours, 24 minutes) with all of the same settings as before but instead of encoding to x.264 I encoded to regular M4v and it is amzing at the speed difference in encoding.

To encode to x.264 my G5 Dual 2.7 encoded at about 12 fps (12 hours)
But to encode to M4v it encoded at about 160 fps (40 minutes) that's with 2 pass encoding.
Geez! What a difference.

Now, any suggestions as to the best, easiest way to export/import a .vob into Quicktime Pro.

The difference in file size was pretty extreme though:
x.264 = 1.2 Gigabytes
MP4 = 5.6 Gigabytes
 

DavidCar

macrumors 6502a
Jan 19, 2004
525
0
bbyrdhouse said:
I then used Handbrake to encode....I found that my 15" Powerbook with 1 Gig of Ram, 1.67 Ghz proc was on it's second pass at 20% done
While my Dual 2.7 G5, with 1.5 Gig of Ram was only at 80% of the first pass.

I found a note on http://xlr8yourmac.com/ that Handbrake was updated with "Multithreaded H.264 encoding with x264." I wonder if that would change your results.
 
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.