Folding GPU (esp. AMD) is where it's at...

[Louis Wu]

I think I saw that but I wanted to try a single board with all 4 of them just to see, the power bill would be less than with 2 boards running the 4 Titans. I will still have 2 old boards that I can use if needed, and if I decided I could afford some more Titans I can still use the old boards. I have 3 extra psu's, ram, and lots of drives to use but nothing high end except the soon to be 4 Titans. I'm hoping that will give me about 600k ppd, we will see what happens.

In my younger days I could never have spent money like this but I figure I don't spend much on anything else so why not. Plus: no wife, no dog, no kids=it's mine to do with as I wish! It's for a good cause too!

I was #1 on the team for a while but I got busy with other things and let it go. I have kept the Mac Pro going all this time for a few points but wanted to take the ppd up to try to match twoodcc and maybe even catch up to him!

Back in the day our team only got 2 to 6 million Points per week, I think, we are doing much better now with your help and others. It's more interesting with competition. It would be nice to get some more interest in the team, and I know it's a Mac website but it's hard to get points with Mac hardware. At least Linux is more useful now, I hate using windows!

I never imagined we would get to a billion points, wow! but we are almost... There...

you sure were #1 on the team when I first started -- saw 40 million and thought I'd never be at that level... you were -- and are -- a great inspiration!

I could never get more than 2 cards going on one board; I have a 3-way board but cooling was such an issue. right now temps are fine; I just keep losing the connection to the servers so finished WUs stick around not being uploaded, and GPUs sit idle; often for hours.

your titan machine will be very power effective, that's for sure! my 4p board sucks 500w and sometimes only produces 90 k ppd on the dreaded 8101 WUs.

glad to hear you're able to contribute so much for this worthy cause! good to be at that point in your life for sure!

I'm getting a little more familiar with linux but am using Win7 for the GPUs so I can boost clocks and fan speeds and such. the talk of flashing bios on GPUs scares me a little.

now I have to consider flashing the bios on the 4p board to overclock these entry-level chips I have. That's going to be nerve-wracking!

anyhow, onwards we go

 

[ChristianVirtual]

I'm getting a little more familiar with linux but am using Win7 for the GPUs so I can boost clocks and fan speeds and such. the talk of flashing bios on GPUs scares me a little.

You can do same with Linux ... I'm using standard nvidia driver and change the fan speed to increase cooling. "Cool Bits" is the keyword to achieve that. I could change also clock speed etc. Later I don't do based on the suggestion that core 17 is sensitive on overclocking. No flushing of bios required. You can try in dual boot and switch back to Windows.

 

[Louis Wu]

You can do same with Linux ... I'm using standard nvidia driver and change the fan speed to increase cooling. "Cool Bits" is the keyword to achieve that. I could change also clock speed etc. Later I don't do based on the suggestion that core 17 is sensitive on overclocking. No flushing of bios required. You can try in dual boot and switch back to Windows.

thanks for the info!

I might just try that on the nVidia GPU box -- I'll get around to it at some point!

 

[twoodcc]

I wouldn't worry about it... (folding that is; I'd just shut it down for the weekend... all of it!)

go have a good time at your buddy's wedding

thanks. weddings really aren't my thing, but i survived. of course i come home and everything has crashed!

Hey have a good time at the wedding!

And next week I will set up my new rig, a four slot board to hold my FOUR Titans.

Then I will start thinking about a 2P/4P system. Geez I didn't realize what other people have running! Some guys are making 4-6 million PPD!!!! on piles of 4P systems. Doubt I will ever be able to afford that or the power bill!

congrats on the 4 titans! looking forward to seeing what kind of points you can put up.

yeah there are a few putting up those numbers. yeah the power bill would be my main concern, after i got all the hardware accumulated

 

[lordonuthin]

Titans

So far I am getting about 120k ppd on each of the Titans. Which means I should get about 480k ppd from 4 of them once they are set up. I should have everything tomorrow so that I can get started with the new rig.

 

[ChristianVirtual]

Please don't forget to make some pics for the picture thread, I really would like to see the rig with four Titan ... In one system ? How do you cool it down ?

 

[Louis Wu]

Please don't forget to make some pics for the picture thread, I really would like to see the rig with four Titan ... In one system ? How do you cool it down ?

I'm guessing liquid nitrogen (just kidding )

 

[lordonuthin]

I'm guessing liquid nitrogen (just kidding )

Ooo, that would be awesome! Dunk it in a fish tank full of liquid nitrogen! it will be more mundane with a fan blowing air at it.

 

[twoodcc]

So far I am getting about 120k ppd on each of the Titans. Which means I should get about 480k ppd from 4 of them once they are set up. I should have everything tomorrow so that I can get started with the new rig.

it seems you should be getting more than 120k, depending on WU. i'm getting close to that with my 780

 

[lordonuthin]

it seems you should be getting more than 120k, depending on WU. i'm getting close to that with my 780

That is about average, I have seen as much as 220k ppd and as little as 80k ppd, it depends on the WU.

I tried to get the new rig up last night and no joy. Hardware seems to be fine, it's just me and linux. I may not have much time tonight to mess with it.

And I didn't know this before but there are 4 power ports on the board which means that my older 1200w ps does not have enough cables for 4 PSIe cards... unless I use drive cables with adapters, which I don't want to do. So, do I get another ps, make due with drive cables or just go 3 up on that board? Oh well, I'll figure it out eventually.

 

[ChristianVirtual]

it's just me and linux. I may not have much time tonight to mess with it.

What works well for me is Ubuntu 13.04 with driver from xorg-edgers 319.34 ... Still need to restart every 24 hours due to driver bug; but else works well with GTX 780 (Titan mini) and 660 Ti.

I think I read that CPU only support 40 lanes on PCIe. Four cards with 16 lanes each dont fit the bill. 2x16 plus 1x8 ... What MB you have ?

 

[Louis Wu]

What works well for me is Ubuntu 13.04 with driver from xorg-edgers 319.34 ... Still need to restart every 24 hours due to driver bug; but else works well with GTX 780 (Titan mini) and 660 Ti.

I think I read that CPU only support 40 lanes on PCIe. Four cards with 16 lanes each dont fit the bill. 2x16 plus 1x8 ... What MB you have ?

Thanks for the Ubuntu info -- I'll give it a shot when I get bored of everything working properly!

I'm running a board that has 2 slots, one with 16 lanes and the other with only 4 -- but it doesn't seem to matter to the transfer rates on the WUs... points are always the same...

 

[twoodcc]

and of course i got another 8101 on the big rig. i guess i can't complain, it's been awhile. but i did have a short power outage earlier today

 

[ChristianVirtual]

and of course i got another 8101 on the big rig. i guess i can't complain, it's been awhile. but i did have a short power outage earlier today

Forgive me: what is the problem with 8101 ? I don't have a bigadv so never run into one of those. Low PPD and lots of watts ?

 

[lordonuthin]

What works well for me is Ubuntu 13.04 with driver from xorg-edgers 319.34 ... Still need to restart every 24 hours due to driver bug; but else works well with GTX 780 (Titan mini) and 660 Ti.

I think I read that CPU only support 40 lanes on PCIe. Four cards with 16 lanes each dont fit the bill. 2x16 plus 1x8 ... What MB you have ?

I have given up on Ubuntu and started win 7 install. ill try again with another system when I have the time. Saved the disk so maybe I won't have to start from scratch.

I got an i5-4430 and Asus Z87-WS for the build, I think it was supposed to have enough guts for 4 cards but maybe I misread... Wouldn't be the first time.

 

[Louis Wu]

Forgive me: what is the problem with 8101 ? I don't have a bigadv so never run into one of those. Low PPD and lots of watts ?

You're exactly right on both counts!

8103, 8104 and 8105 are decent; 8101s take longer and generate fewer points (for the amount of time your rig spends doing them).

I've got one now too... it'll take my rig more than 2 days to complete, and barely gets it done before the deadline.

 

[Louis Wu]

I have given up on Ubuntu and started win 7 install. ill try again with another system when I have the time. Saved the disk so maybe I won't have to start from scratch.

I got an i5-4430 and Asus Z87-WS for the build, I think it was supposed to have enough guts for 4 cards but maybe I misread... Wouldn't be the first time.

don't blame ya. twoodcc knows that I wasted MONTHS trying to get Ubuntu working on my big rig. windows not being an option for bigadv of course.

that's a serious board! can't see why it wouldn't be able to run all 4 titans... the nVidia GPUs do require a lot more CPU muscle to feed properly... but the i5-4430 has 4 cores so one per titan should work...

 

[Louis Wu]

That is about average, I have seen as much as 220k ppd and as little as 80k ppd, it depends on the WU.

I tried to get the new rig up last night and no joy. Hardware seems to be fine, it's just me and linux. I may not have much time tonight to mess with it.

And I didn't know this before but there are 4 power ports on the board which means that my older 1200w ps does not have enough cables for 4 PSIe cards... unless I use drive cables with adapters, which I don't want to do. So, do I get another ps, make due with drive cables or just go 3 up on that board? Oh well, I'll figure it out eventually.

I'd go track down Andy_E on [H] (he's on other forum boards as well) and ask how he did it -- there's lots of speculation out there but very few have 4 titans like you do...

maybe go with 2 x 750W psd units? I'm not familiar with how you get the "slave" psu to turn on - probably a manual jumper-type switch like this:

http://www.performance-pcs.com/cata...ge=product_info&cPath=22_101&products_id=4057

----------

thanks. weddings really aren't my thing, but i survived. of course i come home and everything has crashed!

glad you did. weddings aren't my thing either but they're better than funerals.

our entire team does notice when your big rig crashes so glad to hear that you're back up!

 

[twoodcc]

glad you did. weddings aren't my thing either but they're better than funerals.

our entire team does notice when your big rig crashes so glad to hear that you're back up!

haha, yeah you have a point there.

yeah these 8101s are really cramping my style...

 

[lordonuthin]

Titan's 3 and 4 are up and running the cpu is right at 50% though with nothing else running and folding paused on the cpu. Didn't know it would be so close to 25% to feed each gpu I guess that is maximizing the use of resources cuz it will be pegged if I can get all four titans on there! Hope the pegged cpu won't impact gpu folding

I have decided to try and run the rig with only 3 power cables to the board. Instructions indicate the extra power is needed when more PCIe cards are installed but maybe I can get by with just 3. It just says it may be "unstable" with more PCIe cards and and not enough power to the main board. I will have to try it out this weekend, first with 3 titans then with 4 if 3 goes well.

Once the other rig is free of the titans I can get it's 980x cpu folding again! By the way with 2 titans the 980x runs about 15% with cpu folding paused, I wonder if the PCIe-3 bandwidth has anything to do with the i5-4430, new rig, being run harder or is it just the difference in cpu power? I thought the i5 with 4 cores would be more than enough power to feed all 4 titans. If 3 titans put it much above 75% load I could have trouble with all 4 gpu's. More dilemmas...

It's good to get it going, now maybe I can start stomping on some toes Wear you steel toe shoes boys!

 

[Louis Wu]

Titan's 3 and 4 are up and running the cpu is right at 50%

It's good to get it going, now maybe I can start stomping on some toes Wear you steel toe shoes boys!

Great to hear that you've got all the Titans doing their stuff!

I've heard that a lot of the CPU activity on nVidia GPUs is useless stuff like polling to see if the GPU is active... you could probably peg all 4 CPU cores and have some left over for other tasks... (let us know if that's indeed the case!)

I found this thread if you haven't seen it:

http://foldingforum.org/viewtopic.php?f=38&t=23944

2 Titans per rig sounds well within power limits -- I've heard that each Titan uses not too much more than an overclocked 7970 -- my cheap power meter shows 300+ W for the rig with dual 7970s. don't really trust it though.

I didn't think the GPUs saturated the PCIe pipe at all.

Seeing as I'm already in your dust I guess I can take my Kodiaks off now

 

[twoodcc]

Titan's 3 and 4 are up and running the cpu is right at 50% though with nothing else running and folding paused on the cpu. Didn't know it would be so close to 25% to feed each gpu I guess that is maximizing the use of resources cuz it will be pegged if I can get all four titans on there! Hope the pegged cpu won't impact gpu folding

I have decided to try and run the rig with only 3 power cables to the board. Instructions indicate the extra power is needed when more PCIe cards are installed but maybe I can get by with just 3. It just says it may be "unstable" with more PCIe cards and and not enough power to the main board. I will have to try it out this weekend, first with 3 titans then with 4 if 3 goes well.

Once the other rig is free of the titans I can get it's 980x cpu folding again! By the way with 2 titans the 980x runs about 15% with cpu folding paused, I wonder if the PCIe-3 bandwidth has anything to do with the i5-4430, new rig, being run harder or is it just the difference in cpu power? I thought the i5 with 4 cores would be more than enough power to feed all 4 titans. If 3 titans put it much above 75% load I could have trouble with all 4 gpu's. More dilemmas...

It's good to get it going, now maybe I can start stomping on some toes Wear you steel toe shoes boys!

glad you got them going. at the current rate, i would continue to focus on gpu and not the 980x cpu folding. i mean, points are points, but right now gpu is where it's at. the only cpu folding is AMD 4 processor rigs, that's the only cpu folding i'm doing.

haha, i got my work boots on. let's go!

 

[Louis Wu]

Titan's 3 and 4 are up and running

Congratulations on hitting #1 again!

Looks like you're firing on all cylinders

 

[lordonuthin]

Congratulations on hitting #1 again!

Looks like you're firing on all cylinders

Thanks, but it will be a long time at the current rate to get to #1 again, though I might squeak by with highest ppd. rwh202 and twoodcc are going to be hard to beat!

I got around to adjusting the fans on a pair of the Titans and temps went down so I don't think cooling will be a problem. I think I will leave things as they are for a bit to see how it goes. Later I will move the first two to Titan1 and maybe try a little bit of oc.

glad you got them going. at the current rate, i would continue to focus on gpu and not the 980x cpu folding. i mean, points are points, but right now gpu is where it's at. the only cpu folding is AMD 4 processor rigs, that's the only cpu folding i'm doing.

haha, i got my work boots on. let's go!

Maybe you are right, I will think about what to do next.

It looks like I've just squeaked past you for ppd! By less than 100 points!

Cheers!

 

[twoodcc]

Thanks, but it will be a long time at the current rate to get to #1 again, though I might squeak by with highest ppd. rwh202 and twoodcc are going to be hard to beat!

I got around to adjusting the fans on a pair of the Titans and temps went down so I don't think cooling will be a problem. I think I will leave things as they are for a bit to see how it goes. Later I will move the first two to Titan1 and maybe try a little bit of oc.




Maybe you are right, I will think about what to do next.

It looks like I've just squeaked past you for ppd! By less than 100 points!

Cheers!

glad you got the titans figured out. and the points are looking good.

better check those numbers, looks like i'm on top right now! (of course it won't be for long though)