Mac Pro (Late 2013) - Amfeltec AngelShark Carrier Board - Storage Upgrade

[mikeboss]

I agree. For something like the iMac Pro which is forced to only have one processor on the motherboard I don't see much difference between a high-end consumer CPU and a high-end Xeon CPU. What makes a Xeon rise above a consumer CPU is the scalability of multiple processors on a single board. They could have taken the existing iMac 5K and released a version of it with a Vega board and a PRO SSD and called it good. I feel like the iMac Pro only serves Apple and not their customers. I really hope the next Mac Pro has an option not only for regular PCIe slots but also for multiple CPU.

AFAIK, the Xeons usually have a lot more PCIe lanes than their i5/i7 counterparts.

for example: Core i7 6700 (Skylake) has 16 PCIe lanes, the Xeons in the MacPro6,1 offer 40 lanes which is a good thing if one likes to connect a lot of Thunderbolt devices.

 

[joevt]

https://www.anandtech.com/show/7603/mac-pro-review-late-2013/8
The PCIe for the storage on Graphics Board B is only PCIe 2.0 x4.

Can you show the PCIe layout using the pcitree.sh script at #223 ?

 

[CodeJingle]

AFAIK, the Xeons usually have a lot more PCIe lanes than their i5/i7 counterparts.

for example, Core i7 6700 (Skylake) has 16 PCIe lanes, the Xeons in the MacPro6,1 offer 40 lanes which is a good thing if one likes to connect a lot of Thunderbolt devices.

Lots of PCIe lanes is excellent, and Xeon multi-CPU for even more PCIe lanes. But it is lost on the Mac Pro 2013 with the proprietary 300 pin connectors and lost on the iMac Pro being a solution not made to be opened to upgrade manually. The throughput on the Thunderbolt ports is limited if using them as pass-thru for the display, whereas plugging the display directly into the GPU then those Thunderbolt ports would be available for other purposes (including as pass-thru for PCIe).

 

[mikeboss]

The throughput on the Thunderbolt ports is limited if using them as pass-thru for the display...

are you sure about that? in what way, by what amount, will the throughput be limited? this is new to me.

 

[CodeJingle]

are you sure about that? In what way, by what amount, will the throughput be limited? This is new to me.

Can anyone else corroborate? If I connect three 4k displays to Thunderbolt 2, do I still have the full PCIe bandwidth available for use as pass-thru?

 

[mikeboss]

AFAIK there should be no bandwith limitation because of a display connected to the same Thunderbolt bus. what should be avoided is connecting two Thunderbolt devices to the same bus.

 

[RyanXM]

Can anyone else corroborate? If I connect three 4k displays to Thunderbolt 2, do I still have the full PCIe bandwidth available for use as pass-thru?

The Anandtech review should give you an idea of the bandwidth to all devices. There is a PLX PCIe Switch, which according to Anandtech, should give full bandwidth to any device connected, even when a 4K display is connected to each Thunderbolt bus.

https://www.anandtech.com/show/7603/mac-pro-review-late-2013/8
https://support.apple.com/en-us/HT202801

 

[CodeJingle]

AFAIK there should be no bandwith limitation because of a display connected to the same Thunderbolt bus. what should be avoided is connecting two Thunderbolt devices to the same bus.

View attachment 740518

Sounds good. Thanks for the correction. I am still wanting to try and pull half the PCIe lanes off of the two internal graphics cards for my own purposes. They should still work correctly each running at 8 lanes.

 

[William_si]

Uses a PLX PEX8718-AB80B1 Bridge.

Interesting, assuming they knew the port is 2.0 x4 the more expensive 3.0 chip is fairly useless, but the rumour is that 2.0 chips are scarce these days.

 

[jclmavg]

Ryan, any closing remarks and/or updates on this board?

 

[urbanmacUser]

Any update on this?

 

[jclmavg]

Any update on this?

Good point. I'd like to know if the boot time stays normal.

 

[RyanXM]

Holidays were a little crazy around these parts. Back into the swing of things and this is on the list to complete.

 

[RyanXM]

Original post updated with pictures of all benchmarks and unit installed. 10.13.2 didn't change the benchmarks for any of the drives, so it seems that the 960 EVO and 960 PRO are the go to NVMe Drives to use.

 

[RyanXM]

If anyone is wanting one, please PM me. I'm working on getting some better pricing for those who are wanting one.

 

[Kris Kelvin]

Thanks to @RyanXM, an AngelShark found its way to me.

• Works flawlessly so far with an Apple SM0256G (AHCI) and a Samsung 960 PRO 2TB (NVMe) installed at the same time
• Performance as expected (limited by the Mac Pro's PCIe bus speed to about 1.5GB/s for both drives)
• Temperature of the Samsung 960 is slightly elevated (43°C; +1°C) compared to before (when using the Sintech adapter and a heatsink)
• System temperatures are the same or slightly lower except GPU 2 VR Proximity, which is now about 50°C (+5°C)
• No changes in fan speed (still 790rpm)
• SSD power consumption has doubled to about 0.96A (makes sense, since there are now two SSDs installed)

 

[Wolfman Jack]

I don't have enough posts to PM, but I'd be interested in the discounted pricing details if you could PM me.

I had a couple of additional questions:

- it's not clear to me on the first post how the 300MB/s write limitation resolved itself (if it did?)
- did you also try the SSPOLARIS SSD as the Apple SSD?
- I'm also wondering how this solution works with the "pre-boot checklist" boot delay referenced on the Sintech/PC-Parts NVMe thread. Is that resolved either by (a) using the SSPOLARIS SSD rather than a non-Apple one; or (b) using this board instead of the Sintech/NVMe one?

 

[Kris Kelvin]

If I understand correctly, the speed varied between SSD manufacturers. My Samsung 960 PRO 2TB and 960 EVO 256GB both work flawlessly.

Unfortunately, I don't have a SSPOLARIS, but Apple's AHCI drive works fine. I don't see any reason why the SSPOLARIS shouldn't.

The boot delay exists for me with both the Sintech adapter and the AngelShark when the 2TB SSD is installed (didn't try with the 256GB 960 EVO only).

 

[Kris Kelvin]

Some information about running Windows from a separate SSD (Samsung 960 EVO):

• Boot Camp Assistant on my MacPro6,1 wouldn't allow me to create an USB flash drive with the Windows installer, informing me that "You cannot partition an external disk, a RAID disk or an internal disk on which FileVault encryption is in progress." I have neither a RAID nor FileVault enabled on this machine.
• After preparing the USB flash drive with the Boot Camp Assistant on a MacBook, the Mac Pro booted happily from it and showed all three SSDs attached to the AngelShark.
• As with my MacPro5,1, the Windows installer wouldn't succeed as long as there is more than one SSD installed. After physically removing the other two SSDs, the installer ran properly and quickly.
• Once Windows is installed, the other drives can be attached again and everything is working as expected.

When the Mac Pro gets hot (sustained load for CPU and GPU, enclosure temperature above 36°C or so), a red light appears that is visible trough the bottom holes. Everything continued to work fine though. Once temperatures drop, the light vanishes.



The same light was (sometimes?) also visible after waking from sleep under macOS before I installed the third SSD. The manual doesn't say anything about that, so I'm not sure what it means.

 

[RyanXM]

@Kris Kelvin I forgot to mention the lights! It's alive! On first power on, it will show blue, red, and green lights if I remember correctly. Glad you got Windows installed and working. I always just used it in a VM for my needs.

 

[Wolfman Jack]

Just one last question on the boot delay: if you have both the nVME and the Apple AHCI SSD installed, you observe it?

 

[Kris Kelvin]

Just one last question on the boot delay: if you have both the nVME and the Apple AHCI SSD installed, you observe it?

Yes. However:
Booting from Samsung NVMe SSD: 120s (all drives mounted after boot)
Booting from Apple AHCI SSD: 30s (only AHCI SSD mounted after boot; manually mounting the NVMe SSD takes ~90s)

 

[Earl Urley]

Looks like RobArt of BareFeats got his hands on one, here's the results of his tests:

http://barefeats.com/hard230.html

Of course there are no mentions of any difficulties he had, I am glad that Kris Kelvin let us know about the overheating issues and boot delays. Dunno if I'd want to sink half a grand into just an adapter for NVMe compatibility, but having three slots is a definite plus.

 

[RyanXM]

Looks like RobArt of BareFeats got his hands on one, here's the results of his tests:

http://barefeats.com/hard230.html

Of course there are no mentions of any difficulties he had, I am glad that Kris Kelvin let us know about the overheating issues and boot delays. Dunno if I'd want to sink half a grand into just an adapter for NVMe compatibility, but having three slots is a definite plus.

I was never able to get the slow speeds to go the quoted specs for those given SSDs. I believe it to be an issue related to the drivers in macOS. Given that Apple OEM SSDs, both AHCI and NVMe, are typically Samsung based, those drives seem to work the best. XP941 and SM951 being the exceptions.

 

[ActionableMango]

It is nice to see expansion upgrades for the tube. I didn't think it would happen.