MP All Models Bootable PCI SSD Cards/Drives for Mac Pro (NVMe M.2, U.2, EDSFF E3.S, etc. (was Micron 9300 Pro 15.36TB U2))

[ZombiePhysicist]

Thanks. I've got a Micron 9300 pro 7.68 in a Helios Thunderbolt enclosure now. It's great. I'm just wondering where to go when eventually I need something bigger.

Well if 15tb is enough, the 15TB 9300 Pro works fine. Ive been using that for over 4 years now.

 

[ZombiePhysicist]

Hmm these high point cards claim Mac compatibility with a bunch of Ssd providers:

E1.S form factor

Loading…

www.highpoint-tech.com

M.2 form factor

Loading…

www.highpoint-tech.com

I think the largest m.2 sticks are still 8tbl so doesn’t work as a boot drive For me asits too small.

Does anyone know if there are any E1.s 30thb drives? For some reason the high point cards seem to work in the Mac and can work, apparently, with a wide variety of e1.s driver providers.

 

[ZombiePhysicist]

Wow. So Sonnetech's latest February version of their compatibility chart lists the Solidigm D5-P5316 30TB drive as macOS compatible!

My guess this drive might be a touch old as it is often still branded as Intel and not Solidigm. That said, it seems Solidigm might be the only make that looks completely or highly compatible on macOS:

 

[ZombiePhysicist]

Welp. I ordered one. Probably won’t get here until the end of next week. Fingers crossed!

 

[ZombiePhysicist]

OMG just shoot me.

So good news is the Solidigm/Intel SSDPF2NV307TZN1 D5-P5316 30.72 TB is seen by macOS via both my HighPoint cards (7120 (PCIe3) and 1580 (PCIe4)):
I got the drive. It shows right up. You can format it. You can use it.

The bad news:
The performance when you format at is as an APFS drive is horrible:

Yet, if you format it as exFAT it works much better:

This drive in theory should write at 3600MB/s and read at 7000MB/s. The 7,1 is PCIe3 so it should limit the speed. However, the 9400Pro, in windows, showed read writes at over 5000MB/sec, so I was expecting better reads and writes on this drive over my 15TB Micron 9300 Pro (which does about 2600MB/s read/write with FileVault on (faster with it off)). Frankly, I could live with the read/write speeds I get with exFAT as it's as fast or faster than my current drive, but I cannot live with teh slow 500MB/sec writes when formatted with APFS.

BTW, I moved my 15TB 9300 Pro to the 2nd port on the Highpoint card, and it still boots fine and it still read/writes at the same speed. I put this new drive on port0 on the card where the 15TB used to be.

ARG! This is so unbelievably frustrating.

I'm going to swap the cables just to see if that has any effect, but considering the wild differences based on formatting, I have my doubts.

Does anyone have any idea what could be messing up the drive read/writes so much, especially just based on formatting?

 

[ZombiePhysicist]

I found these articles and not sure if it's related, but not sure it explains why my old 15TB drive still works faster?

UPDATED: SSD and ExFAT Speeds are Much Slower in macOS Ventura & Reformatting Drives is Trickier | Larry Jordan

https://www.reddit.com/r/MacOS/comments/18sglj3

 

[avkills]

OMG just shoot me.

So good news:
I got the drive. It shows right up. You can format it. You can use it.

The bad news:
The performance when you format at is as an APFS drive is horrible:

View attachment 2363072

Yet, if you format it as exFAT it works much better:
View attachment 2363073

This drive in theory should write at 3600MB/s and read at 7000MB/s. The 7,1 is PCIe3 so it should limit the speed. However, the 9400Pro, in windows, showed read writes at over 5000MB/sec, so I was expecting better reads and writes on this drive over my 15TB Micron 9300 Pro (which does about 2600MB/s read/write with FileVault on (faster with it off)). Frankly, I could live with the read/write speeds I get with exFAT as it's as fast or faster than my current drive, but I cannot live with teh slow 500MB/sec writes when formatted with APFS.

BTW, I moved my 15TB 9300 Pro to the 2nd port on the Highpoint card, and it still boots fine and it still read/writes at the same speed. I put this new drive on port0 on the card where the 15TB used to be.

ARG! This is so unbelievably frustrating.

I'm going to swap the cables just to see if that has any effect, but considering the wild differences based on formatting, I have my doubts.

Does anyone have any idea what could be messing up the drive read/writes so much, especially just based on formatting?

What happens if you format as HFS+?

That is frustrating.

 

[ZombiePhysicist]

So I booted into Boot Camp and windows, and I don't know if this is good news or bad news, but it performed basically as poorly in windows as it did on the Mac. It wrote around 400 MB per second, and read around 4000 MB per second using the eXFAT partition that was formatted on the Mac.

Interestingly if I run just 1GB test files it performs better on the AJA test, but if I increase it to 4GB, it comes down to the Mac speed (so I'll be testing this with bigger files). When I increase to 16GB files, the writes go up to 775MB/sec and reads to over 5300MB/sec under the AJA test. Really nice reads, probably as fast as PCIe3 can take. I dont get why the writes are such a dud.

But when I formatted as NTFS and windows, it basically performed as slowly as AFS did on the Mac. I'll come back and upload the screen grabs. I will reboot into the Mac and try the HFS plus to see if that makes any difference.

Truly confounding.

When I got the drive it was supposed to be new, but I looked and it had 1hour of use and 18 cycles on it. Perhaps that is just some testing. Is there any chance it's a bum drive being resold as new?

 

[ZombiePhysicist]

Thes tests were on windows:

This was the exFAT formatted on the mac but run on windows:

This is on the mac HFS+:

And impressively this was the AJA test on the mac formatted as HFS+:

This is the mac APFS:

This is mac with APFS:

So the size of the test file and format makes a big difference on read speed. But the bad write speeds are consistent.

This makes no sense. I'd love any theories why this performs so poorly and what can fix it. On the off chance that having both U.2 drives on the same card are an issue, or the cable, I'm going to try the test with just one drive and the original cable to see if that makes any difference. :/

 

[ZombiePhysicist]

The only thing I can think of is maybe it works with the Dual Sonnet card?

Fusion Dual U.2 SSD PCIe Card - SONNETTECH

Two U.2 NVMe SSD slots on a PCIe 3.0 x16 card. Add your own U.2 SSDs for Mac, Windows, and Linux.

www.sonnettech.com

I ordered the Sonnet dual fusion card (should get here friday) and several other more generic PCIe 4/U.2 adapter cards. Will give all of them one last try. Maybe one of them will click.

I also checked with the Solidigm utility in windows, and it reported some weird stuff, but purported that I have the latest firmware. I tried installing a driver but it said no Solidigm drive was found and i could install the driver later. My guess is maybe there is some incompatibility with the Highpoint card, but weirdly, the Intel drive and all its info shows up everywhere in Windows (and on Mac profiler too). So who knows what's going on...

 

[ZombiePhysicist]

One interesting test. I put in the Highpoint 7120 card (PCIe 3) and got these results:

So for whatever reason, it's 2.5 times faster on the HighPoint 7120 (PCIe 3) than on the HighPoint 1580 (PCIe 4) card, although the read speed obviously suffered going down from 5000MB/s to 3000MB/s.

If the writes would just come up to over 2000MB/sec I probably could live with this waiting for an M3 or M4 Mac Pro update. :/

Just weirdness all around.

 

[ZombiePhysicist]

BTW I also tried to use this drive with the Sabrent U.2 USB adapter. It would mount and read/write for just a little it and then lock up. Both on my MBP M1max and on the 7,1.

This is the only U.2 to thunderbolt adapter I've found.

 

[avkills]

Wow, that is very interesting. Yeah that is so weird that the write speeds are in the dump.

I've been holding out getting anything additional for my Mac Pro; just to wait out and see what Apple does. Based on the performance of my M3 Max laptop, it sure would be nice to have a M3 Ultra Mac Pro.

 

[ZombiePhysicist]

Wow, that is very interesting. Yeah that is so weird that the write speeds are in the dump.

I've been holding out getting anything additional for my Mac Pro; just to wait out and see what Apple does. Based on the performance of my M3 Max laptop, it sure would be nice to have a M3 Ultra Mac Pro.

It might be. Im waiting to see if they offer an M3Extreme and also if the new machine has PCIe5 and Thunderbolt 5. IF it has the latter, I might settle for the M3Ultra. But if the M3Ultra still has thunderbolt 4 and PCIe 4, I might risk waiting for the M4. Although I wonder if there will even be an M4 Mac Pro. Tough to gauge because apple runs things so poorly, ever Mac Pro may be the last.

That said, not sure if the PCIe 4 of the 2023 Mac Pro would make a difference with regard to this drives speed. If I knew it would, that would be enough reason for me to upgrade.

So beaten down, it's sad that it's gotten to this pathetic state of affairs.

 

[avkills]

I hear you, 2019 got everyone excited again and then the dumpster fire that is the 2023 version. Let us hope they make it up with the next version.

 

[ZombiePhysicist]

Ok more updates. I've now tried the drive (Solidigm/Intel SSDPF2NV307TZN1 D5-P5316 30.72 TB) with the following controller cards, and all exhibit the same slow write speed (wont bore the thread with the exact benchmarks):

DiLiVing PCIe to NVMe U.2 Adapter, PCIe 4X to SFF-8639 Interface for 2.5" NVMe SSD(PEX4SFF8639)

Boost system speed and performance by adding a PCIe NVMe U.2 SSD to your PC Simplify system builds with a U.2 to PCIe adapter that mounts your SSD directly to the adapter, with no need for adapter cables. PCI Express 3.0 x4 lane host adapter Connect your 2.5" NVMe SSD (U.2 / SFF-8639 interface) t...

www.amazon.com

GLOTRENDS PU11 U.2 to PCIe 4.0 X4 Adapter for 2.5 Inch U.2 SSD, PCIe 4.0 X4 Full Speed

GLOTRENDS PU11 U.2 to PCIe 4.0 X4 Adapter for 2.5 Inch U.2 SSD, PCIe 4.0 X4 Full Speed

www.amazon.com

This one mounts but gets errors read/write doing any tests:

GLOTRENDS PU21 Dual U.2 SSD to PCIe 4.0 X8 Adapter, Support 2 x U.2 SSD or Soft Raid, Without PCIe Splitter Function (PCIe Bifurcation Motherboard is Required)

GLOTRENDS PU21 Dual U.2 PCIe 4.0 Adapter, Support 2 x U.2 SSD or Soft RIAD Mode, Without PCIe Splitter Function (PCIe Bifurcation Motherboard is Required)

www.amazon.com

Also, for what it's worth, the HighPoint 1580 gets much better throughput than any of the above by at least 25%.

The absolutely nuts thing is if I format the drive using exFAT in macOS I get full read/write performance on the drive. Yet NTFS under windows, or HFS+/APFS on macOS has this horrendous slowdown in write speed for no discernible reason?

I'm guessing this must be some kind of driver issue?

Anyway, Tomorrow I'm getting the Sonnet fusion card and couple of more PCIe adapters to test. I'm hoping since Sonnet reports the drive as compatible, that the sonnet fusion will do something magical to make it work at full speed, but not holding my breath. :/

 

[avkills]

Thank you for struggling through this. Any insight gained is going to be very helpful for anyone doing future purchases.

 

[reglit]

OMG just shoot me.

So good news is the Solidigm/Intel SSDPF2NV307TZN1 D5-P5316 30.72 TB is seen by macOS via both my HighPoint cards (7120 (PCIe3) and 1580 (PCIe4)):
I got the drive. It shows right up. You can format it. You can use it.

The bad news:
The performance when you format at is as an APFS drive is horrible:

View attachment 2363072

Yet, if you format it as exFAT it works much better:
View attachment 2363073

This drive in theory should write at 3600MB/s and read at 7000MB/s. The 7,1 is PCIe3 so it should limit the speed. However, the 9400Pro, in windows, showed read writes at over 5000MB/sec, so I was expecting better reads and writes on this drive over my 15TB Micron 9300 Pro (which does about 2600MB/s read/write with FileVault on (faster with it off)). Frankly, I could live with the read/write speeds I get with exFAT as it's as fast or faster than my current drive, but I cannot live with teh slow 500MB/sec writes when formatted with APFS.

BTW, I moved my 15TB 9300 Pro to the 2nd port on the Highpoint card, and it still boots fine and it still read/writes at the same speed. I put this new drive on port0 on the card where the 15TB used to be.

ARG! This is so unbelievably frustrating.

I'm going to swap the cables just to see if that has any effect, but considering the wild differences based on formatting, I have my doubts.

Does anyone have any idea what could be messing up the drive read/writes so much, especially just based on formatting?

I heard that Solidigm D5-P5336 61.44TB SSD is also supported on MacOS. It might have better performance and it could be worth trying since it is the maximum capacity a Mac can have.

 

[ZombiePhysicist]

I heard that Solidigm D5-P5336 61.44TB SSD is also supported on MacOS. It might have better performance and it could be worth trying since it is the maximum capacity a Mac can have.

Really? Can you tell me where you heard this and if it worked full speed and if so with what adapter card? I appreciate the data point.

 

[reglit]

Really? Can you tell me where you heard this and if it worked full speed and if so with what adapter card? I appreciate the data point.

I heard it from a U2-to-thunderbolt dock manufacturer in China called WERO. It uses JHL 7440 + JMS583 as thunderbolt controller. From my pointer of view, most manufacturer uses similar controller chip. Hence I think D5-P5336 should work on other expansion card/dock too. It's not on the compatibility list of SONNETTECH probably because it's quite new and hard to find.

 

[ZombiePhysicist]

I heard it from a U2-to-thunderbolt dock manufacturer in China called WERO. It uses JHL 7440 + JMS583 as thunderbolt controller. From my pointer of view, most manufacturer uses similar controller chip. Hence I think D5-P5336 should work on other expansion card/dock too. It's not on the compatibility list of SONNETTECH probably because it's quite new and hard to find.

Thanks. Is there any chance you could find out if it worked at full speed for them? If this 30tb drive doesnt work, I may well try out the 60 even though its a bit more pricy than I was hoping to spend, the extra space won't hurt.

 

[reglit]

Thanks. Is there any chance you could find out if it worked at full speed for them? If this 30tb drive doesnt work, I may well try out the 60 even though its a bit more pricy than I was hoping to spend, the extra space won't hurt.

The read speed is not an issue but the write speed is around 2000mbs according to the feedback from the supplier. It is slower than 9400 pro since P5336 is an QLC ssd.

 

[ZombiePhysicist]

The read speed is not an issue but the write speed is around 2000mbs according to the feedback from the supplier. It is slower than 9400 pro since P5336 is an QLC ssd.

Thanks. That's probably good enough. The write speed on my 9300pro is around 2600MB/s. 2000 is slower than I'd like but at least in the world of tolerable. Sad that the 9400 Pro doesnt work. Thanks so much for the info!

 

[ZombiePhysicist]

Ok, latest update is none of the following improved the write speed problem. All around 400MB-550MB/s depending on the card:

Ableconn PEXU3-132 NVMe 2.5-inch U.3 SSD PCIe 4.0 x4 SFF-TA-1001 Carrier Adapter Card - for 2.5-inch U.3 NVMe SSD - U.3 to PCIe 4.0 - PCIe Adapter for U.3 Drive

Ableconn PEXU3-132 NVMe 2.5-inch U.3 SSD PCIe 4.0 x4 SFF-TA-1001 Carrier Adapter Card - for 2.5-inch U.3 NVMe SSD - U.3 to PCIe 4.0 - PCIe Adapter for U.3 Drive

www.amazon.com

Sonnet Technologies Fusion Dual U.2 SSD PCIe Card

Part No. FUS-U2-2X4-E3 The Most Economical Way To Add Ultra-Fast, Extra-Large Capacity SSD Storage Install Two U.2 NVMe SSDS Into a PCIe Card Slot – Ultra-fast Data Transfers If you need more storage capacity for your computer, and super high-speed throughput plus maximum IOPS (Input/Output Opera...

www.amazon.com

USB4 40Gbps Compatible with Thunderbolt 3 to U.2 NVMe SSD Enclosure, Up to 3000Mb/s Type C to U.2(SFF-8639) PCIe Adapter, Aluminum External 2.5” U.2 SSD Case Compatible with Thunderbolt 3/4 USB3.2/3.1

USB4 40Gbps Compatible with Thunderbolt 3 to U.2 NVMe SSD Enclosure, Up to 3000Mb/s Type C to U.2(SFF-8639) PCIe Adapter, Aluminum External 2.5” U.2 SSD Case Compatible with Thunderbolt 3/4 USB3.2/3.1

www.amazon.com

And this card wouldn't mount the drive at all:

StarTech.com U.2 to PCIe Adapter - x4 PCIe - For 2.5" U.2 NVMe SSD - SFF-8639 PCIe Adapter - U.2 SSD - PCIe SSD - U.2 drive (PEX4SFF8639)

"Get ultra-fast data access by adding an NVMe U.2 SSD to your computer. This U.2 to PCIe adapter lets you add a high-speed, U.2 (4-lane PCIe) 2.5"" NVMe SSD, such as the Intel 750 Series drive, to a PC with an available x4 PCI Express expansion slot. You can mount the PCIe U.2 drive directly onto...

www.amazon.com

The USB4 to U.2 case was actually pretty impressive. While the write speed was no faster, the read speed was around 2900MB/s, not bad at all for an external case.

Again, the weird thing with all the cards, and cases, if I format the drive exFAT, the write speeds zip right up to around 2500-2800MB/sec. But APFS/HFS+ and NTFS (in windows) all slow down to around 400-550MB/sec write speeds. It makes no sense.

BTW, the Highpoint 1580 is by far the fastest cards. Read speeds were around 5200MB/s with the highpoint card, while they maxed out at around 3200MB/s for all other cards, including the Sonnet.

I asked Sonnet if they can report if there is some known issue in how to get the Solidigm/Intel SSDPF2NV307TZN1 D5-P5316 30.72 TB writing at full speed.

I'm not sure what is the most likely culprit. The Micron 9300 Pro is not experiencing this writ slow down at all (I tested it along side with the Intel drive). At this point I either have a bum drive, or there is just some driver problem with both windows and macOS with this drive.

So what to do next?
-Option A is maybe exchange this drive and see if there is a problem with the replacement.
-Option B is maybe order the 60TB Solidigm drive as it has reports, above of working at 2000MB/s (it's rated for 3000 while this drive is rated at 3600MB/s, and once you turn on fileVault performance drops by about 25%) so that might be the way to go.

Any ideas?

 

[ZombiePhysicist]

So I got an interesting response from Sonnet!

I am sorry that you are not getting the expected speeds out of the Intel drive. I am honestly a little surprised this 30TB drive even works based off the issues I have seen with macOS and large capacity drives. Here is the tech note we have about this:
"As of 2023, the current maximum capacity of U.2SSDs is 30.72TB. The current maximum capacity supported by macOS is 15.36TB. This product will support higher-capacity SSDs as they become available."
As a result of this and the fact that the Micron drive works at full speed, I would say this is likely an issue with the interaction of the OS and the large capacity drive, so unfortunately I can not offer any tips to getting better write speeds with this drive. I would stick with the 15TB Micron drives if possible.

I do not know what to make of all this craziness. I very much appreciate the Sonnet folks at lest being aware of some of these issues. But considering we have reports, above, that the 61TB Solidigm drive "seems" to work at full write speeds, I'm even more baffled as to what it all means.

What I don't get about the above statement is that it is clearly not true with regard to capacity as 20+TB spinning drives work just fine in macOS. So not sure why there is a capacity limitation for SSDs?

Would appreciate anyone with any thoughts on any of this. I'm prepping to send the 30tb intel drive back, and was going to look for a 61tb Solidigm, but now wondering if I should bother... but with a hearsay report of the 61TB drive working, perhaps that's all we got to go on? Man, I cannot believe the dearth of information about such a simple thing. So weird.