Sidebar Sidebar
Become a MacRumors Supporter for $50/year with no ads, ability to filter front page stories, and private forums.

MP 1,1-5,1 installing windows 10 / 11 on Mac Pro 5.1

kk3mpp

kk3mpp

macrumors newbie
Original poster
Sep 26, 2023
3
0
hello there, i'm just wondering if i can get windows 10 or 11 on my Mac Pro 5.1...
my Mac Pro 5.1: 3.06 GHz 6-Core Intel Xeon / 16GB memory / GTX 680 2GB (non-flashed) / Big Sur on NVMe SSD 512GB (using OCLP)
i had windows 10 (with bootcamp drivers installed via brigadier) on SATA disk, which could be switched normally from Mojave on NVMe SSD.
But after installing Big Sur using OCLP on NVMe SSD, System Preference keeps showing "An error occurred while setting “BOOTCAMP” as the startup disk: The operation couldn’t be completed. (SDErrorDomain error 104.)" and can't get to windows SSD forever :(((((((
so i wanna ask you:
  1. how i can get "WORKING-WITH-OCLP-WINDOWS"?
  2. and can i boot into that "W-W-O-W" WITHOUT BOOT PICKER (SINCE I DON'T HAVE FLASHED GPU AND OCLP BOOT PICKER)
P.S. if there's a way to get OCLP boot picker with my non-flashed GTX 680, please tell me...
 
kk3mpp

kk3mpp

macrumors newbie
Original poster
Sep 26, 2023
3
0
h9826790 said:
No offence, but the first question in my mind is "why don't you flash the GTX680"?
Click to expand...
ty for your reply!!
i was thinking about flashing my gpu too, and i tried using from-windows way but it wasn't working for me...
also i recently bought XFX RX 5600 XT so i thought its no use to flash for 3-4 days use lol
 
h9826790

h9826790

macrumors P6
Apr 3, 2014
16,614
8,546
Hong Kong
kk3mpp said:
ty for your reply!!
i was thinking about flashing my gpu too, and i tried using from-windows way but it wasn't working for me...
also i recently bought XFX RX 5600 XT so i thought its no use to flash for 3-4 days use lol
Click to expand...
If that's a new 5600XT, and your OpenCore was setup correctly. Then may be it can give you the UEFI GOP boot screen straight away.

XFX is well known has firmware compatibility issue on cMP. The boot picker may be in weird colour, but at least you can still see the screen and use it accordingly.

To flash a GTX680, you can use your existing Windows, DOS, or Linux. It should be as simple as download the Mac EFI GTX680 ROM, backup the existing ROM from your card, then flash it.

If you can't switch to your Windows now, you may try Linux, you can boot Linux from a LiveCD / LiveUSB directly. No need to install, very handy.

And from my own experience, flashing in Linux also works better than in Windows, less unknown error to deal with.
 
kk3mpp

kk3mpp

macrumors newbie
Original poster
Sep 26, 2023
3
0
ty again for your reply!
it's a used one so i'm a bit scared if i can use it for my cMP :( and yes i read some documents that XFX RX 5xx series isnt't working on cMP, but i also read this thread that XFX RX 5600 XT had a compatibility with macOS. so maybe gpu won't be like not-working-completely ig...
ty for your quite understandable method of flashing GTX 680 :) i'll try it laterr.
it seems that you are an expert of customizing cMP so if you don't mind, plz let me ask you about problems that i can't figure out.

P.S. is "An error occurred while setting “BOOTCAMP” as the startup disk: The operation couldn’t be completed. (SDErrorDomain error 104." this error occur in Monterey too?? i'm thinking about doing clean install of OCLP Monterey and installing windows in UEFI mode
 
N

N008

macrumors newbie
Jan 19, 2024
18
0
kk3mpp said:
hello there, i'm just wondering if i can get windows 10 or 11 on my Mac Pro 5.1...
my Mac Pro 5.1: 3.06 GHz 6-Core Intel Xeon / 16GB memory / GTX 680 2GB (non-flashed) / Big Sur on NVMe SSD 512GB (using OCLP)
i had windows 10 (with bootcamp drivers installed via brigadier) on SATA disk, which could be switched normally from Mojave on NVMe SSD.
But after installing Big Sur using OCLP on NVMe SSD, System Preference keeps showing "An error occurred while setting “BOOTCAMP” as the startup disk: The operation couldn’t be completed. (SDErrorDomain error 104.)" and can't get to windows SSD forever :(((((((
so i wanna ask you:
  1. how i can get "WORKING-WITH-OCLP-WINDOWS"?
  2. and can i boot into that "W-W-O-W" WITHOUT BOOT PICKER (SINCE I DON'T HAVE FLASHED GPU AND OCLP BOOT PICKER)
P.S. if there's a way to get OCLP boot picker with my non-flashed GTX 680, please tell me...
Click to expand...


Hi kk3mpp,

My answer here comes with a slight delay, but maybe it helps someone else as well. I am a newbie with Mac Pro 5,1(specs at the end) and I had same issue. If I had already settled with the idea and prepared all the tools for flashing GPU, I found other way - you can flash your ROM with "EnableGOP method", which broadens your options to use many more GPU models.


Bmju said:
You wait ten years for a driver to enable pre-boot graphics on unsupported GPUs in EFI era Macs, and then two come along at once...

This post is to announce a firmware driver to enable the native Apple boot picker and early macOS boot progress bar (plus other parts of the firmware UI: target disk mode; firmware password; internet recovery on machines which have it; etc.) on non-natively supported GPUs, before, or even without, the rest of OpenCore.

The basic idea is: if your GPU already works in but not before OpenCore, then with this driver added to your vBIOS or firmware it should be able to work before OpenCore, too.

It has now been fairly widely tested, on both iMac and Mac Pro, but nevertheless it is a firmware driver:
  • If for any reason it fails (or perhaps is incorrectly installed) it can completely brick your machine.
  • If you are not comfortable modifying and burning updates to firmware or vBIOS, and recovering from bricks of either of these (which will involve additional hardware) - do not proceed. (Or at the very least, wait for clear and replicable instructions and proven success stories for your specific system, from other users who do have this expertise, before proceeding.)
Instructions and required files can be found in the /Utilities/EnableGop directory of the current OpenCore release. For the very latest build (if any recent updates have been made) see the first couple of posts in the thread below.

Background​


This driver has a completely separate origin and development history from @Syncretic's current impressive work. This driver aims to be lightweight and standalone. It comes naturally out of the huge amount of work already done in OpenCore (and mainly finished a couple of years ago, except for these additional steps) to support the OpenCore boot menu on non-natively supported GPUs, within OpenCore. It is also - as part of OpenCore - open source.

After reversing enough of the firmware to work out how to link OpenCore’s GOP to the Apple firmware UI protocol, it seemed worth trying to enable this before OpenCore as well - i.e. to try to get 'as native' support for these cards. A little bit more experimentation made it clear that the best way to do this would be to piggy-back off the existing, very well tested work already done in OpenCore to support these cards - basically to package up the required parts of OpenCore (ForgeUefiSupport, ProvideConsoleGop and the recent code to connect this to the Apple firmware UI) into a firmware injectable driver, and then figure out a way to deliver its 'payload' (particularly the ProvideConsoleGop part) at the right time. Early versions also used OpenCore’s ReloadOptionRoms, as OpenCore has to do, to load any option ROM which needs ForgeUefiSupport - but with the correct approach it was possible to avoid this, letting the firmware do it for us, which turned out to make the driver much more stable.

So after considerable reversing, some additional new code, and a lot of helpful testing and input from those listed below, this is now working.

Tested Mac models:​

  • MacPro4,1/5,1
  • MacPro5,1
  • iMac10,1
  • iMac11,1
  • iMac11,2
  • iMac11,3
  • iMac12,2
Notes:
  • The current version of the driver is believed to be safe on all 2009-2012 iMacs and on the MacPro4,1/5,1 and MacPro5,1
    • iMac12,1 should be supported but not yet known: a confirmed test result would be welcome
  • Most recent available firmware in all cases
  • Other Mac models not yet tested, and outside the ones listed above probably will not work (since the required patches apply to the listed firmware only) - please PM me to discuss support

NOT Supported:​

  • The current version of the driver is NOT compatible with the MacPro3,1, it will make the boot process hang and should not be installed there

Tested GPUs:​

  • GT610 (with original or added GOP in VBIOS)
  • GT640
  • GT710
  • GT720 (EnableGopDirect)
  • GT730
  • GTX60
  • GTX660
  • GTX670
  • GTX750Ti
  • GTX780
  • GTX960
  • GTX1050Ti
  • GTX1070
  • GTX1080Ti
  • HD 7770
  • HD 7970 (all except EnableGop(Direct) version 1.2)
  • K2000
  • K2000d
  • K420
  • K600 (with GOP addition in VBIOS)
  • M4000, M5100, W5170M (Venus, MXM)
  • M6100, W6150M, W6170M (Saturn, MXM)
  • NVS 510
  • P3000
  • Radeon VII
  • RX460
  • RX480 (PCI and MXM)
  • RX5500XT (PCI and MXM)
  • RX5700XT
  • RX580 Nitro+
  • RX6800
  • RX6900XT
  • W6600
  • Vega 56
  • Vega (687F)
  • RX560, RX570, RX580, RX590
  • S7100X (MXM)
  • WX4130, WX4150, WX4170, WX7100 (MXM)

Notes​

  • All GPUs work with EnableGop, unless explicitly listed as requiring EnableGopDirect
  • Some GPUs listed above may need additional firmware - such as a GOP driver for older GPUs which do not come with one; or other patches - in order for them to work with OpenCore in the first place (hence to be eligible to work with EnableGop in firmware); try searching for the card in this thread or the following iMac specific threads:
  • The driver should also work fine with natively supported GPUs such as GT120 (tested) (e.g. when installed in main firmware and swapping cards)
  • It should work with OpenCore (of course) and with RefindPlus
  • OpenCore settings which this driver already implements can be, but do not have to be, disabled

Releases of EnableGop in not-yet-released versions of OpenCore may be obtained as per the first couple of posts in the thread below. Older versions may be downloaded as required from the named OpenCore release.

If you find you are short on space when flashing to vBIOS - which can be especially a problem for AMD GPUs - then try EnableGop 1.1 (available with the 0.9.0 release of OpenCore).

EnableGop version (released with OpenCore version):

1.4 (0.9.3)​

  • Incorporates recent updates to OpenCore console control code, but no difference in behaviour compared to version 1.3 is expected on any supported systems.

1.3 (0.9.2)​

  • Included fix to GopBurstMode for non-standard frame buffer information on AMD Radeon HD 7970 and similar
  • Applied GopBurstMode even on natively supported cards, as it can provide a noticable speed up
Note: GopBurstMode is applied by all versions of this driver for the speed-up it gives, so it may be advisable to test OpenCore with GopBurstMode enabled before burning to main firmware or vBIOS. However the driver has been fairly widely tested now, and there are no known remaining issues on any systems listed as supported, after the fix mentioned above.

1.2 (0.9.1)​

  • Added GopBurstMode support
Note 1: This should provide faster GOP rendering on all EnableGopDirect systems; and rendering at least at the same speed as before, and on some systems noticeably faster than before, on almost all EnableGop systems.

Note 2: The compressed driver for version 1.2 is 1KB larger than for version 1.1, so for AMD vBIOSes which are tight on space version 1.1 may be used instead to avoid the need for VGA stripping to make additional space.

1.1 (0.9.0)​

  • Fixed early verbose boot lines appearing over picker
  • Added EnableGop version number to UI section

1.0 (0.8.9)​

  • Initial public release

The vBIOS insertion script vBiosInsert.sh now supports both AMD and Nvidia cards.

In the case of AMD, considerably less space is normally available than with Nvidia, due to a strict limit of 128k for legacy and EFI parts of the potentially larger ROM image (the rest of which is only usable internally by the card itself).

So far, there has largely been enough spare space on desktop format (PCIe) cards for Mac Pro, and not enough space on iMac format (MXM) cards. If there is not enough space (i.e. script reports ROM data exceeds the 128k limit) then it is necessary to strip some legacy VGA parts of the vBIOS, or check on the iMac threads listed in the next spoiler to see if this has already been done for your card.

You can also inject EnableGop into the main system firmware instead of the GPU vBIOS - see the README.md file in the Utilities/EnableGop directory of the most recent OC builds, or here. In that case the AMD vBIOS limit does not matter.

There has been an active community of iMac users making updates and modifications to iMac GPU vBIOSes since long before EnableGop was written. They have kindly adopted EnableGop (and helped to test it), so for many iMac GPUs, you can already find a modified version including EnableGop (and with legacy VGA parts stripped if necessary, see previous spoiler), if you search for your GPU in these threads:

The script relies on the EDK-II EfiRom tool to compress the driver EFI file into option ROM format, and on UEFIRomExtract as part of verifying the modified ROM.
  • A current version of EfiRom can be obtained from the Utilities/BaseTools directory of OpenCore releases (0.9.0+)
    • Just for the purposes of obtaining EfiRom in this way, you should avoid the Dortania build server (for technical reasons, which don't affect the rest of OpenCore, it produces an Apple-silicon EfiRom); use releases (or other builds, if need be) from the Acidanthera OpenCore site.
  • UEFIRomExtract can be obtained here: https://github.com/andyvand/UEFIRomExtract/tree/master/Release
These additional tools should be on your path rather than in the same directory as the script. For example, you could:
  • mkdir ~/MyTools
  • Copy the required tools into ~/MyTools
  • Add export PATH=~/MyTools:$PATH to ~/.bashrc or ~/.profile
  • Close and re-open your shell window
Also useful, if you want to starting looking inside vBIOS roms:
And for examining main firmware:

Credits​

Click to expand...
Other words:
"EnableGOP - is the firmware driver to enable the native Apple boot picker and early Mac OS Boot progress bar (plus other parts of the firmware UI: target disk mode, firmware password, internet recovery on machines, which have it etc.) on non-natively supported GPU’s, before, or even without, the rest of OpenCore.
(it’s a firmware patch so you can flash your Mac Pro’s internal boot ROM or / you can flash your GPU or other hardware component)"

Just some days after my preparations I saw Lance's video about flashing your ROM with EnableGOP method. Of course I was very scared with the idea of "bricking my mac", but with some encouragement from Joerg Henninges I grew more balls and... I did it successfully like Lance from Mac Sound Solutions inc. showed in his tutorial and... BOOM, it works very nice so far I can use boot selector and get the apple booting screen .

This worked for me. Still I have to mention...
!!! IT IS YOUR RISK WHICH METHOD YOU CHOOSE - EVEN IF YOU FOLLOW ALL THE STEPS!!!


Here are the links for "EnableGOP" videos:


Special thanks to the:
Martin Lo https://www.youtube.com/@h9826790
Macschrauber https://forums.macrumors.com/thread...es.2333460/page-4?post=32055801#post-32055801
Dosdude (Collin) https://dosdude1.com/apps/
tsialex https://forums.macrumors.com/threads/macpro5-1-bootrom-thread-144-0-0-0-0.2132317/
Lance https://www.youtube.com/@MacSoundSolutions
Joerg https://www.youtube.com/c/JoergHenningesWelcome2MyWorld/featured

My set up:
Mac Pro 5,1 2010(ROM flashed - EnableGOP method)
- Mojave 10.14.6
- 2 x Intel(R) Xeon(R) CPU X5690 @ 3.47GHz (Westmere)
- 64Gb DDR3 EEC 1333MHz (8 x 8 Gb)
- 256Gb SSD
- Radeon PULSE RX 580 4Gb
- superdrive
 
Last edited:
N

N008

macrumors newbie
Jan 19, 2024
18
0
Hi Martin h9826790,

I tried to install Windows 11 with your OC version by Lance's video
So actually up to 6:50 everything ran nice. I did exactly everything he said in the video, even used the same Rufus release, but I ended with following situation :
1. Rebooting the 5,1
2. Getting bootpicker with 3 options [ Moneterey; EFI(with disk image) and EFI(with USB image ]
3. Selecting EFI(with disk image)
4. Getting bootpicker with 3 options [ Moneterey(with double disk image); Recovery 12.7.3(dmg); UEFI_NTFS(with yellow external disk image)
5. As soon as I hit the UEFI_NTFS boot I get 5 minutes black screen*** and after that 5,1 reboots back to the bootloader... and I can restart the process infinite times... still no Windows installer logo.
***(only pressing enter or some other keys get me back to the bootloader selection)

1) What could be the possible issue and what could be the solution?
2) Do I have to flash the rx580?(But I have flashed the ROM chip already with EnableGOP method)


I have:
Mac Pro 5,1 2010(ROM flashed - EnableGOP method)
- Bay 1: Monterey 12.7.3 + OC Martin Lo
- Bay 2: For Windows 11
- Bay 3: Mojave 10.14.6(removed for the Windows 11 installation)
- 2 x Intel(R) Xeon(R) CPU X5690 @ 3.47GHz (Westmere)
- 64Gb DDR3 EEC 1333MHz (8 x 8 Gb)
- 256Gb SSD
- Radeon PULSE RX 580 4Gb
- superdrive
 
Last edited:
N

N008

macrumors newbie
Jan 19, 2024
18
0
And here is my 09.03 update.
(Pictures in attachments are ordered by time and course of action, below)
(I can add more pictures of the whole process if necessary)
Well 09.03.
In the moment I managed to advance in my installation process (step 4) I wanted to say “Don’t ask how I did it”, but here it comes…

I tried with both (Monterey and Mojave) drives. And here’s what worked for me:

Before install:
I made deep NVRAM reset (not related to following procedures, I just checked that its working. But I don't know does this have anything to do with my successful installation.)


1. I installed OCLP on Mojaves ESP.
2. Rebooted: selected EFI(OCLP’s bootloader) > NTFS(windows 11 USB drive made with Rufus)
3. Got the blue screen with message(I can share the pic, not going to write all that message out here):
“Your PC/Device needs to be repaired” and error message
/…/

File: \windows\sustem32\winload.efi
Error code: 0xc000000e

/…/

Press Enter to try again
Press F1 to enter Recovery Environment
Press F8 for Startup Settings
Press F9 to use a different operating system

4. I pressed fn + F9
(before this attempt I had tried the same procedure when I had Monterey drive connected and any of those key combinations did not work, so I had low expectation it would work with Mojave drive)

5. I proceeded again with jensd_be video
10 : 00 (I actually forgot to check this time that, could I proceed with Lance’s video from 10:01 by hitting next, because in one of my previous attempts the button was gray and using delete or format didn’t work - it gave also the error message, but unfortunately I forgot to take picture)

6. So everything went fluent til the part of selecting the “Set up the personal use” or “Set up for work or school” (the video was bit too fast forwarded for that and maybe some frames were cut), but I was messing for few minutes to get passed creating the Microsoft account.

7. Babysitting the rest of the installing process:

1st autoreboot
bootloader
win screen loaderwheel

2nd autoreboot
bootloader
win screen loaderwheel

3rd autoreboot
bootloader
win screen loaderwheel
Just a moment

4th autoreboot(maybe there were 1 autoreboot less, I might of lost the count)
bootloader
win screen loaderwheel
Welcome
“Name your Device”

5th autoreboot
bootloader
win screen loaderwheel
Welcome

Creating Windows user

8. Windows update

1st Update restart

2nd autorestart
autoreboot
bootloader
win screen loaderwheel
Log in screen


MY QUESTIONS TODAY(after the install):
9. I Would like to know is everything as it should be:
- Is the BootROM ok?
- Is the NVRAM ok?

So for the final confirmation, I wish to have all the (OCLP; Monterey drive; Windows drive) set up as it should be.
- Does my ESP’s have the contents they have to have? (Mojave ESP; Windows drive ESP; Monterey ESP)
- What do I have to do to connect Monterey in Bay 1(ESP contents, Can I delete Mojave’s ESP finally, Do I have to have ESP partition for the Windows drive(Microsoft folder location)?

+
Secondary matters(no point to deal with them before I can be sure in step 9. - that I don't have to reinstall, because I don't have ESP partition on Windows drive(I don't know yet, do I need?) or etc.):
10. Still don’t have the bluetooth and audio(audio is only via screen speaker/3.5mm)

(Pictures in attachments are ordered by time and course of action):
3. "Your PC/Device needs to be repaired" Mojave - OCLP result.jpg
9. Mojave EFI after windows install.jpg

I have:
Mac Pro 5,1 2010(ROM flashed - EnableGOP method)
- Bay 1: Monterey 12.7.3 + OCLP(disconnected currently)
- Bay 2: For Windows 11
- Bay 3: Mojave 10.14.6 + OCLP(+ Microsoft folder)
- 2 x Intel(R) Xeon(R) CPU X5690 @ 3.47GHz (Westmere)
- 64Gb DDR3 EEC 1333MHz (8 x 8 Gb)
- 256Gb SSD
- Radeon PULSE RX 580 4Gb
- superdrive
 

Attachments

  • 3. "Your PC:Device needs to be repaired" Mojave - OCLP result.jpg
    3. "Your PC:Device needs to be repaired" Mojave - OCLP result.jpg
    607.8 KB · Views: 24
  • 9. Mojave EFI after windows install.jpg
    9. Mojave EFI after windows install.jpg
    49.9 KB · Views: 24
Last edited:
You must log in or register to reply here.
Register on MacRumors! This sidebar will go away, and you'll see fewer ads.
Top Bottom