Proxmox VE OSX Guide discussion

[Driftwood]

 

1 hour ago, TheDaemonBarber said:

EB|`B:WFDW] Err(0xE), 0 @ LocHB 71B4903C-14EC-42C4-BDC6-CE1449930E49

 

 

Edited June 6, 2020 by Driftwood

[TheDaemonBarber]

Hey Driftwood. Yeah, I tried that, and a bunch of other related Quirks tweaks I found with similar searches on the topic. None of them helped, but I'm going to try this Clover route now and see if it helps.

*Trying everything from scratch again, because the Clover route gave me the same error as Opencore*.

[fabiosun]

@TheDaemonBarberexplain please 🙂

are you passing entire disk where are you booting?OSX and EFI I mean?

 

[TheDaemonBarber]

@fabiosun I've tried with both a VM created virtual drive as per the basic install on your guide/Nick Sherlock's guide. I've also tried with deleting that and passing an empty and waiting NVME drive, which works passing through to a Windows VM, but both methods are failing.

 

I've essentially followed your guide now, from the beginning to create a new VM container, got VM assistant to download and create the Clover ISO, the Catalina ISO... loaded those as IDE drives...

 

"failed to start Boot0003 "EFI Internal Shell".

I'm getting really disheartened with all this, but will push forward.

 

Maybe I need to yet again destroy Proxmox and start from complete scratch.

[fabiosun]

no, how to you upload EFI on Proxmox?

EFI.ziptry attached efi if you want 😉

 

[Rocket88]

21 hours ago, Rox67er said:

I checked running Proxmox disk on the USB-C port and passing the 47:00.1 and 47:00.3 USB Controllers. (As long as I block bluetooth / wifi MacOs works fine, as soon as I try to load the intelUSB kext my Aquantia controller gets messed up. My wifi/BT upgrade has not arrived yet...)

 

However, in this config I can only boot the VM once. If I shut down and then start the VM again, it locks up with the 99 error on the motherboard. It may be related to older AGESA / BIOS since we are still on 1.0.0.3 B if I understand correctly, Fabiosun has been on 1.0.0.3 C and is now on 1.0.0.4.

 

Could also be the reset bug for the Radeon VII.

I am getting the 99 error on my Asrock TRX40 Creator also. I thought it was just me. I get this error approximately every 3rd time I power up. This includes booting with Windows by hitting F11 on startup. To make the error go away, I have to unplug (or power off) the power supply, start the computer, which drains the capacitors, plug in the power supply, and then start again. Everything works great after that. The other issue that I can't seem to solve is Proxmox will only start on HDMI, not DisplayPort.  

[Driftwood]

3 hours ago, Rocket88 said:

I am getting the 99 error on my Asrock TRX40 Creator also. I thought it was just me. I get this error approximately every 3rd time I power up. This includes booting with Windows by hitting F11 on startup. To make the error go away, I have to unplug (or power off) the power supply, start the computer, which drains the capacitors, plug in the power supply, and then start again. Everything works great after that. The other issue that I can't seem to solve is Proxmox will only start on HDMI, not DisplayPort.

 

I reckon theres something up with this board. We're all getting, or have had similar issues. I noticed if I plug my QNAP TR004 USB-C 4 bay 'spinner' drive box in, I get freezes every now and again - I'm trying to narrow it down but I think it happens when screen saver kicks in. I also get the occasional USB disappearance on a Corsair Gaming iCue keyboard. *Note: I have 'Put drives sleep' and 'Wake on network' disengaged in sleep in sys preferences. But generally screen saver and screen sleep works.

 

I think the APCi stuff needs to be sorted in BIOS or we need a fw update asap. We should all hit the ASRock site and forum and get them to look into it.

 

I knew I had to clear down memory after a shutdown. Shutdown Mac, then Shutdown Now on Proxmox once Mac has closed. Pull plug or back switch, wait 5 seconds, turn on and reboot.

 

However, things have been more stable at running 32 cores and at least 8Gb memory switched out of your total. 

 

Has anyone done USB/dsdts yet in OpenCore?

Edited June 6, 2020 by Driftwood

[Driftwood]

ASRock Creator owners please report your problem here and we can get some action - you dont have to mention Mac!

 

https://event.asrock.com/tsd.asp

3 hours ago, Rocket88 said:

The other issue that I can't seem to solve is Proxmox will only start on HDMI, not DisplayPort

I can boot on Displayport or/and with HDMI no problem.

[Driftwood]

5 hours ago, TheDaemonBarber said:

I'm getting really disheartened with all this, but will push forward.

Dont get disheartened, we've all been there!

[TheDaemonBarber]

Hmm. Still getting the following error that stops me booting any further. 

    

EB|`B:WFDW] Err(0xE), 0 @ LocHB 71B4903C-14EC-42C4-BDC6-CE1449930E49

I've put my backed up, working, baremetal install onto the passed through NVME drive. I replaced the EFI on that drive with the one @fabiosun posted just above. I can start the VM, the GPU seems to pass correctly, and Opencore loads on the Guest monitor. I choose to load the Mac up, and Opencore starts its boot sequence and once again gets stuck on the above error.

 

Now, I found this guy talking about how he turned off "Above 4G encoding" in the BIOS, to get past this error. I tried that, but now starting the VM just results in a black screen.

    https://amd-osx.com/forum/viewtopic.php?t=10503

I've tried it with 4G encoding on and off a few times. Off means the VM boots to a black screen and I can't see anything. No logs generated in the EFI either. On, and I get the error.

I'm not sure if it's relevant, but the VM Status shows 97% of the 32GB RAM I assign to it, when it hits this error.

 

I think there's a good chance I'm suffering from a BIOS issue. Does anyone have any tips for an X570 board, or even an ASROCK one?
I'm going to be going through the BIOS with a fine tooth comb soon, so if anyone can save me any time with some tips, I'd be very grateful.

 

 

[Rox67er]

4 hours ago, Rocket88 said:

I am getting the 99 error on my Asrock TRX40 Creator also. I thought it was just me. I get this error approximately every 3rd time I power up. This includes booting with Windows by hitting F11 on startup. To make the error go away, I have to unplug (or power off) the power supply, start the computer, which drains the capacitors, plug in the power supply, and then start again. Everything works great after that. The other issue that I can't seem to solve is Proxmox will only start on HDMI, not DisplayPort.  

Which video card do you use? Do you load lilu.kext and whatevergreen.kext in OpenCore? Should solve most HDMI / Displayport output issues.

 

Thanks for confirming the 99 error, at a point I thought I had a bad mother board. Hope ASRock can improve in the biosfirmware and next Proxmox version should include new kernel with a lot of AMD improvements. 

[iGPU]

On 6/5/2020 at 1:22 PM, Rox67er said:

I checked running Proxmox disk on the USB-C port and passing the 47:00.1 and 47:00.3 USB Controllers. (As long as I block bluetooth / wifi MacOs works fine, as soon as I try to load the intelUSB kext my Aquantia controller gets messed up. My wifi/BT upgrade has not arrived yet...)

 

I'm using a swapped BT/Wfi card. I do pass 48:00.1 and 48:00.3.

 

I too had all sorts of trouble with the Aquantia, so I ended up leaving it for the host and passing the Intel I211 and using the usual SmallTree kext.

 

But I thought my problem may have been because I initially ran WIndows 10 and I updated various drivers. From old posts about Aquantia (I think dating from 2017 or thereabouts, when Sierra (High?) began supporting Aquantia), there were indications that once WIndows wrote a driver to the Aquantia chip, it wouldn't work under macOS. But that if you used macOS up until Mojave 10.14.3, macOS could write to the Aquantia chip (but then Aquantia wouldn't work under WIndows).

 

So I don't know if the Aquantia problem I had is related to having written a WIndows driver to it or some other glitch. Anyhow, I don't have a 10G Ethernet system, so I'm content with the I211 port working.

Edited June 6, 2020 by iGPU

[iGPU]

Driftwood,

 

I got the same model StarTech FireWire card as you use. However, it too generates a bridge and is unusable in macOS. I give up (I now have so many FireWire cards...).

 

I truly do not understand how you have no bridge appearing, unless the bridge issue is unique to mobos, and we do have different mobos. But this doesn't quite make sense as I thought most of the mobo behavior is dictated by the chip, in our case, the TRX40. Nevertheless, as long as a bridge shows up on a PCIe device, we cannot pass that device via VM (and to repeat what I've posted earlier, this seems to be the reason that Thunderbolt cards cannot be passed).

 

If I understand correctly (but I may not be), some of the things being written about the latest Linux kernels, indicate that Linux may soon support the passing of bridges. But Proxmox seems to be a year or so behind using the latest kernels. (This makes me want see if I can get a VM working in AcroLinux.)

[Driftwood]

Here's my OC settings n stuff. Hope it helps!

OC-Driftwood.zip

 

Firewire in MacOS

 

 

Edited June 6, 2020 by Driftwood

[Rocket88]

9 minutes ago, Rox67er said:

Which video card do you use? Do you load lilu.kext and whatevergreen.kext in OpenCore? Should solve most HDMI / Displayport output issues.

 

I am using a single MSI Vega64 card in the first slot. I have lilu.kext and whatevergreen.kext installed and I am running OpenCore 0.58

I had to add this line to grub to make the video pass thru:

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt amd_iommu=on video=efifb:off"

 

To start my VM, I either need another computer or I set onboot: 1 in my 100.conf file

[Driftwood]

11 minutes ago, iGPU said:

I got the same model StarTech FireWire card as you use. However, it too generates a bridge and is unusable in macOS. I give up (I now have so many FireWire cards...)

 

If it has Cheetah bridge its not going to work . You need the single chip LP version. Send that one back!

 

Does it show one or two device addresses in lspci -nn -k? 

Edited June 6, 2020 by Driftwood

[Rox67er]

58 minutes ago, Driftwood said:

ASRock Creator owners please report your problem here and we can get some action - you dont have to mention Mac!

 

https://event.asrock.com/tsd.asp

I can boot on Displayport or/and with HDMI no problem.

I just filed the 99 error, hope they pick this up since MSI boards don't seem to suffer the issue but HW wise are very similar.

[Driftwood]

My GRUB line 

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on video=efifb:off"

[Rox67er]

24 minutes ago, iGPU said:

 

I'm using a swapped BT/Wfi card. I had all sorts of trouble with the Aquantia, so I ended up leaving it for the host and passing the Intel I211 and using the usual SmallTree kext. I do pass 48:00.1 and 48:00.3.

Interesting, I have a 94360NG on order. Should be a swap in replacement for the onboard version. If I don't use WIFI/BT my aquantia plays perfect. I patched it to internal and even iMessage works.

 

Regarding my 99 error, are you able to shutdown VM and start it again while passing through the USB controllers? Since you also run the Radeon VII this would rule out the Reset Bug as a cause but point at differences in bios. Maybe should have gone for the MSI board... 🤬

[Driftwood]

5 minutes ago, Rox67er said:

If I don't use WIFI/BT my aquantia plays perfect. I patched it to internal and even iMessage works.

You need to rip out the BT/Wifi card from the board. It interferes with proxmox/Mac.

You need to get the Realtek 8125 Linux driver into Proxmox boot or the Aquantia or the 8169 or whatever is will go missing.

 

here's my Dmesg for it

 

[    1.212042] r8125: loading out-of-tree module taints kernel.
[    1.212860] r8125 2.5Gigabit Ethernet driver 9.003.04-NAPI loaded
[    1.212874] r8125 0000:46:00.0: enabling device (0000 -> 0003)
[    1.214183] ahci 0000:48:00.0: version 3.0
[    1.216264] firewire_ohci 0000:4c:00.0: enabling device (0000 -> 0002)
[    1.227995] r8125: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    1.230006] r8125  Copyright (C) 2020  Realtek NIC software team <nicfae@realtek.com> 
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>. 
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>. 
[    1.244004] r8125 0000:46:00.0 enp70s0: renamed from eth0
 

Edited June 6, 2020 by Driftwood

[Rox67er]

10 minutes ago, Rocket88 said:

I am using a single MSI Vega64 card in the first slot. I have lilu.kext and whatevergreen.kext installed and I am running OpenCore 0.58

I had to add this line to grub to make the video pass thru:

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt amd_iommu=on video=efifb:off"

 

To start my VM, I either need another computer or I set onboot: 1 in my 100.conf file

You can probably remove the video=efifb:off if you use my hook script at the start of the VM, it effectively does the same as the grub setting but at least gives you normal boot with capability to start the VM.

Look here: https://github.com/iroger/ASRock-TRX40-proxmox/tree/master/var/lib/vz/snippets

Add to your VM 100.conf like:

hookscript: local:snippets/reset-gpu.pl

 

If you leave out the last command in the reset script:

system("rtcwake -m mem -s 3");

 

You don't get the 3 second system stand-by (To reset GPU from D3 power state) but effectively get the same as video=efifb:off

[Driftwood]

40 minutes ago, TheDaemonBarber said:

I've tried it with 4G encoding on and off a few times. Off means the VM boots to a black screen and I can't see anything. No logs generated in the EFI either. On, and I get the error.
 

Yeah I install Proxmox with Above 4G off, and put it on later.

 

Sometimes, you can pull the Displayport out (or HDMI) from the back and give it a few seconds and put it back in... then wait and give it a minute and see if screen comes alive.

[iGPU]

25 minutes ago, Driftwood said:

 

If it has Cheetah bridge its not going to work . You need the single chip LP version. Send that one back!

 

Does it show one or two device addresses in lspci -nn -k? 

 

03:00.0 PCI bridge [0604]: Texas Instruments XIO2213A/B/XIO2221 PCI Express to PCI Bridge [Cheetah Express] [104c:823e] (rev 01)
04:00.0 FireWire (IEEE 1394) [0c00]: Texas Instruments XIO2213A/B/XIO2221 IEEE-1394b OHCI Controller [Cheetah Express] [104c:823f] (rev 01)
        Subsystem: Device [3412:7856]
        Kernel driver in use: firewire_ohci
        Kernel modules: firewire_ohci

 

I see no other style FireWire cards to order... Every single one has the Cheetah bridge.

[cj750]

14 minutes ago, Driftwood said:

You need to rip out the BT/Wifi card from the board. It interferes with proxmox/Mac.

You need to get the Realtek 8125 Linux driver into Proxmox boot or the Aquantia or the 8169 or whatever is will go missing.

 

here's my Dmesg for it

 

[    1.212042] r8125: loading out-of-tree module taints kernel.
[    1.212860] r8125 2.5Gigabit Ethernet driver 9.003.04-NAPI loaded
[    1.212874] r8125 0000:46:00.0: enabling device (0000 -> 0003)
[    1.214183] ahci 0000:48:00.0: version 3.0
[    1.216264] firewire_ohci 0000:4c:00.0: enabling device (0000 -> 0002)
[    1.227995] r8125: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    1.230006] r8125  Copyright (C) 2020  Realtek NIC software team <nicfae@realtek.com> 
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>. 
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>. 
[    1.244004] r8125 0000:46:00.0 enp70s0: renamed from eth0
 

Reading the the BT/WIFI does not work unless i get a internal  adapter. Do the 10g and 2.5 network adapter work as designed on the board?

[iGPU]

21 minutes ago, Driftwood said:

You need to rip out the BT/Wifi card from the board. It interferes with proxmox/Mac.

You need to get the Realtek 8125 Linux driver into Proxmox boot or the Aquantia or the 8169 or whatever is will go missing.

 

here's my Dmesg for it

 

[    1.212042] r8125: loading out-of-tree module taints kernel.
[    1.212860] r8125 2.5Gigabit Ethernet driver 9.003.04-NAPI loaded
[    1.212874] r8125 0000:46:00.0: enabling device (0000 -> 0003)
[    1.214183] ahci 0000:48:00.0: version 3.0
[    1.216264] firewire_ohci 0000:4c:00.0: enabling device (0000 -> 0002)
[    1.227995] r8125: This product is covered by one or more of the following patents: US6,570,884, US6,115,776, and US6,327,625.
[    1.230006] r8125  Copyright (C) 2020  Realtek NIC software team <nicfae@realtek.com> 
                This program comes with ABSOLUTELY NO WARRANTY; for details, please see <http://www.gnu.org/licenses/>. 
                This is free software, and you are welcome to redistribute it under certain conditions; see <http://www.gnu.org/licenses/>. 
[    1.244004] r8125 0000:46:00.0 enp70s0: renamed from eth0
 

 

The Realtek 2.5Gigabit Ethernet port is different than most TRX40 mobos. Did you say earlier that you got this kext from PAVO? This is the first that I've heard of 2.5GB devices working on Hackintoshes.

 

Many TRX40s seem to use the Intel I211 (1 GB) or the Aquantia AQC107 10 GB devices. The Aquantia is natively supported in Mac; the Intell I211 needs the SmallTree82576 kext. The Realtek kext doesn't work with either of these Ethernet devices.

Edited June 6, 2020 by iGPU