Proxmox VE OSX Guide discussion

[iGPU]

26 minutes ago, fabiosun said:

 

I did as you showed and got the exact results you show here. Thanks!

[fabiosun]

38 minutes ago, iGPU said:

 

Can you show your "/usr/share/qemu-server/pve-q35-4.0.cfg" file?  I'd like to compare with mine. Thanks.

Out now

bur I am using default one which should be the same for all

[meina222]

On 7/22/2020 at 1:51 PM, Driftwood said:

Can you enter that equipment info in your profile so its always visible below posts (like ours). It's useful for us when we see your posts to be reminded of that hardware is employed - saving us going back to a thread post! Thanks 🙂

With some delay due to being occupied lately. Point taken. Added lots of info that may help others who look at similar hardware. I don't think it's part of my signature as it is too much text, but hopefully if you click or hover on the profile, the full setup is visible.

Edited August 1, 2020 by meina222

[meina222]

@fabiosun - I finally managed to fix BigSur Beta 3 to work on my VM. Latest OC/Lilu versions indeed support passing CPU as host.

 

Alas, 64 core 1 socket is the max. The booter hangs very early if you try to pass 128. I played with the -v parameter in OC and the release build shows little to no useful logging that says why - it is just a halt very early in the process. One probably needs OC DEBUG to find out more. I think it's safe to assume nothing changed in that regard since Catalina for VM world.

 

The 64 core 3990x benchmark Geekbench on Big Sur beta3 / pve-5.44.2 on MacPro7,1 looks even better than Catalina, especially multicore. Pleased with Beta3 despite 1 crash/freeze involving plugging and unplugging USB devices.

https://browser.geekbench.com/v5/cpu/3136798

 

 

 

 

 

Edited August 1, 2020 by meina222

[iGPU]

52 minutes ago, fabiosun said:

Out now

bur I am using default one which should be the same for all

 

Why I ask is that I do not see any built-in ALC1220 device despite using AppleALC kext, and I was wondering what I'm missing.

 

In my /usr/share/qemu-server/pve-q35-4.0.cfg file, only the following are active (the rest were commented out):

Spoiler

[device "ich9-pcie-port-1"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.0"
  port = "1"
  chassis = "1"

[device "ich9-pcie-port-2"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.1"
  port = "2"
  chassis = "2"

[device "ich9-pcie-port-3"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.2"
  port = "3"
  chassis = "3"

[device "ich9-pcie-port-4"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.3"
  port = "4"
  chassis = "4"


[device "pcidmi"]
  driver = "i82801b11-bridge"
  bus = "pcie.0"
  addr = "1e.0"

[device "pci.0"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "1.0"
  chassis_nr = "1"

[device "pci.1"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "2.0"
  chassis_nr = "2"

[device "pci.2"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "3.0"
  chassis_nr = "3"

[device "pci.3"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "4.0"
  chassis_nr = "4"

 

 

These items were commented out:

Spoiler

#[device "ehci"]
#  driver = "ich9-usb-ehci1"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.7"

#[device "uhci-1"]
#  driver = "ich9-usb-uhci1"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.0"
#  masterbus = "ehci.0"
#  firstport = "0"

#[device "uhci-2"]
#  driver = "ich9-usb-uhci2"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.1"
#  masterbus = "ehci.0"
#  firstport = "2"

#[device "uhci-3"]
#  driver = "ich9-usb-uhci3"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.2"
#  masterbus = "ehci.0"
#  firstport = "4"

#[device "ehci-2"]
#  driver = "ich9-usb-ehci2"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.7"

#[device "uhci-4"]
#  driver = "ich9-usb-uhci4"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.0"
#  masterbus = "ehci-2.0"
#  firstport = "0"

#[device "uhci-5"]
#  driver = "ich9-usb-uhci5"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.1"
#  masterbus = "ehci-2.0"
#  firstport = "2"

#[device "uhci-6"]
#  driver = "ich9-usb-uhci6"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.2"
#  masterbus = "ehci-2.0"
#  firstport = "4"

 FIXME: Remove this audio0 device at the next possible time
#     see: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038417.html
#          https://pve.proxmox.com/pipermail/pve-devel/2019-July/038428.html
#[device "audio0"]
#  driver = "ich9-intel-hda"
#  bus = "pcie.0"
#  addr = "1b.0"



##
# Example PCIe switch with two downstream ports
#
#[device "pcie-switch-upstream-port-1"]
#  driver = "x3130-upstream"
#  bus = "ich9-pcie-port-4"
#  addr = "00.0"
#
#[device "pcie-switch-downstream-port-1-1"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.0"
#  port = "1"
#  chassis = "5"
#
#[device "pcie-switch-downstream-port-1-2"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.1"
#  port = "1"
#  chassis = "6"

 

 

Edited August 1, 2020 by iGPU

[fabiosun]

5 hours ago, iGPU said:

 

Why I ask is that I do not see any built-in ALC1220 device despite using AppleALC kext, and I was wondering what I'm missing.

 

In my /usr/share/qemu-server/pve-q35-4.0.cfg file, only the following are active (the rest were commented out):

  Hide contents

[device "ich9-pcie-port-1"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.0"
  port = "1"
  chassis = "1"

[device "ich9-pcie-port-2"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.1"
  port = "2"
  chassis = "2"

[device "ich9-pcie-port-3"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.2"
  port = "3"
  chassis = "3"

[device "ich9-pcie-port-4"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.3"
  port = "4"
  chassis = "4"


[device "pcidmi"]
  driver = "i82801b11-bridge"
  bus = "pcie.0"
  addr = "1e.0"

[device "pci.0"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "1.0"
  chassis_nr = "1"

[device "pci.1"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "2.0"
  chassis_nr = "2"

[device "pci.2"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "3.0"
  chassis_nr = "3"

[device "pci.3"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "4.0"
  chassis_nr = "4"

 

 

These items were commented out:

  Hide contents

#[device "ehci"]
#  driver = "ich9-usb-ehci1"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.7"

#[device "uhci-1"]
#  driver = "ich9-usb-uhci1"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.0"
#  masterbus = "ehci.0"
#  firstport = "0"

#[device "uhci-2"]
#  driver = "ich9-usb-uhci2"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.1"
#  masterbus = "ehci.0"
#  firstport = "2"

#[device "uhci-3"]
#  driver = "ich9-usb-uhci3"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1d.2"
#  masterbus = "ehci.0"
#  firstport = "4"

#[device "ehci-2"]
#  driver = "ich9-usb-ehci2"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.7"

#[device "uhci-4"]
#  driver = "ich9-usb-uhci4"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.0"
#  masterbus = "ehci-2.0"
#  firstport = "0"

#[device "uhci-5"]
#  driver = "ich9-usb-uhci5"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.1"
#  masterbus = "ehci-2.0"
#  firstport = "2"

#[device "uhci-6"]
#  driver = "ich9-usb-uhci6"
#  multifunction = "on"
#  bus = "pcie.0"
#  addr = "1a.2"
#  masterbus = "ehci-2.0"
#  firstport = "4"

 FIXME: Remove this audio0 device at the next possible time
#     see: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038417.html
#          https://pve.proxmox.com/pipermail/pve-devel/2019-July/038428.html
#[device "audio0"]
#  driver = "ich9-intel-hda"
#  bus = "pcie.0"
#  addr = "1b.0"



##
# Example PCIe switch with two downstream ports
#
#[device "pcie-switch-upstream-port-1"]
#  driver = "x3130-upstream"
#  bus = "ich9-pcie-port-4"
#  addr = "00.0"
#
#[device "pcie-switch-downstream-port-1-1"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.0"
#  port = "1"
#  chassis = "5"
#
#[device "pcie-switch-downstream-port-1-2"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.1"
#  port = "1"
#  chassis = "6"

 

 

mine:

Spoiler

[device "ehci"]
  driver = "ich9-usb-ehci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.7"

[device "uhci-1"]
  driver = "ich9-usb-uhci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.0"
  masterbus = "ehci.0"
  firstport = "0"

[device "uhci-2"]
  driver = "ich9-usb-uhci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.1"
  masterbus = "ehci.0"
  firstport = "2"

[device "uhci-3"]
  driver = "ich9-usb-uhci3"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.2"
  masterbus = "ehci.0"
  firstport = "4"

[device "ehci-2"]
  driver = "ich9-usb-ehci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.7"

[device "uhci-4"]
  driver = "ich9-usb-uhci4"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.0"
  masterbus = "ehci-2.0"
  firstport = "0"

[device "uhci-5"]
  driver = "ich9-usb-uhci5"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.1"
  masterbus = "ehci-2.0"
  firstport = "2"

[device "uhci-6"]
  driver = "ich9-usb-uhci6"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.2"
  masterbus = "ehci-2.0"
  firstport = "4"

# FIXME: Remove this audio0 device at the next possible time
#     see: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038417.html
#         https://pve.proxmox.com/pipermail/pve-devel/2019-July/038428.html
[device "audio0"]
  driver = "ich9-intel-hda"
  bus = "pcie.0"
  addr = "1b.0"

[device "ich9-pcie-port-1"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.0"
  port = "1"
  chassis = "1"

[device "ich9-pcie-port-2"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.1"
  port = "2"
  chassis = "2"

[device "ich9-pcie-port-3"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.2"
  port = "3"
  chassis = "3"

[device "ich9-pcie-port-4"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.3"
  port = "4"
  chassis = "4"

##
# Example PCIe switch with two downstream ports
#
#[device "pcie-switch-upstream-port-1"]
#  driver = "x3130-upstream"
#  bus = "ich9-pcie-port-4"
#  addr = "00.0"
#
#[device "pcie-switch-downstream-port-1-1"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.0"
#  port = "1"
#  chassis = "5"
#
#[device "pcie-switch-downstream-port-1-2"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.1"
#  port = "1"
#  chassis = "6"

[device "pcidmi"]
  driver = "i82801b11-bridge"
  bus = "pcie.0"
  addr = "1e.0"

[device "pci.0"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "1.0"
  chassis_nr = "1"

[device "pci.1"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "2.0"
  chassis_nr = "2"

[device "pci.2"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "3.0"
  chassis_nr = "3"

[device "pci.3"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "4.0"
  chassis_nr = "4"

 

@meina222 if you have time post also Cinebench 20 results

thank you

 

 

[fabiosun]

3 hours ago, iGPU said:

 

Why I ask is that I do not see any built-in ALC1220 device despite using AppleALC kext, and I was wondering what I'm missing.

 

 

  Reveal hidden contents

 

 

Sorry I do not understand well this sentence

our audio is a particular USB audio you should see Realtek USB 2.0 Audio and it is not related to AppleAlc kext

 

removing in cfg you posted above ich9 audio multimedia you will not see in your PCI device:

also if you comment usb part always in that file you will miss all intel (faked) USB useful to pass as single device via web interface any device you connect without passing an entire USB controller:

[meina222]

4 hours ago, fabiosun said:

mine:

  Reveal hidden contents

[device "ehci"]
  driver = "ich9-usb-ehci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.7"

[device "uhci-1"]
  driver = "ich9-usb-uhci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.0"
  masterbus = "ehci.0"
  firstport = "0"

[device "uhci-2"]
  driver = "ich9-usb-uhci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.1"
  masterbus = "ehci.0"
  firstport = "2"

[device "uhci-3"]
  driver = "ich9-usb-uhci3"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.2"
  masterbus = "ehci.0"
  firstport = "4"

[device "ehci-2"]
  driver = "ich9-usb-ehci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.7"

[device "uhci-4"]
  driver = "ich9-usb-uhci4"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.0"
  masterbus = "ehci-2.0"
  firstport = "0"

[device "uhci-5"]
  driver = "ich9-usb-uhci5"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.1"
  masterbus = "ehci-2.0"
  firstport = "2"

[device "uhci-6"]
  driver = "ich9-usb-uhci6"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.2"
  masterbus = "ehci-2.0"
  firstport = "4"

# FIXME: Remove this audio0 device at the next possible time
#     see: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038417.html
#         https://pve.proxmox.com/pipermail/pve-devel/2019-July/038428.html
[device "audio0"]
  driver = "ich9-intel-hda"
  bus = "pcie.0"
  addr = "1b.0"

[device "ich9-pcie-port-1"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.0"
  port = "1"
  chassis = "1"

[device "ich9-pcie-port-2"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.1"
  port = "2"
  chassis = "2"

[device "ich9-pcie-port-3"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.2"
  port = "3"
  chassis = "3"

[device "ich9-pcie-port-4"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.3"
  port = "4"
  chassis = "4"

##
# Example PCIe switch with two downstream ports
#
#[device "pcie-switch-upstream-port-1"]
#  driver = "x3130-upstream"
#  bus = "ich9-pcie-port-4"
#  addr = "00.0"
#
#[device "pcie-switch-downstream-port-1-1"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.0"
#  port = "1"
#  chassis = "5"
#
#[device "pcie-switch-downstream-port-1-2"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.1"
#  port = "1"
#  chassis = "6"

[device "pcidmi"]
  driver = "i82801b11-bridge"
  bus = "pcie.0"
  addr = "1e.0"

[device "pci.0"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "1.0"
  chassis_nr = "1"

[device "pci.1"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "2.0"
  chassis_nr = "2"

[device "pci.2"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "3.0"
  chassis_nr = "3"

[device "pci.3"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "4.0"
  chassis_nr = "4"

 

@meina222 if you have time post also Cinebench 20 results

thank you

 

 

CPU Cinebench attached as requested (Big Sur beta 3 on pve-kernel 5.44-2). ~21000 give or add a 100 depending on run

Edited August 2, 2020 by meina222

[meina222]

While on the topic of benchmarks I decided to see if there is any effect from "pinning" the host CPU threads given to the VMs. "Isolation" or "pinning" is something Linux supports - the full understanding of this term requires knowing about NUMA and OS scheduling but essentially it means that once a CPU "resource" such as a logical core is given to a process (the VM) by the host, it cannot be "taken away". This would be mostly of importance if you run 2 VMs and want maximum performance on both - the host won't be able to bounce threads between the two.

 

Anyways, I was curious if the equivalent of "pinning" would benefit benchmarks. In Proxmox "pinning" is achieved via "tasksets". Here's how to enable it as an example of my Big Sur VM id 101

 

1. Start VM

2. "Pin" your CPUs to the started VM. In my case I have CPU id's 0-63

Run this command (change CPU range and VM id according to your setup)

taskset --cpu-list  --all-tasks --pid 0-63 "$(< /run/qemu-server/101.pid)"

Edit: What this did for my 64C/128T CPU is to essentially fix the mac OS VM threads to run on 0-63 and never be scheduled on 64-127 of the logical cores. This benefit may only be visible in 3970x if you run 32 cores in macOS. In that case you would pin 0-31 and see benefit but I am not sure this would work if you already give macOS all of your 64 logical cores. This is an example of how the extra cores of the 3990x can help without trying to run VM arg -smp 128 for macOS, which doesn't seem to be possible.

 

For more info on how to hook this to your VM on every start:

https://forum.proxmox.com/threads/cpu-pinning.67805/

 

Now I run Geekbench again. Multicore score went up by more than 1000 points. I believe this is more than just variance from about 28000 to 29500. Single core didn't change.

 

https://browser.geekbench.com/v5/cpu/3142197

 

Cinebench benefited a bit less. From 21000 to 21100.

 

Of course the increases in both cases are relatively minor and there is still a chance this could be variance, but if you care about benchmarks and you're like me who doesn't want to kill the power line by overclocking, it's a nice trick to get extra 2-3%. The benefit would be much bigger if you run X VMs and want to "isolate" one from the other. Then you can split the cores in X task sets and give them to each VM.

Edited August 2, 2020 by meina222

[fabiosun]

 

to have a dedicated thread for our experimental TB3 support I have created a new thread where we can write and discuss about this subject

Thank you your partecipation

 

[tomnic]

@iGPU here it is my pve-q35-4.0.cfg

 

[device "ehci"]
  driver = "ich9-usb-ehci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.7"

[device "uhci-1"]
  driver = "ich9-usb-uhci1"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.0"
  masterbus = "ehci.0"
  firstport = "0"

[device "uhci-2"]
  driver = "ich9-usb-uhci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.1"
  masterbus = "ehci.0"
  firstport = "2"

[device "uhci-3"]
  driver = "ich9-usb-uhci3"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1d.2"
  masterbus = "ehci.0"
  firstport = "4"

[device "ehci-2"]
  driver = "ich9-usb-ehci2"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.7"

[device "uhci-4"]
  driver = "ich9-usb-uhci4"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.0"
  masterbus = "ehci-2.0"
  firstport = "0"

[device "uhci-5"]
  driver = "ich9-usb-uhci5"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.1"
  masterbus = "ehci-2.0"
  firstport = "2"

[device "uhci-6"]
  driver = "ich9-usb-uhci6"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1a.2"
  masterbus = "ehci-2.0"
  firstport = "4"

# FIXME: Remove this audio0 device at the next possible time
#     see: https://pve.proxmox.com/pipermail/pve-devel/2019-July/038417.html
#          https://pve.proxmox.com/pipermail/pve-devel/2019-July/038428.html
#[device "audio0"]
#  driver = "ich9-intel-hda"
#  bus = "pcie.0"
#  addr = "1b.0"


[device "ich9-pcie-port-1"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.0"
  port = "1"
  chassis = "1"

[device "ich9-pcie-port-2"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.1"
  port = "2"
  chassis = "2"

[device "ich9-pcie-port-3"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.2"
  port = "3"
  chassis = "3"

[device "ich9-pcie-port-4"]
  driver = "pcie-root-port"
  x-speed = "16"
  x-width = "32"
  multifunction = "on"
  bus = "pcie.0"
  addr = "1c.3"
  port = "4"
  chassis = "4"

##
# Example PCIe switch with two downstream ports
#
#[device "pcie-switch-upstream-port-1"]
#  driver = "x3130-upstream"
#  bus = "ich9-pcie-port-4"
#  addr = "00.0"
#
#[device "pcie-switch-downstream-port-1-1"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.0"
#  port = "1"
#  chassis = "5"
#
#[device "pcie-switch-downstream-port-1-2"]
#  driver = "xio3130-downstream"
#  multifunction = "on"
#  bus = "pcie-switch-upstream-port-1"
#  addr = "00.1"
#  port = "1"
#  chassis = "6"



[device "pcidmi"]
  driver = "i82801b11-bridge"
  bus = "pcie.0"
  addr = "1e.0"

[device "pci.0"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "1.0"
  chassis_nr = "1"

[device "pci.1"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "2.0"
  chassis_nr = "2"

[device "pci.2"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "3.0"
  chassis_nr = "3"

[device "pci.3"]
  driver = "pci-bridge"
  bus = "pcidmi"
  addr = "4.0"
  chassis_nr = "4"

Just commented the audio0 section. In X570 chipset based mobos it is important to comment it out as long as the Audio chipset is not converted into an usb one, and as a PCI resource lilu cannot correctly address it

[Ploddles]

On 7/19/2020 at 11:26 PM, meina222 said:

p.s.

As CCC doesn't copy the MacOS main disk EFI by default, you need to do that manually once only, when happy with your OpenCore config, else it won't boot of course. OC crashes badly if your try to use the NVME as a Time Machine or CCC destination and forget to fix the EFI. Then the only way to boot is to remove the NVME PCI, but that defeats the purpose of using it as a backup, so don't forget this step.

 

Have a look HERE,  at Step 13 Spoiler. There is a script in there that you can add to your CCC clone job to automatically clone your EFI partition to your CCC disk's EFI. There is a TEST_SWITCH so you can run it without actually copying the files but it outputs a list of what it would copy. Just change the switch to 'N' so that it does the copy if you are happy with the test.

[tsongz]

Hi all, thanks to everyone in this community for publishing all your findings and being immensely helpful in sharing your knowledge. I've been through all 38 pages of this thread - many times over to refer to tidbits of information.

 

I wanted to share my own experiences, what worked for me and what didn't, in a concise post so that someone who may stumble on this in the future might benefit from it. I'm not entirely done yet, but I'm further ahead than I previously was in a different system build.

 

EDIT: added more notes

 

As of this post, my system configuration is:

 

MacOS Big Sur Beta 3
OpenCore 0.5.9
AsRock TRX40 Creator
AMD Threadripper 3970X [Be Quiet Dark Rock Pro TR4]
64GB DDR4 3200MHz Team T-Force Vulcan Z (2 x 32GB - maxing out later)
Radeon Vega Frontier Edition 16GB @ PCIe Slot 2
- 1080p HDMI Monitor
- 4K Displayport Monitor
Intel 660p 1TB NVME* (don't buy this, note on this later)
Samsung 500GB SSD
- LED Fans* (note on this later)


To be added on:
Gigabyte Alpine Ridge (for use with current Thunderbolt Display)

 

Scripts used will be posted at the bottom of the post.

 

The purpose of my build is as a primary workstation for Machine Learning (NLP), which is my primary job day-to-day. I currently use cloud VMs with GPUs in order to do this, as my existing Macs aren't up for the job. However, this is a tough requirement in itself, because anyone in ML will know, Nvidia is the de-facto GPU due to it's CUDA cores, and well, Apple =/= Nvidia. My existing system is:

 

- 5k iMac Late 2014 (Mojave 10.14.6)
- 3.5GHz 4-core i5
- 24GB DDR3
- 2GB R9 M290X

- 2010 Mac Pro (10.14.4 - Last Supported Version)
- 3.46 GHz 6-core Xeon
- 64GB DDR3 ECC
- 8GB RX 580
- 1GB NVME

 

The reason I chose the Frontier Edition is specifically due to it's 16GB of HBM, which is very rare in consumer grade GPUs, as the GPUs I access daily all have 16GB, making the work with ML more accessible. For all other intents and purposes, the Radeon VII seems to have more support, and if you were to pick one and didn't need the 16GB of GPU RAM, I would lean towards the Radeon VII or others.

 

I had previously started with Nick Sherlock's Proxintosh guide, along with many different vanilla guides with the last build, which was with a 2x E5-2680v2 Xeon and a Huanan X79 motherboard. I would not suggest venturing into motherboards that are not widely supported or have community support around them. This led to 2 weeks of my time down the drain.

 

This time I started with Pavo's Proxintosh guide on Github, while also using Nick Sherlock's guide in parallel. Pavo's guide is a great starting point - but skips over a big step (under the assumption you know what you're doing) of preparing the bootloader and installer.

 

Assuming you have successfully installed Proxmox, and at least have gotten into the web UI, I will start from there. At this stage, I had identical configs from Pavo's repo:

- /etc/modprobe.d/blacklist.conf

- /etc/modprobe.d/kvm.conf

Minor edits to:

- /etc/modprobe.d/vfio.conf (only GPU Passthrough)

 

Additionally, I use Remote SSH with VSCode in order to better work with editing the configs/files, without having the terminal close on me. You will need to add your ssh key to Proxmox through the web console first, and then add the Proxmox instance to your VSCode config for easy connection. At this step I would start a spreadsheet to write down your pci configurations as you will refer to this many times over.

 

The Installer

 

For Big Sur, you will need to sign up and request developer access. Since my Apple account was already a developer account, this wasn't difficult. I don't know what it would be like for non-developer accounts. It will download a dmg which is an installer application to verify your hardware compatibility before downloading Big Sur.

 

I tried this with my 2010 Mac Pro first, but hardware verification didn't allow me to, so I ended up having to download the installer again to my iMac, which was allowed to.

The process took a couple of hours, and then I copied over my 'Install macOS Big Sur Beta.app' to my Mac Pro since I offload intensive work over to it.

 

I used Nick's script  with minor modifications:

- In the script 'hdiutil create -o "$dmg_path" -size 16g -layout GPTSPUD -fs HFS+J' was modified (from -size 12g to -size 16g) as the beta 3 in my own device (or at the time of writing) was not big enough.

- It looks to be modified now, but previously where 'in_path=/Applications/Install\ macOS\ Beta.app' was modified to reflect the correct path -> 'in_path=/Applications/Install\ macOS\ Big\ Sur\ Beta.app'

 

Once I had the dmg, I renamed to iso, and uploaded it into proxmox through the web UI.

- Datacenter -> Storage -> local -> Content -> Upload -> ISO Image

Additionally, I pulled the latest OpenCore installer and moved it there as well. (I didn't have access to Downloads in this forum at the time of writing)

 

Then following the steps of the remainder of Nick's guide, got Big Sur installed (will take 4-5 reboots for B3) through Proxmox's VNC Viewer.

 

Notes:
- At this time, no PCIe passthrough
- Only using Pendryn Emulated CPU
- Passed through only 16 Cores / 16 GB
- Installed on a qemu raw disk of 64 GB
- macOS installed as iMacPro1,1

 

Within the VM, I installed OpenCore configurator, and copied the EFI folder from the OpenCore bootloader into the VM Storage.

 

After this step, I wanted to get my GPU passthrough to work.

 

Important Vega Specific Note:

- In my experience, you absolutely must have a gpu reset script in order to get GPU passthrough to work successfully for Vega (specifically for my Frontier Edition).

- I used the reset-gpu.pl script from someone who shared theirs on here, and have made minor modifications to it since. In this script, the part where "rtcwake -m mem -s 4" is important, as it is the function that restarts your system in order for the GPU to be passed through. Additionally, I added the "post-stop" for rebooting proxmox, which can be removed, but whenever I was shutting down the VM - the machine itself would start having fans turboing up like crazy. I haven't isolated the component but safer to do a full reboot. [1]

- Another consideration was that after a few times of this, the RGB LED controller stopped working. I have two sets of these controllers, ones that came with the fans, and another that came with the case. I didn't have enough fan headers so I used both. The case fan controller still worked, but the secondary fan controller did not. (Only the RGB was affected, the fans still worked). They were both independently plugged into the motherboard in their respective RGB header pins. I fixed this by taking the non-working secondary RGB controller pin header and plugging it into the case RGB controller slot, and now it works as normal.

- I have seen notes regarding the latest kernel header but many have pointed out that this does not solve the problem, so I have not upgraded the linux kernel.

- I also downloaded vega rom bios  and added it to /usr/share/kvm/vegafrontier.rom

 

Now I'm able to add the two lines to my 101.conf

 

- hookscript: local:snippets/reset-gpu.pl

- hostpci0: 4d:00,pcie=1,x-vga=1,romfile=vegafrontier.rom

 

NVMe Passthrough (failed)

 

After GPU Passthrough, I wanted to pass through my entire NVMe (Intel 660P - brand new, out the box) to macOS. After several different attempts, including adding "rombar=0", adding additional boot args, and getting an error

- "failed to add PCI capability 0x11[0x50]@0xb0: table & pba overlap, or they don't fit in BARs, or don't align"

- https://patchwork.kernel.org/patch/10707761/

- Apparently several NVME drives, and in my case, specifically the Intel 660P lack a certain module allowing for ease of use with VM passthrough. Stick to NVME drives that have been seen within members' sigs that have proven to work.

 

I decided to just mount the drive and use it as a qemu. This works with no problems.

 

If you've gotten to this point, I would recommend using Proxmox's Backup feature (backup tab in VM View) to completely back up the VM in the event something goes wrong.

- You may have to increase the limits of backups.

- Unless you've already partitioned your primary Proxmox drive, would recommend putting it somewhere there's more available storage.

- If you add a large secondary drive to MacOS that does not contain the OS, remember to remove it first from 101.conf prior to this, otherwise the backup will take a very long time as it will save the entire volume. A 64GB VM-mounted drive should take 5-7 mins, and be around 1/3 of the size (22-23GB in my case)

 

PCIe Passthrough

 

I had first tried to go all in and pass through all the PCIe ports all at once, but this led to macOS not booting up (with -v args in config.plist). Don't try to pass everything all at once. I would recommend going through one by one in order to isolate and identify which works and which don't. At this point, you should have additional items in your vfio.conf and done:

- update-initramfs -u -k all && update-grub && pve-efiboot-tool refresh

 

In my case, specifically for the ASRock Creator motherboard, one of the USB Controller headers could not be passed through successfully. (22:00.3, 22:00,4 - Asrock's USB 3.0 and HD Audio)

 

Currently I have passed through successfully.

 

hostpci1: 43:00.0,pcie=1 [USB 3.2 Controller]
hostpci2: 47:00.1,pcie=1 [USB 3.0 Controller]
hostpci3: 47:00.3,pcie=1 [USB 3.0 Controller]
hostpci4: 45:00.0,pcie=1,rombar=0 [Intel Network - thanks to rombar tip from iGPU]
hostpci5: 44:00.0,pcie=1 [Aquantia Ethernet]
usb0: host=0461:4dd7 [USB Keyboard passed through]

 

Network & Bluetooth (not working)

 

I am still using VMXnetwork from Proxmox qemu to access the internet, but have been able to start getting Bluetooth and Wifi viewable within macOS in System Report. I am still using Intel AX200 (which is what comes with the motherboard) and trying to use itlwm kexts to get it working. (as well as previous posts by iGPU)

 

Current MacOS System

 

GeekBench 5 CPU: 1156 Single-Core, 16350 Multi-Core

GeekBench 5 GFX (Metal): 61914

Passed Through HW:

- 64 Cores (Penryn)

- 32 GB RAM

- Vega Frontier

 

PCIe Tab does not display anything - guessing due to not passing it through yet.

 

 

 

Bluetooth looks to be enabled, but doesn't actually pick up any devices

 

 

OpenCore Configurator

 

I use OpenCore Configurator to mount and modify the config.plist and add/edit new settings. I modified the system from iMacPro1,1 to MacPro7,1 and got a lift of 1000 points in geekbench in multithreaded. I am currently working on setting up the full host CPU passthrough without Pendryn, as iGPU has referenced in the previous posts with using the kernel patches for Pendryn, cpumask, and not leaf7. Planning on upgrading to 6.0 after xcode finishes downloading on MacOS BS.

 

Still WIP (I/O):

 

- Full networking (Wifi/Bluetooth) passthrough as mentioned above, if not, then going the route of getting the BCM card

- Full NVME passthrough (new drive)

- Upgrading to OC 6.0 with Pavo's OCBuilder.

 

Questions/WIP (Performance Enhancement):

 

For full CPU Passthrough of the TR3970X, in the CPU Mask in OpenCore configurator, I've seen a few different lines used. In my existing config.plist (vanilla from Nick's OC), in:

Kernel -> Add -> Cpuid1Mask
- Current: FFFFFF00 00000000 00000000 00000000
- Enabled: DummyPowerManagement
- Pavo's: FFFFFFFF 00000000 00000000 00000000
- Enabled: PanicNoKextDump, PowerTimeoutKernelPanic

 

As far as I've read, would I need to have the system-specific ACPI compiled prior to using full Host CPU? Or would the change to the patches (enabling) take care of that?

 

There's been several discussions related to GPU Performance with specific SSDTs (which I've tried to understand, but I get lost every time) https://forums.macrumors.com/threads/tired-of-low-geekbench-scores-use-radeonboost.2231366/

 

I'm not 100% confident in compiling the system SSDT, as you would generally do a compilation on the host system, not the virtualized system, and it seems like PCI mapping is a bit off.

 

My Vega FE shows as PCI0@0/SE0@1C in IORegistry.

 

I'm wondering whether there's value in compiling the SSDT, because in my use-case, there will be times when the GPU will be pushed to 100% for hours at a time, and so optimizing the performance in both temp control and speeds will be important as to not fry the system.

 

I had started with KGP's X299 System SSDTs  and his guide using achive.org to try to better understand the process, but because the PCIe slots aren't lining up, it's confusing me a bit.

 

BIOS Setting Modifications (AsRock TRX40 Creator, thanks to Rox67er):

 

- Advanced > Onboard Devices Configuration > SR-IOV Support > Enabled
- Advanced > AMD CBS > NBIO Common Options > IOMMU > Enabled
- Advanced > AMD CBS > NBIO Common Option > Enable AER Cap > Enable
- Advanced > AMD CBS > NBIO Common Option > ACS Enable > Enable
- I had tried to disable Bluetooth in BIOS but led to Bluetooth not being detected in MacOS.

 

Current Kexts (Kext Order is VERY important, although not sure mine is correct, or if I'm missing some. V indicates vanilla from the default installer):

 

- Lilu (Enabled - V)
- VirtualSMC (Disabled - V)
- Whatevergreen (Enabled - V)
- AGPMInjector (Enabled - V)
- AGPMInjectorVega (Disabled - for Vega Specifically from Pavo's AGPMInjector tool)
- USBPorts (Enabled - V)
- MCEReporterDisabler (Enabled - V)
- AppleALC (Enabled - V)
- itlwm (Enabled - for intel networking)
- IntelBluetoothInjector (Enabled)
- IntelBluetoothFirmware (Enabled)
- LucyRTL8125Ethernet (Enabled)
- itlwmx (Disabled, for newer intel networking, not sure which one AX200 falls in yet)

 

Helpful One-liners

 

# Mount EFI Folder within MacOS - Make sure disk1s1 is where your efi is located.

 

sudo mkdir /Volumes/EFI && sudo mount -t msdos /dev/disk1s1 /Volumes/EFI

 

# Update Grub and all after vfio.conf edits

 

update-initramfs -u -k all && update-grub && pve-efiboot-tool refresh

 

Helpful Tools (within MacOS VM)

 

- VSCode (preferred Editor, but any others would work)

- OpenCore Configurator - not linking this one bc there seems to be a lot of debate around the source.

- OCBuilder (used for compiling to 6.0)

- Kexts Folder (library of Kexts for a majority of not-as-easy to find Kexts)

- Hackintool

- AGPMInjector

- Apple's Developer Downloads (used to download ioregistryexplorer under Additional Tools for Xcode 12 Beta 3 - does not require Xcode to run, but will need Xcode to compile other stuff, so might as well grab it as well)

- Screen Sharing (controlling VM Mac from iMac, native Mac OS Application)

- MaciASL (for working with SSDTs and stuff)

 

Things I wish I knew before wasting a lot of time

 

- If this is your first time, do it exactly as it says first to get things working. Don't be tempted to modify anything that you aren't instructed to first. (bad habit of mine). Then back up often. You will mess up a lot before you get things right.

 

- Make changes one at a time. Test it. Then continue. This approach will save you a lot more time in the end, because if you're like me and tempted to change a lot of different settings, inevitably, something will cause things to fail, and then you're in a situation where you have no clue what failed - and in the worst of cases, unrecoverable failure, making all the hard work you put in, wasted. Set goals/milestones to reach, and then back up your VM when that is reached. (GPU Passthrough successful, PCIe Passthrough Successful, etc).

 

- Be prepared to spend at the very least, 3-4 full days getting things to work. It won't be an overnight thing. I'm fairly comfortable with VMs, working with them for at least 4-5 years in cloud environments. However, they were ubuntu kernels and not debian kernels, so there are minor differences. Despite this, my first go at this (prior to this) - I spent a full 1-2 weeks trying to, and failing to, get things working. This time around, it took me 3 days, and still ongoing. This is not an easy project, and don't be fooled by Youtubers who claim it is. To them, it probably is, but for the majority of people who don't do this for a living, or have not dived into this side of things before, there is a significant learning curve. Because we are human, we tend to want to get things done the easiest way possible, and will likely skip a step here or there (or trying something someone said - and it works for them), and could cause you more problems, which will lead you to then try to figure out how to fix that new problem instead of the original problem. 

 

- There are likely zero one-size-fits-all guides/configs/kexts that matches your hardware exactly. Even using 99% of the same hardware, there will likely be differences. For example, in my configuration, I have my GPU in PCIe Slot 2 in order to reach the fan headers easier, and be able to actually pop out the GPU via the PCIe slot. Slot 1 made it nearly impossible to do so. Because of that, my lscpi IDs are different than other members here who have the identical motherboard. Even within the same, identical motherboards, these IDs will be different. Driftwood and Rox67er who both have the ASRock Creator, have different IDs for their PCIe slots than I do.

 

- Pavo had warned me specifically about ensuring I have the latest BIOS versions early on, so this was one of the first things I did, and because I updated the BIOS (1.7.0) which had AGESA 1.0.0.4 - I did not run into many issues. BIOS is often not discussed in great detail, but plays a crucial role in whether you're going to have lots of problems early on or not.

 

- Research your GPU support - and then double check it before you buy it. I bought my Vega FE on ebay for a great deal ($350) without doing a ton of Due Diligence. This caused me a lot of problems in the first go at this with the GPU reset bug which I didn't find out about for several days trying to figure out what was going wrong. Don't just look at the specs, google search "[your gpu] proxmox bug" and see what comes up, and anticipate dealing with it.

 

- Figure out your workflow. You will probably have dozens of runs at this, including reinstalling proxmox from scratch several times because something borked, so make sure you're able to access things easily, especially things like your vfio config, lspci outputs, in the case that you need to wipe your system. When Proxmox reinstalls, it wipes the entire drive, so you will lose everything you did not save (if on the same drive). I have over 40 tabs open related to different bugs/things to read through.

 

[1] reset-gpu.pl in /var/lib/vz/snippets/reset-gpu.pl

 

#!/usr/bin/perl

# Hook script for PVE guests to reset AMD GPU before VM start
# chmod +x /var/lib/vz/snippets/reset-gpu.pl
# qm set 101 -hookscript local:snippets/reset-gpu.pl
# https://www.macos86.it/topic/2675-proxmox-ve-osx-guide-discussion/page/10/

use strict;
use warnings;

print "HOOK: " . join(' ', @ARGV). "\n";

# First argument is the vmid

my $vmid = shift;

# Second argument is the phase

my $phase = shift;

if ($phase eq 'pre-start') {

print "HOOK: unbind vtconsole and reset GPU.\n";
system("echo 0 > /sys/class/vtconsole/vtcon0/bind");
system("echo 0 > /sys/class/vtconsole/vtcon1/bind");
system("echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind");
system("rtcwake -m mem -s 4");

} elsif ($phase eq 'post-stop') {

print "HOOK: $vmid is shutting down. Rebooting System\n";
system("sudo reboot");

} else {
print "HOOK: got unknown phase '$phase'\n";
}

exit(0);

 

[2] 101.conf in /etc/pve/qemu-server/101.conf

 

args: -device isa-applesmc,osk="****(c)AppleComputerInc" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu Penryn,kvm=on,vendor=GenuineIntel,+kvm_pv_unhalt,+kvm_pv_eoi,+hypervisor,+invtsc,+pcid,+ssse3,+sse4.2,+popcnt,+avx,+avx2,+aes,+fma,+fma4,+bmi1,+bmi2,+xsave,+xsaveopt,check
balloon: 0
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 64
cpu: Penryn
efidisk0: local-lvm:vm-101-disk-0,size=4M
hookscript: local:snippets/reset-gpu.pl
hostpci0: 4d:00,pcie=1,x-vga=1,romfile=vegafrontier.rom
hostpci1: 43:00.0,pcie=1
hostpci2: 47:00.1,pcie=1
hostpci3: 47:00.3,pcie=1
hostpci4: 45:00.0,pcie=1,rombar=0
hostpci5: 44:00.0,pcie=1
localtime: 1
machine: q35
memory: 32768
name: BigSur
net0: vmxnet3=A6:95:0E:F2:D3:A6,bridge=vmbr0,firewall=1
numa: 1
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=****
sockets: 1
usb0: host=0461:4dd7
vga: none
virtio0: local-lvm:vm-101-disk-1,discard=on,size=75G
virtio1: macos-nvme:101/vm-101-disk-0.raw,cache=unsafe,size=750G
vmgenid: ******

 

[3] vfio.conf in /etc/modprobe.d/vfio.conf

 

# 4d:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6863]
# 4d:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]

options vfio-pci ids=1002:6863,1002:aaf8 disable_vga=1
# 47:00.1, 47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
options vfio-pci ids=1022:149c
# 22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
options vfio-pci ids=1022:148c
# 43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
options vfio-pci ids=1b21:3242
# 42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
options vfio-pci ids=8086:f1a8
### Bluetooth Configuration ###
# 46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
# options vfio-pci ids=10ec:8125
# 45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
# Subsystem: Intel Corporation Device [8086:0084]
options vfio-pci ids=8086:2723,8086:0084
# Bus 007 Device 006: ID 8087:0029 Intel Corp.
options vfio-pci ids=8087:0029

# 47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
# options vfio-pci ids=1022:1485

# 44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
options vfio-pci ids=1d6a:07b1
# 48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
options vfio-pci ids=1022:7901

 

[4] blacklist.conf in /etc/modprobe.d/blacklist.conf

 

blacklist snd_hda_codec_hdmi
blacklist snd_hda_intel
blacklist snd_hda_codec
blacklist snd_hda_core
blacklist radeon
blacklist amdgpu
blacklist btusb
blacklist snd-usb-audio
# blacklist bcma
blacklist xhci_hcd

Edited August 3, 2020 by tsongz

[meina222]

@tsongz:

 

A few remarks to make you get where you want to be:

 

Bluetooth 

    Save yourself the trouble with AX200 and get either a MacOS native PCIE x1 card or a Bluetooth USB dongle with a Broadcom chip supported by macOS.

    For example I have verified the following to work in a Proxmox VM (including my MacOS keyboard and trackpad, iPhone pairing, AirDrop, Continuity etc), which as a bonus includes native WiFi:

    https://www.amazon.com/gp/product/B082X8MBMD/

    If you don't have a free PCIE slot (my motherboard has a x1 slot so I don't have to use a full slot but the ARock may not have it) use a dongle:

    https://www.amazon.com/GMYLE-Bluetooth-Dongle-V4-0-Dual/dp/B007MKMJGO/

   ⚠️ Make sure to disable the AX200 1st in MacOS by running 'sudo nvram bluetoothHostControllerSwitchBehavior=never' or else the AX200 will get picked up and show up as Ericsson 00-00-00-00-00-00 device. When you restart the BCM chip should get picked up.

 

Big Sur and CPU

    Host works now. I would suggest to use with a later build OC and Lilu. Start with the minimum kexts - my Big Sur only works with Lilu and even if there are no power savings on the 5700XT I am not bothered by it. You would need USB kext to use keyboard in VM mode but that's about all to start booting. Add Whatevergreen and other stuff as you need.

For example my BS VM config only has this for CPU:

 

args: -device isa-applesmc,osk="[redacted apple haiku - fill yourself]" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu host,+invtsc,vendor=GenuineIntel

 

You will get significant performance boost from that.

 

Since you mentioned Pavo - start with this config.plist and add kexts as you need. It's barebones and it actually works with Big Sur.

 

https://github.com/Pavo-IM/Proxintosh/tree/master/Big_Sir

 

To get iMessage to work, ditch the vmx adapter and pass through one of the Ethernets. I have Intel I210 which MacOS recognizes - you need to ensure it's designated as 'built-in' - after you pass it through check the PCIE id in macOS and add it to the config.plist as 'built-in' and you need to add the ROM / Mac address in the config.plist as described in https://dortania.github.io/OpenCore-Post-Install/universal/iservices.html#generate-a-new-serial.

 

Use your VGA VM mode to do installs/upgrades/tests to avoid the reset bug, then when satisfied, get the graphics card in and enjoy.

[meina222]

Today Linus Torvalds released Linux Kernel 5.8. I downloaded the git submodule and try to link it to pve-edge-kernel and build it. The 5.8 kernel built just fine, but ZFS build failed as the May 8.4 release used in Proxmox / pve-edge is not compatible with 5.8. The failure I encountered is https://github.com/openzfs/zfs/pull/10422

 

Unfortunately Torvalds is not a fan of ZFS, so he could care less about breaking it.  We may have to wait for a ZFS release to be cut from the openzfs project. I can try merging the current ZFS master branch and rebuild Promox with that from https://github.com/fabianishere/pve-edge-kernel, but that's a dicey approach - on the other hand if you (unlike me) don't use ZFS, you probably won't care. I'll give it a shot and keep you posted. Might get in touch w Fabian too, to see if we can cut experimental ZFS, but I suspect they will wait for an official release from openzfs.

 

5.8 is a huge release and I am looking forward to a 5.8 kernel Hypervisor in the coming weeks. 

 

Edit:

News release. Linus Torvalds now owns a Threadripper 3970x so lots of AMD love coming to Linux, only some of which is mentioned briefly here

ttps://www.theregister.com/2020/08/03/linux_5_8_released/

Edited August 3, 2020 by meina222
Add news release link

[tsongz]

@meina222 thanks for those tips!

 

I had actually started with that Big Sur repo first, but ran into issues (namely, the steps regarding plugging in the USB, and my NVME passthrough) so I had used Nick's guide to get started, and referred back to that one. When I switched over to that config.plist on my first go, after a successful installation, I ran into a kernel panic, and had to redo things (which is why I have 101.conf).

 

Also note for anyone who did the same thing I did - Nick Sherlock's guide uses VBoxHfs, while others use HfsPlus - make sure you have the right one.

 

RE CPU configuration: for clarification, the cpuid1mask is a tad different on my existing vs Pavo's config

 

Kernel -> Add -> Cpuid1Mask
- Current: FFFFFF00 00000000 00000000 00000000
- Enabled: DummyPowerManagement

- Pavo's: FFFFFFFF 00000000 00000000 00000000
- Enabled: PanicNoKextDump, PowerTimeoutKernelPanic

It should be the second one if passing through all the CPU cores right?

 

RE Ethernet Port: Currently macos sees the Aquantic Ethernet port, which leads me to a question. If Proxmox is connected to one port, would the macos need a second ethernet connection via the Aquantic port?

 

RE Bluetooth: I most likely will - just want to exhaust all options first. I've been looking at this post where they've been able to use the AX201 on the Z490 board, with aml modifications. I looked all around the forums and didn't see a lot of discussion around ssdts, so I've been using the ACPIs from @Driftwood and @Pavo as a starting point. 

[fabiosun]

dear @tsongz,

I was tempted to delete your initial post on this forum, but after thinking about it a bit I decided to leave it without deleting anything you wrote, because this forum has some no restrictive politics and I would live to give also a second chance to understand better 😉

. however, it seems ungrateful to me to mention all the experiences you have had during your learning process for several reasons:

 

1) mention Sherlocks' guides. try reading in the comments of catalina or highsierra and you will understand who was the first to tell the hackintosh community how to solve the problem of the ruined text in verbose mode and beyond with clover..using Opencore I mean

 

2) you also mention Pavo's github, I often compare myself with Pavo who started his journey by reading my guide (read his credits ), by the way do it too, you could find some interesting ideas from this

 

3) you also mention KGP, but I prefer them to fly over as there is little to say.

 

the main thing that bothered me and that is disrespectful for the forum that hosts us is the advertising of "important" tools to be able to install OSX. Reading the guide carefully you will discover that it is possible to have everything using only two applications in our download area (and without modesty whatsoever, often copied with or without authorization and published on other international forums) I am writing this not to magnify or incense my work but to give due credit to the efforts made by our admin once I had found the way to have OSX installed with Proxmox on our "UNVANILLA" system 🙂 said this: welcome to our forum and I hope you will stay and actively participate in its growth

😉

[meina222]

3 minutes ago, tsongz said:

@meina222 thanks for those tips!

 

I had actually started with that Big Sur repo first, but ran into issues (namely, the steps regarding plugging in the USB, and my NVME passthrough) so I had used Nick's guide to get started, and referred back to that one. When I switched over to that config.plist on my first go, after a successful installation, I ran into a kernel panic, and had to redo things (which is why I have 101.conf).

 

Also note for anyone who did the same thing I did - Nick Sherlock's guide uses VBoxHfs, while others use HfsPlus - make sure you have the right one.

 

RE CPU configuration: for clarification, the cpuid1mask is a tad different on my existing vs Pavo's config

 

Kernel -> Add -> Cpuid1Mask
- Current: FFFFFF00 00000000 00000000 00000000
- Enabled: DummyPowerManagement

- Pavo's: FFFFFFFF 00000000 00000000 00000000
- Enabled: PanicNoKextDump, PowerTimeoutKernelPanic

It should be the second one if passing through all the CPU cores right?

 

RE Ethernet Port: Currently macos sees the Aquantic Ethernet port, which leads me to a question. If Proxmox is connected to one port, would the macos need a second ethernet connection via the Aquantic port?

 

RE Bluetooth: I most likely will - just want to exhaust all options first. I've been looking at this post where they've been able to use the AX201 on the Z490 board, with aml modifications. I looked all around the forums and didn't see a lot of discussion around ssdts, so I've been using the ACPIs from @Driftwood and @Pavo as a starting point. 

 I also used Nick Sherlock as a start, then after doing research realized that OC and the Hackintosh community had moved beyond - but my 1st VM 2 months ago was from Nick's article. But the person who popularized Threadripper + MacOS was @fabiosun. It was his post (I wasn't registered at the time) that I saw in Google that hinted this was a thing.

[tsongz]

Hi @fabiosun - my apologies, that definitely wasn't the intent at all. I wrote everything out in one sitting to get everything out as fast as possible while it was fresh on my mind because I know if I didn't today, I likely wouldn't have done it at all - and I have exclusively shared this entire writeup only here and nowhere else, because I truly do value all the work you guys have done. 

 

You and Pavo deserve all the credit in the world. I actually found this thread through your original post, and it is one of my tabs. 80% of my tabs are this forum.

 

I wanted to share my own experiences with exactly what worked and what didn't work for me personally, because I've found it extremely rare that people share that 'what didn't work' part, and then they wonder what they did wrong if they messed up something. I will certainly be more mindful of that going forward. I just lost count of which post exactly contained the part of information that I used for something specifically.

[meina222]

>RE Ethernet Port: Currently macos sees the Aquantic Ethernet port, which leads me to a question. If Proxmox is connected to one port, would the macos need a second ethernet connection via the Aquantic port?

 

@tsongz - the way I do this is as follows:

 

I have 2 Ethernets. I set up a VM Bridge with the gateway/router using 1 Ethernet for the host and pass the 2nd Ethernet to the VM (Catalina). That way both the host and the guest access the internet. When I set up Big Sur VM2 I wanted to use a different ROM / Mac address than Catalina so I swapped the Ethernets. That unfortunately meant that while Big Sur is up, the host loses the connection (but Big Sur still has it) as the gateway is only associated with 1 of the ethernets. The reason I did this is because I did not want to have Apple system record same ROM form 2 different Mac's trying to activate - likely won't work but didn't try it. This is only an issue with 2 Mac VMS though. You do need ethernet to activate iMessage, but once activated you can go back to vmx or wireless if you need your Ethernet back - I choose to leave it.

[fabiosun]

Misunderstood clarified! 🙂

I took it a bit since in December 2019 (on December 6 amazon gave me the cpu)

I started thinking about finding a way to make this system work under osx

The community said it should have worked with old patches

I made myself available to Algrey to perfect all the tests he wanted to do, also buying serial cards to make a serial output of the problem and often taking fish in the face from other users who said that I was not able to do a serial output and to send a trx40 chip to the developers

I lost hope when Algrey also told me to think about linux, which was perhaps the best solution

I tried the road with the OpenCore developers, who, with difficulty, gave me some steps to do, and with these we understood that the problem was patches (which they called "borked"

This is to tell you that during this process of knowledge I caught several fish in the face (Italian idiom) but I went ahead managing to have already since January the first hack made with TRX40 perfectly functional and performing then I decided to write this guide (you can find it in guides) which is not perfect, but it is certainly the most detailed and free that you can find around today

happy with the clarification you had, when possible it would be useful to see how your frontier performs, perhaps with the test on davinci resolve, cinebench 15 and with luxmark3.1

Thanks and welcome again 🙂

Edited August 3, 2020 by fabiosun
text formatting

[tsongz]

I can certainly relate to that feeling. A lot of my day to day work is just like that, since I'm building models that hasn't historically been done before. I will get those benchmarks for you once the system is stable and becomes my main system. I was actually planning on building a tool specifically for this using machine learning once I got things up and running, and providing it to this community for free. The short version of it is that it would look at your hardware configuration (CPU, motherboard, GPU, PCIe slots + addons, etc), intended OS, and populate the correct config.plist as much as possible.

 

@meina222 I was able to get CPU Host Passthrough Successfully! 

 

 

Few things I did first, I inspected all the ASCPI files from the existing ones from the install, and the ones from Driftwood's, and added the correct ones for this motherboard first, and added them to the config.plist. 

 

Specifically,

SSDT-DTGP.aml (still not quite sure what this one does)

SSDT-GFX.aml (GPU Power and mapping, after recompiling for Vega, and correcting the address in _ADR and the definition block)

SSDT-PLUG.aml (CPU Power - no correction needed as layout was identical)

SSDT-SBUS.aml (replaced SSDT-EHCI.aml - no correction needed)

SSDT-EC.aml (for USB Power - no correction needed)

 

I replaced all 0.5.9 OC files with 0.6.0 OC files from Pavo's OCBuilder, and then disabled all previous kexts and kept only:

- Lilu

- WhateverGreen

- AGPMInjector (Vega specific from Pavo's AGPMInjector tool)

- MCEReporterDisabler

 

Changed Cpuid1Mask from FFFFFF00 00000000 00000000 00000000 -> FFFFFFFF 00000000 00000000 00000000

Ensured that all the Kernel patches were correctly enabled. 

- Aquantia Patch

- algrey - cpu_topology_sort -disable _x86_validate_topology

- algrey - cpuid_set_cpufamily - force CPUFAMILY_INTEL_PENRYN

 

Went through all the enabled/disabled settings and matched them. 

 

Boot args were set at -v

 

GB5 Results: Surprisingly, my Metal Compute score went down, whereas the CPU score went up

 

CPU: 1241 (up from 1156) 16361 (up from 16350)

Metal: 58920 (down from 61914)

 

My SSDT-GFX.aml for Vega FE. _SB_.PCI0.SE0 is where it's reported in IORegistryExplorer. 

/*
 * Intel ACPI Component Architecture
 * AML/ASL+ Disassembler version 20200110 (64-bit version)
 * Copyright (c) 2000 - 2020 Intel Corporation
 * 
 * Disassembling to symbolic ASL+ operators
 *
 * Disassembly of iASLuDEBV3.aml, Sun Aug  2 23:19:49 2020
 *
 * Original Table Header:
 *     Signature        "SSDT"
 *     Length           0x00000213 (531)
 *     Revision         0x01
 *     Checksum         0x08
 *     OEM ID           "APPLE "
 *     OEM Table ID     "GFX"
 *     OEM Revision     0x00001000 (4096)
 *     Compiler ID      "INTL"
 *     Compiler Version 0x20200110 (538968336)
 */
DefinitionBlock ("", "SSDT", 1, "APPLE ", "GFX", 0x00001000)
{
    External (_SB_.PCI0, DeviceObj)
    External (_SB_.PCI0.SE0_, DeviceObj)

    Scope (_SB.PCI0)
    {
        Device (PEGP)
        {
            Name (_ADR, 0x001C0000)  // _ADR: Address
            Device (GFX0)
            {
                Name (_ADR, Zero)  // _ADR: Address
                Method (_DSM, 4, NotSerialized)  // _DSM: Device-Specific Method
                {
                    If ((Arg2 == Zero))
                    {
                        Return (Buffer (One)
                        {
                             0x03                                             // .
                        })
                    }

                    Return (Package (0x06)
                    {
                        "ATY,EFIVersionB", 
                        Buffer (0x10)
                        {
                            /* 0000 */  0x31, 0x31, 0x33, 0x2D, 0x44, 0x33, 0x36, 0x30,  // 113-D360
                            /* 0008 */  0x30, 0x32, 0x30, 0x30, 0x2D, 0x31, 0x30, 0x36   // 0200-106
                        }, 

                        "ATY,EFIVersion", 
                        Buffer (0x2B)
                        {
                            /* 0000 */  0x30, 0x31, 0x36, 0x2E, 0x30, 0x30, 0x34, 0x2E,  // 016.004.
                            /* 0008 */  0x30, 0x30, 0x30, 0x2E, 0x30, 0x33, 0x38, 0x2E,  // 000.038.
                            /* 0010 */  0x30, 0x31, 0x31, 0x37, 0x31, 0x37, 0x2E, 0x44,  // 011717.D
                            /* 0018 */  0x33, 0x36, 0x30, 0x30, 0x32, 0x30, 0x30, 0x2E,  // 3600200.
                            /* 0020 */  0x31, 0x30, 0x36, 0x2E, 0x31, 0x37, 0x34, 0x32,  // 106.1742
                            /* 0028 */  0x33, 0x36, 0x32                                 // 362
                        }, 

                        "hda-gfx", 
                        Buffer (0x0A)
                        {
                            "onboard-1"
                        }
                    })
                }
            }

            Device (HDAU)
            {
                Name (_ADR, One)  // _ADR: Address
                Method (_DSM, 4, NotSerialized)  // _DSM: Device-Specific Method
                {
                    Return (Package (0x02)
                    {
                        "hda-gfx", 
                        Buffer (0x0A)
                        {
                            "onboard-1"
                        }
                    })
                }
            }
        }
    }

    Scope (_SB.PCI0.SE0)
    {
        Device (GFX1)
        {
            Name (_ADR, Zero)  // _ADR: Address
            Method (_DSM, 4, NotSerialized)  // _DSM: Device-Specific Method
            {
                If ((Arg2 == Zero))
                {
                    Return (Buffer (One)
                    {
                         0x03                                             // .
                    })
                }

                Return (Package (0x08)
                {
                    "ATY,EFIVersionB", 
                    Buffer (0x10)
                    {
                        /* 0000 */  0x31, 0x31, 0x33, 0x2D, 0x44, 0x33, 0x36, 0x30,  // 113-D360
                        /* 0008 */  0x30, 0x32, 0x30, 0x30, 0x2D, 0x31, 0x30, 0x36   // 0200-106
                    }, 

                    "ATY,EFIVersion", 
                    Buffer (0x2B)
                    {
                        /* 0000 */  0x30, 0x31, 0x36, 0x2E, 0x30, 0x30, 0x34, 0x2E,  // 016.004.
                        /* 0008 */  0x30, 0x30, 0x30, 0x2E, 0x30, 0x33, 0x38, 0x2E,  // 000.038.
                        /* 0010 */  0x30, 0x31, 0x31, 0x37, 0x31, 0x37, 0x2E, 0x44,  // 011717.D
                        /* 0018 */  0x33, 0x36, 0x30, 0x30, 0x32, 0x30, 0x30, 0x2E,  // 3600200.
                        /* 0020 */  0x31, 0x30, 0x36, 0x2E, 0x31, 0x37, 0x34, 0x32,  // 106.1742
                        /* 0028 */  0x33, 0x36, 0x32                                 // 362
                    }, 

                    "hda-gfx", 
                    Buffer (0x0A)
                    {
                        "onboard-1"
                    }
                })
            }
        }

        Device (HDAU)
        {
            Name (_ADR, One)  // _ADR: Address
            Method (_DSM, 4, NotSerialized)  // _DSM: Device-Specific Method
            {
                Return (Package (0x02)
                {
                    "hda-gfx", 
                    Buffer (0x0A)
                    {
                        "onboard-1"
                    }
                })
            }
        }
    }
}

Also confirmed Audio passthrough works with Realtek USB2.0 (plugged in headphones in the front Audio port on the PC Case) 

 

 

Edited August 3, 2020 by tsongz
added SSDT-GFX.aml and Audio Passthrough

[fabiosun]

@tsongz

are you sure you need all those ssdt to have cpu as host booting fine?

In my MSI board no need for any of these ssdt, only to configure vm config and config.plist in open core properly (for owned hardware I mean)

 

[fabiosun]

and also for me is not useful to block all that stuff in blacklist.conf and in vfio.conf

but I can't say for sure because many of you and Proxmoxwiki advice to do that...

maybe I am lucky 🙂

 

[tsongz]

I am not 100% sure yet, I plan on doing process of elimination of the SSDTs one-by-one after backing up. I wanted them all just in case since it was moving from non-Vanilla. Will update and edit with which ones are eliminated.