Proxmox VE OSX Guide discussion

[meina222]

Here's the Catalina EFI (this one is OC 0.6.3 I believe). Again, missing just the "Resources" subfolder due to size. Needs serials and I have commented PCI DeviceProperties specific to me.

 

My VM config in /etc/pve/qemu-server/100.conf:

 

args: -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu host,+invtsc,vendor=GenuineIntel
balloon: 0
bios: ovmf
boot: cdn
bootdisk: virtio0
cores: 64
cpu: Penryn
efidisk0: aorus:vm-100-disk-1,size=1M
hookscript: local:snippets/vmhook.sh
hostpci0: 85:00,pcie=1
hostpci1: 84:00,pcie=1
hostpci2: 88:00,pcie=1
hostpci3: 43:00,pcie=1,romfile=vbios.bin,x-vga=1
hostpci4: 02:00,pcie=1
hugepages: 1024
ide2: local:iso/OpenCore.iso,size=150M
machine: q35
memory: 196608
name: catalina
numa: 1
ostype: other
scsihw: virtio-scsi-pci
smbios1: uuid=2dc4e309-f3cc-4b49-b09f-ca3e89f7a697
sockets: 1
vga: none
virtio0: aorus:vm-100-disk-0,cache=unsafe,discard=on,size=250G
vmgenid: 0c7cc702-74ba-4d8e-ba4b-d52d5fe53847

 

 

EFI.zip

[23d1]

@meina222 thanks for sharing! Gonna give it a whirl in a bit. ✌️

[meina222]

@23d1, any luck?

[dtek]

On 11/30/2020 at 12:49 AM, meina222 said:

I re-enabled the re-binding of the framebuffer and vtconsole - the highlighted commands were previously commented out - this is needed since otherwise you can't unbind in the pre-start phase when you restart.

 

When you shutdown your VM, you'd need an external device to restart it as main display won't come back to host (still not sure how to achieve that) but you can re-start VM from web console on a different device and display should come back. Similarly, rebooting a VM should just work and GPU should reset properly (for me DisplayPort sound stops working but this is WIP by developer).

 

#!/bin/bash

vmid="$1"
phase="$2"

if [[ "$phase" == "pre-start" ]]; then
    clear
    echo "Starting VM $vmid - please wait..."
    IFILE=/var/lib/vz/snippets/interfaces.$vmid
    if [[ -f "$IFILE" ]]; then
        cp /var/lib/vz/snippets/interfaces.$vmid /etc/network/interfaces
        systemctl restart networking
    fi          
    echo 0 > /sys/class/vtconsole/vtcon0/bind
    echo 0 > /sys/class/vtconsole/vtcon1/bind
    echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/unbind
elif [[ "$phase" == "post-start" ]]; then
    main_pid="$(< /run/qemu-server/$vmid.pid)"

    cpuset="0-63"
    #cpuset="$(< /etc/pve/qemu-server/$vmid.cpuset)"

    taskset --cpu-list  --all-tasks --pid "$cpuset" "$main_pid"
elif [[ "$phase" == "post-stop" ]]; then
    echo "Post-stop VM $vmid"
    #reboot
    #shutdown -h now

    sleep 1 
    echo efi-framebuffer.0 > /sys/bus/platform/drivers/efi-framebuffer/bind
    echo 1 > /sys/class/vtconsole/vtcon0/bind
    echo 1 > /sys/class/vtconsole/vtcon1/bind
fi
vmhook.sh (END)

 

 

 

Alternatively, you can get a cheap GPU for the host and not worry about the framebuffer unbind/rebind in the hook and manipulate the display output selector.

@meina222

Should I use this script for 2 separate VMs (Big Sur and Win10) and 2 GPUs (6800xt and 3060ti) passthrough?  Are there any modifications that is needed?

[meina222]

You mean the VM config?

[dtek]

36 minutes ago, meina222 said:

You mean the VM config?

Yeah the hookscript,  should I make any changes to the script to use on separate VMs?

[meina222]

For the GPU that will be used by Proxmox as primary display, you need to unbind the efi framebuffer but the other should be w/out the part referring to the framebuffer. So no, the hookscript won't be shared as identical copy. In your BIOS you should be able to select which PCIE slot is primary display. The other GPU should pass without any tweaking of the framebuffer.

 

If you want to run both VMs at the same time, it may get tricky. Make sure you are stable with 1 VM at a time.

[meina222]

My 3090 ended up on backorder btw (so I withdraw my TigerDirect recommendation, it doesn't seem honest), but I got a non-XT 6800 from Newegg that was charged and will ship tomorrow. So I may try to dual GPU VMs (6800 for Win 10 as BigSur doesn't seem to have the driver enabled yet and the 5700XT for MacOS).

[dtek]

19 hours ago, meina222 said:

My 3090 ended up on backorder btw (so I withdraw my TigerDirect recommendation, it doesn't seem honest), but I got a non-XT 6800 from Newegg that was charged and will ship tomorrow. So I may try to dual GPU VMs (6800 for Win 10 as BigSur doesn't seem to have the driver enabled yet and the 5700XT for MacOS).

I did a passthrough with dual GPU VMs(5700xt for Big Sur and  6800xt for Windows 10) and they're are both stable if only one VM is running at a time.  As soon as both VMS are started,  the machine froze up and rebooted on it's own.  I'm thinking it could be the 5700xt reset bug that might be causing the reboot so I got a Nvidia 3060 ti for further testing.  I didn't get a chance to install the 3060ti yet,  I'll keep you posted.  Please update me on your progress with the dual GPU VMs.

[meina222]

I think the issue may be with AMD's PCIE management or board IOMMU groups. I recall that to run many VM's with many GPUs, one ideally goes with an Intel platform. Will try and let you know, my 6800 shipped today so I should get it by early next week.

[fabiosun]

bump 🙂

 

[muifaha]

Hello I susses full install BigSur 11.4 in Ryzen 3600 with Proxmox 6.4 .My  GPU RX 5500 XT, usb and sata suscesfull passthrough, but passthrough onboard audio is like not working, I have AppleALC.kext in my EFI and also alcid in boot args, any solution about that?

[fabiosun]

@muifaha

have you passed audio controller as pci in proxmox?

do you see on board audio inside OSX?

 

[23d1]

Hey guys, it's been a while since I last posted. Finally picking this back up after a loooong time.

I've gotten my hands on a 6800 XT that I initially intended to use as an eGPU, but Apple's Metal API makes rendering on eGPUs very slow, most likely for some clandestine security reasons. Very annoying. In any case, I've gotten Big Sur up and running as a KVM via QEMU on Arch. The intent is to use my Arch install primarily for simulation and rendering, and macOS for compositing and other design tasks (Adobe CC and so forth, and potentially lookdev in 3D using the 6800XT to render).

 

For some odd reason that I don't know if, I manage to passthrough the GPU with no fuss, but upon loading macOS from the OpenCore boot menu, it simply crashes as it starts loading up. No error or anything, just freezes up and I have to SSH into the machine and kill the QEMU process to get back my USBs and disks that i passthrough (which works great without GPU).

 

Anyone run into this issue and can shed some light?

[fabiosun]

@23d1welcome back

Sorry I have left virtualisation way many months ago and I am not useful to solve this problem 

I hope other users could help you

[23d1]

On 9/19/2021 at 11:38 PM, fabiosun said:

@23d1welcome back

Sorry I have left virtualisation way many months ago and I am not useful to solve this problem 

I hope other users could help you

Ah, you've gone fully bare metal? That would be an option also, but I'm assuming I would run into similar issues. Guess I'll pick it back up, I do prefer bare metal, just the annoyance of turning off hyperthreading on a 64 core CPU. 🙂

Edited September 21, 2021 by 23d1

[fabiosun]

@23d1

yes very happy with bare metal

My software pipeline works at its best

Davinci Resolve

ADOBE creative suite (AE,PS,PR,ME) *

 

you will have same problems with your 3990x CPU (the limit is 64 core (no HT) or 32c +32t)..and this is an old and annoying OSX limitation.

 

*

 

[fabiosun]

@23d1when you are ready to bare metal we could test a thing if you like

new patches have a chance to declare number of cores and we could try to limit it there without entering in motherboard bios

I do not think we can cheat OSX cores limit in this way ..but it could be worth to try

 

[23d1]

On 9/20/2021 at 11:31 PM, fabiosun said:

@23d1

yes very happy with bare metal

My software pipeline works at its best

Davinci Resolve

ADOBE creative suite (AE,PS,PR,ME) *

 

you will have same problems with your 3990x CPU (the limit is 64 core (no HT) or 32c +32t)..and this is an old and annoying OSX limitation.

 

*

 

Oh, awesome—similar pipeline as me. I use Adobe CC, Autodesk Maya, Redshift and so forth. Lots of 3D and motion.

 

On 9/20/2021 at 11:31 PM, fabiosun said:

@23d1

yes very happy with bare metal

My software pipeline works at its best

Davinci Resolve

ADOBE creative suite (AE,PS,PR,ME) *

 

you will have same problems with your 3990x CPU (the limit is 64 core (no HT) or 32c +32t)..and this is an old and annoying OSX limitation.

 

*

 

I'll jump in the bare metal thread when I have some time (hopefully this weekend) to go crazy and nerd out. 🙂 Grazie mille!

[23d1]

Hopping back in this thread to see if anyone has some info in regards to stalling on "PCI Configuration Begins" in the kernel when booting with a passed through GPU in Big Sur? Would love to be able to boot via qemu as well as bare metal, which is working. I seem to be getting about 15% better performance via qemu kvm. Single GPU passthrough with all the hooks and everything is working, launching from within my Arch install, so it's odd to get stuck in the kernel.

I tried setting the boot flag npcie=0x2000 but did nothing.

 

Edit;

Disabling "Above 4G Encoding" in BIOS resolved the issue. Strangely, my KVM is functioning better/faster than my bare metal boot. Just some kinks to sort out and pass through all the proper ports and devices.

Edited October 14, 2021 by 23d1
Formatting

[Vell123]

Hi can i get some extra help with my setup

 

do this work with proxmox 7.1

 

[blirtec]

wonder 2

[g2v7b5n]

ASUS AMD Dual Radeon RX 6600 - working macOS 12.1 Monterey

Edited April 8, 2022 by g2v7b5n