Proxmox VE OSX Guide discussion

[Rox67er]

@Driftwood from your VFIO I see you are passing the 05:00.3 and 25:00.3 USB as well? Are you using a patched kernel for this?

 

[Driftwood]

15 minutes ago, Rox67er said:

your VFIO I see you are passing the 05:00.3 and 25:00.3 USB as well? Are you using a patched kernel for this

 

No. But Pavo patched the 8125 driver for me. We're going to work on a guide to include all this stuff over the next few days.

Edited June 17, 2020 by Driftwood

[iGPU]

Another source of USB problems may be Catalina... 

 

To  quote this link: "Apple has a USB 2.0 issue with either the chipset they are using, or a Catalina bug regarding the handling and refreshing of USB 2.0 devices....Transparent proxies take the USB 2.0 input and present it as USB 2.0 to the ‌MacBook Pro‌. The Mac or Catalina then will do something wrong and the USB 2.0 devices will freeze / become unresponsive at some point (minutes or hours after being attached)."

***

 

I'm still experiencing problems with stability that I'm working on (hence my silence). I'm trying to further sort out issues one by one.

 

(And  Newegg sent me the exact same Firewire card I'd already tried from Amazon. Newegg's ad showed the older style card, but that's not what they shipped; so it went back.)

[fabiosun]

15 hours ago, Driftwood said:

 

No. But Pavo patched the 8125 driver for me. We're going to work on a guide to include all this stuff over the next few days.

Could you explain how 8125 ethernet driver is related to those usb controllers that @Rox67eris asking for in above his message?

i think if you are following also pavo’s github or private advice you are using some kernel patched deb...with 148c id

this thing helps to pass above usb controller

it is a well known patch named Flr patch and you can find it on reddit from january of this year, or better..there you can find the one useful for x570 chipset, then i have asked to Pavo to compile for me the ones useful for us trx40 users and he very kindly compiled the patch as i requested to him.

waiting your guide in a few days to understand better😉

[fabiosun]

6 hours ago, iGPU said:

Another source of USB problems may be Catalina... 

 

To  quote this link: "Apple has a USB 2.0 issue with either the chipset they are using, or a Catalina bug regarding the handling and refreshing of USB 2.0 devices....Transparent proxies take the USB 2.0 input and present it as USB 2.0 to the ‌MacBook Pro‌. The Mac or Catalina then will do something wrong and the USB 2.0 devices will freeze / become unresponsive at some point (minutes or hours after being attached)."

***

 

I'm still experiencing problems with stability that I'm working on (hence my silence). I'm trying to further sort out issues one by one.

 

(And  Newegg sent me the exact same Firewire card I'd already tried from Amazon. Newegg's ad showed the older style card, but that's not what they shipped; so it went back.)

Searching on the web for mine highsierra audio problem i have found many users using usb audio on mac with the same problems or pretty similars one

i am convinced that problem is not related to Vm also because in a proxmox windows vm i have not problems at all.

 

if you want and you have time explain which problems do you have and if they are playable also for me using same software or procedure

if these problems happen randomly...it could be a difficult task.

 

[Driftwood]

21 hours ago, iGPU said:

(And  Newegg sent me the exact same Firewire card I'd already tried from Amazon. Newegg's ad showed the older style card, but that's not what they shipped; so it went back.)

Like I linked some pages back 'Laptops Direct UK' seem to have the single chip fw card:

 

https://www.laptopsdirect.co.uk/startech.com_3_port_2b_1a_low_profile_1394_pci_express_firewire_card_adapte_pex1394b3lp/version.asp?refsource=ldadwords&mkwid=sghpO99t9_dc&pcrid=293700874314&product=PEX1394B3LP&pgrid=59244711832&ptaid=pla-363324503738&channel=search&gclid=EAIaIQobChMIwYa056vu6QIVFWHmCh0TTAjGEAQYAyABEgJOvvD_BwE#!/%23specs

[Driftwood]

On 6/18/2020 at 7:20 AM, fabiosun said:

is a well known patch named Flr patch and you can find it on reddit from january of this year

I wasnt interested in the USB controller with the audio on as Im not using it. So Im not sure if Pavo patched my kernel for that as I never asked but maybe he did. I will check. The only thing I was originally interested in getting working correctly was the Realtek 8125 for the host use.

 

Im only using the two USBs as found in my spreadsheet and diagrams shown earlier.

 

Also, its been found that with the latest BIOS beta firmware we dont need flr patch. As I quote from that site: USB passthrough works fine on AGESA 1.0.0.4 Patch B without needing this patch.

Edited June 19, 2020 by Driftwood

[iGPU]

For the past couple of weeks, both Mojave and Catalina have been unstable for me on this VM. After logging in, I often didn't see a background image; instead, only  a black background was present. The normal background would pop into view after a few minutes. If I did a Re-Start, the background would instantly (normally) appear. Also, after the background popped into view, the mouse cursor would eventually disappear and the computer would lock-up, forcing a shutdown. There was no correlation with AppleALC or WEG kexts.  I had no stability issues if running under Console mode; instability was only present when passing the GPU(s).

 

Now, I finally seem to  have stability...

 

The critical steps for me were:

1.  cut  out some of the SSDTs that I was using to rename devices (a habit I've picked up from bare-metal builds). When present, boot time is prolonged.

2. replace my power supply (I don't think the brand is as important as simply having a good one). I normally used EVGA, but this last one was a lemon.

3. reluctantly, I added 'blacklist amdgpu'. This was probably the most important step for stability. It also fixed the missing background on log-in as well as

         making the boot time much faster when the GPUs were being passed.

 

Unfortunately, when blacklist amdgpu is used, the host monitor output disappears. As long as the GUI interface works, this isn't a problem. But as I found out a few weeks ago, after moving the Proxmox NVMe drive to another slot, the GUI can also disappear. When that happens, you're totally locked out. The only solution is to then do a Proxmox re-install.

Edited June 19, 2020 by iGPU

[fabiosun]

14 hours ago, Driftwood said:

I wasnt interested in the USB controller with the audio on as Im not using it. So Im not sure if Pavo patched my kernel for that as I never asked but maybe he did. I will check. The only thing I was originally interested in getting working correctly was the Realtek 8125 for the host use.

 

Im only using the two USBs as found in my spreadsheet and diagrams shown earlier.

 

Also, its been found that with the latest BIOS beta firmware we dont need it. As I quote from that site: USB passthrough works fine on AGESA 1.0.0.4 Patch B without needing this patch.

I know you are not interested in those usb (by the way, audio for our motherboards is a bit different from others system like x570 chipset)

I was quoting you because @Rox67erdid a precise question about two usb controllers..and both of them need FLR patch (1022:148c id)

I am also very surprised from your Opencore EFI, two aspects should be clarified for others users (ie ALS ssdt useful for what in your rig and other stuff like 7.1 SMBIOS misconfiguration ram popup problem if you are not using any kext o stuff to avoid this well known problem)

 

interesting the AGESA part of your message..if true, it is another little step to patch less things possible and have a perfect proxmox/OSX system! I have to try also with mine 1.0.04 agesa MSI firmware

 

@iGPUwhen you loose GUI shell interface I think is also possible to boot with a minimal working shell during blu Proxmox menu choice if you do not ave automatise all the booting process

Also I would like to say that AppleALC.kext is not necessary for USB audio, also for integrated one (Realtek 2.0 audio) or a cheaper USB audio pen
In my case Apllealc is useful for hdmi/dp audio toghter Nvidia web driver correct installation process

Edited June 19, 2020 by fabiosun
added iGPU part :)

[iGPU]

8 hours ago, fabiosun said:

@iGPUwhen you loose GUI shell interface I think is also possible to boot with a minimal working shell during blu Proxmox menu choice if you do not ave automatise all the booting process

Also I would like to say that AppleALC.kext is not necessary for USB audio, also for integrated one (Realtek 2.0 audio) or a cheaper USB audio pen
In my case Apllealc is useful for hdmi/dp audio toghter Nvidia web driver correct installation process

 

Yes, AppleALC provides HDEF seen in IORegistryExplorer, so I continue using.

 

My list  of kexts are:

 

Essential:

Lilu

VirtualSMC  (a branch of FakeSMC + GPU support kext is better for getting temps off of a Radeon VII)

 

Used:

AppleALC----------------------------needed for HDEF / HDMI

AirportBrcmFixup----------\

BrcmBluetoothInjector----\

BrcmFirmwareData---------|---needed for swapped BT/Wifi card (some users don't need these; I have no BT if they're not present)

BrcmPatchRAM3------------/

BTLEContinuityFixup-----/

SmallTreeIntel2576--------------needed for I211 Ethernet (different kexts needed for different Ethernet controllers)

 

Used (with mixed feelings):

WhateverGreen (when enabled does cause issues with 5700XT and Radeon VII on Intel platform; not necessarily same on VM)
 

Potentially useful (but not yet vetted for this VM):

NVMeFix

RadeonBooster (v1.6)

 

Not used (but could be useful in future if modified for Threadripper):

AMDRyzenCPUPowerManagement

SMCAMDProcessor

 

 

***

 

While the build now seems stable, I am experiencing audio drop-out (duration 0.5 to 5 seconds) while watching YouTube using a USB Audio Interface. I want to proceed using this build for a Davinci based video project I'm working on. The audio interface is required for recording, so fixing the drop-out is important.

 

It is not a regular drop-out, but somewhat random in frequency. I'm trying various remedies. Varying the USB ports (whether ASMedia USB-C or stock) does not help: drop-out is the same. Some areas to yet check are disabling BT and WiFi, along with  trying some of the USB related efi driver files provided with OpenCore builds.

 

Edited June 19, 2020 by iGPU
Audio Drop-out

[fabiosun]

@iGPUfor me audio problem was solved by Catalina ..I have not used it intensively in it because I have no acceleration with GFX in it (no web driver) if you are in Mojave OSX maybe you can solve with 10.15.5..

Someone solves with an USB hub independent of tsc variation of pc chipset..but it could be a serious problem

I use davinci in High Sierra with HDEF audio, and there no problem at all

USB audio is a problem for now

 

[iGPU]

2 hours ago, fabiosun said:

@iGPUfor me audio problem was solved by Catalina ..I have not used it intensively in it because I have no acceleration with GFX in it (no web driver) if you are in Mojave OSX maybe you can solve with 10.15.5..

Someone solves with an USB hub independent of tsc variation of pc chipset..but it could be a serious problem

I use davinci in High Sierra with HDEF audio, and there no problem at all

USB audio is a problem for now

 

 

Thanks for reply. I have this issue whether Mojave 10.14.6 or latest Catalina 10.15.6-beta2.

 

I'm not using any built-in audio, but instead a USB-interface that is plugged into a USB-port. All ports seem to behave the same with these drop-outs.

 

BTW, I do NOT see this problem when running ArcoLinux on this same mobo with same USB-interface, nor do I see it with Intel Hackintoshes using same interface. So this problem is most likely related to the VM and USB ports.

Edited June 19, 2020 by iGPU

[fabiosun]

32 minutes ago, iGPU said:

 

Thanks for reply. I have this issue whether Mojave 10.14.6 or latest Catalina 10.15.6-beta2.

 

I'm not using any built-in audio, but instead a USB-interface that is plugged into a USB-port. All ports seem to behave the same with these drop-outs.

 

BTW, I do NOT see this problem when running ArcoLinux on this same mobo with same USB-interface, nor do I see it with Intel Hackintoshes using same interface. So this problem is related to the VM and USB ports.

Yes agreed

it is a VM USB / OSX USB driver

Beta 10.15.6 introduce newly a little problem with audio as I have had with 10.15.4 release

10.15.5 was perfect for both (internal Realtek audio or external USB pen)

In a proxmox windows VM audio is perfect..for this I am confident it is OSX related problem

 

[Rox67er]

Have been trying the suggestion of @Driftwood regarding the XHCI biosd setting, found this link also suggesting to put them on "enable" in stead of "auto". https://forums.unraid.net/topic/87557-guide-asrock-x570-taichi-vm-w-hardware-passthrough/

 

I do see effect, after adding the setting I now am able to boot MacOs up to the login screen, and if I then shutdown and re-start the VM, all is fine. If I however login to MacOs and THEN shutdown I still run into the 99 error after second reboot. Have been trying other options (CSM / Above 4G / ...)  but did not find a combination that worked...

[Driftwood]

12 hours ago, iGPU said:

AirportBrcmFixup----------\

BrcmBluetoothInjector----\

BrcmFirmwareData---------|---needed for swapped BT/Wifi card (some users don't need these; I have no BT if they're not present)

BrcmPatchRAM3------------/

BTLEContinuityFixup-----/

 

SO I just got the same Broadcom BCM4360 PCIe M2 WIFI/BT as @Rox67er and its installed and after a bit of a play with moving IOMMUs Ive got wifi on the vm working which I dont really need but hey! its the BT I need. How have you guys got the Bluetooth working? Is it necessary for all those kexts for the Broadcom? Wireless works out of the box and I was expecting BT to...

 

Any pointers would be of help, thanks fellas.

 

 

 

 

Edited June 20, 2020 by Driftwood

[iGPU]

25 minutes ago, Driftwood said:

SO I just got the same Broadcom BCM4360 PCIe M2 WIFI/BT as @Rox67er and its installed and after a bit of a play with moving IOMMUs Ive got wifi on the vm working which I dont really need but hey! its the BT I need. How have you guys got the Bluetooth working? Is it necessary for all those kexts for the Broadcom? Wireless works out of the box and I was expecting BT to...

 

Any pointers would be of help, thanks fellas.

 

Both BT and Wifi work on the swapped card if you use all the kexts I indicated. (The first is for WiFi, the others for BT.)

 

If this doesn't work, then:

1. don't have the card properly seated in its socket

2. the antennas are not properly connected to the card or the rear panel

3. you didn't pass the USB power for BT (I wrote about this once or twice on this thread)

On the other hand, as I posted on this thread, if you ONLY need BT, then the stock BT/Wifi card (no swap needed) works just fine if you  use the 2 kexts I uploaded earlier on this thread.

[Driftwood]

8 hours ago, iGPU said:

On the other hand, as I posted on this thread, if you ONLY need BT, then the stock BT/Wifi card (no swap needed) works just fine if you  use the 2 kexts I uploaded earlier on this thread

My wifi works out the box... Just need to black list the BT I reckon plus maybe some of the kexts... Lets try this out.

[fabiosun]

@iGPU

in archLinux you have no problem with audio

you mean with also an OS X vm loaded with it?

 

It could be interesting if in an intel vm with OS X is present the same problem

i see many people suffering audio problem also in vanilla amd forum Someone solve it forcing all pcie slot to 3.0..

 

 

[Rox67er]

1 hour ago, Driftwood said:

My wifi works out the box... Just need to black list the BT I reckon plus maybe some of the kexts... Lets try this out.

@Driftwood It should work "out of the box" I have NO Broadcom kext loaded. Only LiLu / WEG / Virtual SMC / AppleALC.

blacklist.conf added:

blacklist btusb

blacklist bcma

 

vfio.conf added:

# WiFi PCI broadcom

options vfio-pci ids=14e4:43a0

 

In your VM forward:

hostpci: 45:00.0, pcie=1.    #This will give you the wifi

 

Bluetooth is connected on USB so make sure you either forward the USB device or completely pass the USB controller to your VM (it is connected on 47:00.1 USB controller)

lsusb gives you the BT info:

Bus 007 Device 003: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth)

 

If you don't forward the USB controller use the hardware passthrough in Proxmox to select the right device, lsusb somehow gives other data? 

• 
  • 
     

Edited June 20, 2020 by Rox67er
added System info screenshot

[Driftwood]

11 minutes ago, Rox67er said:

Bluetooth is connected on USB so make sure you either forward the USB device or completely pass the USB controller to your VM (it is connected on 47:00.1 USB controller)

 

This is the missing piece of the jigsaw perhaps as Im not passing this controller.

[Rox67er]

1 minute ago, Driftwood said:

This is the missing piece of the jigsaw perhaps as Im not passing this controller.

Yep, it can be confusing but on old Mac Pro 5.1  I remember you had to connect a separate USB cable to get bluetooth working so I knew it needed a pci port for wifi AND a USB port for bluetooth.

 

Mine lists as manufacturer Apple Inc.

[Driftwood]

Stupid me I forgot from old hack that BT needs USB so Ive passed 48:00.1 again (which I didn't need those USBs but lets see how it shapes up with my audio under the latest fw beta).

FYI: My addresses have shifted up one from 47:00.1 to 48:00.1 for anyone following the history of this build. 

 

Anyways BT finally working. Thanks @Rox67er and @iGPU for the pointers. No kexts required. Blacklisted bcma, btbcm, and btusb for those who are following.

 

 

blacklist.conf

 

Spoiler

blacklist.conf nano /etc/modprobe.d/blacklist.conf
# This file contains a list of drivers we want to disable from the kernel so we can h$
blacklist amdgpu
blacklist snd_hda_intel
blacklist xhci_hcd
blacklist r8169
blacklist atlantic
blacklist xhci_hcd
blacklist nvme
blacklist bcma
blacklist btbcm
blacklist btusb 

 

Bluetooth working:

Edited June 20, 2020 by Driftwood

[iGPU]

For the past couple of weeks, I've been playing with getting a VM working on ArcoLinux while, in parallel, working on problems in Proxmox. I did this to check on 2 issues: getting Thunderbolt to work and getting rid of the USB dropout problem.

 

The potential advantage of ArcoLinux, unlike Proxmox or UnRAID, is that it gets the latest kernels; maybe a year or two faster than Proxmox or UnRaid. I've finally got a Catalina VM working (see image below). However, I've not yet passed the GPUs and I'm presently only passing 8 cores to the macOS VM.

 

While the installation of ArcoLinux is simple and not much different than Proxmox, setting up a VM is more involved. In ArcoLinux, there is more support for VirtualBox (which doesn't seem to want to play nicely with macOS), but Virt-Manager seems to be a more flexible choice. However, it takes a few more steps to get it working. (I did not document what is involved, nor do I plan on presenting a tutorial; this post is more of an FYI.)

 

Once Virt-Manager is installed and the usual vfio files are created (same as with Proxmox), running the VM is straightforward. There are a few differences, such as the devices seen with IORegistryExplorer having different addresses, which appear more ordered. (I'll later update an image or two to this post.)

 

As with Proxmox, I was not able to get Thunderbolt working for the same reasons: no ability as of yet to pass the bridge portions. Although, I thought I'd read that in the near future, the Linux kernel will allow passing whole devices. (And since ArcoLinux kernels seem, a VM running under ArcoLinux might be expected to pass TB before other distros.)

 

With regards to the USB audio dropout problem, it is about the same as with Proxmox. However, I was able to noticeably reduce, if not eliminate, the dropout by turning off BT and WiFi. (I tried this once before in Proxmox and did not notice an improvement; but I'll revisit it again.)

 

Catalina VM under AcroLinux:

Edited June 22, 2020 by iGPU

[Driftwood]

31 minutes ago, iGPU said:

With regards to the USB audio dropout problem, it is about the same as with Proxmox. However, I was able to noticeably reduce, if not eliminate, the dropout by turning off BT and WiFi. (I tried this once before in Proxmox and did not notice an improvement; but I'll revisit it again.)

 

You've been where I have been. I didn't want to turn on 'that' particular USB controller - the one on my 48:00.1 (not doing 48:00.3) which is probably the same as your 47:00.1 & 3 to get BT working as it requires the power from this controller to work. But without it switched on its glitch free as far as firewire goes. With it on, yes BT works (you don't need it on for wifi) but then it introduces the errors with USB NAS dropouts, the stutter of BT if you switch off wifi, and the occasional audio glitching on my firewire card/fireface 800.

 

Ive made a point of this as a moan to ASRock to see if they have any more info out of AMD or other board manus as this problem seems across the board. Especially for those who need .3 switched on for HDAudio.

 

Ive looked at similar glitches on Linux unix boxes, and Windows. Its definitely a problem with this particular USB controller. The others are fine!

 

If theres no fix coming soon I think Im going to stop passing 48:00.1 and forget about BT and the USB ports it offers.

 

UPDATE: Just going to see if @Rox67er Add USB Device method is any better without passing the controller on 48:00.1 

 

Edited June 21, 2020 by Driftwood

[Driftwood]

@Pavohas just released his OC-X EFI /OC generator app in BETA 🙂

 

Give it a whirl if you're confused about OC and EFI setups.

 

Support for Ryzen in this.

 

Pavo-IM 'OC-Gen-X' App

Edited June 21, 2020 by Driftwood