Proxmox VE OSX Guide discussion

[fabiosun]

do not update for now

now also my minimal Vm hangs on ++++++++ end seed both with clover 5107 and opencore 0.58

try to revert to old kernel by now

i hope it is enough to start again my VM

😂

upgrading deleted my lvm.conf file so this was the reason for the hang on ++++++++ phase (MSR related)

[Driftwood]

On 5/10/2020 at 2:37 PM, Rocket88 said:

After a week of messing around, I finally got my Threadripper 3970x, TRX40 Creator, MSI Radeon Vega 64 to work. I only have one video card.

Here are the changes I had to make:

/etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on textonly video=amdgpu video=efifb:off"    <<< I need efifb:off or I get the BAR 0 error.

*** Note: My monitor will not show anything past the Ramdisk message with video=efifb:off" set

 

/etc/modprob.d/vlio:

# Make vfio-pci a pre-dependency of the usual video modules
softdep amdgpu pre: vfio-pci           <<< Not sure if this is still needed

options vfio-pci ids=1002:687f,1002:aaf8 disable_vga=1
options kvm_amd avic=1                <<< Not sure if this is still needed

 

/etc/pve/qemu-server/101.conf

hostpci0: 03:00,pcie=1,x-vga=on,romfile=MSI.RXVega64.8176.170719.rom    <<< romfile goes in   /usr/share/kvm/

*** Set the boot flag. Without the boot flag set, the display hangs up after the Ramdisk message.

 

I also made the following BIOS changes, and I have not checked yet to see if they are still needed:

 

 

1.     SR-IOV Support. 

2.     In bios under Advanced\AMD CBS\NBIO Common Options

   a.     Enable AER Cap

   b.     Enable ACS Enable.  (not visible until a. is done)

   c.      Turn on ARI

 

 

Now when I power up the machine, It jumps to the PROXMOX screen, then I can click on the icon of my MacOS, and then I can boot up my new AMD Macintosh. 

Things I need to do:

1. Create a hook script for powering down my hardware. For example something that periodically checks to see if the vm is running and if not, send "shutdown -h now" command. 

2. Everything else. I have days (weeks) of tinkering to do.

 

Once again I would like to thank all of you for your posts!!!!

 

 

which boot flag? In proxmox or opencore or ? My boot screen only gets to "ramdisk..." too. even though it boots correctly, it would be nice to see complete proxmox messages!

 

[fabiosun]

Ok, it is a not new kernel problem related..Probably new QEMU has some conflict with our bootloaders...

for now stay away from this or try with a solid backup

 

[fabiosun]

ok solved 🙂

root@pve:~# pveversion -v
proxmox-ve: 6.2-1 (running kernel: 5.4.34-1-pve)
pve-manager: 6.2-4 (running version: 6.2-4/9824574a)
pve-kernel-5.4: 6.2-1
pve-kernel-helper: 6.2-1
pve-kernel-5.4.34-1-pve: 5.4.34-2
ceph-fuse: 12.2.11+dfsg1-2.1+b1
corosync: 3.0.3-pve1
criu: 3.11-3
glusterfs-client: 5.5-3
ifupdown: 0.8.35+pve1
ksm-control-daemon: 1.3-1
libjs-extjs: 6.0.1-10
libknet1: 1.15-pve1
libproxmox-acme-perl: 1.0.3
libpve-access-control: 6.1-1
libpve-apiclient-perl: 3.0-3
libpve-common-perl: 6.1-2
libpve-guest-common-perl: 3.0-10
libpve-http-server-perl: 3.0-5
libpve-storage-perl: 6.1-7
libqb0: 1.0.5-1
libspice-server1: 0.14.2-4~pve6+1
lvm2: 2.03.02-pve4
lxc-pve: 4.0.2-1
lxcfs: 4.0.3-pve2
novnc-pve: 1.1.0-1
proxmox-mini-journalreader: 1.1-1
proxmox-widget-toolkit: 2.2-1
pve-cluster: 6.1-8
pve-container: 3.1-5
pve-docs: 6.2-4
pve-edk2-firmware: 2.20200229-1
pve-firewall: 4.1-2
pve-firmware: 3.1-1
pve-ha-manager: 3.0-9
pve-i18n: 2.1-2
pve-qemu-kvm: 5.0.0-2
pve-xtermjs: 4.3.0-1
qemu-server: 6.2-2
smartmontools: 7.1-pve2
spiceterm: 3.1-1
vncterm: 1.6-1
zfsutils-linux: 0.8.3-pve1
root@pve:~# 

it was simple..a file....kvm.conf 🙂 🙂 🙂

 

no improvement for our common problem for now

I would like also to remark:

 

useful file for my rig to modify after a clean Proxmox VE 6.2 iso:

 

• etc/modules

vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd

 

• etc/modprobe.d/kvm.conf

options kvm ignore_msrs=1

 

• /etc/default/grub.cfg 

quiet iommu=pt amd_iommu=on

 

then I have used my previous VM.conf file recreating in it only EFI disk

 

 

 

[Driftwood]

So its safe to upgrade now?

[fabiosun]

I did..in my case yes..no improvement but it is latest available so could have some benefit I cannot see by now

I f you are unsure stay with old version

[iGPU]

I've got TB3 tree working in this build!

 

More things to work out; details to follow. The teaser is the tree below with details appearing in the PCI/Thunderbolt section. This card was previously flashed with firmware. (I also inserted the GB4 window to show that it is indeed the 3970X CPU.) Interestingly, I only passed 15eb and 15ec. The other TB3 sections I did not even try to pass-thru. The card is in slot-4 and has the TB header connected, the USB-2 connection and the power jacks applied.

 

I think I was blocking the appearance of the TB device while re-naming USB devices, and each part was appearing on the PCI0 tree. Now, the entire TB tree is appearing within SB0, which contains the NVMe (M2M slot) drive located at 1:00.0. This drive contains macOS Mojave. Next, I'll work on the SSDT; once this is working TB3 should become functional.

 

Unfortunately, after I made the image below with passed-thru Radeon VII, I made many changes to config.plist file and now I cannot boot into macOS while passing the GPU. (Nuts.) Also, my water cooler pump is making a loud racket and I think I'll need to replace with an air-cooler.

 

BTW, fabiosun, how to you monitor temps; the usual SMCAMDProcessor kext doesn't work on the TRX40?

 

 

Edited May 13, 2020 by iGPU

[fabiosun]

@iGPU I monitor my temps with my motherboard led code display

in bios there is an option to monitor temps

trulyspinach app does not work

one kext seems to output a missing cpu Id in verbose

 

[paulthesparky]

i'm following this thread with utmost interest and respect for what you guys have achieved so far, using Proxmox 6 VM to run OSX using Thread ripper 3970x. Exciting times indeed!

 

I myself have yet to do my first ever Hackintosh, but given your trail blazing success so far, and the power advantage of AMD Ryzen over a Intel I9, I'm willing to give this a go, although I see your all wisened Hackintosh professionals, compared to a noob like myself. Fortune favours the brave, I say.

 

If iGPU can get TB3 running smoothly, I'll probably buy the same parts list GB TRX40 Designare M/Board etc., to run Mojave for a music/ audio production. 

 

Is anyone running Logic x? Any benchmarks similar to Morganaut on you tube, running 540 tracks on a 3970x? 

 

 

 

 

[fabiosun]

@paulthesparkywelcome here 

yep I did test you said

without screen software recording I have 580 tracks as I said in this thread or in the main guide thread 🙂

take a look if you want also in this channel:

 

 

FYI we have here this system working from first days of February 🙂 

so a bit first of Theresa 😛 

we did a free guide to build from yourself if you have one 🙂 🙂

[paulthesparky]

Cheers Fabio, Thanks for the welcome. You guys rock my world.

 

Those results (580 tracks) absolutely blow my mind. I've heard reports from other users with non-VM AMD Hackintosh 'Vanilla' builds (up to 3950x max.), who  seem to be running into stability issues when using Logic, whilst  running 3rd party plugins like Waves Mercury Bundle. 

 

Have you had any stability issues yourself on Logic using 3rd party plugins? 

 

I intend to use a lot of 3rd party plugins like Acustica Audio, Fabfilter, Izotope, Soundtoys, Valhalla etc., so interesting to find out how well it's performed?

 

Any other audio/ sleep/ bluetooth/ wi-fi issues I should know about, before I dive into the AMD VM rabbit hole?

 

 

 

 

 

Edited May 13, 2020 by paulthesparky
grammar error

[fabiosun]

1 hour ago, paulthesparky said:

Cheers Fabio, Thanks for the welcome. You guys rock my world.

 

Those results (580 tracks) absolutely blow my mind. I've heard reports from other users with non-VM AMD Hackintosh 'Vanilla' builds (up to 3950x max.), who  seem to be running into stability issues when using Logic, whilst  running 3rd party plugins like Waves Mercury Bundle. 

 

Have you had any stability issues yourself on Logic using 3rd party plugins? 

 

I intend to use a lot of 3rd party plugins like Acustica Audio, Fabfilter, Izotope, Soundtoys, Valhalla etc., so interesting to find out how well it's performed?

 

Any other audio/ sleep/ bluetooth/ wi-fi issues I should know about, before I dive into the AMD VM rabbit hole?

 

 

 

 

 

 

Hi Paul, this "hole" for audio related part is a bit risky by now.

Internal audio of TRX40 mb is bridged (maybe) with usb controller so this for now it is problematic

I have bought also a cheap USB sound adapter and also this has problems.

Display Port audio from my Nvidia GFX is perfect and I have tested with it audio features ( I am not an audio expert) but I can affirm which Proxmox/osx you can use any plugin without problem if this plugin works well in OSX

Proxmox/osx installation is not affected from problem you may have with Vanilla patches method..

 

[paulthesparky]

2 hours ago, fabiosun said:

 

Hi Paul, this "hole" for audio related part is a bit risky by now.

Internal audio of TRX40 mb is bridged (maybe) with usb controller so this for now it is problematic

I have bought also a cheap USB sound adapter and also this has problems.

Display Port audio from my Nvidia GFX is perfect and I have tested with it audio features ( I am not an audio expert) but I can affirm which Proxmox/osx you can use any plugin without problem if this plugin works well in OSX

Proxmox/osx installation is not affected from problem you may have with Vanilla patches method..

 

Fabio, That’s great news on plugin/ DAW stability, and heard similar good reports from VM users vs Vanilla patches. As long as I can rely on using my RME PCIE MADI card connected to a RME UFX+ interface, for audio, I should be fine. I don’t need to use a 3.5mm Audio Jack, DP/HDMI/USB ports for sound, video only. Similarly, I don’t require thunderbolt, but still like the idea of it, and looking forward to iGPU writing a SSDT, as it’s another milestone on the journey.

[Driftwood]

So with the new proxmox update we have greater than 6 maximum PCIE devices to pass through right? Is it okay to Update using the Proxmox web console Update button?

[fabiosun]

I have had also before more than 10 pci device passed

i do not see big improvement for us

backup your main proxmox config files

in my case I have updated via shell

then I did also a fresh installation with new iso without problems

[iGPU]

7 hours ago, paulthesparky said:

Fabio, That’s great news on plugin/ DAW stability, and heard similar good reports from VM users vs Vanilla patches. As long as I can rely on using my RME PCIE MADI card connected to a RME UFX+ interface, for audio, I should be fine. I don’t need to use a 3.5mm Audio Jack, DP/HDMI/USB ports for sound, video only. Similarly, I don’t require thunderbolt, but still like the idea of it, and looking forward to iGPU writing a SSDT, as it’s another milestone on the journey.

 

As I mentioned here I'm using an Audient USB interface without problems. There are many good USB interfaces now. A new Apogee Symphony USB desktop interface is supposedly coming to market soon (here) and will be bundled with various plugins.

 

A TB interface may be possible if I can activate it. The TB device tree is nested along with the NVMe drive and 2 other nodes, and so far has eluded me in creating an SSDT (I've written SSDT files for other nested USB nodes on this build without difficulty). I continue working on it. There may be a juggling act to balance the NVMe positions and how they and the TB device are assigned addresses, so next step is to try different locations for each part and see if a more amenable address is assigned to permit a functioning SSDT.

Edited May 13, 2020 by iGPU

[Driftwood]

Skylake 1080 ti vs the dual Radeon VII cards + Threadripper 3970X. I havent passed thru the cpu, nvme yet and USBs etc... just the GPUs. Looking on track though in initial tests - here Skylake versus Threadripper Ryzen 3970x. Disk SPeed plus 8K 12:1 RAW tests.

 

[iGPU]

10 minutes ago, Driftwood said:

Skylake 1080 ti vs the dual Radeon VII cards + Threadripper 3970X. I havent passed thru the cpu, nvme yet and USBs etc... just the GPUs. Looking on track though in initial tests - here Skylake versus Threadripper Ryzen 3970x. Disk SPeed plus 8K 12:1 RAW tests.

 

Interesting!

 

I didn't think dual GPUs worked with Hackintoshes, as they do on the Windows/Linux side. Are the scores noticeably improved with the 2nd GPU (I think I saw 120 on an earlier post vs 146 here)?

Edited May 13, 2020 by iGPU

[fabiosun]

@Driftwood if better results are obtained with a VM file disk..they are not accurate (sorry for this 🙂 )

if instead they are achieved with a passed drive or entire controller...are very good almost impossible but very good

 

@iGPUyou can use multiple gpu (not in crossfire or SLI), software like davinci resolve and other have a great benefit with dual gpu system

[lvtz]

Hi, I have some questions. They may be dumb for you, but for me and probably many other people may prove crucial to understanding all this.

1. I currently have macOS installed on my separate disk. Can I use it to start using Proxmox? Is a clean installation required? I would like to avoid configuring the system and installing apps from scratch.
2. Do I understand correctly that Proxmox + OpenCore must be on a separate disk and the installed macOS system must be on a different disk?
   1. If so, is it important to place Proxmox + OpenCore on a fast disk, or does it not matter and it may be located on a weaker disk.
3. Are the current methods relatively stable or can I encounter any unexpected issues during installation or the use of macOS itself?
   1. My build is based on: MSI MEG X570 ACE / 3900X / 5700XT
   2. Is there anyone who has achieved similar success with a similar build as mine?
   3. Is 5700XT supported, if so any additional actions will be needed?
4. Is switching to Proxmox worth with AMD 3900X processor? The current solution using OpenCore works OK, but I don't know if switching to Proxmox + OpenCore will bring significant improvement.

If possible, please provide the most detailed answers possible.

 

Of course, if I decide to take further steps, I will publish my feedback and any issues encountered or benchmark on my specific build, so as to support this Proxmox/macOS community as much as possible, which will certainly grow day by day.

 

Thank you for your hard work and for inspiring me to think about doing my Hack once again in a different way.

Edited May 14, 2020 by lvtz

[fabiosun]

2 hours ago, lvtz said:

Hi, I have some questions. They may be dumb for you, but for me and probably many other people may prove crucial to understanding all this.

1. I currently have macOS installed on my separate disk. Can I use it to start using Proxmox? Is a clean installation required? I would like to avoid configuring the system and installing apps from scratch.
2. Do I understand correctly that Proxmox + OpenCore must be on a separate disk and the installed macOS system must be on a different disk?
   1. If so, is it important to place Proxmox + OpenCore on a fast disk, or does it not matter and it may be located on a weaker disk.
3. Are the current methods relatively stable or can I encounter any unexpected issues during installation or the use of macOS itself?
   1. My build is based on: MSI MEG X570 ACE / 3900X / 5700XT
   2. Is there anyone who has achieved similar success with a similar build as mine?
   3. Is 5700XT supported, if so any additional actions will be needed?

If possible, please provide the most detailed answers possible.

 

Of course, if I decide to take further steps, I will publish my feedback and any issues encountered or benchmark on my specific build, so as to support this Proxmox/macOS community as much as possible, which will certainly grow day by day.

 

Thank you for your hard work and for inspiring me to think about doing my Hack once again in a different way.

 

Hi

1) yes you can use it*

1b) no no clean installation is required

2) if you want to use an exist OSX instalaltion yes, if you use a disk in a virtual file no

3)

success dipend from your skill and patience. It is not simple but many users with your same rig (a vanilla Patches supported system) are trying proxmox having less problem with compatibility for app (0 problem) and better gpu performances

It is Tricky but it could work 🙂

 

* you have to pass your disk controller or single disk (SATA, NVME..)

 

To have a detailed answer you have to read all the post here and also in main guide..

you can also check @Pavo GitHub , there you will find a system more similar to yours

 

 

[AllubzV]

17 hours ago, iGPU said:

 

Interesting!

 

I didn't think dual GPUs worked with Hackintoshes, as they do on the Windows/Linux side. Are the scores noticeably improved with the 2nd GPU (I think I saw 120 on an earlier post vs 146 here)?

 

@iGPU The MacPro7,1 also utilizes multiple GPU's (basically Radeon VII, the Radeon Pro Duo = 2x 40cu Radeon VII which has 36cu), but it's not CrossFire anymore. AMD's recent Pro cards have a variant of Infinity Fabric (like the SLI / CF connector, but $100-200). macOS doesn't mind multiple GPU's, and it sees them typically as separate units, but allows them to work in tandem if the application supports it.

 

I'm excited to see your TB3 progress. I know the PCI-E / PCI ID layout can change drastically when changing components. This basically means that if you can't use the host to edit anything (like me, I'm always cut off and have to use the network web client), if you change a a part there's a chance LAN devices will switch names, and all that jazz. I've locked myself out a few times now, so I make practice to keep a recent backup of the few main files; it's easy to put them back if you have 'm written down.

 

Initially I was able to reboot my system within the VM without running into a "macOS disk not found"-type of message in the macOS verbose bootlog. I changed NVME's around, so maybe that's the  reason. I believe I had it in 01:00.0 first, and now it's in the other part, with 43/44:00.0. There's many iffy things to look at, and the struggle of changing around components when trying passthrough, especially when already configured, is a real pita.

@fabiosun I can imagine using a supported USB or PCI-E audio card will get us ahead very quickly, compared to fighting against AMD's funky PCI-E/USB audio layout, ProxMox its layers, and macOS its unofficial kext support. With the TB3 card working (which is good news for any system that has TB3 and wants passthrough), I think our options of having a sound card are good. I see why you want to use the onboard though. Such a "simple device" should "just work", right!?

 

 

To all, a general tip in addition to the fact that you can edit the Clover / Opencore ISO file when finally arriving in macOS: I keep a "failsafe" copy of the initial OpenCore.iso, so I can easily swtich back and forth between my edited EFI, and the failsafe EFI. Plus to that, I have zero trouble booting from the NVME disk its EFI partition, when all VM disks are disabled, and only my datadisk is passed through. QEMU finds the right EFI only then, as we know setting it as a primary boot device in QEMU doesn't really seem to work (or did I miss a memo?)

 

 

PS: I have sucessfully upgraded to 6.2, but I can't get my VM to boot at the moment, so working on that.

Edited May 14, 2020 by AllubzV

[fabiosun]

@AllubzV yes i think so for internal device..also i have bought a cheaper USB audio device and it has the same problem

 

For your problem try to see if you have your KVM.conf file (MSR related)

in my case that was the problem

 

[fabiosun]

ok a big improvement for me..

trying to solve audio problems I have had an idea to try to pass impassable USB controller in my case they had an id like this:

10:22,148c

so I asked @Pavoto compile for me new kernel adding this patch:

 

diff --git a/drivers/pci/quirks.c b/drivers/pci/quirks.c
index 308f744393eb..9806bff34b9b 100644
--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -4999,6 +4999,9 @@ static void quirk_intel_no_flr(struct pci_dev *dev)
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1502, quirk_intel_no_flr);
 DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_INTEL, 0x1503, quirk_intel_no_flr);
 
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x148c, quirk_intel_no_flr);
+DECLARE_PCI_FIXUP_EARLY(PCI_VENDOR_ID_AMD, 0x1487, quirk_intel_no_flr);
+
 static void quirk_no_ext_tags(struct pci_dev *pdev)
 {
     struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
-- 
2.24.1

He compiled it successfully and I have inserted in my Proxmox

now all controller are passed under vfio driver (in my case no need to declare it on vfio.conf or block any drivers)

 

Pavo added this patch in his GitHub .

I think this could also be useful to pass other problematic controllers (ie thunderbolt?? ) 🙂

below my passed controller now:

 

04:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: Micro-Star International Co., Ltd. [MSI] Starship USB 3.0 Host Controller [1462:7c60]
        Kernel driver in use: vfio-pci
        Kernel modules: xhci_pci
23:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: Micro-Star International Co., Ltd. [MSI] Starship USB 3.0 Host Controller [1462:7c60]
        Kernel driver in use: vfio-pci
        Kernel modules: xhci_pci
46:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486]
        Kernel driver in use: vfio-pci
        Kernel modules: xhci_pci
46:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
        Kernel driver in use: vfio-pci

for now I haven't tested if I have additional problem using this patch....

we are on the bleeding edge..using not tested hardware for our task 😉

 

[Driftwood]

@fabiosun I'm getting https://enterprise.proxmox.com/debian/pve buster InRelease' is not signed / unauthorized IP etc... when trying to upgrade - did the proxmox thing pasting in the following;-

# PVE pve-no-subscription repository provided by proxmox.com,
# NOT recommended for production use
deb http://download.proxmox.com/debian/pve buster pve-no-subscription

 

Any ideas, and does your proxmox versio show v 6.2?

 

Edit: Sorted: Ran this through https://serenity-networks.com/how-to-update-proxmox-ve-when-you-do-not-have-a-valid-subscription/
ssh 'Shutdown -r Now' , Then ran Update off the Datacenter/PVE/Updates button. Sorted to

Virtual Environment 6.2-4

 

 

Edited May 14, 2020 by Driftwood