Proxmox VE OSX Guide discussion

[fabiosun]

2 hours ago, Rox67er said:

@fabiosun @Driftwood

 

I Just posted my config files and the IOMMU / USB pdf file on GitHub: https://github.com/iroger/ASRock-TRX40-proxmox

 

I will try to keep it up to date with my latest config so others can use and learn as well.

 

Things planned:

* Swap out my intel wifi/USB for a fenvi BCM94360 NGFF version (It is in the mail now and should be a drop in replacement) since the intel one gives headaches.

* Diagnose an USB 7-5 error I get occasionally when booting proxmox, when I get this error and then restart I get the Dr debug 99 error and have to switch off the power supply to get it booting again. (USB 7-5 is the internal LED controller)

* Complete the USB overview, which port goes where

* Go to a cleaner OpenCore config, probably with passing "CPU host" 

 

Any remarks and tips are welcome!

 

seeing your VM conf I could imagine which some problem could happen to your system because you allocate (maybe) to much memory for VM if you have in total 128Gb..in my case in my previous attempts I have had put about 122000..and vm not was stable in that case

I am not using also huge page stuff.. is it useful for you? and so when/where do you see benefit?

 

[Rox67er]

3 hours ago, Driftwood said:

Removed the BT kext...good thinking. 

 

The only thing I'm suffering is the Radeon Shutdown/restart problem. So it solves that?

If you use the hook script it should work.

 

I also noticed in your EFI you use the MacPro7,1 setting. Any reason for that? Benefits?

 

[Rox67er]

2 hours ago, fabiosun said:

seeing your VM conf I could imagine which some problem could happen to your system because you allocate (maybe) to much memory for VM if you have in total 128Gb..in my case in my previous attempts I have had put about 122000..and vm not was stable in that case

I am not using also huge page stuff.. is it useful for you? and so when/where do you see benefit?

 

Good remark, I will look into that. I allocate 120GB (120*1024 = 122880, thought leaving 8GB for the VM would be sufficient.) and everything seems to be stable and running well.

The huge pages suppose to give somewhat better performance, since the page table where ram allocation is kept is much smaller if you allocate huge chunks. (https://forum.proxmox.com/threads/huge-pages-support.15849/)

 

I did change my 100.conf to use the "CPU host" config and tweaked my opencore config with insight from @Driftwood this gives 8-10% better performance in GeekBench than before!

 

@Driftwood, is there a reason for the DSDT in your opencore ? I think they can both be removed. (I removed both and don't see a difference, plus the DSDT-ALS is I think for Ambient Light Sensor which is often a laptop thing)

 

Anyhow, boot is now reliable although I did see the 99 error again so I have not gotten to the bottom of that one. I will try to remove some HW passthrough (maybe USB?)  and see if I can narrow down the issue.

 

@fabiosun I saw you also pass SMBus controller to MacOS, what is the benefit of this?

 

Edited May 28, 2020 by Rox67er

[fabiosun]

@Rox67er no benefit to pass SMBus. I have tried to pass all controllers I can during my test

about 99 error, have you tried to reflash your bios and clear CMOS before configure it as you need?

 

[Driftwood]

4 hours ago, Rox67er said:

there a reason for the DSDT in your opencore

You mean SSDT?

[Rox67er]

2 minutes ago, Driftwood said:

You mean SSDT?

SSDT in deed. 😎

[Driftwood]

3 hours ago, fabiosun said:

reflash your bios and clear CMOS

 

I experience the same problems as Rox67er on the reboot/shutdown issue - I often have to switch off, pull power and sometimes I might wait 10 seconds before repowering/reboot in.

 

Basically the ASRock Creator seems to have problems with restart from Mac or a shutdown that the MSI board doesn't experience. Unless Fabiosun you have seen this too?

Edited May 28, 2020 by Driftwood

[Driftwood]

Hang on our shutdown error is GPU based of course... so... Which Radeon vii card u got? Looks like only the expensive Sapphire one is the only one which works properly in Mac/hax. Need to see if Sapphire ROM can be flashed to it in Windows. Did you try that already?

Edited May 28, 2020 by Driftwood

[Rox67er]

6 minutes ago, Driftwood said:

Which Radeon vii card u got? Looks like only the expensive Sapphire one is the only one which works properly in Mac/hax. Need to see if Sapphire ROM can be flashed to it in Windows. Did you try that already?

I have the Sapphire Radeon VII so that gives the same result unfortunately. I switched my config over to CPU host and removed wifi / bluetooth and am quite happy now, boot reliable apart from the shutdown - start issue when booting windows, shutting down and then starting Proxmox without power off I get the USB 7-5 error and that leads to the error 99. Somehow low level settings which don't get reset in the bios. It might be related to BIOS and agesa version.

 

I added my external U-Phoria UMC404HD audio interface and also got the stuttering audio. (didn't add the IRQ setting to VM yet, will try later) It works fine however if I plug it into my USB hub which is connected to the rear USB-C dedicated controller. So it is definitely something USB related. Do we need any SSDT or kext for the USB since we are passing them directly?

[fabiosun]

1 hour ago, Driftwood said:

 

I experience the same problems as Rox67er on the reboot/shutdown issue - I often have to switch off, pull power and sometimes I might wait 10 seconds before repowering/reboot in.

 

Basically the ASRock Creator seems to have problems with restart from Mac or a shutdown that the MSI board doesn't experience. Unless Fabiosun you have seen this too?

If I understand well it seems more probable it is related to amd Gpu reset bug?

[fabiosun]

@pavo and other user seem to have solved audio problem with this parameter to add in grub :

pcie_aspm=off
In my case it is not necessary but just in case you can try

to use only 10.15.5 solved for me this problem

@Rox67er

audio problem is surely related to usb or as usb audio drivers works in OS X with our audio double bridged 

[eMac]

I recently changed my motherboard to a B450 from Asrock, as I wanted to be able to use the ASMedia USB controller instead of the matisse, but due to the limitations of the firmware and AGESA, the IOMMU grouping was awful. I've now got an MSI B450 that has an ACS Feature to section off IOMMU groups better, so I will be testing this out at some point this week. Another thing I noticed was if you put CPU Type Under SMBIOS in Opencore on Bare Metal it changes the CPU to Cascade Lake and shows in system profiler as a Xeon W 12-Core Processor which has given me some performance increase on my 3900X, but the CPUType doesn't seem to take on a Proxmox Installation, Any ideas why CPU type wouldn't work bearing in mind the HOST CPU is being used and not an emulated Penryn.... Any Ideas?

 

Edited May 28, 2020 by eMac

[Jaidy]

I'm so envious of all of you who have the macOS going on Proxmox. Someone kindly help me set up mine as well. Is there a way to get the drivers for my wifi into Proxmox? I also have a TP Link Archer T9E wifi adapter..

[Rox67er]

3 hours ago, fabiosun said:

@pavo and other user seem to have solved audio problem with this parameter to add in grub :

pcie_aspm=off
In my case it is not necessary but just in case you can try

to use only 10.15.5 solved for me this problem

@Rox67er

audio problem is surely related to usb or as usb audio drivers works in OS X with our audio double bridged 

I tried it and also irq setting but both did not help with audio issue. 

[Rox67er]

3 hours ago, fabiosun said:

If I understand well it seems more probable it is related to amd Gpu reset bug?

Will try to investigate further, I’m pretty sure I didn’t have the issue when only GPU and nvme passed through. Did you see changes when passing things in different order?

[iGPU]

In follow up, I transferred the 3970X to a new MSI Creator. Things looked really good (although, this is my first MSI product and BIOS rather different than ASUS and GB and ASRock). But after 30 min, the same error codes began. Exact same error codes and then it crashed.

 

So I have a bad CPU. Interestingly, it started out okay, but then gradually got worse and worse. Fortunately, I had 3 days left to do an exchange, so I'll get a replacement 3970X next week.

[Driftwood]

40 minutes ago, iGPU said:

So I have a bad CPU. Interestingly, it started out okay, but then gradually got worse and worse. Fortunately, I had 3 days left to do an exchange, so I'll get a replacement 3970X

Good luck iGPU with the new cpu dude. 

[fabiosun]

5 hours ago, Rox67er said:

I tried it and also irq setting but both did not help with audio issue. 

Catalina 10.15.5?

5 hours ago, Rox67er said:

Will try to investigate further, I’m pretty sure I didn’t have the issue when only GPU and nvme passed through. Did you see changes when passing things in different order?

No

[fabiosun]

All problem we have (except gpu amd reset bug) are related to proxmox—->OS X usb driver version

then stuff like WiFi or Bt could also be related to our weird problem in conjunction with passed usb controllers

later I will try to explain more in detail and I would grateful if some of you could do some tries to confirm this

 

@Jaidy to start you have to connect to cabled ethernet

it seems your WiFi card is not seen by default proxmox driver

you can also find a driver for it but in this way you are adding additional difficulty to your starting proxmox/Linux learning curve

[fabiosun]

@iGPUare your memory tested?

I think also you have verified this failure in a different os environment than proxmox/OS X 

[iGPU]

16 minutes ago, fabiosun said:

@iGPUare your memory tested?

I think also you have verified this failure in a different os environment than proxmox/OS X 

 

I tried an old Windows 10 NVMe drive that I move between computers, and it booted and ran just fine. I left it on for a couple of hours over dinner. No crash.

 

So why is suddenly Proxmox creating problems? Today I used 6.1 and tonight I tried 6.2 --- all crash while I'm trying to get the VM files edited and ready to go. And I'm doing fresh installs. Hmmm... all the installs are on the same Samsung SATA SSD (its a new 500GB drive). I'll try installing on a different drive. I'll also test UnRAID. I have a test set up for it. I'll also install another flavor of Linux like Mint or Elementary to see how it behaves.

 

I have a few days before replacement comes. If this CPU is okay, I will keep and return the other one, unopened.

[fabiosun]

@iGPU i cant say where your problem is but i am pretty confident is about proxmox

if you see also on proxmox forum the staff answer are often not clear..like “you are on bleeding new hardware untested and so on “

apart of this i think proxmox is the best linux distro for our goal and maybe more simple to master in many aspect. As advise for all and also for me is to pass less things possible related to usb..because main problem are there in relation with osx driver and qemu pve.cfg.

 

i loose so many time to have a working audio in highsierra when in windows it was working well and the solutions was in front of my eyes...driver.

catalina solved this problem and i am using the same linux and vm config previously used.

and also catalina is using audio driver in a worst way than windows..but now it works. So this is a fact not in our hands.

 

i can now using my system professionally with highsierra using dp audio but i have to accept some limitation (usb 3 device are not seen if i pass controllers or single device) but system is solid rock and fast as windows in cpu/gpu jobs

 

Osx for me has a sense only with highsierra and catalina for now it is a no solution for me..

also in catalina i could no accept amd gpu reset bug!

but in general our needs could differ

 

[cj750]

Thank you to Driftwood, iGPu, fabiosun,Rox67er,Rocket88,Pavo,

 

Wow! I read the entire 15 pages slowly like a movie. This would be a great movie. Driftwood said it best. All of you are pioneers. This help me understand how hard it really is to get a system like mine working. Even thou i have not gone down the rabbit hole yet. I have allot of respect for all the work you have already done. But i know where it started once you get these board running like a fine tuned F1 race car. 

 

 

[fabiosun]

@cj750 welcome here

I will happy you also partecipate with your point of view. It is interesting to see also newbie (in this task) experience..because maybe we are doing the same errors 🙂

If you have time put your rig in signature so we can see better motherboard and cpu you use 🙂

Thank you

[paulthesparky]

1 hour ago, cj750 said:

Thank you to Driftwood, iGPu, fabiosun,Rox67er,Rocket88,Pavo,

 

Wow! I read the entire 15 pages slowly like a movie. This would be a great movie. Driftwood said it best. All of you are pioneers. This help me understand how hard it really is to get a system like mine working. Even thou i have not gone down the rabbit hole yet. I have allot of respect for all the work you have already done. But i know where it started once you get these board running like a fine tuned F1 race car. 

 

 

I would have to agree totally. I've read all 15 pages as well. Total respect, gratitude and appreciation for all involved in this thread, for having the tenacity, problem solving ability and teamwork to see this through.

 

It's so great to see how successful each individual build becomes, as the community works together and continue to resolve the issues/ pitfalls.

 

The popularity of this 3970x/ TRX40 build has the potential to absolutely skyrocket, as the level of success increases. It's exciting to see.

 

Reminds me a bit like the early days of internet web browsers, where Yahoo/Google etc. were fighting it out for superiority.

 

Interesting to see whether Proxmox becomes the VM king. Time will tell.