Proxmox VE OSX Guide discussion

[Driftwood]

Hi guys, So Im about to go again after network nics just would not respond to allow me to log on. It seems the only way was for a re install of proxmox to achieve access to my home network!

In the vm before the network problem, I had installed Catalina onto the first nvme which was in m2 slot 2 and Ive now moved to m2 slot 0. My SSD for Proxmox is on sata 0.

 

After reinstalling to a new ssd, and able to root login, Im trying to recreate my old vm101 and all the other confs which Ive done, but for some reason when I start the vm it cannot connect getting 'No VNC' and cannot connect to server. Im under chrome but safari fails too. So Im wondering if Ive failed to do something or do I have to start all over again re-installing isos and catalina onto the nvme because something has gone amiss?

Edited May 18, 2020 by Driftwood

[iGPU]

3 hours ago, Driftwood said:

Hi guys, So Im about to go again after network nics just would not respond to allow me to log on. It seems the only way was for a re install of proxmox to achieve access to my home network!

In the vm before the network problem, I had installed Catalina onto the first nvme which was in m2 slot 2 and Ive now moved to m2 slot 0. My SSD for Proxmox is on sata 0.

 

After reinstalling to a new ssd, and able to root login, Im trying to recreate my old vm101 and all the other confs which Ive done, but for some reason when I start the vm it cannot connect getting 'No VNC' and cannot connect to server. Im under chrome but safari fails too. So Im wondering if Ive failed to do something or do I have to start all over again re-installing isos and catalina onto the nvme because something has gone amiss?

 

It seems like I may have seen this when I forgot to add:

echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

 

Also, when you move drives, you need to re-run "lspci -nnk" as the addresses change when you move anything on the mobo. At the very least, the drive will change to a new address, or at worse, everything gets shifted. Also, these movements change where macOS loads the devices as seen in IORegistryExplorer; so some SSDT files or DeviceProperties may need adjustment too.

[Driftwood]

1 hour ago, iGPU said:

 

It seems like I may have seen this when I forgot to add:

echo "options kvm ignore_msrs=1" > /etc/modprobe.d/kvm.conf

 

Also, when you move drives, you need to re-run "lspci -nnk" as the addresses change when you move anything on the mobo. At the very least, the drive will change to a new address, or at worse, everything gets shifted. Also, these movements change where macOS loads the devices as seen in IORegistryExplorer; so some SSDT files or DeviceProperties may need adjustment too.

 

Cool @IGPU: Ill give it a go! Might save some trouble! My kvm.conf I just rebuilt with nano editing and saved...  so it should be the same, I need to lspci-nnk though as youre right things may have changed!

 

Edit: Still getting 'No VNC' and cannot connect to server. I may have to take a look at all the other configs I rebuilt...

Edited May 19, 2020 by Driftwood
update

[fabiosun]

@Driftwood novnc problem is related to passed gpu and none in display VM config line

if you access to your amd from another pc/device you have to change that none and block gpu bybass (maybe)

in my experience any problem your rig can have (if it has been working well before) is caused from an user problem 😊

network problem could be tricky

if you installed Proxmox with vmxnet3 in vm config and if you pass the same network controller you will have proble or to navigate or to access to webinterface because passed ethernet will take your ip (ip you used in installation step) and conseguentially it assign a different ip to your vmxnet3 device (assuming you are in dchp for both)

 

@iGPU this should be last question about audio😂

do you know which driver is using your working audio device?

 

[fabiosun]

Gigabyte TRX40 Designare

 

ASrock TRX40 Creator

I would also use this USB port to have a proxmox disk connected and installed with all useful things (like VM and so on)
this helps to pass more easily all other controllers

[fabiosun]

a discover...not useful but...

Our internal USB Realtek USB 2.0 audio does no need any additional kext for "working"...

it works (bad) with only OSX system kext

 

[Driftwood]

6 hours ago, fabiosun said:

@Driftwood novnc problem is related to passed gpu and none in display VM config line

if you access to your amd from another pc/device you have to change that none and block gpu bybass (maybe)

in my experience any problem your rig can have (if it has been working well before) is caused from an user problem 😊

network problem could be tricky

if you installed Proxmox with vmxnet3 in vm config and if you pass the same network controller you will have proble or to navigate or to access to webinterface because passed ethernet will take your ip (ip you used in installation step) and conseguentially it assign a different ip to your vmxnet3 device (assuming you are in dchp for both)

 

@iGPU this should be last question about audio😂

do you know which driver is using your working audio device?

 

Thanks @fabiosun. Yeah I understand that, but for some reason Proxmox will not allow me to use the Intel nic, there's two nics plus wifi (which proxmox doesn't support) on this ASRock Creator trx40 Mobo. The Aquantia works in Mac out of the box so I wanted to use that for the VM. But the host has it, and bridged it, checking hosts the Aquantia is UP has been taken and no sign of Intel. Indeed Intel is DOWN.

 

What's the ideal scenario here? Get Intel nic UP for host and leave Aquantia nic for Mac vm? How have any other trx40 users  setup proxmox in the first instance?

 

[fabiosun]

Have you put kext for your intel?

otherwise OS X do not see it

or maybe I miss your problem completely😂

[iGPU]

3 hours ago, Driftwood said:

Thanks @fabiosun. Yeah I understand that, but for some reason Proxmox will not allow me to use the Intel nic, there's two nics plus wifi (which proxmox doesn't support) on this ASRock Creator trx40 Mobo. The Aquantia works in Mac out of the box so I wanted to use that for the VM. But the host has it, and bridged it, checking hosts the Aquantia is UP has been taken and no sign of Intel. Indeed Intel is DOWN.

 

What's the ideal scenario here? Get Intel nic UP for host and leave Aquantia nic for Mac vm? How have any other trx40 users  setup proxmox in the first instance?

 

 

I don't know your exact mobo. Intel is supported on macOS side for 1GB Ethernet, but not 2.5GB (1GB = I210. I211 and I219, but these need kexts to work). So if you have 2.5G ports, I don't think any kexts allow them to work. My mobo has two I210 and needs SmallTreeIntel82576.kext to work. I pass one of the two (the 2nd one) for macOS and leave the first one for host.

 

If you look at PVE/Network via the GUI, you'll probably see that the first one in list is active at your IP (CIDR section) and gateway. The first will be named something like enp68s0, and the second, enp69s0. (if you  have a 3rd controller, it will be named in sequence, enp70s0). Since the the first is active, you'll want to pass the 2nd one in list. This port will also be the 2nd controller listed when running "lspci -nnk". If you pass the active port, then you'll lose the VM and host connection. (BTW, I run without any "net0: vmxnet3" bridge port; hardware only.)

 

Meanwhile, if you have Aquantia AQC107, it is natively supported on macOS side: no extra kexts needed. I'd pass the Aquantia controller and leave the others for host.

Edited May 19, 2020 by iGPU

[fabiosun]

searching on the net also real Mac have some problems (not solved for all) USB audio problem this is a partial log when audio starts to fail

 

[Driftwood]

@fabiosun or anyone: I'm reinstall ing proxmox, it's hanging on nics again. Lights are flashing on off on both nics but they've gone awol and the install is stuck? This is true of proxmox 6.2 and 6.1

Edited May 19, 2020 by Driftwood

[iGPU]

On 5/9/2020 at 12:19 AM, fabiosun said:

again....benchmarking:

 

Final Cut X 10.4.6 (works in High Sierra)

Bruce X benchmark test

about 18s

 

I have followed method published here:

https://blog.alex4d.com/2013/10/30/brucex-a-new-fcpx-benchmark/

 

I think Nvidia GFX is not appropriate in FCPX..but results are not so bad

Maybe it improves in a newer Metal OSX version (No web drivers so no test for me)

 

 

I just ran 5K Bruce X test. First run was just over 9 sec, next 2 runs were each under 2 sec; maybe 1.8 sec (my reaction time  is too slow to stop timer; I tried ChronoX was didn't seem to work correctly). This is with one Radeon VII, no WEG, no SoftPowerTable injection, but with Radeon kext all under Mojave.

Edited May 19, 2020 by iGPU

[Driftwood]

8 hours ago, iGPU said:

Meanwhile, if you have Aquantia AQC107, it is natively supported on macOS side: no extra kexts needed. I'd pass the Aquantia controller and leave the others for host.

IGPU: Thanks for your points. Whats weird is (I think) this got screwed up when I flashed the bios to latest version for this mobo. v 1.06 the latest. Though I cant prove it now... so Im in a bit of a dilemma here. I cant remember if both nics originally worked on first install of Proxmox. I need the Aquantia nic in the Mac OS VM. Judging by other posts on Proxmox  I need to get a 2.5G nic driver perhaps from linux.git and find the correct one to support it for host networking. There's also the wifi /BT one (Intel Wi-Fi 6 802.11ax (2.4Gbps) + BT 5.) has anyone got that working?

For now, on Proxmox USB install, the only one that seems to work is the one which recommends 192.168.1.134 address (Im guessing its Aquantia or how can I find out the one which Proxmox 'likes'?), the other ethernet nic places a Mac address as its IP! (IPv6?) which I tried manually editing in Managenment Interfaces on the last setup page where you put in your network info.

 

Just read manual and the other nic (the 2.5g one) is Dragon RTL8125AG

Does Proxmox/Linux support this?

 

Its confusing not knowing enough about Linux!

Edited May 19, 2020 by Driftwood

[Rox67er]

On 5/18/2020 at 5:32 PM, Driftwood said:

I can only 'restart' fine from macos, but 'shutdown' will cause the aforementioned bug. Where do you run that from shell console

?

 

It is a so called hook script, save it in the location: /var/lib/vz/snippets

and add it to your conf file with: 

hookscript: local:snippets/reset-gpu.pl

 

[Rox67er]

17 hours ago, fabiosun said:

Gigabyte TRX40 Designare

 

ASrock TRX40 Creator

I would also use this USB port to have a proxmox disk connected and installed with all useful things (like VM and so on)
this helps to pass more easily all other controllers

Good tip! Interesting things happen when you passthrough the SATA controller which has the Proxmox disk hooked up. 🙂

 

12 hours ago, Driftwood said:

Thanks @fabiosun. Yeah I understand that, but for some reason Proxmox will not allow me to use the Intel nic, there's two nics plus wifi (which proxmox doesn't support) on this ASRock Creator trx40 Mobo. The Aquantia works in Mac out of the box so I wanted to use that for the VM. But the host has it, and bridged it, checking hosts the Aquantia is UP has been taken and no sign of Intel. Indeed Intel is DOWN.

 

What's the ideal scenario here? Get Intel nic UP for host and leave Aquantia nic for Mac vm? How have any other trx40 users  setup proxmox in the first instance?

 

He driftwood, I had the same issue, you need to update Proxmox to support the Realtek. It was added recently. Look here: https://forum.proxmox.com/threads/linux-kernel-5-4-for-proxmox-ve.66854/

 

I think you can now do a normal update - upgrade step since the kernel is released now.

Edited May 19, 2020 by Rox67er

[Rox67er]

For those on ASRock TRX40 creator running into network issues, I locked myself out at one point with an autostating VM with aquantia passthrough. I had to use a linux recovery disk (USB stick) and mount the Proxmox root partition to access the network file and change it to the other NIC to get into Proxmox again. More info here: https://pve.proxmox.com/wiki/Network_Configuration

 

[Driftwood]

1 minute ago, Rox67er said:

For those on ASRock TRX40 creator running into network issues, I locked myself out at one point with an autostating VM with aquantia passthrough. I had to use a linux recovery disk (USB stick) and mount the Proxmox root partition to access the network file and change it to the other NIC to get into Proxmox again. More info here: https://pve.proxmox.com/wiki/Network_Configuration

 

So when you install Proxmox (Ive a Proxmox 6.2 USB bootloader now) why is it one of my nics gets a IPV6 mac address in the last page of the install, I then click the alternative (Atlantic) one and get a proper home.pve network address assigned and not a mac address.... which is which?  On the install you're kindof blind as to which nic Proxmox recomends. You dont know if its chosen Aquantia or the Realktek! If someone knows please enlighten me! I want two IP4 addresses please!!!!

 

Getting a bit fed up of 'doing the reinstall' thinking its other things, wiping disks because of partition info from previous install is in same place, etc... etc... very frustrating. I was so close to getting things passthroughed and finalised, then the network lockup screwed everything up and I was locked out. I even reformatted the nvme install of catalina thinking something was amiss there.... very frustrating. :-)
Ah well, biugg sigh... here goes...

[iGPU]

2 hours ago, Driftwood said:

IGPU: Thanks for your points. Whats weird is (I think) this got screwed up when I flashed the bios to latest version for this mobo. v 1.06 the latest.

There's also the wifi /BT one (Intel Wi-Fi 6 802.11ax (2.4Gbps) + BT 5.) has anyone got that working?

 

Yes, I posted this some time ago: here. I also included the BT kexts to allow it  to work. I repeated comments about it here.

 

Again, so far, only BT works, but there are people actively working on the Wifi components, so only a matter of time before it all should work.

[fabiosun]

@Driftwood your Realtek ethernet in osx should supported from this kext by Mieze:

 

[fabiosun]

for your problem (main problem) have you checked in updated bios if some new voice is there about IOMMU I mean?

If you do not have done any error it is the only answer to explore because @99% it is an user errors (some time I miss myself things and I break my head to understand..

 

FYI I have reverted all mid Debian files to original ones, because for me passing them with FLR patch is not useful to latest my goal on this system ( internal audio 🙂) others task are reached and all controllers are passed well
I hope I can find a solution for audio, but I am convinced it is an OSX problem and if you search also many original Mac are affected from this usb audio bug
A solution seems to use an Audio with DAC capability or also @iGPUexternal audio
I have not tested personally so if people could share experiences in this subject I will more than happy 🙂

[Rox67er]

Last night first time had macOS and Windows running in parallel. 😃

 

I just completed my first test result with:

1) VM running Catalina on Radeon VII, 48 cores, 96 Gb memory  Geekbench 5  single core ~1150 / multicore ~22000

2) VM running Windows 10 on GTX980, 16 cores, 24 Gb memory  Geekbench 5 single core ~1200 / multicore ~10500

 

Both running in parallel with dedicated GPU / monitor and dedicated disk. 🙂

 

Now figuring out how to split resources (USB / SATA)

 

I’m working on a table showing which USB port goes to which physical port and is on which USB-hub / pci will share once completed. 👍🏻

[Rox67er]

8 hours ago, Driftwood said:

So when you install Proxmox (Ive a Proxmox 6.2 USB bootloader now) why is it one of my nics gets a IPV6 mac address in the last page of the install, I then click the alternative (Atlantic) one and get a proper home.pve network address assigned and not a mac address.... which is which?  On the install you're kindof blind as to which nic Proxmox recomends. You dont know if its chosen Aquantia or the Realktek! If someone knows please enlighten me! I want two IP4 addresses please!!!!

 

Getting a bit fed up of 'doing the reinstall' thinking its other things, wiping disks because of partition info from previous install is in same place, etc... etc... very frustrating. I was so close to getting things passthroughed and finalised, then the network lockup screwed everything up and I was locked out. I even reformatted the nvme install of catalina thinking something was amiss there.... very frustrating. 🙂
Ah well, biugg sigh... here goes...

I also get a ipv6 but don’t have an issue with it. Maybe disable ipv6 on your router? I have not use install 6.2 yet, just upgraded from my installed 6.1 version.

 

I did see some issues when passing through hardware to the guest which is also using resources in the host (e.g. Proxmox drive) it then messed up my pci in such way that a reboot wouldn’t help. Keep getting an “99” error on the DrBios on the motherboard. Bios defaults seemed to work but ass soon as I enabled the xmp profile for memory it froze again on boot. Had to switch of my power supply and restart to get it resolved. I also have seen that leaving the Bluetooth module on auto gives me a bios rom error for Bluetooth when booting Proxmox. If I set it to enabled it does work. 

[Driftwood]

After a long evening of trying to sort this motherboard out (ASRock Creator TRX40) and some fantastic tips from all here (plus some eye opening conversations with @iFPU who I am indebted to)  Ive finally managed to get network connections and other issues sorted out and am slowly rebuilding the VM that went pear shaped the other day!

 

On my reinstall of Proxmox (which was 6.2 for trhe first trime for me), I was getting a IPv6 Mac address offered as the IP address! on one of the nics, the secondarey nic in the dropdown correctly offered me 192.168.1.134 for my home network and then I could get a working install. Still dont know why I was being offered the mac address - maybe something to do with the realtek nic kindof being 'discovered' under the 5.4 kernel of Proxmox 6.2 - who knows?! Anyway got thru install.

 

Insiode the box and after creating a new VM, If anyone is getting no access to the vnc console in order to restart ISO installation of a OS ensure your args in your .conf are correct. Ensure IOMMU in BIOS is set to Enabled and not just Auto, obviously make sure SMT is enabled for virtualisation (which it was!) and check everything else - you dont neccesarily need to have Above 4G ENabled (you can keep csm and Above 4G Disabled - so long as you are booting from UEFI you're good to go for that matter (if anyone has any advantages let me know), etc.

 

You don't need a bridged net adapter as I learned from iGPU, so the setup vm in conf where you get net0 blah blah blah vmbro bridge can go. Youre going to passthrough along with the correct IP address anyway. 

 

The mosty important thing is to re-read items that have been mentioned before here (reminder to self) as even though your board might differ from one of the more experienced guys here, there's still many similarities you need to follow.

 

Anyways, Im back in the game and thank you once again for all support here. This is a great site with a reservoir of info and helpful guys.

Edited May 20, 2020 by Driftwood

[iGPU]

I just ran 8K Bruce X test. First run was just over ~18 sec, next 2 runs were each under 2 sec ~1.7 sec. I followed the instructions as given here.

 

Attached is the file that contains the 8K as well as two 5K versions of Bruce X. It is interesting that the times for 5K and 8K are the same. So the times are not 'limited' by the resolution but by maybe the software. Whatever, this build is pretty fast!

 

Again, this is with one Radeon VII, no WEG, no SoftPowerTable injection, but with Radeon kext all under Mojave. (I plan on adding a 2nd Radeon VII in a few days; a very fast machine even if there is no TB.)

 

***

 

I added a connector to the USB3 plug on the mobo as shown in photo. It was bought off Amazon here. This passes-thru with the back panel USBs (~ 47:00 on my mobo). This gives much better USB performance, using the same drive that had the very delayed opening when plugged into the USB3 jack on the rear panel. Both connectors are part of the same 47:00 group that is being passed-thru, so I don't understand why one is faster than the other.

 

However, at times, after copying some folders or scrolling within a window on this external USB drive, the whole computer would freeze. But after 20 to 30 seconds, it would un-freeze and behave normally----as long as I didn't interact with the external USB drive. The internal drive is just fine and everything is very fast. Not so with the external USB drive: there is still something strange about USB external drives on my build.

 

Bruce X.fcpbundle.zip

Edited May 21, 2020 by iGPU

[fabiosun]

12 hours ago, iGPU said:

***

I added a connector to the USB3 plug on the mobo as shown in photo. It was bought off Amazon here. This passes-thru with the back panel USBs (~ 47:00 on my mobo). This gives much better USB performance, using the same drive that had the very delayed opening when plugged into the USB3 jack on the rear panel. Both connectors are part of the same 47:00 group that is being passed-thru, so I don't understand why one is faster than the other.

 

 

about this

I am also using both internal USB3 connector using my 4 usb3 external Core X9 case  ports

I would suggest to check in ioreg how your system sees those ports also back plate ports..some time it does some weird things..I think it is for some VM compatibility USB check

see also this file in your proxmox:

/usr/share/qemu-server/pve-q35-4.0.cfg

here you can find many declaration about old USB ports..if you do not pass entire usb controller you should see in web interface/hardware/usb port or device disks or stuff you connect to your rig.

In my opinion those lines could create some problems in our modern (and unsupported AMD) rig