Proxmox VE OSX Guide discussion

[meina222]

12 minutes ago, tsongz said:

@iGPU thanks for sharing that! Per Fabio and meina's suggestion, I disabled all the SSDTs and was able to boot with no issues. I am curious though whether they make a difference at all when it comes to performance, and/or optimizing the GPU, since there was a big drop in my Vega FE.

 

I also share your OCD-ness for the cosmetic appearance. Does disabling MCEReporterDisabler affect the memory module warning? That's one cosmetic thing that has been bothering me, even though it doesn't affect performance.

 

 RE TB3 - I may have something for you here. I purchased my GB Alpine Ridge from a seller who flashed(?) it and provided OC instructions for it. Publicly, it's pw protected, but the pw was provided in the instructions when I received it. Attaching the main essentials in the zip here. Main files are the SSDTs (TR, TR2, AR) and their respective OC configs. I have not tested this yet on my own machine, and the note is that this was specifically compiled for a genuine Mac Pro (<2010) - but hopefully can give you some insight.

 

 

thunderbolt_files.zip 117 kB · 0 downloads

Re: GPU benchmarks. The only wait to find out is to re-enable them and try again.

Re: memory warnings. Are you referring to MacPro7,1 memory misconfiguration popup? There is a kext for that. I think it's called macpromemomrynotiticationdisabler.kext. The way I avoid this error is by giving 192GB to my VM which at 12 vritual dimms each of 16GB populates all slots of the MacPro so it stops complaining, but if you don't have this amount of memory I guess the kext woudl do. Never tried.

No expertise on TB3 - I think @iGPU would comment on, but I think you can't reuse a baremetal SSDT. There seems to be also some fundametal issues about how the PCIE bridge is passed or not. Why don't you try it? This is beyond my expertise atm, but interested in at least passing through a leaf device via TB3

[tsongz]

Thanks @meina222 - I had ran across that kext before, but never got to a point where I could actually use it. Will revisit. I don't have that much memory yet, only 64GB but plan on adding more later. 

 

RE the TB3 SSDTs - from looking at them briefly, if you remap the ports from the decompiled aml files and match them to your system PCIe slots (I think) - it should in theory work, or perhaps remapping other ports with a mask. I haven't gotten around to it yet mainly bc I found out I still need a TB3 -> TB2 wire in order to use it for my TB Display, so I could only validate whether it shows or not, not actually working.

[iGPU]

1 hour ago, fabiosun said:

 

I also updated the Proxmox system using "pve-edge-kernel-5.8" folder. When running uname -r, I got the same result as fabiosun.

 

System seems to have booted into Big Sur just fine. 😁

 

(I downgraded to ß2 several days ago and ß3 was not stable to me.)

[meina222]

@tsongz - on GPUs (which I haven't had time to benchmark and configure much) - try adding only Whatevergreen kext and see if that bumps your benchmarks. I don't fully understand the implications of these kexts and SSDT to ACPI power management and framebuffer interactions - especially in VM setting. I think the answer can only be found via testing and experiments. I am not sure if anyone can recommend a setup for that. I have a 5700XT. Last time I benchmarked it scored only 43K-45K in Geekbench 5, but also I am not sure the boosts some people report in baremetal are real - e.g. would you actually see better rendering performance in real life apps, or better power consumption. Espeically in VM that is tricky and needs measurement.

[iGPU]

41 minutes ago, tsongz said:

@iGPU thanks for sharing that! Per Fabio and meina's suggestion, I disabled all the SSDTs and was able to boot with no issues. I am curious though whether they make a difference at all when it comes to performance, and/or optimizing the GPU, since there was a big drop in my Vega FE.

 

I also share your OCD-ness for the cosmetic appearance. Does disabling MCEReporterDisabler affect the memory module warning? That's one cosmetic thing that has been bothering me, even though it doesn't affect performance.

 

 RE TB3 - I may have something for you here. I purchased my GB Alpine Ridge from a seller who flashed(?) it and provided OC instructions for it. Publicly, it's pw protected, but the pw was provided in the instructions when I received it. Attaching the main essentials in the zip here. Main files are the SSDTs (TR, TR2, AR) and their respective OC configs. I have not tested this yet on my own machine, and the note is that this was specifically compiled for a genuine Mac Pro (<2010) - but hopefully can give you some insight.

 

 

thunderbolt_files.zip 117 kB · 1 download

 

If you're adding GPU functions via the SSDT or DeviceProperties, you could adversely impact GPU function. I have some mods for Vega 56 and Vega 64 that I can share, but I'm not familiar with Vega FE.

 

MCEReporterDisabler was supposedly needed for booting Catalina, but doesn't seem to be necessary for our VM approach. The memory warning is a problem when using SMBIOS of MacPro7,1. I think safer to use iMacPro1,1 SMBIOS.

 

I'll look over TB files later tonight after work. I don't think writing custom SSDT for our build will be a problem and I'll happily share it here. The problem that I addressed in this post and others is passing through the complete device (esp the pci-bridge sections of the TB). This is also a problem with other 'bridged' devices (USB and FireWire cards for example), which I've previously posted.

 

***

 

BTW, I just did the  5.8 kernel update and the TB device still splits into 2 parts without passing through the pci-bridge sections (although I did not specifically try and pass them through; normally passing those parts gives and VM error, preventing the VM from running). I'll test more later.

[tsongz]

@meina222 - I plan on doing those benchmarks later tonight and will share the results. I just received and installed the PCIe WiFi/BT (PCIe slot 1) card you mentioned prior, and now I'm running into an issue with Proxmox where the NIC isn't being correctly passed, so I'm not able to connect to proxmox and with VFIO passthrough, I can't access it via normal KB. Any tips on fixing this? I had tried to do a suggestion on the proxmox forums adding "net.ifnames=1" to grub command line prior to boot but it's not doing anything.

[fabiosun]

@tsongzif it is a PCIE card it could be happen it changed your passed id number..overall if your mb PCIE slot are "shared"

 

[meina222]

3 minutes ago, tsongz said:

@meina222 - I plan on doing those benchmarks later tonight and will share the results. I just received and installed the PCIe WiFi/BT (PCIe slot 1) card you mentioned prior, and now I'm running into an issue with Proxmox where the NIC isn't being correctly passed, so I'm not able to connect to proxmox and with VFIO passthrough, I can't access it via normal KB. Any tips on fixing this? I had tried to do a suggestion on the proxmox forums adding "net.ifnames=1" to grub command line prior to boot but it's not doing anything.

 

My Proxmox host is connected via cat cable on both Ethernets. Do you have a video card connected to the host or are you running 'headless'? If the former, you can login in the console directly to host and check the contents of '/etc/network/interfaces' e.g. less /etc/network/interfaces

 

I have 2 interfaces - enp70s0 and enp68s0. I also have virtual bridge vmbr0 which acts as a switch connected to my router - gateway belowvia enp68s0 and this gives me access to the host via the internet.

 

# network interface settings; autogenerated
# Please do NOT modify this file directly, unless you know what
# you're doing.
#
# If you want to manage parts of the network configuration manually,
# please utilize the 'source' or 'source-directory' directives to do
# so.
# PVE will preserve these directives, but will NOT read its network
# configuration from sourced files, so do not attempt to move any of
# the PVE managed interfaces into external files!

auto lo
iface lo inet loopback

iface enp70s0 inet manual

iface enp68s0 inet manual

auto vmbr0
iface vmbr0 inet static
        address [IP redacted]
        gateway [IP redacted]
        bridge-ports enp70s0
        bridge-stp off
        bridge-fd 0

[meina222]

20 minutes ago, fabiosun said:

@tsongzif it is a PCIE card it could be happen it changed your passed id number..overall if your mb PCIE slot are "shared"

 

That too. But also you'd lose host connectivity if you pass the ethernet that happens to be associated with the gateway. In my example above that would be the card associated with enp68s0. If you have 2nd ethernet you should pass the other card or swap the one that is connected to your gateway.

 

The ASROCK has 2 ethernet cards. Aquantia and Realtek. Connect both to your router, make sure your vmbr0 (or 1 if you have 2nd bridge) has the card that you plan to keep for the host associated with the gateway IP, and identify the PCIE id of the 2nd card and pass that through. If you want to keep the Realtek to host, make sure it is configured as the bridge-ports device for the host/vmrb0. The Aquantia will be the 'free' one

 

For more details read this:

https://pve.proxmox.com/wiki/Network_Configuration

Edited August 4, 2020 by meina222

[tsongz]

Sorry - to clarify, I am not able to access my proxmox host instance via it's IP address :8006 (i've ran into this issue before, and I typically just reinstall proxmox). My guess is that with the new card, or when I unplugged it, my router might have assigned it a new IP address. I have it running headless, but also have a monitor plugged in to observe it.

 

The current problem is that even with the monitor plugged in, because I have the VGA blacklisted on proxmox boot, I am not able to access the typical commandline once it's up (only the 2 lines). So my two theories in trying to get it to work:

 

- How would I disable VGA blacklist so that I can re-access the commandline from host

- How to disable/change the static IP that Proxmox is configured to use from grub (because of #1)

[fabiosun]

4 minutes ago, tsongz said:

 

 

- How would I disable VGA blacklist so that I can re-access the commandline from host

-

remove it from vfio.conf 

also you can try to unblacklist amdgpu from blacklist.conf

in my rig I do not use any block

and it works

 

[meina222]

With blacklist or not, you should always be able to boot into the video output of the host. With blacklist you will get default resolution maybe larger text. Don't reinstall Proxmox! You just need to double check the contents of etc/network/interfaces 

 

I agree with @fabiosun - no need to blacklist. Blacklisting is only useful if you boot headless. Easiest is to have 2 video cards - one for host (I have old GT 240) and one for guest - your Vega 64. But to fix connectivity you should always have access after reboot to host via console regardless of blacklist.

[tsongz]

How would I edit a file from within grub commandline? That's the main problem right now. I can't access any of the typical files because grub's cmdline is limited.

I've tried edit files with

- vi

- nano

- edit

 

But grub's accessible commands are very different, and in several posts, I've seen it said that you can't edit files while within grub, only the grub.cfg with 'E' during boot selection

[fabiosun]

I am using only my guest to use my thread ripper machine

Sometimes during my "bleeding edge test" I have problem and I have to use a spare pc to solve

normally I turn on my thread ripper and when shell is available I execute from it qm start 100 to start my main VM

when I have used to add in vfio.conf  my GPU or to blacklist nouveau or Nvidia drivers.. my system to boot needs of a spare pc or an automatic vm 100 booting option enable

 

you can see it in macos86.it you tube channel 🙂

check framed window..there only 2 verbose initial lines and then vm starts in automatic way in big window no block are there and I see regular verbose and shell

 

[meina222]

@tsongz I am confused. Why can't you login direct to host and manipulate  cmd line? Are you running something in your root ~/.bashrc that turns off the video output as soon as you do?

Edited August 4, 2020 by meina222

[fabiosun]

I think he is blocking to much stuff

it is also in Proxmox wiki..but we can do in a different way 🙂

 

[meina222]

6 minutes ago, fabiosun said:

I think he is blocking to much stuff

it is also in Proxmox wiki..but we can do in a different way 🙂

 

 

What I think happened here is as follows:

 

By adding the broadcom card, Promox re-assigned the interface ID's othe the ethernets. Say enp68s0 became enp69s0.

Now his /etc/network/interfaces is invalid. And he runs a script at logon to prevent the host to use the single video card.

 

@tsongz - unplug the broadcom, remove any logon scripts if you manage to regain access and then plug the broadcom again - this time around access the host direct and fix your network bridge.

[tsongz]

Was able to fix it without removing the broadcom card.

 

EDIT: what @meina222 mentioned is really a key observation, which you should be able to see with "dmesg | grep eth" - in the logs it showed that the new broadcom card was added as enp68s0 whereas my Realtek NIC was added as enp70s0. But my /etc/network/interfaces was correctly configured.

 

- Added second GPU / removed power from primary GPU to access host with keyboard.

- Ran the following commands in normal /bin/bash (not grub... ugh):

# will display your network nics

ip link



# will display what (if any) NICs have been renamed. 

# Make note of what the names are to ensure that they match with your /etc/network/interfaces in the next step

dmesg | grep eth



# will disable all NICs / so be careful of this

ifdown -a



# modify your network names if it changed. In my case, the NIC names were correct but internet access.

# So what I did was a bit weird.

# I changed the IP address of my vmbr0 192.168.5.157 -> 192.168.5.158, ran ifup -a

# led to no changes, then ifdown -a, changed it back to 192.168.5.157, did another ifup -a

# and pinged google.com and it reconnected. 



nano /etc/network/interfaces



# after changes are made

ifup -a



# ping a website to see if it's connected. If it timed out, you don't have a connection yet, so try again

ping google.com

 

Edited August 4, 2020 by tsongz

[fabiosun]

Post your vfio.conf

also lspci -nnk

but i think if you followed pavos github you block things he is blocking because he starts or started in the past his guest from web interface on another pc

i can be wrong...

[tsongz]

Switched back to primary GPU, all is good with networking.

Also added a new Sabrent Rocket Nvme 1 Tb (based on several member's sigs that indicate this works well, as the intel 660p does not)

 

EDIT: Sabrent NVME passed through with zero problems with only adding it to vfio.conf, and now reinstalling MacOS on the same VM by reattaching the BigSurBeta.iso from Proxhox Host in 101.conf, which will now show "Install MacOS" in the selection screen. Did not make any modifications to EFI nor remove any existing disks that are attached.

 

vfio.conf

# This file contains all the device-ids and vendor-ids of components we want to passthrough to the macOS VM using vfio.
# reload with update-initramfs -u -k all && sudo reboot
options vfio-pci ids=1002:6863,1002:aaf8 disable_vga=1
# 47:00.1, 47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
options vfio-pci ids=1022:149c
# 22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
options vfio-pci ids=1022:148c
# 43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
options vfio-pci ids=1b21:3242
# 42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
options vfio-pci ids=8086:f1a8
### Bluetooth Configuration ###
# 46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
# options vfio-pci ids=10ec:8125
# 45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
#                         Subsystem: Intel Corporation Device [8086:0084]
options vfio-pci ids=8086:2723,8086:0084
# Bus 007 Device 006: ID 8087:0029 Intel Corp.
options vfio-pci ids=8087:0029

# 47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
# options vfio-pci ids=1022:1485

# 44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
options vfio-pci ids=1d6a:07b1
# 48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
options vfio-pci ids=1022:7901

#options vfio-pci ids=1002:66af,1002:ab20 disable_vga=1
#options vfio-pci ids=1987:5016
#options vfio-pci ids=144d:a804
#options vfio-pci ids=144d:a802
#options vfio-pci ids=14e4:43b1
#options vfio-pci ids=10ec:8168
#options vfio-pci ids=1022:1487

 

lspci -nnk

00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: ASRock Incorporation Starship/Matisse Root Complex [1849:1480]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        Subsystem: ASRock Incorporation FCH SMBus Controller [1849:ffff]
        Kernel driver in use: piix4_smbus
        Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
        Subsystem: ASRock Incorporation FCH LPC Bridge [1849:ffff]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 0 [1022:1490]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 1 [1022:1491]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 2 [1022:1492]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 3 [1022:1493]
        Kernel driver in use: k10temp
        Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 4 [1022:1494]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 5 [1022:1495]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 6 [1022:1496]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 7 [1022:1497]
01:00.0 Network controller [0280]: Broadcom Limited BCM4360 802.11ac Wireless Network Adapter [14e4:43a0] (rev 03)
        Subsystem: Apple Inc. BCM4360 802.11ac Wireless Network Adapter [106b:0111]
        Kernel driver in use: bcma-pci-bridge
        Kernel modules: bcma
02:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
03:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
03:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
20:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
20:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
20:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
20:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
21:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
22:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
22:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Kernel driver in use: ccp
        Kernel modules: ccp
22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
22:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:d102]
        Kernel modules: snd_hda_intel
40:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
40:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
40:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
40:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
41:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57ad]
        Kernel driver in use: pcieport
42:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
        Subsystem: ASMedia Technology Inc. Device [1b21:3242]
        Kernel driver in use: xhci_hcd
44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
        Subsystem: ASRock Incorporation AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1849:d107]
        Kernel driver in use: atlantic
        Kernel modules: atlantic
45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
        Subsystem: Intel Corporation Device [8086:0084]
        Kernel driver in use: iwlwifi
        Kernel modules: iwlwifi
46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
        Subsystem: ASRock Incorporation Device [1849:8125]
        Kernel driver in use: r8169
        Kernel modules: r8169
47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
47:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486]
        Kernel driver in use: xhci_hcd
47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
        Kernel driver in use: xhci_hcd
48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
49:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
        Subsystem: Intel Corporation Device [8086:390d]
        Kernel driver in use: nvme
4b:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E12 NVMe Controller [1987:5012] (rev 01)
        Subsystem: Phison Electronics Corporation E12 NVMe Controller [1987:5012]
        Kernel driver in use: nvme
4c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1470]
        Kernel driver in use: pcieport
4d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1471]
        Kernel driver in use: pcieport
4e:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6863]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6b76]
        Kernel modules: amdgpu
4e:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Kernel modules: snd_hda_intel
4f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
50:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
60:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
60:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
60:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
60:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
61:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
62:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]

 

Edited August 4, 2020 by tsongz
adding clean nvme details

[tsongz]

@meina222 - the new broadcom card gets picked up at PCIe 01:00.0 with IDS [14e4:43a0] and [106b:0111]. On the topic of vfio and blacklisting, do you add these as well or leave them out? as well as the kernel module 'bcma'

 

In your last post, you mentioned adding it as a 'built-in', which I am understanding as adding In config.plist

 

As the device on host is on Pci slot 1:

DeviceProperties -> add -> PciRoot(0x1)/Pci(0x10,0x1)/Pci(0x0,0x0) -> built-in: 1

 

I plan on doing a fresh install on the nvme card with the Sabrent drive

[meina222]

20 minutes ago, tsongz said:

@meina222 - the new broadcom card gets picked up at PCIe 01:00.0 with IDS [14e4:43a0] and [106b:0111]. On the topic of vfio and blacklisting, do you add these as well or leave them out? as well as the kernel module 'bcma'

 

In your last post, you mentioned adding it as a 'built-in', which I am understanding as adding In config.plist

 

As the device on host is on Pci slot 1:

DeviceProperties -> add -> PciRoot(0x1)/Pci(0x10,0x1)/Pci(0x0,0x0) -> built-in: 1

 

I plan on doing a fresh install on the nvme card with the Sabrent drive

You shouldn't need to black list it.

"built-in" should be your ethernet en0, not the broadcom.

Your devices should look like this en0 (built-in ethernet), en1 (wireless) en2 (bluetooth) (you can swap bluetooth and wireless but not ethernet - must be en0)

Get Hackintool and verify on the "Peripherals" tab under your System selection

 

Should show something like this for ethernet (this shot misses bluetooth and wireless) - this older gui, it's called Peripherals now, not Misc

 

 

 Depending on what order your add the ethernet and bluetooth and what PCI they get assigned, the ethernet may jump to en1 or end2 - you need to find an order where it's en0. Then add built-in.

I followed the OC guide - https://dortania.github.io/OpenCore-Post-Install/universal/iservices.html#generate-a-new-serial

 

Edited August 4, 2020 by meina222

[meina222]

3 hours ago, iGPU said:

 

I also updated the Proxmox system using "pve-edge-kernel-5.8" folder. When running uname -r, I got the same result as fabiosun.

 

System seems to have booted into Big Sur just fine. 😁

 

(I downgraded to ß2 several days ago and ß3 was not stable to me.)

If things look stable I wil create a githubclone and update the 5.8.xx kernels and ZFS releases as they come online. I need some time to structure the whole thing as modules so that we can automatically pick up any changes from pve or linux or zfs. I'll try to put a README that will allow for anyone to build the packages.

 

This is going to be until Proxmox migrates their edge github to 5.8 themselves which may take a while.

[iGPU]

2 hours ago, tsongz said:

How would I edit a file from within grub commandline? That's the main problem right now. I can't access any of the typical files because grub's cmdline is limited.

I've tried edit files with

- vi

- nano

- edit

 

But grub's accessible commands are very different, and in several posts, I've seen it said that you can't edit files while within grub, only the grub.cfg with 'E' during boot selection

 

I've had same issue many times with Proxmox and even on their forum, they could not help.

 

Solution is never blacklist GPU, otherwise command line disappears from host monitor and if any thing goes wrong with GUI, the only solution is to re-install Proxmox (which I've had to do so many times, I've lost count...). I've had GUI disappear after simply changing the Proxmox NVMe drive to a different slot.

 

For ethernet, I leave one port (Aquantia) for host and the other I transfer to macOS. I don't VM any other ethernet ports. To manually adjust the ethernet, I ran "nano /etc/network/interfaces" and entered the following code (and after changing,  ran the command:  /etc/init.d/networking restart):

 

Spoiler

auto lo
iface lo inet loopback

allow-hotplug enp69s0

auto enp69s0
iface enp69s0 inet manual

allow-hotplug enp71s0

auto enp71s0
iface enp71s0 inet static
        address 192.168.1.155/24
        netmask 255.255.255.0
        gateway 192.168.1.1

auto vmbr0
iface vmbr0 inet static
     address  192.168.1.153
     netmask  255.255.255.0
     bridge_ports enp69s0
     bridge_stp off
     bridge_fd 0
     mtu 1500

 

 

As for BT/Wifi, I never got a PCIe card to work. I eventually swapped out the built-in card for macOS compatible card.

Edited August 4, 2020 by iGPU

[tsongz]

Thanks @iGPU - that's extremely helpful to know. So you remove amdgpu/radeon from blacklist.conf and keep it in vfio.conf right? Do you also adjust grub? 

 

Based on your config, this is what I have - does this look right?

enp70s0: Realtek (host Proxmox)

enp68s0: Aquantic (passthrough)

 

auto lo
iface lo inet loopback

allow-hotplug enp70s0
auto enp70s0
iface enp70s0 inet manual

allow-hotplug enp68s0
auto enp68s0
iface enp68s0 inet static
    address 192.168.5.115/24
    netmask 255.255.255.0
    gateway 192.168.5.1

auto vmbr0
iface vmbr0 inet static
    address 192.168.5.157
    netmask 255.255.255.0
    gateway 192.168.5.1
    bridge_ports enp70s0
    bridge_stp off
    bridge_fd 0