tsongz

On 8/11/2020 at 1:10 PM, meina222 said:

Do you use patched ROM as described here:

 

https://github.com/joeknock90/Single-GPU-Passthrough

No, I only use the rom from the download as you mentioned. I tested a few different PPM configurations in Device Properties and although I did notice a bit of a performance jump, (+5-7%), it made it really unstable, but at least was able to validate that modifying Power Tables does affect GPU performance without kexts/ACPIs. 

I've been following that thread - I'm a bit torn about going bare metal, mainly because of the docker compatibility with AMD CPUs, specifically on MacOS, and VirtualBox isn't yet supported on Big Sur.

Finally got around to running benchmarks after working through several Vega specific threads based on @meina222's device property patch, and combing through IORE

<key>DeviceProperties</key>
    <dict>
        <key>Add</key>
        <dict>
            <key>PciRoot(0x1)/Pci(0x1C,0x0)/Pci(0x0,0x0)</key>
            <dict>
                <key>AAPL,slot-name</key>
                <string>Slot-1</string>
                <key>ATY,Copyright</key>
                <string>Copyright AMD Inc. All Rights Reserved. 2005-2020</string>
                <key>ATY,DeviceName</key>
                <string>Vega Frontier Edition</string>
                <key>ATY,EFIVersion</key>
                <string>017.001.000.042.012566</string>
                <key>ATY,FamilyName</key>
                <string>Radeon</string>
                <key>ATY,Rom#</key>
                <string>113-D1820501-102</string>
                <key>CAIL_EnableLBPWSupport</key>
                <integer>0</integer>
                <key>CAIL_EnableMaxPlayloadSizeSync</key>
                <integer>1</integer>
                <key>CFG_APER_MODE</key>
                <integer>1</integer>
                <key>CFG_CAA</key>
                <integer>0</integer>
                <key>CFG_FB_LIMIT</key>
                <integer>0</integer>
                <key>CFG_FORCEMAXDPM</key>
                <false/>
                <key>CFG_FORCE_MAX_DPS</key>
                <false/>
                <key>CFG_GEN_FLAGS</key>
                <integer>0</integer>
                <key>CFG_INT_SSPC</key>
                <integer>25</integer>
                <key>CFG_NODM</key>
                <true/>
                <key>CFG_NON_ZERO_MC_LOC</key>
                <true/>
                <key>CFG_NO_HDCP</key>
                <false/>
                <key>CFG_NO_MSI</key>
                <false/>
                <key>CFG_NO_MST</key>
                <false/>
                <key>CFG_NO_PP</key>
                <false/>
                <key>CFG_NO_SLS</key>
                <false/>
                <key>CFG_PAA</key>
                <integer>0</integer>
                <key>CFG_PULSE_INT</key>
                <true/>
                <key>CFG_TRANS_WSRV</key>
                <true/>
                <key>CFG_USE_AGDC</key>
                <true/>
                <key>CFG_USE_CP2</key>
                <true/>
                <key>CFG_USE_DPT</key>
                <true/>
                <key>CFG_USE_FBC</key>
                <false/>
                <key>CFG_USE_FBWRKLP</key>
                <true/>
                <key>CFG_USE_FEDS</key>
                <true/>
                <key>CFG_USE_HDMI20</key>
                <true/>
                <key>CFG_USE_LPT</key>
                <false/>
                <key>CFG_USE_SRRB</key>
                <false/>
                <key>CFG_USE_STUTTER</key>
                <true/>
                <key>CFG_USE_SWIP</key>
                <true/>
                <key>DALReadDelayStutterOff</key>
                <integer>4</integer>
                <key>DALUseUrgencyWaterMarkOffset</key>
                <integer>0</integer>
                <key>Force_Load_FalconSMUFW</key>
                <true/>
                <key>PP_DisableClockStretcher</key>
                <integer>1</integer>
                <key>PP_DisableDIDT</key>
                <integer>1</integer>
                <key>PP_DisablePowerContainment</key>
                <integer>1</integer>
                <key>PP_DisableULPS</key>
                <integer>1</integer>
                <key>PP_DisableULV</key>
                <integer>1</integer>
                <key>PP_DisableVoltageIsland</key>
                <integer>0</integer>
                <key>PP_EnableUploadFirmware</key>
                <integer>0</integer>
                <key>PP_Falcon_QuickTransition_Enable</key>
                <integer>1</integer>
                <key>PP_FuzzyFanControl</key>
                <integer>0</integer>
                <key>PP_GfxOffControl</key>
                <integer>0</integer>
                <key>PP_PhmSoftPowerPlayTable</key>
                <data>
                tgIIAQBcAOEGAAAILAAAGwBIAAAAgKkDAPBJAgAyAAgA
                AAAAAAAAAAAAAAAAAAIBXABPAkYClACeAb4AKAF6AIwA
                vAEAAAAAcgIAAJAAqAJtAUMBlwHwSQIAcQICAgAAAAAA
                AAgAAAAAAAAABQAHAAMABQAAAAAAAAABCCADhAO2A+gD
                GgRMBGUEfgQBAUYFAQGEAwAIYOoAAABAGQEAAYA4AQAC
                3EoBAAOQXwEABAB3AQAFkJEBAAbA1AEABwEI0EwBAAAA
                gAAAAAAAAByDAQABAAAAAAAAAABwpwEAAgAAAAAAAAAA
                iLwBAAMAAAAAAAAAAMDUAQAEAAAAAAAAAABEIwIABQAA
                AAABAAAA8EkCAAYAAAAAAQAAAKhVAgAHAAAAAAEAAAAA
                BWDqAAAAQBkBAACAOAEAANxKAQAAkF8BAAAACChuAAAA
                LMkAAAH4CwEAAoA4AQADkF8BAAT0kQEABdCwAQAGwNQB
                AAcACGw5AAAAJF4AAAH8hQAAAqy8AAADNNAAAARobgEA
                BQiXAQAG7KMBAAcAAWg8AQAAAQQ8QQAAAAAAUMMAAAAA
                AIA4AQACAADEgAEABQAAAQgAmIUAAEC1AABg6gAAUMMA
                AAGAuwAAYOoAAJQLAQBQwwAAAgDhAACUCwEAQBkBAFDD
                AAADeP8AAEAZAQCIJgEAUMMAAARAGQEAgDgBAIA4AQBQ
                wwAABYA4AQDcSgEA3EoBAFDDAAAGAHcBAAB3AQCQXwEA
                UMMAAAeQkQEAkJEBAAB3AQBQwwAAARgAAAAAAAAAC+QS
                vAIkEy0ACgBUA5ABkAGQAZABkAGQAZABAAAAAAACCDEH
                9wD3APcAUQEAAFkAaQBKAEoAXwBzAHMAZABAAAAAl2CW
                AJBVAAAAAAAAAAAAAAAAAAAAAAACAtQwAAACEGDqAAAC
                EA==
                </data>
                <key>PP_PhmUseDummyBackEnd</key>
                <integer>0</integer>
                <key>PP_ThermalAutoThrottlingEnable</key>
                <integer>1</integer>
                <key>PP_ToolsLogSpaceSize</key>
                <integer>524288</integer>
                <key>PP_WorkLoadPolicyMask</key>
                <integer>32</integer>
                <key>built-in</key>
                <data>
                AA==
                </data>
                <key>device_type</key>
                <string>gpu-controller</string>
                <key>hda-gfx</key>
                <string>onboard-1</string>
                <key>model</key>
                <string>Radeon Vega Frontier Edition</string>
                <key>name</key>
                <string>ATY_GPU</string>
            </dict>
        </dict>
        <key>Delete</key>
        <dict/>
    </dict>

11 hours ago, fabiosun said:

(aquantia is seen by default but no cable connected message)

@fabiosun Is the message you're seeing saying "Self Assigned IP" and gives you a bad IP, or does it not show as built in? I've ran into this problems several times, especially if I shift pci devices around, but there's generally a method I use that works to clear it. 

In Proxmox MacOS, my work around is through:

a) making sure the ethernet device is added to device properties with "built-in": 01

b) clearing MacOS's network interfaces cache with script [1] below

c) shutdown of MacOS, on boot, run cleanNVRAM, shutdown

d) full shutdown of Host

clear_net.command - I have this as a bash script with chmod +x and an alias set up in MacOS to clear it with "clearnet", but you can use it in terminal with those two commands.

#!/bin/bash

sudo rm /Library/Preferences/SystemConfiguration/NetworkInterfaces.plist 

sudo rm /Library/Preferences/SystemConfiguration/preferences.plist

I know this thread is Vanilla, but just in case. 

Wanted to give back, so I worked on a python script to simplify how we can get all the information on our Proxmox configs.

https://gist.github.com/trisongz/05c0681583a97d442538c8737c74b8b5

It's not the cleanest, but it works for me, and hopefully makes our lives a bit easier with debugging.

You can copy that file or create a new file in your Proxmox and call it with

python3 vmcheck.py --vmid [your VM ID]

It should produce results like the following:

@iGPU - I don't have a second hackintosh yet, though I will likely rebuild my Xeon e5-2680v2 here soon, so I can test it on that. I was planning on plugging it into my real 2010 Mac Pro (mojave) to see if it's recognized. It currently uses another Broadcom chipset, so I think that should validate it. 

Have a question on CPU Flags running cpu as host for nested virtualization. Currently I'm trying to get Docker up and running on Big Sur b4. I've tested every method, including Docker for Desktop (Mac), docker-machine (cli + virtualbox), and all have crashed. I think it's likely due to Docker not seeing some intel specific flags on macos, as AMD uses different virtualization names. (smx, ept). 

One thing I haven't tried yet is unloading the VBoxHfs.efi from my EFI Driver, as I only use purely an NVME passthrough now, which could be the reason it's causing VirtualBox to crash, but I was wondering if anyone's played around with the flags yet to get nested virtualization working on host cpu

sysctl -a | grep machdep.cpu on Hackintosh

Proxmox CPU Flags

sysctl -a | grep machdep.cpu on i5 iMac, which can run Docker
 

@iGPU - I did exactly as you mentioned - and it's still not showing currently.

Hackintool

Kexts

Device Properties

Results

@meina222 - That's awesome - going to try it out! I have VegaFE kexts and ACPI info that I've pulled and collected since they're pretty rare, and there's a specific RadeonBoost.kext that still has the Vegas supported (I believe newer versions aren't supported). Do you find that the kext itself makes a significant difference?

Oh - so is it that we're masking BCM4360 -> BCM4355 for compatibility using DeviceProperties so that it loads the BCM kexts that is supported?

Actually - @iGPU, I went to grab the latest AirportBrcmFixup kext and saw this note

I don't quite follow 100% - I opened the kext and did find it in Plugins, but would you set the kext path to those two and then add a block to the kernel option?

7 minutes ago, meina222 said:

Nice you just build the kernel :).

 

You can actually build it now every time Linus Torvalds updates it. I will post instructions on that too  - it's a simple git command. Let me fix the missing dpkg-dev add-in in step 1. Thanks for the help.

This might be easier if done through a docker container, I'll write a quick Dockerfile to compile it so that it doesn't interfere with the host dependencies.

3 hours ago, iGPU said:

 

 

 

You might try adding this for DeviceProperties (changing the address based on your Hackintool results for the Broadcom card in your mobo:

  Reveal hidden contents

            <key>PciRoot(0x1)/Pci(0x10,0x1)/Pci(0x0,0x0)</key>
			<dict>
				<key>AAPL,slot-name</key>
				<string>Built in</string>
				<key>built-in</key>
				<data>
				AA==
				</data>
				<key>compatible</key>
				<string>pci14e4,43ba</string>
				<key>device-id</key>
				<data>
				ukMAAA==
				</data>
				<key>device_type</key>
				<string>Network Controller</string>
				<key>model</key>
				<string>BCM4352 802.11ac Network Adapter</string>
				<key>name</key>
				<string>Broadcom BCM4352 BT / Wifi Adapter</string>
			</dict>

 

 

 

 

I'm trying to match device-id (ukMAAA== > BA430000 )  from your config, where in Hackintool would you find that?

For some reason my CPU is displayed as an i5 3.7GHz - does this affect how programs read from the host? (which may be part of the problem below)

I currently don't have MacProMemoryReporterDisabler enabled yet, which I will do with the same fix as the BT/WiFi, but wanted to share that a program that polled sysinfo prior to launch crashed due to RAM configuration (48/64 GB Passthrough). I think this is a really important note. 

3 hours ago, iGPU said:

And set the memory to "memory: 32768" (that will show up as 32GB on the Mac side; the memory intervals I use are 32768 / 65536 / 131072).

Here was the System Report for it

So after a full 1 1/2 days of basically redoing everything from scratch, using VM Assistant (thanks fabiosun and genki84) over the Proxmox installer (latest BS), and running into almost every problem I had originally thought I solved, I now have BS4(?) running and booting purely on the NVME drive with no virtual disk, or EFI from proxmox.

Everything is running stable, except the WiFi from the Broadcom card, which I realized that it's actually not the card itself, but Big Sur. Apparently Big Sur bricked WiFi for a ton of BC cards.

Post Discussing Fixing the Kext Fix.

It also involves accessing the root volume, getting CSR disabled, and a lot more fun. I did all the steps for the fix, but it's still not working currently for me, but I'm using ethernet from the ethernet port, so I'm not stressed. All iServices work as well.

I personally would not recommend BS currently - it seems like with the new method of how they are handling root disk access, there's likely going to be a lot of future changes that may or may not break things. 

My ethernet also kept reverting to a self-assigned nic, so it set things back a bit.

One of the early major roadblocks was actually taking a working VM that I had thought I had everything working through console, and then when passing back the GPU, it would not boot and was constantly stuck on [PCIE Passthrough] on the bootloader, and ultimately (half a day later), fixed it by having to blacklist amdgpu again - it would not boot past that no matter what I tried, and blacklisting was the last resort. 

Currently not passing through the full USB port, but just USB devices (keyboard/mouse). Will likely pass it through later.

03:00 is GPU, 4a:00 is NVME, 4b:00 is Broadcom PCIe, 44:00 is Aq Ethernet

vm.conf

args: -device isa-applesmc,osk="**" -smbios type=2 -device usb-kbd,bus=ehci.0,port=2 -cpu host,+invtsc,vendor=GenuineIntel

balloon: 0

bios: ovmf

boot: c

bootdisk: virtio0

cores: 64

cpu: host

hookscript: local:snippets/reset-gpu.pl

hostpci0: 03:00,pcie=1,x-vga=1,romfile=vegafrontier.rom

hostpci1: 4a:00.0

hostpci2: 4b:00.0,pcie=1

hostpci3: 44:00.0,pcie=1

localtime: 1

machine: q35

memory: 49152

name: mac-x

numa: 1

ostype: other

scsihw: virtio-scsi-pci

smbios1:

sockets: 1

usb0: host=0461:4dd7

usb1: host=1d57:5280

usb2: host=05ac:828d

usb3: host=0a5c:4500

vga: none

vmgenid

blacklist.conf

blacklist snd_hda_codec_hdmi 

blacklist snd_hda_intel

blacklist snd_hda_codec 

blacklist snd_hda_core 

# blacklist radeon 

blacklist amdgpu 

blacklist btusb 

blacklist snd-usb-audio 

blacklist bcma 

blacklist xhci_hcd

You have a very valid point, and I think that discussion is more meta and observing how most of us (at least in my own experiences) got here, and what we were told to do when first starting out, before we got here. When it came to PCIe passthrough with VFIO, it was very much - "make sure you blacklist things" as common knowledge (esp with GPUs), and more often than not, it worked. It was the most heavily emphasized 'fact' in regards to this. Most topics won't ever suggest "not unblacklisting" as a solution, and if so it's very rare.

So since it's the first things we learned, it's most likely an 'intuitive' thing to do. For example in earlier discussions around getting this specific BT/WiFi card to work, it was mentioned to blacklist the drivers. Then as it works, it tends to accumulate more than it gets removed. Currently, I don't have anything blacklisted. 

After removing commenting out blacklist items, disabling kexts, full system reboot

Also another note that may shine some insight. On the host, whenever the VM is started, it logs this error:

pcieport 0000:00:01.1: AER PCIe Bus Error: severity=Uncorrected (Non-Fatal), type=Transaction Layer, (Requester ID)

pcieport 0000:00:01.1: AER device [1022:1483] error status/mask=001000000/04400000

pcieport 0000:00:01.1: AER [02] UnSupReq     (First)

pcieport 0000:00:01.1: AER TLP Header: 34000000 010000010 000000000 883c883c

*not exactly confident on the 0's - typing this from an image

@iGPU 

Currently:

Adding my lspci while I make the changes you suggested.

00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: ASRock Incorporation Starship/Matisse Root Complex [1849:1480]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        Subsystem: ASRock Incorporation FCH SMBus Controller [1849:ffff]
        Kernel driver in use: piix4_smbus
        Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
        Subsystem: ASRock Incorporation FCH LPC Bridge [1849:ffff]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 0 [1022:1490]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 1 [1022:1491]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 2 [1022:1492]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 3 [1022:1493]
        Kernel driver in use: k10temp
        Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 4 [1022:1494]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 5 [1022:1495]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 6 [1022:1496]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 7 [1022:1497]
01:00.0 Network controller [0280]: Broadcom Limited BCM4360 802.11ac Wireless Network Adapter [14e4:43a0] (rev 03)
        Subsystem: Apple Inc. BCM4360 802.11ac Wireless Network Adapter [106b:0111]
        Kernel driver in use: vfio-pci
        Kernel modules: bcma
02:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
03:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Kernel driver in use: vfio-pci
03:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
20:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
20:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
20:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
20:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
21:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
22:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Kernel driver in use: vfio-pci
22:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Kernel driver in use: ccp
        Kernel modules: ccp
22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
22:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:d102]
        Kernel modules: snd_hda_intel
40:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
40:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
40:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
40:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
41:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57ad]
        Kernel driver in use: pcieport
42:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
        Subsystem: ASMedia Technology Inc. Device [1b21:3242]
        Kernel driver in use: vfio-pci
44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
        Subsystem: ASRock Incorporation AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1849:d107]
        Kernel driver in use: vfio-pci
        Kernel modules: atlantic
45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
        Subsystem: Intel Corporation Device [8086:0084]
        Kernel driver in use: iwlwifi
        Kernel modules: iwlwifi
46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
        Subsystem: ASRock Incorporation Device [1849:8125]
        Kernel driver in use: r8169
        Kernel modules: r8169
47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Kernel driver in use: vfio-pci
47:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486]
        Kernel driver in use: vfio-pci
47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
        Kernel driver in use: vfio-pci
48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
49:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
        Subsystem: Intel Corporation Device [8086:390d]
        Kernel driver in use: nvme
4b:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E12 NVMe Controller [1987:5012] (rev 01)
        Subsystem: Phison Electronics Corporation E12 NVMe Controller [1987:5012]
        Kernel driver in use: vfio-pci
4c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1470]
        Kernel driver in use: pcieport
4d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1471]
        Kernel driver in use: pcieport
4e:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6863]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6b76]
        Kernel driver in use: amdgpu
        Kernel modules: amdgpu
4e:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Kernel modules: snd_hda_intel
4f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
50:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Kernel driver in use: vfio-pci
60:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
60:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
60:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
60:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
61:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
62:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Kernel driver in use: vfio-pci

So I'm currently a little stuck. I can't seem to get WiFi working on the card, however, Bluetooth works, and I have my ethernet at en0, with internet connection.

I've tested several arrangements of the hostpci slot settings, having 'built-in' enabled in config.plist - tested it with all the different combinations of USB slots (which I believe only affects Bluetooth, not WiFi), have tested with kexts and without kexts and nothing's getting it to show up. 

Currently using the BCM4360 PCIe card in 01:00.0

Shows the Bluetooth being picked up from the card

Has WiFi information?

Confirms Broadcom

Confirming Device Path in Hackintool

Have tested with and without Kexts

101.conf

hostpci0: 4b:00.0
hostpci1: 44:00.0,pcie=1
hostpci2: 43:00.0,pcie=1
hostpci3: 47:00.0,pcie=1
hostpci4: 47:00.3,pcie=1
hostpci5: 01:00.0,pcie=1

blacklist.conf

blacklist snd_hda_codec_hdmi
blacklist snd_hda_intel
blacklist snd_hda_codec
blacklist snd_hda_core
blacklist btusb
blacklist snd-usb-audio
blacklist bcma
blacklist xhci_hcd

vfio.conf

options vfio-pci ids=1002:6863,1002:aaf8 disable_vga=1
# 47:00.1, 47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
options vfio-pci ids=1022:149c
# 43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
options vfio-pci ids=1b21:3242


# 4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
options vfio-pci ids=8086:f1a8



# 4b:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E12 NVMe Controller [1987:5012] (rev 01)
#        Subsystem: Phison Electronics Corporation E12 NVMe Controller [1987:5012]
options vfio-pci ids=1987:5012



# 01:00.0 Network controller [0280]: Broadcom Limited BCM4360 802.11ac Wireless Network Adapter [14e4:43a0] (rev 03)
#        Subsystem: Apple Inc. BCM4360 802.11ac Wireless Network Adapter [106b:0111]
#        Kernel driver in use: bcma-pci-bridge
#        Kernel modules: bcma
options vfio-pci ids=14e4:43a0



# 44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
options vfio-pci ids=1d6a:07b1
# 48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
options vfio-pci ids=1022:7901

11 minutes ago, meina222 said:

Setting sudo nvram bluetoothHostControllerSwitchBehavior="never". and clearing the files containg "Bluetooth" in ~/Library/Preferences/ByHost  /Library/Preferences  to get rid of MacOS cached stuff did the trick for me.

That's what I was missing - the second part of that. I had ran the previous command along with resetting nvram, but AX200 kept showing up. Despite that though, my BT worked and in System Preferences had Broadcom showing, while Hackintool showed AX200 still present. Tested it connecting to my AirPods successfully, but WiFi wasn't working. (while still having VMXNet port)

Thanks @iGPU - that's extremely helpful to know. So you remove amdgpu/radeon from blacklist.conf and keep it in vfio.conf right? Do you also adjust grub? 

Based on your config, this is what I have - does this look right?

enp70s0: Realtek (host Proxmox)

enp68s0: Aquantic (passthrough)

auto lo
iface lo inet loopback

allow-hotplug enp70s0
auto enp70s0
iface enp70s0 inet manual

allow-hotplug enp68s0
auto enp68s0
iface enp68s0 inet static
    address 192.168.5.115/24
    netmask 255.255.255.0
    gateway 192.168.5.1

auto vmbr0
iface vmbr0 inet static
    address 192.168.5.157
    netmask 255.255.255.0
    gateway 192.168.5.1
    bridge_ports enp70s0
    bridge_stp off
    bridge_fd 0

@meina222 - the new broadcom card gets picked up at PCIe 01:00.0 with IDS [14e4:43a0] and [106b:0111]. On the topic of vfio and blacklisting, do you add these as well or leave them out? as well as the kernel module 'bcma'

In your last post, you mentioned adding it as a 'built-in', which I am understanding as adding In config.plist

As the device on host is on Pci slot 1:

DeviceProperties -> add -> PciRoot(0x1)/Pci(0x10,0x1)/Pci(0x0,0x0) -> built-in: 1

I plan on doing a fresh install on the nvme card with the Sabrent drive

Switched back to primary GPU, all is good with networking.

Also added a new Sabrent Rocket Nvme 1 Tb (based on several member's sigs that indicate this works well, as the intel 660p does not)

EDIT: Sabrent NVME passed through with zero problems with only adding it to vfio.conf, and now reinstalling MacOS on the same VM by reattaching the BigSurBeta.iso from Proxhox Host in 101.conf, which will now show "Install MacOS" in the selection screen. Did not make any modifications to EFI nor remove any existing disks that are attached.

vfio.conf

# This file contains all the device-ids and vendor-ids of components we want to passthrough to the macOS VM using vfio.
# reload with update-initramfs -u -k all && sudo reboot
options vfio-pci ids=1002:6863,1002:aaf8 disable_vga=1
# 47:00.1, 47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
options vfio-pci ids=1022:149c
# 22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
options vfio-pci ids=1022:148c
# 43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
options vfio-pci ids=1b21:3242
# 42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
options vfio-pci ids=1022:57a4
# 4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
options vfio-pci ids=8086:f1a8
### Bluetooth Configuration ###
# 46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
# options vfio-pci ids=10ec:8125
# 45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
#                         Subsystem: Intel Corporation Device [8086:0084]
options vfio-pci ids=8086:2723,8086:0084
# Bus 007 Device 006: ID 8087:0029 Intel Corp.
options vfio-pci ids=8087:0029

# 47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
# options vfio-pci ids=1022:1485

# 44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
options vfio-pci ids=1d6a:07b1
# 48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
options vfio-pci ids=1022:7901

#options vfio-pci ids=1002:66af,1002:ab20 disable_vga=1
#options vfio-pci ids=1987:5016
#options vfio-pci ids=144d:a804
#options vfio-pci ids=144d:a802
#options vfio-pci ids=14e4:43b1
#options vfio-pci ids=10ec:8168
#options vfio-pci ids=1022:1487

lspci -nnk

00:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: ASRock Incorporation Starship/Matisse Root Complex [1849:1480]
00:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
00:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
00:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
00:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
00:14.0 SMBus [0c05]: Advanced Micro Devices, Inc. [AMD] FCH SMBus Controller [1022:790b] (rev 61)
        Subsystem: ASRock Incorporation FCH SMBus Controller [1849:ffff]
        Kernel driver in use: piix4_smbus
        Kernel modules: i2c_piix4, sp5100_tco
00:14.3 ISA bridge [0601]: Advanced Micro Devices, Inc. [AMD] FCH LPC Bridge [1022:790e] (rev 51)
        Subsystem: ASRock Incorporation FCH LPC Bridge [1849:ffff]
00:18.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 0 [1022:1490]
00:18.1 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 1 [1022:1491]
00:18.2 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 2 [1022:1492]
00:18.3 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 3 [1022:1493]
        Kernel driver in use: k10temp
        Kernel modules: k10temp
00:18.4 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 4 [1022:1494]
00:18.5 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 5 [1022:1495]
00:18.6 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 6 [1022:1496]
00:18.7 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship Device 24; Function 7 [1022:1497]
01:00.0 Network controller [0280]: Broadcom Limited BCM4360 802.11ac Wireless Network Adapter [14e4:43a0] (rev 03)
        Subsystem: Apple Inc. BCM4360 802.11ac Wireless Network Adapter [106b:0111]
        Kernel driver in use: bcma-pci-bridge
        Kernel modules: bcma
02:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
03:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
03:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
20:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
20:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
20:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
20:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
20:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
21:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
22:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
22:00.1 Encryption controller [1080]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP [1022:1486]
        Kernel driver in use: ccp
        Kernel modules: ccp
22:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Starship USB 3.0 Host Controller [1022:148c]
        Subsystem: ASRock Incorporation Starship USB 3.0 Host Controller [1849:ffff]
        Kernel driver in use: xhci_hcd
22:00.4 Audio device [0403]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:1487]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller [1022:d102]
        Kernel modules: snd_hda_intel
40:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
40:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
40:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:01.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:01.3 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:03.2 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge [1022:1483]
        Kernel driver in use: pcieport
40:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
40:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
40:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
41:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57ad]
        Kernel driver in use: pcieport
42:02.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:03.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:04.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:05.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a3]
        Kernel driver in use: pcieport
42:08.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:09.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
42:0a.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:57a4]
        Kernel driver in use: pcieport
43:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242]
        Subsystem: ASMedia Technology Inc. Device [1b21:3242]
        Kernel driver in use: xhci_hcd
44:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02)
        Subsystem: ASRock Incorporation AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1849:d107]
        Kernel driver in use: atlantic
        Kernel modules: atlantic
45:00.0 Network controller [0280]: Intel Corporation Device [8086:2723] (rev 1a)
        Subsystem: Intel Corporation Device [8086:0084]
        Kernel driver in use: iwlwifi
        Kernel modules: iwlwifi
46:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01)
        Subsystem: ASRock Incorporation Device [1849:8125]
        Kernel driver in use: r8169
        Kernel modules: r8169
47:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
47:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486]
        Kernel driver in use: xhci_hcd
47:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c]
        Kernel driver in use: xhci_hcd
48:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
49:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51)
        Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901]
        Kernel driver in use: ahci
        Kernel modules: ahci
4a:00.0 Non-Volatile memory controller [0108]: Intel Corporation Device [8086:f1a8] (rev 03)
        Subsystem: Intel Corporation Device [8086:390d]
        Kernel driver in use: nvme
4b:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E12 NVMe Controller [1987:5012] (rev 01)
        Subsystem: Phison Electronics Corporation E12 NVMe Controller [1987:5012]
        Kernel driver in use: nvme
4c:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1470]
        Kernel driver in use: pcieport
4d:00.0 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Device [1022:1471]
        Kernel driver in use: pcieport
4e:00.0 VGA compatible controller [0300]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6863]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 XTX [Radeon Vega Frontier Edition] [1002:6b76]
        Kernel modules: amdgpu
4e:00.1 Audio device [0403]: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Subsystem: Advanced Micro Devices, Inc. [AMD/ATI] Vega 10 HDMI Audio [Radeon Vega 56/64] [1002:aaf8]
        Kernel modules: snd_hda_intel
4f:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
50:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
60:00.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Root Complex [1022:1480]
60:00.2 IOMMU [0806]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse IOMMU [1022:1481]
60:01.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:02.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:03.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:04.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:05.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:07.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
60:08.0 Host bridge [0600]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Host Bridge [1022:1482]
60:08.1 PCI bridge [0604]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Internal PCIe GPP Bridge 0 to bus[E:B] [1022:1484]
        Kernel driver in use: pcieport
61:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse PCIe Dummy Function [1022:148a]
62:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]
        Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485]

Was able to fix it without removing the broadcom card.

EDIT: what @meina222 mentioned is really a key observation, which you should be able to see with "dmesg | grep eth" - in the logs it showed that the new broadcom card was added as enp68s0 whereas my Realtek NIC was added as enp70s0. But my /etc/network/interfaces was correctly configured.

- Added second GPU / removed power from primary GPU to access host with keyboard.

- Ran the following commands in normal /bin/bash (not grub... ugh):

# will display your network nics

ip link



# will display what (if any) NICs have been renamed. 

# Make note of what the names are to ensure that they match with your /etc/network/interfaces in the next step

dmesg | grep eth



# will disable all NICs / so be careful of this

ifdown -a



# modify your network names if it changed. In my case, the NIC names were correct but internet access.

# So what I did was a bit weird.

# I changed the IP address of my vmbr0 192.168.5.157 -> 192.168.5.158, ran ifup -a

# led to no changes, then ifdown -a, changed it back to 192.168.5.157, did another ifup -a

# and pinged google.com and it reconnected. 



nano /etc/network/interfaces



# after changes are made

ifup -a



# ping a website to see if it's connected. If it timed out, you don't have a connection yet, so try again

ping google.com

How would I edit a file from within grub commandline? That's the main problem right now. I can't access any of the typical files because grub's cmdline is limited.

I've tried edit files with

- vi

- nano

- edit

But grub's accessible commands are very different, and in several posts, I've seen it said that you can't edit files while within grub, only the grub.cfg with 'E' during boot selection

Sorry - to clarify, I am not able to access my proxmox host instance via it's IP address :8006 (i've ran into this issue before, and I typically just reinstall proxmox). My guess is that with the new card, or when I unplugged it, my router might have assigned it a new IP address. I have it running headless, but also have a monitor plugged in to observe it.

The current problem is that even with the monitor plugged in, because I have the VGA blacklisted on proxmox boot, I am not able to access the typical commandline once it's up (only the 2 lines). So my two theories in trying to get it to work:

- How would I disable VGA blacklist so that I can re-access the commandline from host

- How to disable/change the static IP that Proxmox is configured to use from grub (because of #1)

@meina222 - I plan on doing those benchmarks later tonight and will share the results. I just received and installed the PCIe WiFi/BT (PCIe slot 1) card you mentioned prior, and now I'm running into an issue with Proxmox where the NIC isn't being correctly passed, so I'm not able to connect to proxmox and with VFIO passthrough, I can't access it via normal KB. Any tips on fixing this? I had tried to do a suggestion on the proxmox forums adding "net.ifnames=1" to grub command line prior to boot but it's not doing anything.

Thanks @meina222 - I had ran across that kext before, but never got to a point where I could actually use it. Will revisit. I don't have that much memory yet, only 64GB but plan on adding more later. 

RE the TB3 SSDTs - from looking at them briefly, if you remap the ports from the decompiled aml files and match them to your system PCIe slots (I think) - it should in theory work, or perhaps remapping other ports with a mask. I haven't gotten around to it yet mainly bc I found out I still need a TB3 -> TB2 wire in order to use it for my TB Display, so I could only validate whether it shows or not, not actually working.

@iGPU thanks for sharing that! Per Fabio and meina's suggestion, I disabled all the SSDTs and was able to boot with no issues. I am curious though whether they make a difference at all when it comes to performance, and/or optimizing the GPU, since there was a big drop in my Vega FE.

I also share your OCD-ness for the cosmetic appearance. Does disabling MCEReporterDisabler affect the memory module warning? That's one cosmetic thing that has been bothering me, even though it doesn't affect performance.

 RE TB3 - I may have something for you here. I purchased my GB Alpine Ridge from a seller who flashed(?) it and provided OC instructions for it. Publicly, it's pw protected, but the pw was provided in the instructions when I received it. Attaching the main essentials in the zip here. Main files are the SSDTs (TR, TR2, AR) and their respective OC configs. I have not tested this yet on my own machine, and the note is that this was specifically compiled for a genuine Mac Pro (<2010) - but hopefully can give you some insight.

thunderbolt_files.zip