iGPU

So we have exact same issue. I've been trying to attack this problem on several fronts. I'll have another go tonight. I think the problem is that Linux does not natively support Thunderbolt. A key, I believe is in activating TB for Linux. I tried this 2 days ago (and was going to write about it yesterday when I alluded to the complicated reply), but it didn't work out. I've now got a variation to try out and will report if this works. I've been scouring the Internet for Linux/TB and found this one comment about PCI pipes/tubes not being established until TB is authorized. If I cannot authorize my card, then I'll ask you to try and authorize your un-flashed card. (I'll write in detail how; there are a few tools to install and a couple of odd-ball commands.) If you can then authorize, then I'll know my problem is failure to authorize due to flashed firmware. In which case I'll re-flash my card, authorize and try again to pass-thru TB. In any event, I'll report back later.

To get sorted IOMMU information, do the following. 1. nano iommu.sh 2. enter following code, the save (control-X): #!/bin/bash shopt -s nullglob for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*} printf 'IOMMU Group %s ' "$n" lspci -nns "${d##*/}" done | sort -V 3. type: chmod +x iommu.sh 4. run: ./iommu.sh My results are in Spoiler below. This sorts the groups and makes looking at the TB section easier.

Do you mean that your argument section is only contains "args: -cpu host" ? Initially, I had both "cpu: Penryn" and "args: -cpu Penryn", then I changed both to "host". It seems I had slightly better test scores with Penryn. Did you notice any differences in that regard? My latest VM config file:

Before you flash firmware, you might want to try some other things. I'll explain below in a later post as it is a bit involved, but this new avenue to get TB working isn't working for me and I don't know if it's because I am using flashed firmware. (Unfortunately, my flash programmer died and I've not got a replacement up and running.) Anyhow, I'd also like to see you run some Linux commands to see how your un-modified TB card behaves. I've used Fenzi BT cards, but like having the built in BT work; it is more powerful and keeps a better connection for me. Since I use Ethernet, I don't normally use Wifi and so don't miss having it. As for the Linux TB commands, could you please provide results, when TB card is passed, for "lspci -nnk" and "find /sys/kernel/iommu_groups/ -type l"? Also, does the VM shutdown when trying to pass all sections of the TB, as I described above in this post? Thanks.

I've seen a few posts using Radeon VII (I have this GPU in another build, but water cooled so cannot swap), and will be changing to this GPU from Vega 56 in a few days once the VII has arrived. Also some are using 5700XT which behaves very similarly on macOS to Radeon VII.

Here is a flashing link: CH341A . More repository data: here. More importantly for you, specific info for ThunderboltEX 3 here along with how to adjust DROM. This latter step is complicated and I can help later as this step is activating within macOS, which we cannot do until we can pass the device. I just found an old ThunderboltEX 3, so I'll flash it and see how it looks from my end. I think we'll face exact same issues with passing as I see with the Titan Ridge, just with different device ID codes for vfio.conf. If anyone else reading this in interested in flashing, each device needs a different modified firmware to flash, so the firmware for EX 3 ≠ Alpine Ridge ≠ Titan Ridge ≠ native mobo. All are unique, so don't flash what I uploaded to any other TB card or device.

A bit more expensive, but I'm using this unit from the UK: Audient Evo 4. It works very well. When combined with this SoundControl app to adjust EQ in real time, it's all quite nice. I'm running off either of the 2 USB ports beneath the RJ45 jack on rear panel (the right-most RJ-45 jack, when facing the rear panel). Attached is the bin file for flashing an ASUS TB-EX3. You'll also need to modify an SSDT, but we can address that later. AlpineRidgeEX3-NVM26-NATA.bin.zip

The TB passthru error I'm seeing: "vfio 0000:49:00.0: failed to open /dev/vfio/58: No such file or directory" is curious and may be the problem. The file "58" refers to the IOMMU group, to which address 49:00.0 belongs. In the folder, /dev/vfio/ are many files for the IOMUU groups that are being passed: 13 16 30 31 33 52 53 54 56 57 63 64 66 77 vfio. Running the command, "find /sys/kernel/iommu_groups/ -type l", gives the following (Spoiler). The groups for the un-passables are here: 58 for 49:00, 59 for 4a:00, 60 for 4a:01, 61 for 4a:02, and 62 for 4a:04. Those for the passable TB addresses are also visible: 63 for 4b:00, 64 for 53:00 For some reason, which I'm trying to research, there are no such files created for the problematic addresses, 49:00 and 4a:00. If we can force their creation, maybe we can pass the whole TB device.

Thanks for your reply (and esp for all of your help)! The resource you've created here are very much appreciated! As for BIOS, from the start, I had CSM disabled and 4G enabled. Until yesterday, I also kept XMP off, just to keep things as simple as possible. As for Thunderbolt, you can use your card without a TB header with almost any mobo. You only need to jumper (I soldered) two pins on back of card (see Spoiler below). However, to get functionality in macOS, you'll have to flash the chip on the card. I can provide links and help if you're interested (I've collected a repository of modified firmware for different cards and can supply one for your card). If you have the small piece of equipment to flash (bought off Amazon), it is really simple to do. But even without flashing to get macOS functionality, the jumper will still allow you to test passing addresses via VM without a TB header. Your USB setup is very similar to mine even though our mobos are different. I too get 4 USB ports on rear panel activated. They belong to those 2 groups we cannot pass (I think actually from the 4.00 group; I think I can pass 4:00.0, but not 4:00.3). However, my mobo does not have any ASMedia controllers. The USB slowness is odd; I still don't understand why I see it. Once the windows from the USB device have populated with contents, I don't notice the slowness any more. It's as if some buffer needs filling and does so slowly, but once filled, then all behaves rapidly after that. The initial slowness is even displayed by the spinning beachball.

I mentioned some SSDT and Device Properties in previous post. in case anyone wants to use them, I'm attaching them here. The SSDTs are for the GB TRX40 Designare but I think will work with all TRX40 mobo, only requiring some slight adjustments. Look at the addresses for the devices inside the SSDTs, comparing them to what you see on IORegistryExplorer, and adjust accordingly. For example, I chose to use only 1 Ethernet port: 44:00.0 (not 43:00.0), so the supplied data is for that port, not the other); I passed both SATA controllers and both Matisse USB controllers. As for the DeviceProperties, it is supplied for OpenCore and can be copied and pasted between this file and your current config file. There are 2 sets for GPUs: one is active for Vega 56 and the other has a "#" symbol at the front (see Spoiler image below), leaving the other set in-active. This other set is for the Radeon VII. Both include CMMChris's PowerPlayTable data. (BTW, if you place "#" in front of any item in OC, it will inactivate that item.) The other items provide various USB, SATA, Ethernet, and internal BT data (not for substituted card) that you can see on PCI section in bottom Spoiler. You'll most likely need to change the name of the NVMe SSD from the one I'm using in the slot closest to the CPU (the other M2Q slot houses the Proxmox SSD and is not visible). DeviceProperties section: System Information/PCI section (using above DevProp data):

I spent a few days trying to passthru the GPU. All variations suggested on this thread didn't help. Most other things were relatively easy to pass but I was stymied with the GPU. Finally it worked: thanks to reading PAVO's GitHub. I was not booting Proxmox into UEFI mode, but standard mode. (Mea Culpa!) Once I switched methods, the GPU easily passed and did not require a GPU-rom file nor any unbinding code. This UEFI boot issue needs to be high-lighted for those of us who over look details. 😉 *** One odd behavior I noted on this build is that high capacity USB connected drives open very slowly (the window opens quickly but is empty and only after 20 sec or more, do the contents appear in the window). Finder functionality seems otherwise okay. I've now been working on getting IORegistryExplorer info organized with better SSDT and DeviceProperties; mostly labelling, cosmetic issues (like assigning XHC, XHCI, etc). Hackintool does not see any USB ports but from what I see, there is no more than 10 USB ports on a given USB device, so I think no kexts or UIAC SSDTs are required to limit ports. *** Next, I started work on the GB Titan Ridge PCIe card. I flashed modified firmware onto it (I have these flashed cards working well on the X299 and on-board TB on a Z390 mobo). Unfortunately, it does not yet work. Below is the TB tree(s) in first Spoiler. The 8086,15eb is the NHI TB section; the 8086,15ec, the USB TB section. These are normally branches off the main trunk, instead of 2 separate devices. So usual custom SSDT starts to fill out the NHI section, but leaves the USB section as is. So no active TB. We need the whole tree. The problem seems to stem from passing some addresses. To clarify, the card was placed in the PCIe slot farthest from CPU and gave addresses as shown in next Spoiler (your addresses may vary). There are several sections associated with TB cards: the nodes of the TB tree are each at different addresses (and in different IOMMU groups; shown next). As can be seen in the address data, only 4b:00.0 and 53:00.0 are passed based on being able to substitute "vfio-pci" for the Thunderbolt drivers. The other ports, 49:00.0 and 4a:00 when passed in the VM cause a failed start, giving an error: "kvm: -device vfio-pci,host=0000:49:00.0,id=hostpci7,bus=ich9-pcie-port-8,addr=0x0: vfio 0000:49:00.0: failed to open /dev/vfio/58: No such file or directory". (If 49:00.0 is removed and 4a:00.0 is left, then the same error with 4a:00.0 is given; VM start only can occur if both 49 and 4a are removed.) I then tried blacklisting "pcieport", but that did not help. Lots of internet searching also turned up no help: there is no clear indication that anyone has successfully passed TB devices (some posts talk about using TB for eGPU usage, but no one has indicated passing a TB card to use for proper audio or TB drive connections). There is also a similar problem when trying to pass the Audio section which is something like 25:00.4 (on most of our TRX40 mobos), and also when trying to pass 2 other USB controllers, the two located at 4:00.3 and 25:00.3 (adjacent to Audio controller). When trying to pass 4:00.3, 25:00.3 or the above TB addresses are passed, one cannot start VM. But I think the Audio and USB problems are due to different reasons. When using IORegistryExplorer, the inability to pass the Realtek Audio device and the 2 other USB devices may be that Proxmox is already using them and 'silently' passing them. A section from IORegistryExplorer is shown in the next Spoiler. Here you canl see the Realtek device, with all channels, is being passed via USB. So Proxmox has tight control over these addresses and that is why we cannot pass them? (Personally, I use either external USB or TB devices to process audio, so I don't use the Realtek device.) Anyhow, getting back to TB, I think it will work if we can get those other 2 sections properly passed. Any ideas? lspci -tv gives a different view of the TB addresses as compared to other addresses:

Thanks for your help. I'm slowly getting things to work. (I'll attack the monitor last.) i've had the kernel modules and grub.cfg files properly filled out. There are 2 files for one of the kernel modules that I've seen used, but I think I only need one of them since they both contain same thing. And these are the files named "/etc/modprobe.d/vfio.conf" and "/etc/modprobe.d/vfio-pci.conf". I think I don't need the latter. You were correct with the Audio being a problem. It was causing some of the crashes. Even after doing 'mod' described below for hostpciX, trying to pass-thru the 23:00:4 audio section results in a freeze (and adding in adjacent USB @ 23:00:3 did not help either). It turns out that I cannot get error free pass-thru with Intel I210 Ethernet device if I use "hostpci3: 44:00.0,pcie=1". This gives an error on Proxnox monitor of "Invalid PCI ROM header signature". Instead, the better way for my mobo is using "hostpci3: 44:00.0,pcie=1,rombar=0". This 'rombar=0' made all the difference. It also allowed finally passing BT... and so I've now got native BT (Intel WiFi 6 AX200; probably same on most of our mobos) working (but no WiFi) using pass-thru BT! This is native mobo BT, not an add-on card! Native BT requires 4 things: a) the above 'rombar=0' addition in VM config file. b) USB power pass-thru (described below). c) Most important: two special kexts (a work in progress; see GitHub); I've attached versions that worked. These simply go into OC/Kexts folder (and of course, enabled inside config.plist file). d) And once booted into macOS, open BT preferences and remove all BT items. Then re-connect each one as desired. You need to pass the Network + the USB for BT (on my mobo, as shown in my previous post above), it is at 45:00:0. The associated USB is found with command "lsusb". On my mobo, results are shown in spoiler below. So I pass 8086:2723,8086:0084 for BT and 8087:0029 for the USB power in "/etc/modprobe.d/vfio.conf". Then run "update-initramfs -u -k all" and reboot Proxmox. The other problem I found is that while many things seem to need to be blacklisted, the kernel drivers for Ethernet, SATA and BT should not be entered into the blacklist file. I found out the hard way: I blacklisting 'igb' and on re-boot, I couldn't communicate from laptop to Proxmox: the blacklisted Ethernet driver shut down all host Ethernet. But since I kept 2nd monitor and keyboards connected, I was able to correct the blacklist file and get things working again. (This is why I'll pass the GPU as the last device after I've made most of my mistakes.) And as for the two Ethernet ports on my mobo, I have both connected to my hub at home, but only pass-thru one (which is all I need on Mac), so as to keep communication lines open from Mac to Proxmox. *** One thing I've noticed is that Proxmox passes all the various mobo devices to macOS as USB devices (including the Realtek 1220 audio device). I can see almost all of them using IORegistryExplorer once Mac has booted. Therefore, it makes sense to pass-thru as many as possible to minimize USB usage (and later so we can limit each USB device to 15 entries). results of command "lsusb": BT-kexts.zip

I finally got all the parts for this build a few day ago and after a couple of days of learning about Proxmox, the computer can boot onto an NVMe SSD previously cloned using Mojave 10.14.6 from the X299 build. The bootloader is OC v058 on the EFI partition of the NVMe drive (the drive is assigned to sata2 for booting from the conf file). Parts are listed in signature at bottom. I presently only have 64GB total DDR4 and so pass only 28-32GB. My intention was to pass the Vega 56 but when things started acting strangely, I ended up moving the Vega 56 to PCIe slot 3 (TB card will later be in slot 4), and placing an RX580 in PCIe slot 1 for Proxmox (slot 2 is empty). My problem is that I can only use the Console from my laptop to run the VM, I cannot pass through the GPU. In fact, my problem is worse: I cannot pass anything from the mobo. The VM freezes if I try to pass through any device. I've tried only passing 1 device, like the Realtek 1220 Audio device and still it freezes/crashes. I've gone over all items pretty carefully, but since so many of you have pass through working, I've must have made a mistake somewhere. I believe I've properly configured the /etc/pve/qemu-server/102.conf and the /etc/modprobe.d/vfio.conf files (both shown below) , as I can see "Kernel driver in use: vfio-pci" properly substituted for the chosen devices (see lspci -nn -k, spoiler below), except for 2 devices. The Ethernet port (44:00:0) and the NVMe (01:00:0): neither show substituted kernel drivers. BTW, Cinebench R15 gives about 7200 on average, so about par for this build and shows that the VM is basically working. Any ideas what I'm doing wrong that prevents pass through? Thanks for any help. lspci -nn -k : /etc/modprobe.d/vfio.conf : /etc/pve/qemu-server/102.conf :

Hackintoshes typically don't work with on-board Intel BT/WiFi, requiring the use of add-on PCIe cards, like a Fenvi T919. Does this VM method now allow (interface with) MacOS to use on-board Intel BT/WiFi? And any experience about using Thunderbolt PCIe cards? I presently have these working with flashed firmware on X299 and X570 builds, and I'm planning on using the GB TRX40 Designare mobo.

Thanks! I hope all of you stay healthy! The Hackintoshes occupy more of my time while staying at home.

I've made a few Hackintoshes, the most recent was an AMD 3950X. Now, I'm curious about the 3970X and the TRX40 platform and studying what fabiosun is doing.