Driftwood

Removed the BT kext...good thinking. The only thing I'm suffering is the Radeon Shutdown/restart problem. So it solves that?

There you go! Open Core stuff. Yeah the Matisse USB has been problematic. If you are getting what Im getting, avoid it. You're limited to 4 USB devices in Proxmox. So you may be able to pass through some off them. If you get success I'd like to know! I'd like to get my 3.2 gen 2s working 🙂 Currently the front USB stuff is knocked out and the bottom two USB 3 ports are too at the rear, plus the USB3.2s. Also its quite possible you have to get the r8125 Linux driver installed like I got as its more 'friendly' shall we say than Proxmox default driver which just seems hit and miss for this board. Another tip: Flash back to 1.50 fw on the bios, roll back your Proxmox install to an earlier than 6.2 and update later when you get in. 6.2 it appears is troublesome on install for us ASRock Creator users. When you're in correctly you can always flash back to 1.60. Ensure BIOS is set to defaults when you install (ie leave CMS alone for now), get boot order correct, etc. EFI.zip

Apologies @fabiosun. My mouse on an old Hack Mac running the host web console proxmox is faulty, I should have copied out to end of line! It is corrected now. I think the config is good for my case (because I use Decklink and Firewire pcie cards) but of course don't take things literally as other people's build / mobos are different. The ASRock Creator has come with a few headaches and I am keen in the future to see what other boards are out there - like the msi. Of course, when there's time for me, I will investigate further the problem with the other USB hub to see if we can knock that out of the equation as Im missing the 3 x USB 3.2 Gen2 (1 Front Type-C, 2 Rear) with this config. Apologies once again, I did not mean to offend you in any way. Thank you @Pavo for coming in to answer whilst I was afk.

I value this site more than anyone. I hardly know about Discord and not much Linux. Yourds was the first post I saw fabiosun so I want to say it was you who caught my eye first. I needed a fast video compression/editing machine for raw files first and foremost and didn't want to spend a fortune on a Mac so it was fab to see your post. The first 🙂 This thread ids great and I value all the guys here, most of all you. As I learn more about Proxmox and Linux in my free time I will be able to answer better. It won't take long as Ive learned a lot here, and with Pavo. This thread and group is wonderful I value what you say and the path of discovery. I tried to get all this kit working inside a week - spending unto 12 hours a day (sleeping at 5am!) to try and get this working. But to learn the knowledge of you and others will take me time. But Ill get there. I understand all the principles. After all, we all want our big investment to work for us. Its not just geekery stuff.

VM conf above VFIO conf below Thats my latest VM conf and VFIO conf. Noticed Ive now removed the other problematic USB, the Args line has the IRQ statement, and Ive ensured Proxmox is booting with at least 4Gb and have lowered RAM to 48Gigs until I get more RAM! Safety measure. The firewire and Decklink dont produce weird errors anymore like crackle and erratic mouse lag - now they just work! I think any one of those changes was responsible for problems. Ive also got monitor screen saver working without going black after coming back from its sleep. Hope this helps anyone. Pavo has been instrumental in getting this working and he clearly knows what he is doing, so rather than spend days sorting this (i understand more what he tells me all the time) I think advice from an expert is advise-able if you haven't got the time atm - got jobs on. I have to try that shutdown script @Roxer67 when I get the time. But my system is VERY stable now and Im very happy. It flies! ID 4d:00 and 4c:00 are the black magic Decklink and firewire cards

So here it is without the crackling - get rid of the nasty USB/USB audio controllers and viola - no more crackling. Spent an hour or two sorting this with Pavo. And he suspected the 47:00 Matisse USB 3 Controller was the culprit. Got rid of and now firewire has no crackle and mouse moves fluidly without lag. Also, in vm conf's 'Args' at the top, try kernel_irqchip=on, as this may have helped. Looking good to start using this beast now 🙂 Pic below showing 64 core/threads barely at work!

@Rocket88 Can you give me EXACT installation details below. Is cron part of Proxmox Debian? Do we need to install anything with 'Get' . Please type out exact method to install and how to copy it to snippets, when I run 'Crontab -e ' I get an editor, where do I go from there? Sorry Im a Linux noob!

Ah yes, I see the problem you describe @iGPU about the pci bridging. The card you have chosen to buy is the latest incarnation of mine (same chipset with extra port) Fingers crossed it works for you. Do you have RME audio like me? Ive discovered a problem. Everything works well in my setup until the monitor goes to screen saver. On return to desktop and relogging in to Mac I notice crackles and pops in audio and USB mouse is very jumpy. Any ideas what this could be? As if I have to reboot / switch off everytime just to use the Mac its really a lot of hassle! Ive tried changing from Rombar=1 or 0 with no difference so its not the firewire card, maybe its USB? Maybe its something I havent thought about. I have IOMMU set to enable in BIOS and CSM is disaabled (so is 4G for the sake of Proxmox) Update: even if i turn off screen saver and any type of sleep, this vm will begin to have slight crackles in audio playback and then laggy mouse...so if anyone has ideas? UPDATE2: All fixed now - dodgy USB controller with the sound HDAs on them. Disable them they're useless and interrupt everything else.

I have to admit I went through two different firewire cards before finding this one that worked @iGPU on my old regular hack Skylake Mac. So I already knew it should work in this vm.

Mobo: ASRock TRX40 Creator (good mid range board that is temperamental with ethernet Proxmox pickup on install. CPU: Ryzen 3970X. Fast and furious! RAM: 64GB Ram HyperX HX4361C17PB3K4/64 Predator 3600 Display: 2 x Radeon VII 16Gb GPU cards, 1st for Mac Apps. 2nd for Davinci/rendering packages m2/NVME: 2 x MP600 2TB nvme, 1 x Samsung 960 Pro nvme Video Capture: Black Magic Decklink 8K Pro Capture Card - Sets up well with BM Desktop Video Mac fw/drvr. Comes with 4 audio drivers too. Audio: Startech cheap Firewire Card for 2 x RME Fireface 800 audio boxes (classics!), Input: Corsair K65 and Corsair Mouse (iCue works) So my latest running build in vm Catalina. I guess the only thing I need to tune up now is USB. lspci -nnk VFIO.conf VM100.conf

Firewire StartechPEX1394B3LP with RME FIREFACE800 working with PCI passthrough on lane 2 in Catalina. Bottom pic, Premiere Pro working with Firewire 800. I don't fancy using the Realtek ADACS hence why I need pro cards to work in my box. But its good to know the Firewire works (and Adobe)

@Driftwood @Pavo maybe. Didn't go to him to for audio problems. Just that rogue nic. Ask him about it. It was trouble. It got to a point where it was beyond me and couldn't even get in. I even booted Windows on another drive to check. It found it in Deep Sleep. So I think I'd like to know what happens during an aborted prox install ? Proxmox shows a rollback of the nics if you watch, but what is going on here? It appears to fail the writeback to default dec ids for the realtek? It could well be that 6.2 straight install is the problem. I think after a failed install or abort this particular nic (the Realtek 8125G) just goes into deep sleep or just isn't supported 100% in by the included 8169 driver that proxmox uses for it? Who knows @fabiosun ?! I just needed to get on as have a big video edit to do and wanted to crack on so I got the experience of Pavo involved and he was a gent. As to the weird usb pcie loopback audio driver - I don't know whats going on - all I hear is hiss - but Im not in need of that as I use firewire devices and RME800s so wasnt so fussed. Pavo did give it a look and we checked over a couple of I/Os and it a sounded hissy... Ask Pavo I guess? Also Im a Discord noob (and Linux noob) Sometimes its quicker to just let someone with experience get on with it!

Keep a close eye on hardware dev/ids/IRQs changing order in lspci nnk when shifting devices around in the PCIe slots thats for sure! My Decklink re-install shifted from 4c:00 to 4d:00; and, after removing the unsupported wifi/bluetooth card (which occupied enp69s0) adding the Realtek 8125g driver, the Realktek now becomes enp69s0. So keep an eye on shifting sands :-) I'm now going to replace that Intel 6 Wifi/BT with a supported wifi bluetooth card). Wasted so many hours on this build looking for problems which were simply driver issues...!

@iGPU I had a bit of that during hardware installation with the Decklink card. This worked for me:- 1. Shutdown inside macOS like normal 2. ssh into the IP from your MacBook and type Qm stop 100 (ie your VM ID). 3. If no boot screen on reboot or freeze. the only option was to Power down, Switch off at back, pull the power chord, give it 5 secs, replug in power and switch back on - should restart without losing BIOS settings. Negated having to flash/cmos reset which just gets messy.

Okay, had a bit of a twiddle with that rogue nic on the ASRock Creator TRX40 board. Pavo took a look at it and totally sorted the problem. Even though I changed it in a post above, it was still going awol. Basically, the problem resides in the Realtek 8125 2.5g (the secondary nic in the mobo) and it goes missing sometimes (that's because it's using Proxmox's 'compatible' 8169 driver). The 'Atlantic' nic or (Aquantia) is always seen but you need to keep that unused for your passthrough OS as the Mac supports this nic. During my re-installs of Proxmox I never got the option for an IP home network address for the Realtek 8125g - when it was found it was providing only IPv6 mac addresses and gateway grief. The elegant (yet long-winded approach) method to overcome this problem is to grab hold of the Realtek 8125g driver from their site Realtek 8125g drvr - you'll find it under the Linux section. Bind that into your Debian/Proxmox install by ssh and re-edit / ifconfig it to become your default gateway nic. Its a bit tricky but with the massive help of the legendary Pavo he was able to sort it and viola! Everything is looking sweet now. With his top class authority we whipped thru the rest of the build and got everything working tip top. So Dual Radeon VIIs working, two MP600s, USBs, and a Black Magic Decklink 8K Pro all going very well. Messages works etc. Using 7,1 smbios in OpenCore with more tweaks from Pavo. Got a lot to think about but after installing my Firewire card (which should be a breeze as it already works in my HS Hack) we should be good to go. Top speed flyer... so all in all the Creator TRX40 from ASRock is not a bad budget board. Though I do like the look of the MSI with tons of m2 nvme reservations... tempted. PCI10 is the Black Magic Decklink. Notice you have to ROMbar=1 to get it working.

Exactly what I did on my old High Sierra Hack, and Ill be doing it on this too. Windows on a separate boot drive and F11 to boot it.

That's good to know. You're on a slightly different board to me but Ill keep an eye on that. As to backups yeah once I get all these devices passed Ill be doing that. Incidentally, are you using NVMEs to boot Mac OS and a secoindary data nvme? Im trying to stick to chip based storage over spinners. But will use Spinners to backup data to. Once Ive got a blueprint written down for every device in this system Ill certainly be wriuting a pdf just in case I have to restart the process. We are pioneers in this crazy game!

Black Magic Decklink Cards working very well in KVM virtualisation Machines without a problem. Here, a Decklink 8K Pro PCIe card on the 4th lane working no problems inside MacOS Catalina VM. Tested one of the SDI ports 'In 1' on the Decklink , loaded BMD's drivers, quick reboot, ran Decklink Xpress, plugged in a camera, click Record panel, and viola! We have images & Audio. 🙂 VM.conf hostpci2: 4c:00.0,pcie=1 root@:~# nano /etc/modprobe.d/vfio.conf: # Decklink 8K Pro Black Magic Design options vfio-pci ids=bdbd:a14b Live Video Audio

@Rox67er Did you finish that USB table? Also when trying to get your hook script working If I select and copy your perl script text what are using to create the actual script and saving ... nano?

For anyone wanting to swap their nics around for Proxmox. For some reason of mis-selection Proxmox installed its default gateway on my fast Aquantia nic, and I wanted to swap it over to the other Realtek nic in my mobo so I could passthrough the Aquantia to MacOS). Here's how. Use 'lspci' to check which network card is which. In Proxmox use the pve> 'Network' tab and the SHELL cli : Use 'ip a' to get a listing of current Default Gateway and IP being used by Proxmox. Take a look at your current /etc/network/interfaces (use nano but dont change stuff here, CTR X without saving), notice here in this example that interfaces use enp68s0 and is bridge ported to vmbr0. I want to be able to swap/change enp68s0 (A fast 10G Aquantia nic currently 'taken' by the proxmox host) to a 2.5G Realtek on enp70s0. Note enp70s0 is currently not active. The Linux Bridge using old nic selection (BEFORE) ----------------------BEFORE --------------------------- auto lo iface lo inet loopback iface enp68s0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.134 netmask 255.255.255.0 gateway 192.168.1.254 bridge_ports enp68s0 bridge_stp off bridge_fd 0 iface enp70s0 inet manual -------------------------------------------- Lets issue a ip link show command and see whats going on BEFORE the change we're about to swapover nics. root@pve:~# ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp70s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff 3: enp68s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff 4: wlp69s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 50:eb:71:77:e7:fa brd ff:ff:ff:ff:ff:ff 5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff Now Ill show you how to to edit in Network tab, or CLI to discover how I got to the results of AFTER the swapover below;- Edit each network device to make sure they each have their own IP (the enp68s0 will keep its current details so you can leave it alone, but remove its gateway because the other nic is going to be default gateway). The nic you are swapping to will now have a Default Gateway to your home network's gateway once youve entered an address (e.g. 198.162.1.254) so enter that in this net device. Note: the enp70s0 wont need IP info if using a Linux bridge as the bridge will hold the details. You will have noticed I have a Linux Bridge in use on vmbr0 so we're going to reassign it to enp70s0: So take a note of the vmbr0 Linux Bridge details, remove & re-add/or just edit existing details to read enp70s0 in place of enp68s0 and bridge ports to = new nic to be used - ie the enp70s0. The enp70s0 will now become vmbr0, input that nic card's address in ipv4 (eg in my case 192.168.1.144/24), set Gateway ipv4 to your home network gateway (e.g. in my case 192.168.1.254) and finally Set to autostart by checking its box) So you have basically swapped the information around from one nic to another and on reboot you've activated the new nic assignment and this will become the default used by proxmox host. You can see by me BEFORE (see top) and AFTER (see below) details as to how it all now looks. (Note: to do above you can alternatively use CLI with nano /etc/network/interfaces if you prefer to edit directly) -----------AFTER--------------- auto lo iface lo inet loopback auto enp70s0 iface enp70s0 inet manual iface enp68s0 inet static address 192.168.1.134/24 auto vmbr0 iface vmbr0 inet static address 192.168.1.144/24 gateway 192.168.1.254 bridge-ports enp70s0 bridge-stp off bridge-fd 0 ----------------- ip a AFTER --------------------------- root@dwd:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp70s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff 3: enp68s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff 4: wlp69s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 50:eb:71:77:e7:fa brd ff:ff:ff:ff:ff:ff 5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff inet 192.168.1.144/24 brd 192.168.1.255 scope global vmbr0 valid_lft forever preferred_lft forever inet6 2a00:23c6:ab84:8b01:aaa1:59ff:fe16:3385/64 scope global dynamic mngtmpaddr valid_lft 314957848sec preferred_lft 314957848sec inet6 fd65:76f8:30e8:1:aaa1:59ff:fe16:3385/64 scope global dynamic mngtmpaddr valid_lft forever preferred_lft forever inet6 fe80::aaa1:59ff:fe16:3385/64 scope link valid_lft forever preferred_lft forever --------- DONE ------------- Hope this helps anyone who was confused and reluctant to change nics for fear of losing guest web access to the host like I was! AFTER images And when we passthrough the Aquantia 00:44: nic :- vm101.conf contains this addition: hostpci1: 44:00.0,pcie=1 & nano /etc/modprobe.d/vfio.conf contains this addition: #ethernet - Aquantia 1d6a:07b1 (DeviceName:DeviceID) options vfio-pci ids=1d6a:07b1 Boot to MacOS and viola! you get this in System Report>Network The Aquantia ethernet is automatically recognised by Mac OS Catalina! Finally, you can remove or comment out any install leftovers relating to ethernet in your vm conf file, such as:- # net0: vmxnet3=3E:34:8E:BA:0A:50,bridge=vmbr0,firewall=1

After a long evening of trying to sort this motherboard out (ASRock Creator TRX40) and some fantastic tips from all here (plus some eye opening conversations with @iFPU who I am indebted to) Ive finally managed to get network connections and other issues sorted out and am slowly rebuilding the VM that went pear shaped the other day! On my reinstall of Proxmox (which was 6.2 for trhe first trime for me), I was getting a IPv6 Mac address offered as the IP address! on one of the nics, the secondarey nic in the dropdown correctly offered me 192.168.1.134 for my home network and then I could get a working install. Still dont know why I was being offered the mac address - maybe something to do with the realtek nic kindof being 'discovered' under the 5.4 kernel of Proxmox 6.2 - who knows?! Anyway got thru install. Insiode the box and after creating a new VM, If anyone is getting no access to the vnc console in order to restart ISO installation of a OS ensure your args in your .conf are correct. Ensure IOMMU in BIOS is set to Enabled and not just Auto, obviously make sure SMT is enabled for virtualisation (which it was!) and check everything else - you dont neccesarily need to have Above 4G ENabled (you can keep csm and Above 4G Disabled - so long as you are booting from UEFI you're good to go for that matter (if anyone has any advantages let me know), etc. You don't need a bridged net adapter as I learned from iGPU, so the setup vm in conf where you get net0 blah blah blah vmbro bridge can go. Youre going to passthrough along with the correct IP address anyway. The mosty important thing is to re-read items that have been mentioned before here (reminder to self) as even though your board might differ from one of the more experienced guys here, there's still many similarities you need to follow. Anyways, Im back in the game and thank you once again for all support here. This is a great site with a reservoir of info and helpful guys.

So when you install Proxmox (Ive a Proxmox 6.2 USB bootloader now) why is it one of my nics gets a IPV6 mac address in the last page of the install, I then click the alternative (Atlantic) one and get a proper home.pve network address assigned and not a mac address.... which is which? On the install you're kindof blind as to which nic Proxmox recomends. You dont know if its chosen Aquantia or the Realktek! If someone knows please enlighten me! I want two IP4 addresses please!!!! Getting a bit fed up of 'doing the reinstall' thinking its other things, wiping disks because of partition info from previous install is in same place, etc... etc... very frustrating. I was so close to getting things passthroughed and finalised, then the network lockup screwed everything up and I was locked out. I even reformatted the nvme install of catalina thinking something was amiss there.... very frustrating. :-) Ah well, biugg sigh... here goes...

IGPU: Thanks for your points. Whats weird is (I think) this got screwed up when I flashed the bios to latest version for this mobo. v 1.06 the latest. Though I cant prove it now... so Im in a bit of a dilemma here. I cant remember if both nics originally worked on first install of Proxmox. I need the Aquantia nic in the Mac OS VM. Judging by other posts on Proxmox I need to get a 2.5G nic driver perhaps from linux.git and find the correct one to support it for host networking. There's also the wifi /BT one (Intel Wi-Fi 6 802.11ax (2.4Gbps) + BT 5.) has anyone got that working? For now, on Proxmox USB install, the only one that seems to work is the one which recommends 192.168.1.134 address (Im guessing its Aquantia or how can I find out the one which Proxmox 'likes'?), the other ethernet nic places a Mac address as its IP! (IPv6?) which I tried manually editing in Managenment Interfaces on the last setup page where you put in your network info. Just read manual and the other nic (the 2.5g one) is Dragon RTL8125AG Does Proxmox/Linux support this? Its confusing not knowing enough about Linux!

@fabiosun or anyone: I'm reinstall ing proxmox, it's hanging on nics again. Lights are flashing on off on both nics but they've gone awol and the install is stuck? This is true of proxmox 6.2 and 6.1

Thanks @fabiosun. Yeah I understand that, but for some reason Proxmox will not allow me to use the Intel nic, there's two nics plus wifi (which proxmox doesn't support) on this ASRock Creator trx40 Mobo. The Aquantia works in Mac out of the box so I wanted to use that for the VM. But the host has it, and bridged it, checking hosts the Aquantia is UP has been taken and no sign of Intel. Indeed Intel is DOWN. What's the ideal scenario here? Get Intel nic UP for host and leave Aquantia nic for Mac vm? How have any other trx40 users setup proxmox in the first instance?