Driftwood

@Rox67er Did you finish that USB table? Also when trying to get your hook script working If I select and copy your perl script text what are using to create the actual script and saving ... nano?

For anyone wanting to swap their nics around for Proxmox. For some reason of mis-selection Proxmox installed its default gateway on my fast Aquantia nic, and I wanted to swap it over to the other Realtek nic in my mobo so I could passthrough the Aquantia to MacOS). Here's how. Use 'lspci' to check which network card is which. In Proxmox use the pve> 'Network' tab and the SHELL cli : Use 'ip a' to get a listing of current Default Gateway and IP being used by Proxmox. Take a look at your current /etc/network/interfaces (use nano but dont change stuff here, CTR X without saving), notice here in this example that interfaces use enp68s0 and is bridge ported to vmbr0. I want to be able to swap/change enp68s0 (A fast 10G Aquantia nic currently 'taken' by the proxmox host) to a 2.5G Realtek on enp70s0. Note enp70s0 is currently not active. The Linux Bridge using old nic selection (BEFORE) ----------------------BEFORE --------------------------- auto lo iface lo inet loopback iface enp68s0 inet manual auto vmbr0 iface vmbr0 inet static address 192.168.1.134 netmask 255.255.255.0 gateway 192.168.1.254 bridge_ports enp68s0 bridge_stp off bridge_fd 0 iface enp70s0 inet manual -------------------------------------------- Lets issue a ip link show command and see whats going on BEFORE the change we're about to swapover nics. root@pve:~# ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp70s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff 3: enp68s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff 4: wlp69s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 50:eb:71:77:e7:fa brd ff:ff:ff:ff:ff:ff 5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff Now Ill show you how to to edit in Network tab, or CLI to discover how I got to the results of AFTER the swapover below;- Edit each network device to make sure they each have their own IP (the enp68s0 will keep its current details so you can leave it alone, but remove its gateway because the other nic is going to be default gateway). The nic you are swapping to will now have a Default Gateway to your home network's gateway once youve entered an address (e.g. 198.162.1.254) so enter that in this net device. Note: the enp70s0 wont need IP info if using a Linux bridge as the bridge will hold the details. You will have noticed I have a Linux Bridge in use on vmbr0 so we're going to reassign it to enp70s0: So take a note of the vmbr0 Linux Bridge details, remove & re-add/or just edit existing details to read enp70s0 in place of enp68s0 and bridge ports to = new nic to be used - ie the enp70s0. The enp70s0 will now become vmbr0, input that nic card's address in ipv4 (eg in my case 192.168.1.144/24), set Gateway ipv4 to your home network gateway (e.g. in my case 192.168.1.254) and finally Set to autostart by checking its box) So you have basically swapped the information around from one nic to another and on reboot you've activated the new nic assignment and this will become the default used by proxmox host. You can see by me BEFORE (see top) and AFTER (see below) details as to how it all now looks. (Note: to do above you can alternatively use CLI with nano /etc/network/interfaces if you prefer to edit directly) -----------AFTER--------------- auto lo iface lo inet loopback auto enp70s0 iface enp70s0 inet manual iface enp68s0 inet static address 192.168.1.134/24 auto vmbr0 iface vmbr0 inet static address 192.168.1.144/24 gateway 192.168.1.254 bridge-ports enp70s0 bridge-stp off bridge-fd 0 ----------------- ip a AFTER --------------------------- root@dwd:~# ip a 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever inet6 ::1/128 scope host valid_lft forever preferred_lft forever 2: enp70s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master vmbr0 state UP group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff 3: enp68s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff 4: wlp69s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000 link/ether 50:eb:71:77:e7:fa brd ff:ff:ff:ff:ff:ff 5: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000 link/ether a8:a1:59:16:33:85 brd ff:ff:ff:ff:ff:ff inet 192.168.1.144/24 brd 192.168.1.255 scope global vmbr0 valid_lft forever preferred_lft forever inet6 2a00:23c6:ab84:8b01:aaa1:59ff:fe16:3385/64 scope global dynamic mngtmpaddr valid_lft 314957848sec preferred_lft 314957848sec inet6 fd65:76f8:30e8:1:aaa1:59ff:fe16:3385/64 scope global dynamic mngtmpaddr valid_lft forever preferred_lft forever inet6 fe80::aaa1:59ff:fe16:3385/64 scope link valid_lft forever preferred_lft forever --------- DONE ------------- Hope this helps anyone who was confused and reluctant to change nics for fear of losing guest web access to the host like I was! AFTER images And when we passthrough the Aquantia 00:44: nic :- vm101.conf contains this addition: hostpci1: 44:00.0,pcie=1 & nano /etc/modprobe.d/vfio.conf contains this addition: #ethernet - Aquantia 1d6a:07b1 (DeviceName:DeviceID) options vfio-pci ids=1d6a:07b1 Boot to MacOS and viola! you get this in System Report>Network The Aquantia ethernet is automatically recognised by Mac OS Catalina! Finally, you can remove or comment out any install leftovers relating to ethernet in your vm conf file, such as:- # net0: vmxnet3=3E:34:8E:BA:0A:50,bridge=vmbr0,firewall=1

After a long evening of trying to sort this motherboard out (ASRock Creator TRX40) and some fantastic tips from all here (plus some eye opening conversations with @iFPU who I am indebted to) Ive finally managed to get network connections and other issues sorted out and am slowly rebuilding the VM that went pear shaped the other day! On my reinstall of Proxmox (which was 6.2 for trhe first trime for me), I was getting a IPv6 Mac address offered as the IP address! on one of the nics, the secondarey nic in the dropdown correctly offered me 192.168.1.134 for my home network and then I could get a working install. Still dont know why I was being offered the mac address - maybe something to do with the realtek nic kindof being 'discovered' under the 5.4 kernel of Proxmox 6.2 - who knows?! Anyway got thru install. Insiode the box and after creating a new VM, If anyone is getting no access to the vnc console in order to restart ISO installation of a OS ensure your args in your .conf are correct. Ensure IOMMU in BIOS is set to Enabled and not just Auto, obviously make sure SMT is enabled for virtualisation (which it was!) and check everything else - you dont neccesarily need to have Above 4G ENabled (you can keep csm and Above 4G Disabled - so long as you are booting from UEFI you're good to go for that matter (if anyone has any advantages let me know), etc. You don't need a bridged net adapter as I learned from iGPU, so the setup vm in conf where you get net0 blah blah blah vmbro bridge can go. Youre going to passthrough along with the correct IP address anyway. The mosty important thing is to re-read items that have been mentioned before here (reminder to self) as even though your board might differ from one of the more experienced guys here, there's still many similarities you need to follow. Anyways, Im back in the game and thank you once again for all support here. This is a great site with a reservoir of info and helpful guys.

So when you install Proxmox (Ive a Proxmox 6.2 USB bootloader now) why is it one of my nics gets a IPV6 mac address in the last page of the install, I then click the alternative (Atlantic) one and get a proper home.pve network address assigned and not a mac address.... which is which? On the install you're kindof blind as to which nic Proxmox recomends. You dont know if its chosen Aquantia or the Realktek! If someone knows please enlighten me! I want two IP4 addresses please!!!! Getting a bit fed up of 'doing the reinstall' thinking its other things, wiping disks because of partition info from previous install is in same place, etc... etc... very frustrating. I was so close to getting things passthroughed and finalised, then the network lockup screwed everything up and I was locked out. I even reformatted the nvme install of catalina thinking something was amiss there.... very frustrating. :-) Ah well, biugg sigh... here goes...

IGPU: Thanks for your points. Whats weird is (I think) this got screwed up when I flashed the bios to latest version for this mobo. v 1.06 the latest. Though I cant prove it now... so Im in a bit of a dilemma here. I cant remember if both nics originally worked on first install of Proxmox. I need the Aquantia nic in the Mac OS VM. Judging by other posts on Proxmox I need to get a 2.5G nic driver perhaps from linux.git and find the correct one to support it for host networking. There's also the wifi /BT one (Intel Wi-Fi 6 802.11ax (2.4Gbps) + BT 5.) has anyone got that working? For now, on Proxmox USB install, the only one that seems to work is the one which recommends 192.168.1.134 address (Im guessing its Aquantia or how can I find out the one which Proxmox 'likes'?), the other ethernet nic places a Mac address as its IP! (IPv6?) which I tried manually editing in Managenment Interfaces on the last setup page where you put in your network info. Just read manual and the other nic (the 2.5g one) is Dragon RTL8125AG Does Proxmox/Linux support this? Its confusing not knowing enough about Linux!

@fabiosun or anyone: I'm reinstall ing proxmox, it's hanging on nics again. Lights are flashing on off on both nics but they've gone awol and the install is stuck? This is true of proxmox 6.2 and 6.1

Thanks @fabiosun. Yeah I understand that, but for some reason Proxmox will not allow me to use the Intel nic, there's two nics plus wifi (which proxmox doesn't support) on this ASRock Creator trx40 Mobo. The Aquantia works in Mac out of the box so I wanted to use that for the VM. But the host has it, and bridged it, checking hosts the Aquantia is UP has been taken and no sign of Intel. Indeed Intel is DOWN. What's the ideal scenario here? Get Intel nic UP for host and leave Aquantia nic for Mac vm? How have any other trx40 users setup proxmox in the first instance?

Cool @IGPU: Ill give it a go! Might save some trouble! My kvm.conf I just rebuilt with nano editing and saved... so it should be the same, I need to lspci-nnk though as youre right things may have changed! Edit: Still getting 'No VNC' and cannot connect to server. I may have to take a look at all the other configs I rebuilt...

Hi guys, So Im about to go again after network nics just would not respond to allow me to log on. It seems the only way was for a re install of proxmox to achieve access to my home network! In the vm before the network problem, I had installed Catalina onto the first nvme which was in m2 slot 2 and Ive now moved to m2 slot 0. My SSD for Proxmox is on sata 0. After reinstalling to a new ssd, and able to root login, Im trying to recreate my old vm101 and all the other confs which Ive done, but for some reason when I start the vm it cannot connect getting 'No VNC' and cannot connect to server. Im under chrome but safari fails too. So Im wondering if Ive failed to do something or do I have to start all over again re-installing isos and catalina onto the nvme because something has gone amiss?

I can only 'restart' fine from macos, but 'shutdown' will cause the aforementioned bug. Where do you run that from shell console ?

Luxmark / Cinebench 20

Oh and my 'lspci -nnk'

Ok guys here's my system before I begin the main passthru configs. Only pass thru'd a pair of Radeon VII GPUs so far. There's a bit of tidying up to to do and add the 2nd MP600 to the box. Enjoy! /etc/pve/qemu-server/101.conf etc/default/grub /etc/modprobe.d/kvm.conf /etc/modprobe.d//blacklist.conf /etc/modprobe.d//vfio.conf ./iommu.sh dmesg –kernel find /sys/kernel/iommu_groups/ -type l Anything else you need to know let me know! Proxmox Screens: Hardware & Options

Im currently running MacPro 1,1 do you think Ill be better served running MacPro7,1 for this threadripper 3970x and radeon VII pair? About to begin passthrough of everything outside of the GFX cards. Hoping to see performance gains in NVMEs and rest. By the way, if you have dual GPU use PCIe lane 0 and 2 (first and third slot) Both will guarantee x16 speed.

@fabiosun I'm getting https://enterprise.proxmox.com/debian/pve buster InRelease' is not signed / unauthorized IP etc... when trying to upgrade - did the proxmox thing pasting in the following;- # PVE pve-no-subscription repository provided by proxmox.com, # NOT recommended for production use deb http://download.proxmox.com/debian/pve buster pve-no-subscription Any ideas, and does your proxmox versio show v 6.2? Edit: Sorted: Ran this through https://serenity-networks.com/how-to-update-proxmox-ve-when-you-do-not-have-a-valid-subscription/ ssh 'Shutdown -r Now' , Then ran Update off the Datacenter/PVE/Updates button. Sorted to Virtual Environment 6.2-4

Skylake 1080 ti vs the dual Radeon VII cards + Threadripper 3970X. I havent passed thru the cpu, nvme yet and USBs etc... just the GPUs. Looking on track though in initial tests - here Skylake versus Threadripper Ryzen 3970x. Disk SPeed plus 8K 12:1 RAW tests.

So with the new proxmox update we have greater than 6 maximum PCIE devices to pass through right? Is it okay to Update using the Proxmox web console Update button?

So its safe to upgrade now?

which boot flag? In proxmox or opencore or ? My boot screen only gets to "ramdisk..." too. even though it boots correctly, it would be nice to see complete proxmox messages!

Finally got the Radeon VII working on the host and using DP and HDMI together....Big sigh, next... all the other passthroughs....

Did you have your HD Audio Enabled in BIOS? or AUTO? ENABLED better ?

Its weird as I did note the Aquantia is showing up in Proxmox 'Network' under pve as vmbr0 'enp69s0' (Linux Bridge) so where has my Intel Nic gone?! Hardware doesn't seem to be able to see it. ±Okay after much re-reading and lots of stuff about nic commands being deprecated in Proxmox, I see you can show your nics by typing;- 'ip link show' in proxmox shell Whats 'UP' is live and active, what's 'DOWN' is not active. So it appears no. 3 option is DOWN and is the rogue corrupted nic. 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp69s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master vmbr0 state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff 3: wlp70s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000 link/ether 50:eb:71:77:e7:fa brd ff:ff:ff:ff:ff:ff 4: vmbr0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000 link/ether a8:a1:59:16:33:87 brd ff:ff:ff:ff:ff:ff Now just gotta work out how to change/rename/edit it....

±Nice work Rocket88 :-) Im having strange problems with my first network output. Its gone awol and I can no longer log in to the host root. I think somewhere along the line after a crash it went missing (or was renamed) sometime after by a really slow boot into Mac, followed by out of space error on my Proxmox lvm drive, so Im starting out all over again. Has anyone else found their first nic going missing? ie its not able to be ping'd by the Web interface, and the host doesn't see the web guest either. I used to log in with 198.192.1.134:8006/ on it, now when I reinstall proxmox the install autos to 198.162.100.2 - which is completely unuseable. So my only route into / from web interface is to install and connect to the 2nd nic (the Aquina one) now which luckily auto finds in Proxmox install the correct home network (198.162.1.134:8006/). I'm wondering how to write back the correct nic to the non-working port? I tried reflash my ASRock BIOS but its still the same! It al started when I was messing with passthrough of the Radeon VII card.... then not sure what happened! So wondering if through Proxmox I can bring my first nic back to its default etho0. Incidentally, the nic stiull lights up as though its communicating but that weird address can no longer join my home network ping to/back in any way.

Been a bit slow to get back to this project... Right.. errors > I'm getting a communication failure then host reboot ust trying to get Radeon VII passthrough. Did all the stuff in the guide about rerading in address of pcie for Radeon exactly. Getting log warning: STOPPED UNEXPECTED STATUS QMSTART error kvm: warning: host doesn't support requested feature: CPUID.01H:ECX.pcid [bit 17]kvm: warning: host doesn't support requested feature: CPUID.80000001H:ECX.fma4 [bit 16] followed by an Unable to Read Tail (got 0 bytes) status. Log shows May 09 01:10:51 pve systemd[1]: Startup finished in 34.590s (firmware) + 24.060s (loader) + 4.985s (kernel) + 9.133s (userspace) = 1min 12.769s.May 09 01:10:51 pve kernel: atlantic: link change old 0 new 1000May 09 01:10:51 pve kernel: vmbr0: port 1(enp69s0) entered blocking stateMay 09 01:10:51 pve kernel: vmbr0: port 1(enp69s0) entered forwarding stateMay 09 01:10:51 pve kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vmbr0: link becomes ready So cant seem to get into Catalina to see if the passthrough is working. Any ideas? If I remove the h'Hardware' passthrough pcie raadeon items I can boot back into Catalina so its a pcie radeon vii fault. ..summit I havent done right... do I need a ROM? Becaus ein Hardware it was saying NoRoM

Nice!