Rox67er
-
Posts
181 -
Joined
-
Last visited
-
Days Won
1
Content Type
Profiles
Forums
Events
Downloads
Everything posted by Rox67er
-
Well just took the plunge and tried to get my Proxmox Catalina install booting in bare metal. Used the @iGPU EFI and it almost immediately worked. Had to tweak above 4G to Disabled in bios, I also removed all the BRCM kexts as I replaced the internal wifi with an Apple compatible one. Seems I need to add MMIO whitelist as the system won't shutdown (Getting sleep to work would be awesome) my bootlog is attached below. (Above 4G disabled) 05:165 00:002 OCABC: MMIO devirt start 05:167 00:002 OCABC: MMIO devirt 0xB2500000 (0x81 pages, 0x8000000000000001) skip 0 05:169 00:002 OCABC: MMIO devirt 0xB3580000 (0x81 pages, 0x8000000000000001) skip 0 05:171 00:002 OCABC: MMIO devirt 0xE2100000 (0x181 pages, 0x8000000000000001) skip 0 05:173 00:002 OCABC: MMIO devirt 0xFA180000 (0x81 pages, 0x8000000000000001) skip 0 05:176 00:002 OCABC: MMIO devirt 0xFA300000 (0x100 pages, 0x8000000000000001) skip 0 05:178 00:002 OCABC: MMIO devirt 0xFEA00000 (0x100 pages, 0x8000000000000001) skip 0 05:180 00:002 OCABC: MMIO devirt 0xFEC00000 (0x1 pages, 0x8000000000000001) skip 0 05:182 00:002 OCABC: MMIO devirt 0xFEC10000 (0x1 pages, 0x8000000000000001) skip 0 05:184 00:002 OCABC: MMIO devirt 0xFED00000 (0x1 pages, 0x8000000000000001) skip 0 05:186 00:002 OCABC: MMIO devirt 0xFED40000 (0x5 pages, 0x8000000000000001) skip 0 05:188 00:002 OCABC: MMIO devirt 0xFED80000 (0x10 pages, 0x8000000000000001) skip 0 05:191 00:002 OCABC: MMIO devirt 0xFEDC2000 (0xE pages, 0x8000000000000001) skip 0 05:193 00:002 OCABC: MMIO devirt 0xFEDD4000 (0x2 pages, 0x8000000000000001) skip 0 05:195 00:002 OCABC: MMIO devirt 0xFEE00000 (0x100 pages, 0x8000000000000001) skip 0 05:197 00:002 OCABC: MMIO devirt 0xFF000000 (0x1000 pages, 0x8000000000000001) skip 0 05:199 00:002 OCABC: MMIO devirt 0x10000000000 (0x10400 pages, 0x8000000000000001) skip 0 05:201 00:002 OCABC: MMIO devirt 0x3CB90000000 (0x10400 pages, 0x8000000000000001) skip 0 05:203 00:002 OCABC: MMIO devirt 0x3CBC0000000 (0x10400 pages, 0x8000000000000001) skip 0 05:206 00:002 OCABC: MMIO devirt 0x69750000000 (0x10400 pages, 0x8000000000000001) skip 0 05:208 00:002 OCABC: MMIO devirt end, saved 1087664 KB 05:210 00:002 OCABC: Only 176/256 slide values are usable! 05:212 00:002 OCABC: Valid slides - 80-255
- 3,920 replies
-
- 1
-
-
- amd vanilla patches
- amd kernel patches
- (and 3 more)
-
@Rocket88 I have this exact problem also in my VM proxmox OSX. Switch off your power supply (mains switch) for 10 seconds and then try to boot again. I was not able to solve this but I think it is somehow related to the onboard RGB lighting on the asrock motherboard.
-
I didn't run into any issue yet. My lspci -nnk below. I only use 2 of the three nvme (one for osx other for Proxmox) and pass one sata controller. I hoped the short audio interrupts would reduce with fast NVMe storage for Proxmox (thought it might be latency) but it didn't help. 43:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation E12 NVMe Controller [1987:5012] (rev 01) Subsystem: Phison Electronics Corporation E12 NVMe Controller [1987:5012] Kernel driver in use: nvme 44:00.0 USB controller [0c03]: ASMedia Technology Inc. Device [1b21:3242] Subsystem: ASMedia Technology Inc. Device [1b21:3242] Kernel driver in use: vfio-pci Kernel modules: xhci_pci 45:00.0 Ethernet controller [0200]: Aquantia Corp. AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1d6a:07b1] (rev 02) Subsystem: ASRock Incorporation AQC107 NBase-T/IEEE 802.3bz Ethernet Controller [AQtion] [1849:d107] Kernel driver in use: vfio-pci Kernel modules: atlantic 46:00.0 Network controller [0280]: Broadcom Limited BCM4360 802.11ac Wireless Network Adapter [14e4:43a0] (rev 03) Subsystem: Apple Inc. BCM4360 802.11ac Wireless Network Adapter [106b:0117] Kernel driver in use: vfio-pci Kernel modules: bcma 47:00.0 Ethernet controller [0200]: Realtek Semiconductor Co., Ltd. Device [10ec:8125] (rev 01) Subsystem: ASRock Incorporation Device [1849:8125] Kernel driver in use: r8169 Kernel modules: r8169 48:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485] Subsystem: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP [1022:1485] Kernel driver in use: vfio-pci 48:00.1 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:1486] Kernel driver in use: vfio-pci Kernel modules: xhci_pci 48:00.3 USB controller [0c03]: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:149c] Subsystem: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller [1022:148c] Kernel driver in use: vfio-pci Kernel modules: xhci_pci 49:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] Kernel driver in use: vfio-pci Kernel modules: ahci 4a:00.0 SATA controller [0106]: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] (rev 51) Subsystem: Advanced Micro Devices, Inc. [AMD] FCH SATA Controller [AHCI mode] [1022:7901] Kernel driver in use: ahci Kernel modules: ahci 4b:00.0 Non-Volatile memory controller [0108]: Phison Electronics Corporation Device [1987:5016] (rev 01) Subsystem: Phison Electronics Corporation Device [1987:5016] Kernel driver in use: vfio-pci
-
What was the issue?
-
I didn't experiment with Big Sur yet but are the patches still the same for the new kernel? Found this on Nick Sherlock's Big Sur page, seems to confirm my statement: "The CPU is set to pretend to be Penryn, because the macOS kernel patch that would support -cpu host has not yet been ported to Big Sur. Ensure the args are all on a single line!"
-
@iGPU Be aware that the kernel patches section (in OC) play an important role in being able to set "host" as cpu. I have these two active and I'm pretty sure if I leave them out also Catalina won't boot with "host" setting.
-
yep, I see higher clock and also higher total power. Was around 230W initial, close to 280W with default setting and up to 330W with +200Mhz PBO overclock
-
Well after trying all PBO / overclock settings in bios without result (steady 3.7ghz on all cores) I installed Ryzen master and after one reboot I now see switching frequencies on all cores. Ranging from 3.4 to 4.2ghz. After this also the bios settings seem to work normal now... anyhow temps stay well under 75 degree and even below 85 with aggressive overclock (330 W)
-
Just set PBO to enable but still at 3700MHz ? 😲 Will look into this if I have some time.
-
I booted up Win10 and ran 5 min of OCCT. 100% load but somehow all my cores run @ 3.7Ghz , need to check that. Anyhow temps max out at 72C I run the 360 AIO from Cooler Master: https://www.coolermaster.com/catalog/coolers/cpu-liquid-coolers/masterliquid-ml360-rgb-tr4/ Noise level is very low.
-
Great to see that this path is still possible. 😁
-
thx for trying. macos is picky on topology but I would say if we pretend to be xeon W it should work as that is also in 28 core version..
-
Did anybody find a way to pass 28 cores / Xeon W as the original MacPro 2019 has? I can only boot 28 core with a very odd topology: -smp 56,sockets=7,cores=8,threads=1 Displays as 7 times 8 Core intel core i5 in about this Mac.
-
Last week I replaced my external USB Proxmox drive for an extra internal M2 drive. I bought a XLR8 CS3030 1TB and installed a fresh Proxmox 6.2 on it. Went pretty smooth just copying the configs from my "old" setup. Anyhow everything works as expected again but as expected it didn't help with the USB problem. Some new insight in trying to trouble shoot the USB passthrough problem on 48:00.1 / 48:00.3 while doing some tests. Fact: We are unable to pass the USB controllers but if we only pass the BT USB everything works fine. I have noticed below error (USB passthrough) sometimes when booting Proxmox which after starting the VM always leads to a freeze / 99 error. Somehow linux is trying to initialise the LED controller and USB Audio (On 48:00.1 controller) but cannot access them... So I tried the following: Next to BT I added the USB audio AND LED controller to my VM with: usb0: host=05ac:828d # Bluetooth usb1: host=26ce:01a2 # LED controller usb2: host=26ce:0a01 # USB Audio Booting VM is OK but reboot would hang, so this gives me more or less the same behaviour as when passing the complete controller. Next step I removed the LED controller and have been able to restart the VM a few times. I think it may be related to going from linux to MacOs, back to linux and back to MacOs again... Also the LED and Audio are not recognised in lsusb Bus 009 Device 002: ID 26ce:0a01 Bus 007 Device 006: ID 05ac:828d Apple, Inc. Bus 007 Device 003: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth) Bus 007 Device 002: ID 26ce:01a2 Strange thing is that I disabled the LED in USB but apart from switching off the LED, the LED controller remains visible on USB. Would be nice if we could disable it at least to try if that solves the issue. I tried blocking linux from accessing it but so far not successfull. (Even with VFIO added, linux still boots with the xhci_pci kernel driver. (When starting the VM it is released and attached to vfio)
-
Most of the info is in my overview: https://github.com/iroger/ASRock-TRX40-proxmox/blob/master/Ryzen IOMMU.pdf (2nd page) The port 11 is a USB 3.2 gen 2 which I have connected to front USB-C. It is located on 48:00.3 (It is listed on 47:00.3 in the overview but I added a NVME 1TB drive for Proxmox)
-
Yep, it can be confusing but on old Mac Pro 5.1 I remember you had to connect a separate USB cable to get bluetooth working so I knew it needed a pci port for wifi AND a USB port for bluetooth. Mine lists as manufacturer Apple Inc.
-
@Driftwood It should work "out of the box" I have NO Broadcom kext loaded. Only LiLu / WEG / Virtual SMC / AppleALC. blacklist.conf added: blacklist btusb blacklist bcma vfio.conf added: # WiFi PCI broadcom options vfio-pci ids=14e4:43a0 In your VM forward: hostpci: 45:00.0, pcie=1. #This will give you the wifi Bluetooth is connected on USB so make sure you either forward the USB device or completely pass the USB controller to your VM (it is connected on 47:00.1 USB controller) lsusb gives you the BT info: Bus 007 Device 003: ID 0a5c:4500 Broadcom Corp. BCM2046B1 USB 2.0 Hub (part of BCM2046 Bluetooth) If you don't forward the USB controller use the hardware passthrough in Proxmox to select the right device, lsusb somehow gives other data?
-
Have been trying the suggestion of @Driftwood regarding the XHCI biosd setting, found this link also suggesting to put them on "enable" in stead of "auto". https://forums.unraid.net/topic/87557-guide-asrock-x570-taichi-vm-w-hardware-passthrough/ I do see effect, after adding the setting I now am able to boot MacOs up to the login screen, and if I then shutdown and re-start the VM, all is fine. If I however login to MacOs and THEN shutdown I still run into the 99 error after second reboot. Have been trying other options (CSM / Above 4G / ...) but did not find a combination that worked...
-
@Driftwood from your VFIO I see you are passing the 05:00.3 and 25:00.3 USB as well? Are you using a patched kernel for this?
-
Well only the two in the same group (47:00.1 and 47:00.3 in my case, I think you have 46:00.1 and 3 because you disabled internal wifi) I haven't tried XHCI USB settings. Maybe will give that a try. Thanks for the hint iGPU, I use a Corsair HX1200i but I did have an issue with the supply making strange static noises. (as if it was sparking) It did go away though and I checked all voltages and they seem OK. I don't have a spare on hand but will think about this.
-
@Driftwood well still alive. Been having some short nights sleep over this USB issue. 🥴 I went back to using the USB-C port for my USB connections. I cannot get reliable boot with the internal ports passed through. Went to MacPro 7,1 bios but no improvement, changed the order of hostpci (which does change things in the VM, I had to re-assign the built-in property) but also no improvement. Spent hours tweaking bios settings. No improvement. I still run 1.63 beta bios which seems fine. I think the improvement I had was coming from the bios upgrade where it resets all settings. (Not the same as load defaults in bios) I did a HW bios reset (button on the back) and got a few proper boots with USB passed through then get 99 error again every 2nd reboot. It seems like something gets written in bios which doesn't clear. I can tell if the reboot will work or not by listening to the GPU fans spinning up. If they only have a short whoosh everything is fine, if they already start spinning at the E0 Dr Debug code they spin longer, will boot ok but next reboot gives 99 error. 🤯
-
I found something interesting regarding passing all cores to VM. When passing all 64 cores to OSX VM I noticed a short audio dropout when playing Spotify and in parallel starting up Fusion 360. I also saw the Proxmox disk led blinking indicating some activity. So I tried the same when only passing 32 cores to OSX VM. I started and stopped Fusion 360 several times and don't have one audio dropout. So there definitely is some impact of passing all cores to the VM but I can't figure out what Fusion is doing to cause this. I tried passing 56 cores but MacOs won't start. Seems only 32 and 64 work. Did anybody get other # cores working? How?
-
Well I did run into the 99 error again today, I didn't change anything though. Still needs more testing I guess. I tried performance run with Above 4G on / off results below (Geekbench 5 CPU and GPU Metal score) Above 4G OFF 1282 / 23881 / 61463 (average of 4 runs) Above 4G ON 1282 / 23714 / 60924 (average of 3 runs) I would say no difference in my setup.
-
@Driftwood @fabiosun Well close but not completely. After reading your post I jumped in and started tweaking the bios again.... Using the above settings crashes Proxmox on boot but I noticed that when enabling AER Cap a new option showed up in bios which was disabled. After also enabling that one I can stop-start-reboot and get no 99 error. Tried it now for like 8 times... 😁 So my current bios settings for ASRock TRX40 Creator: (Still using Bios 1.63 but expect no impact from that to be honest) * Start with default UEFI settings * Set XMP profile for your RAM * Tweaked FAN settings * Advanced>Onboard Devices Configuration>SR-IOV Support>Enabled * Advanced>AMD CBS>NBIO Common Options>IOMMU>Enabled * Advanced>AMD CBS>NBIO Common option>Enable AER Cap>Enable * Advanced>AMD CBS>NBIO Common option>ACS Enable>Enable I did not need to set above 4G or anything else. Let's see if this is stable but looks like we nailed it !!! 😎
-
My IOMMU attached. @Driftwood I already assumed he didn't use Proxmox I think it is kernel related how the devices are recognised. (Naming and drivers) ASRock TRX40 IOMMU.xlsx.zip
