meina222
-
Posts
449 -
Joined
-
Last visited
-
Days Won
5
Content Type
Profiles
Forums
Events
Downloads
Everything posted by meina222
-
I haven't had time to do much research on this topic, but here a few things that I came across: 1st check: https://www.kernel.org/doc/Documentation/vfio.txt In particular this section: This device is behind a PCIe-to-PCI bridge [4]_, therefore we also need to add device 0000:06:0d.1 to the group following the same procedure as above. Device 0000:00:1e.0 is a bridge that does not currently have a host driver, therefore it's not required to bind this device to the vfio-pci driver (vfio-pci does not currently support PCI bridges). When I did lspci and checked my TB group id, and then did ls -l /sys/bus/pci/devices/[id]/iommu_group/devices the structure was very similar to what's described there. Essentially the 'leaf' devices are behind a PCI bridge. I don't have the Proxmox console handy now but I will paste the result of this later tonight. I don't think I am saying anything new here - still worth trying and see what happens. I think it would help to reach out to Alex Williamson who is the maintainer of VFIO in the linux kernel https://www.linkedin.com/in/alexwilliamson He might provide helpful tips, or simply tells us not to waste our time as it is not supported. Or this could prompt someone on the Linux kernel team to look into emulating PCI bridge devices. At any rate, he seemed helpful on a reddit board where a user tried to achieve a simpler goal - passthrough a leaf PCIE video card through thunderbolt to his laptop. As you can see this already turned out to be non-trivial at the time You can read the full thread here and see that he provided input:
-
@fabiosun - thank you for your help. It's difficult for me to test this without doing a clean install of Proxmox as I added these packages over time and some might be missing from my list of pre-requisites. Could you check if you have dpkg-dev installed via "apt list dpkg-dev" ? If not do "apt install dpkg-dev". Then retry build via "make clean && make"
-
I made the Proxmox experimental branch with kernel 5.8 available on my personal github https://github.com/mdjabirov/pve-edge-kernel/blob/linux-5.8/README.md I also added instructions on how to build it yourself. If someone has enough diskspace (10-20G for the compiler cache) on his/her Proxmox host, and would like to go through the exercise I'd appreciate it, so I can see if it works and fix any omissions in the steps described. I will not publish official releases. Any official .deb files should be downloaded from Proxmox or Fabian's repo. If you choose to try it, you should download and build it yourself. I can add instructions on how to add patches (such as the AMD reset patch) if anyone is interested.
-
@fabiosun that seems about right - this is what I did too. Try following the steps I posted to purge the 5.8 kernel. I am doubtful this is what's causing issues but one never knows - if your High Sierra stops showing odd issues, it might be. There will be plenty of new versions of 5.8 coming in the weeks ahead to fix issues so I will make builds as they come online. After a cold reboot my sleep is finally working. I suspect that some some device went into a power state that prevented from sleeping - USB controller or other. 100% sure it's the motherboard / devices and not Linux itself. FInally I can try the rtcwake hook - with 5.4 I would not wake up from sleep for reasons unknown.
-
Mine shows. How did you apply it to begin with? And did you try to remove it afterwards? Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. root@designare:~# pve-efiboot-tool kernel list Manually selected kernels: None. Automatically selected kernels: 5.4.44-1-pve 5.4.44-2-pve 5.8.0-1-zen2 root@designare:~# I'd remove by: 1. Select lower kernel at boot 2. list existing edge kernels dpkg --list| grep 'pve-edge*' 3. apt purge -y [the kernel you want from lust in 2] e.g. in my case pve-edge-kernel-5.8.0-1-zen2 Check that the headers are also removed: Run 2. again. If you see pve-edge-headers-5.8.0-1-zen2 remove it with apt purge
-
For the record - the AMD reset bug is still pretty much there in 5.8 - as expected. That sleep issue is preventing me from using the rtcwake trick though. I am starting to think how to engege Gigabyte support as my Ryzen 3700X X570-I sleeps fine with Ubuntu on 5.8
-
@fabiosun apt purge -y [kernel-to-remove] should work. You need to make sure you boot into a different kernel by enabling some timeout in grub so you can select advanced options. Let me spend some time on 5.8 before I can recommend it. It's a bit dangerous as Proxmox is not certified with it. I don't have any critical data connected to it.
-
@tsongz - can you check that wireless of broadcom is working in say Windows if you have one? Just to rule out some card issue. Wireless "should just work". The other step would be to check the IOMMU trees as @iGPU suggested. Let me know how it goes - I was hoping it would be more painless when I recommended that card, but it was BT indeed that was a pain for me. Installed 5.8 + Beta 4 last night. All good. I rebuilt the pve-kernel image with proper ZFS patch - will post that on github over the weekend. Means I feel a bit safer to use patched official release of ZFS that only addressed the 5.8 incompatibility. I was hoping 5.8 would fix sleep issues, where my host would fail to sleep with systemctl suspend or rtcwake -m mem. I doubt it's a Linux issue. I think Designare TRX40 has some BIOS issues that prevent proper sleep support. The AE_ALREADY_EXISTS messages at boot also didn't disappear. Gigabyte hasn't updated their BIOS since early March, but they rolled out a rev. 1.1 of their TRX40 boards with new BIOS.
-
Just upgraded to Big Sur Beta 4. Took really long. It succeeded and booted successfully, but MacOS setup assistant failed miserably. The whole thing is not very smooth still. Also my EFI is still saying 'Preboot' even though install seems to have finished. Once I login things work. Unlike BS beta 3 where I had to reinstall from scratch. So far setup assistant never succeeded for me.
-
You may also need to clear the network preferences plists. Don't recall which ones - google them. Wireless should work - this was the less problematic part for me. Maybe try this https://apple.stackexchange.com/questions/284581/wipe-all-wifi-settings-from-macbook It's OK for Hackintool to show AX200. You didn't unplug it. But MacOS should not load the BT driver for it. If you really want it fully removed you either can try BIOS or if not possible there (in my Designare that option doesn't exist) you can remove it out of its slot, but I'd leave it.
-
I almost gave up on the PCIe card as I couldn't figure how to get rid of the built-in one. Setting sudo nvram bluetoothHostControllerSwitchBehavior="never". and clearing the files containg "Bluetooth" in ~/Library/Preferences/ByHost /Library/Preferences to get rid of MacOS cached stuff did the trick for me. The command above disables the AX200 and allows for the PCIE to load. My AX200 was covered by a shroud and I didn't want to try and unscrew the board to remove it, but luckily above worked. Nothing else would work if you can't disable the AX200 in BIOS.
-
If things look stable I wil create a githubclone and update the 5.8.xx kernels and ZFS releases as they come online. I need some time to structure the whole thing as modules so that we can automatically pick up any changes from pve or linux or zfs. I'll try to put a README that will allow for anyone to build the packages. This is going to be until Proxmox migrates their edge github to 5.8 themselves which may take a while.
-
You shouldn't need to black list it. "built-in" should be your ethernet en0, not the broadcom. Your devices should look like this en0 (built-in ethernet), en1 (wireless) en2 (bluetooth) (you can swap bluetooth and wireless but not ethernet - must be en0) Get Hackintool and verify on the "Peripherals" tab under your System selection Should show something like this for ethernet (this shot misses bluetooth and wireless) - this older gui, it's called Peripherals now, not Misc Depending on what order your add the ethernet and bluetooth and what PCI they get assigned, the ethernet may jump to en1 or end2 - you need to find an order where it's en0. Then add built-in. I followed the OC guide - https://dortania.github.io/OpenCore-Post-Install/universal/iservices.html#generate-a-new-serial
-
What I think happened here is as follows: By adding the broadcom card, Promox re-assigned the interface ID's othe the ethernets. Say enp68s0 became enp69s0. Now his /etc/network/interfaces is invalid. And he runs a script at logon to prevent the host to use the single video card. @tsongz - unplug the broadcom, remove any logon scripts if you manage to regain access and then plug the broadcom again - this time around access the host direct and fix your network bridge.
-
@tsongz I am confused. Why can't you login direct to host and manipulate cmd line? Are you running something in your root ~/.bashrc that turns off the video output as soon as you do?
-
With blacklist or not, you should always be able to boot into the video output of the host. With blacklist you will get default resolution maybe larger text. Don't reinstall Proxmox! You just need to double check the contents of etc/network/interfaces I agree with @fabiosun - no need to blacklist. Blacklisting is only useful if you boot headless. Easiest is to have 2 video cards - one for host (I have old GT 240) and one for guest - your Vega 64. But to fix connectivity you should always have access after reboot to host via console regardless of blacklist.
-
That too. But also you'd lose host connectivity if you pass the ethernet that happens to be associated with the gateway. In my example above that would be the card associated with enp68s0. If you have 2nd ethernet you should pass the other card or swap the one that is connected to your gateway. The ASROCK has 2 ethernet cards. Aquantia and Realtek. Connect both to your router, make sure your vmbr0 (or 1 if you have 2nd bridge) has the card that you plan to keep for the host associated with the gateway IP, and identify the PCIE id of the 2nd card and pass that through. If you want to keep the Realtek to host, make sure it is configured as the bridge-ports device for the host/vmrb0. The Aquantia will be the 'free' one For more details read this: https://pve.proxmox.com/wiki/Network_Configuration
-
My Proxmox host is connected via cat cable on both Ethernets. Do you have a video card connected to the host or are you running 'headless'? If the former, you can login in the console directly to host and check the contents of '/etc/network/interfaces' e.g. less /etc/network/interfaces I have 2 interfaces - enp70s0 and enp68s0. I also have virtual bridge vmbr0 which acts as a switch connected to my router - gateway belowvia enp68s0 and this gives me access to the host via the internet. # network interface settings; autogenerated # Please do NOT modify this file directly, unless you know what # you're doing. # # If you want to manage parts of the network configuration manually, # please utilize the 'source' or 'source-directory' directives to do # so. # PVE will preserve these directives, but will NOT read its network # configuration from sourced files, so do not attempt to move any of # the PVE managed interfaces into external files! auto lo iface lo inet loopback iface enp70s0 inet manual iface enp68s0 inet manual auto vmbr0 iface vmbr0 inet static address [IP redacted] gateway [IP redacted] bridge-ports enp70s0 bridge-stp off bridge-fd 0
-
@tsongz - on GPUs (which I haven't had time to benchmark and configure much) - try adding only Whatevergreen kext and see if that bumps your benchmarks. I don't fully understand the implications of these kexts and SSDT to ACPI power management and framebuffer interactions - especially in VM setting. I think the answer can only be found via testing and experiments. I am not sure if anyone can recommend a setup for that. I have a 5700XT. Last time I benchmarked it scored only 43K-45K in Geekbench 5, but also I am not sure the boosts some people report in baremetal are real - e.g. would you actually see better rendering performance in real life apps, or better power consumption. Espeically in VM that is tricky and needs measurement.
-
Re: GPU benchmarks. The only wait to find out is to re-enable them and try again. Re: memory warnings. Are you referring to MacPro7,1 memory misconfiguration popup? There is a kext for that. I think it's called macpromemomrynotiticationdisabler.kext. The way I avoid this error is by giving 192GB to my VM which at 12 vritual dimms each of 16GB populates all slots of the MacPro so it stops complaining, but if you don't have this amount of memory I guess the kext woudl do. Never tried. No expertise on TB3 - I think @iGPU would comment on, but I think you can't reuse a baremetal SSDT. There seems to be also some fundametal issues about how the PCIE bridge is passed or not. Why don't you try it? This is beyond my expertise atm, but interested in at least passing through a leaf device via TB3
-
Looks good. Just wanted to make sure didn't mess something up.
-
Ok I hope so :). You are now running Proxmox with the kernel dubbed the "the biggest release of all time" by its author https://www.zdnet.com/article/linus-torvalds-linux-kernel-5-8-is-one-of-our-biggest-releases-of-all-time/ Don't expect a magic bullet on AMD reset or TB3 controller VFIO though. I really want to find the time to take a serious look at these. I think the best bet would be to reach to some of the VFIO kernel developers for clues on understanding the current limitations. Do me a favor - check what version command "uname -r" outputs to verify the kernel version
-
Ok. Here is the link. https://mab.to/Kmgg1Igar It contains all .deb files output by the build. You only need to apply the kernel and kernel headers (pve-edge-kernel and pve-edge-headers). Not that I HAVE NOT TRIED it yet. Up to you if you want to hold off another day so I can have time to test it tonight. This is highly experimental and likely Proxmox team haven't tested it themselves yet. I would also be cautious about using ZFS volumes - likelihood of data corruption is low as there many unit tests in the git repo, but this is not built from an official tag - it's development version of latest ZFS git branch. Proxmox applies some patches on top, and I disabled one as it was failing - related to python version scripts (I don't believe it's essential but haven't tested it).
-
Ok I registered. How do I share the file?
-
Thank you. I did build the pve-kernel (5.7.8) with gnif's AMD patch before from Fabian's pve-edge-kernel git. DIdn't work for me. macOS would still not reset. I know the patch was applied correctly - t I did see the card being power cycled and the LED light going off with a click on shutdown. But upon restart the Apple logo would take forever in progress bar. Something with the PCIE bus wasn't properly released. Ultimately I gave up the patch. There is slim hope things might look better with 5.8. I will try patch-less 1st, then add patch if it's not good.
