Driftwood

4 hours ago, cj750 said:

From follow the post. It best to install proxm. On to a SSD and to also leave the iso on them. And leave my M2 drive as scratch drives to run other programs? Am I correct in this logic of setting this up. To use my

Yes as @fabiosun says. Install Proxmox from a bootable USB stick to a small old SSD  for booting the Proxmox host server, storing your ISOs and vm configs. SSDs are more reliable than using a USB stick to install onto. You will attach this drive to the rear Admedia USB C port.

You may have to buy a SATA to USB C adapter. They're easy to find on Amazon. I used my laptop one. 

Then install Mac (use the 2gb DMG install Mac OS .dmg) onto internal SSD on one of your SATA ports or if you have an M2 drive or two you can use one of them.

Then use your spare m2s as data drives for the serious number crunching work like audio and video data!

2 hours ago, Rox67er said:

regarding Radeon VII reset bug, I found this post on a huge number of AMD pulls to kernel 5.5 (Proxmox is now on 5.4) so we may see some improvement to the GPU reset issue on next Proxmox version. https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.5-AMDGPU-More-Features

Sounds promising. We wait in anticipation.

sata port recognition is a joke. I plug in another drive on port 5 or 6 and the ata numbers change - including the proxmox boot on port 1. Random.... left pic with the extra drive, shutdown, pull it off and restart dmesg and the proxmox boot has change ata number (right pic). Guess Ill have to use disk-by-device...

18 minutes ago, fabiosun said:

Are you sure about this?

Yep. I went thru a complete USB/passthru check last night. I even had to correct some Pavo vfio addresses so I was glad I did. 

I'm 100% stable without 47:00.1 and 90% with it enabled (the odd audible crackle even though audio is not addressed in that space). If I enable 47:00.3 (with audio ports) I'm 60% stable and with mouse lag - even though its not on that port!

I'll upload my rear panel diagram shortly.  

4 hours ago, fabiosun said:

Driftwood47:00.3? Don’t use it?

No thats the USB Audio ports. I dont need it - when I do the mouse drag and crackle is worse for my desktop. I have tons of pro audio inputs/outputs on my RME Fireface 800 card connected to the firewire card. Plus 64 I/Os with the Decklink. 2with my DELL HDMI/DP monitor... so Im spoilt!

By the way @iGPU I did a Migration Assistant on my build after install and didn't migrate any of the network stuff (left uinchecked) which would screw the build.

ODDITY
What's weird is, my mobo wouldn't even display sleep on fresh install, now post Migration Assistant  it sleeps fine - going from screensaver to display sleep with wake on mouse/keyboard movement without problem.

Heres my config table for the ASRock Creator TRX40 running Catalina 10.15.5 @fabiosun

The only BIOS setup I use is CSM = Enabled for Proxmox install, and then Disabled with Above 4G checked when Im installed in Catalina.

* Note: You only need to passthru  one copy of the Starship USB 3 address (1022:148c) in your VFIO.conf

** I'm using two MP600 m2 drives on the Phison controllers

Im getting ready to pass SATAs tonight. There appears to be two sata controllers: So I need to discover which controller/address set handles port 1-4 & port 5-8. If I can pass the secondary controller I'll be happy to use the first as boot Proxmox.

1 hour ago, iGPU said:

Tonight I'll pass both GPUs. Instructions on most TRX40 mobos are slots 1 & 3 (as these are 16, not 8, slots) and then monitor for stability.

 

Thats correct. I get no problems with dual GPUs on 1 and 3 with x16 speeds. Everything is stable if I remove the Matisse 47:00 USB/Audio from passthru. Interestingly 47:00.1 is just the rear USB ports and a LED controller (which I don't use). The Matisse USBs on 47:00.1 very nearly work - but I get the occasional crackle in audio so its definitely not happy with firewire co-existence.

The firewire is best on lane 2 and Decklink on lane 4 (all SDI ports working) which only require x8 speeds. And this is what I liked about the ASRock Creator - the PCI lanes are evenly spaced apart with no 'touching' of the firerwire/Decklink to the GPUs, whereas other boards (inc expensive ones) lane 2 & 4 seem too tight to lanes 1 & 3.

1 hour ago, iGPU said:

ArcoLinux, which possibly flashed AMD microcode

Interesting stuff this microcoding the cpu. Did you modprobe cpuid to check if your cpu was?

Detecting available microcode update

It is possible to find out if the intel-ucode.img contains a microcode image for the running CPU with iucode-tool.

1. Install intel-ucode (changing initrd is not required for detection)
2. Install iucode-tool
3. Load the cpuid kernel module:

   # modprobe cpuid

4. Extract microcode image and search it for your cpuid:

   # bsdtar -Oxf /boot/intel-ucode.img | iucode_tool -tb -lS -

5. If an update is available, it should show up below selected microcodes
6. The microcode might already be in your vendor bios and not show up loading in dmesg. Compare to the current microcode running grep microcode /proc/cpuinfo

21 minutes ago, iGPU said:

So while the CPU is working better with Proxmox, macOS is not stable, but I think another issue. If any apps are run (Safari, Hackintool, etc), the OS freezes after a few minutes; and this is running under console, no GPU pass-thru. I will do a fresh install of Catalina and see how that behaves.

 

Ensure GPU (Radeon VII) is on PCIE lane 1 (16x), try NOT passing thru the (47:00.1/ .3s USB or whatever your USB/address with audio shared on - Starship/Matisse), my Mac crackles and is unstable a little after passing that one thru. Just use Asmedia and the other Starship USB.

Then tell me if you have improvement. 

17 hours ago, iGPU said:

What I still don't understand is why my particular CPU was originally working on Proxmox but then gradually became unstable to the point that I couldn't run Proxmox (and I've tried their latest v6.2).

I had similar problems with Proxmox, relating to Ethernet (and wifi before pulling the BT wifi) and getting Pavo to build a proper Dragon / Realtek 8125 driver for linux.  We noticed  the install of Proxmox 6.2 sometimes had issues and going back to 6.1 then updating to 6.2 later worked.

BIOS should be set to pretty much default UEFI settings on install. Later you can add things like disable CSM and Enable Above 4G.

3 hours ago, fabiosun said:

If your system is stable why do you want to change it?

I haven't used any of the SATA ports yet. So Im thinking of passing them all through like yours. Probably going to put a few ssds on the SATAs as it seems a shame to waste them.

My 8Tb\10Tb 'spinners' are on an external QNAP TR04 box thru the Asmedia USB port.

The only drive currently on the SATA port 1 is the Proxmox Host.

@fabiosun said: "...disconnect your proxmox disk and install on a new usb one (I use an external case for a Samsung 840 pro ssd sized 240 Gb)"

I guess I could just move the Proxmox host ssd on SATA port 1 and attach it to the rear usb?

21 minutes ago, fabiosun said:

OT (about Unraid)

to test I have built an Unraid test OSX VM, I did to verify if in there I have a better HighSierra support..it is not 🙂

So Unraid isn't going to be specifically any better than Proxmox on these trx40 Mobo chips too?

My system is stable but am probably going to use a USB stick to boot it now.

@All is there a quick way to back up my working  4gb sized ssd proxmox boot to a bootable USB?

If so, please type FULL & EXACT details below.

This will help anyone following this thread 

Thanks

15 minutes ago, cj750 said:

What card are you using? I must of missed it in the past post

2 x Radeon viis (xfx). Only the first one controlling OS and Display, second one for encoding/Davinci/fcpx

20 minutes ago, fabiosun said:

Apple has worked for me 🙂

catalina 10.15.5 solved my audio problem

Thats interesting. 8K worked for me with the new Cat.

@fabiosun what about USB - all working? NO lag?

8 hours ago, fabiosun said:

are not informed well😀

Audio is working well in latest Catalina and it is not flr patch related

Yeah that was a few days ago. I know you and Pavo have been looking into audio issues together. I saw he had it working initially when he streamed live but there were crackling issues, I'm sure you both have sorted it by now. You've both been working on it, I understand. 🙂

2 hours ago, cj750 said:

Since my system is closer to yours. Do you feel it stable with your setup. Any new issue pop since you been running it for a few days. Any changes?

Nope. All good so far - pondering whether to switch in the SATA's now so I'll update you shortly. Even ran migration assistant on the USB-C rear in record time! 2TB done inside an hour - now you know how long MA normally takes, it waits around for ages checking things - its not really so much hard drive/net speed - its processing/checking off things. This was real quick.

8K Update: Managed to get the Dell 8K monitor working in 8K Catalina 10.15.5 with ResXtreme. My god it was soooo tiny that I had to scale.

5 hours ago, iGPU said:

2. PAVO's site is for the X570; are modifications needed from his download for the TRX40?

That's Pavo's old system. He only just bought the msi. Expect an update from him when he gets the chance but he's busy trying to get audio working on the new trx40 msi mobo.

@Rox67er The two ASRock Creator SATA controllers (8 x SATA3 6.0 Gb/s) which, out of the two FCH address - 48:00 & 49:00 - is the first four?

The reason I ask is I forget if you are booting proxmox from a SATA port or USB? 

I currently have Proxmox booting off SATA port 1 on a SSD.

 Im tempted to go USB (as @fabiosun has successfully done)  but would prefer to boot with SATA from inside the box like Im doing. I currently don't passthrough either of the SATA controllers. I don't mind losing half the controllers to proxmox ssd, but which controller is using which IDs? Is 48:00 ports 1 - 4 ? And 49:00 Ports 5 - 8?

I could do with using one of the controllers I guess.

2 hours ago, iGPU said:

It seems there is a problem with the CPU

You obviously check you had the right amount of cpu thermal grease on it?

2 hours ago, Rox67er said:

I can confirm that IF I pass any of the two (or both) USB controllers (47:00.0 / 47:00.1 / 47:00.3) my system will freeze after shutdown in MacOS and then restarting the VM (even with the GPU reset hook script)

 

I have now removed the passthrough of the USB controllers completely (only use the USB-C rear) and everything works prefect. Also the GPU reset script now works fine so stop-start VM works fine.

I suspected that I might have to do this the other day when talking to Pavo. But after talking, I decided first to get the rear ASMedia USB controller passed through on 44:00 so I was able to use the rear 3.2 gen 2 port - couldn't use it before I passed it which is strange. So are you saying if you remove ALL USB controllers that the rear 3.2 gen 2 USB C connector works without passthrough? Ive actually ordered a Qnap TR4 NAS/Singles USB-C 3.2 gen 1box to stick my spinners in, which is 10Gbs. They also have this https://www.qnap.com/en-uk/product/tl-d800c coming out imminently giving support for USB 3.2 gen 2x2 speeds (20Gbs 8 drive box).

So I might try and go back and un-passthrough the USB controllers/ports in my current config. And use the rear USB C too.

Also I've got plenty of USB-C multi-port adapters from my MBP lappie so all is cool.

Good confirmation, thanks Rox67er.

40 minutes ago, iGPU said:

So I have a bad CPU. Interestingly, it started out okay, but then gradually got worse and worse. Fortunately, I had 3 days left to do an exchange, so I'll get a replacement 3970X

Good luck iGPU with the new cpu dude. 

Hang on our shutdown error is GPU based of course... so... Which Radeon vii card u got? Looks like only the expensive Sapphire one is the only one which works properly in Mac/hax. Need to see if Sapphire ROM can be flashed to it in Windows. Did you try that already?

3 hours ago, fabiosun said:

reflash your bios and clear CMOS

I experience the same problems as Rox67er on the reboot/shutdown issue - I often have to switch off, pull power and sometimes I might wait 10 seconds before repowering/reboot in.

Basically the ASRock Creator seems to have problems with restart from Mac or a shutdown that the MSI board doesn't experience. Unless Fabiosun you have seen this too?

4 hours ago, Rox67er said:

there a reason for the DSDT in your opencore

You mean SSDT?