Gigabyte Titan Ridge on Proxmox/OSX baremetal (WIP)

[iGPU]

5 minutes ago, fabiosun said:

I have this

 

without any ssdt and only with NV23 Elias firmware

 

 

It needs an SSDT for function. NVM23 should still work.

 

[meina222]

Will step away for a few hours. I cleared CMOS and will reconfigure TB from scratch (won't reflash yet) to figure how to get my display going. If there's any extra files you want me to share I will check back in a few hours. Hope you find out how to fix the USB in that DSDT!

 

Update: after I switched the Display Port used for the bridging cable on my 5700XT, I can now boot and have my display working in TB. Weird, I think this is a GB bug. Will happily take one port for now as I don't use 8K. So bridge from DP 1 on 5700XT to mini DP1 on TB causes BIOS 61, but if I bridge Navi's DP2 to mini DP2 it works. Oh well. Maybe this is a quirk of the 390 NVM33 ROM. I guess can't have it all perfect.

Edited September 6, 2020 by meina222

[fabiosun]

1 hour ago, iGPU said:

 

It needs an SSDT for function. NVM23 should still work.

 

Hi know

this day I have put my card on a x299 system i see without any ssdt hs01 hs02 ss 01 and 02 maybe sp like meina i mean

usb works if connected on cold boot

i would like to reach the condition only to see that devices in usb thunderbolt three

i understand we may need of a dsdt job

 

[iGPU]

1 hour ago, meina222 said:

Will step away for a few hours. I cleared CMOS and will reconfigure TB from scratch (won't reflash yet) to figure how to get my display going. If there's any extra files you want me to share I will check back in a few hours. Hope you find out how to fix the USB in that DSDT!

 

Update: after I switched the Display Port used for the bridging cable on my 5700XT, I can now boot and have my display working in TB. Weird, I think this is a GB bug. Will happily take one port for now as I don't use 8K. So bridge from DP 1 on 5700XT to mini DP1 on TB causes BIOS 61, but if I bridge Navi's DP2 to mini DP2 it works. Oh well. Maybe this is a quirk of the 390 NVM33 ROM. I guess can't have it all perfect.

 

I found some interesting things and posted here.

5 minutes ago, fabiosun said:

Hi know

this day I have put my card on a x299 system i see without any ssdt hs01 hs02 ss 01 and 02 maybe sp like meina i mean

usb works if connected on cold boot

i would like to reach the condition only to see that devices in usb thunderbolt three

i understand how his need maybe a dsdt job

 

 

Send me the X299 IORE file and I'll make one for you. I also have X299 system, using ASUS Prime Deluxe (1st edition).

 

Edited September 6, 2020 by iGPU

[fabiosun]

Sorry I have tested only if my card was good 

x299 has a flashed card bought on eBay and it uses nv23 Elias firmware with a Casey’s ssdt I think

 

i have to leave away some doubts I had in my testing

[meina222]

@iGPU,

 

You were right. USB does indeed work on TB, just didn't find it quite where I expected it (not on the DSB subtree).

 

Also, the NVM33 ROM works in Linux, but you need a newer kernel - 5.4 doesn't like it, but 5.8 is perfectly fine with it.

 

Initially I ran into issues that needed like 30-40 reboots to debug. Both my USB 2.0 headers were taken so I used an adapter cable to extend to one of my USB 2.0 ports in the back. Apparently Gigabyte BIOS did not like this at all and this was the reason for getting '61' fail boot codes. It took me a while until I realized it, then I disconnected my front USB 2.0 ports and plugged the TB USB cable in the free header. The other header is taken by Bluetooth. I think I just need to get a USB 2.0 9 pin to 3.2 20 pin adapter and use my free USB 3.2 header instead of the cable.

 

Here is my iPhone connected to the TB USB-C port and bridged to USB 2.0 hub.

 

 

 

Attaching the ROM I use. DISCLAIMER: I don't know if this ROM or the motherboard DSDT is what fixes USB.

 

I should have done more testing with NVM23 1st. But didn't do much testing so possible NMV23 is OK too.

DESIGNARE-Z390-NVM33-Elias64Fr.bin.zip

Edited September 7, 2020 by meina222
Add file

[meina222]

Did some testing on TB with the only proper device I have at the moment - the Thunderbolt 3 port for my Lenovo P32u-10 display.

 

It works... sort of

 

The Apple logo doesn't show up on boot (not a dealbreaker for me)

The display connects and is detected but the resolution is wrong. This is a 4K display but the internal frame buffer rendered is 6K (6400 x 3600). I wish I had an Apple XDR 6K display to test but instead I get scaled resolution of 3200 x 1800 @30hz. Doesn't look bad, but it's not ideal. Moreover MacOs thinks now I have a 61 inch display! So clearly bugs in the driver.

 

I should probably get an Apple rated TB3 cable to test with as I occasionally experienced the infamous random blank on/off flickering but it goes away for long periods of time. This issue has plagued real Macs for years and many claim is down to cable/display compatibility.

 

Conclusion: interesting to experiment with, but not "production" ready out-of-the-box.

 

 

 

Edit:

 

I was able to get 4K by manually selecting resolutions (holding Alt key when selecting "Scaled") but can't get more than 30hz. Must be the mini display port bridge cable from the graphics card to TB or the TB cable itself packaged with the monitor (I wouldn't be surprised if they didn't include top notch cable)

 

 

 

Edit 2:

 

Sounds like trying to get 4K @60hz over just the Thunderbolt 3 cable might be hard if wanting to pair this up with USB functionality over the same wire. My display is only DP 1.2 compatible and not 1.4 too so that could play a role as well.

 

https://www.dell.com/community/XPS/Dell-XPS-15-9560-Unable-to-get-4k-60hz-output-from-thunderbolt-3/td-p/7574431

 

Don't have experience with this but perhaps if I want to use my monitor as TB hub for other devices I can't also hope for TB video at 4K @60hz. On the other hand, if I plug in the DP port cable in the TB DP port, video is 60hz so I will run "dual monitor" setup - TB for data to display and DP for video.

 

 

I also tested sound over TB cable via Audio jack on display (no DP connected from monitor) so sound had to go over TB cable. It works - sound travels over the TB port from the DP bridge between Navi and the Titan Ridge over the TB cable to the headphones.

 

Edit 3:

Display on TB did not work in Linux 5.8 with this firmware. Will have to test other devices. I know this worked with default firmware, so not so good news.

Edited September 8, 2020 by meina222

[meina222]

Something I missed - It seems that reason for my display only hitting 30 hz at 4K is that it runs at legacy TB speeds (TB gen 1.0) and not 3.0.

 

I wonder if the reason is that Z390 firmware doesn't support displays well according to:

 

https://www.reddit.com/r/Thunderbolt/comments/g5as92/apple_thunderbolt_display_titan_ridge_aic_z390_a/

 

I may need to flash NVM23 after all, as I don't like this crippled display limitation.

 

 

 

 

[fabiosun]

Yesterday to solve my gpu problem I have detached all my disk and pcie card 
and I have tested also thunderbolt in all four slot of my motherboard to see if something different happens on usb

no

i can also confirm in my case nv23 patched firmware inhibit a proper use in windows

[meina222]

I don't even have windows installed and don't plan on adding it. But I'd like to understand why my display doesn't run on TB 3.0 but instead TB 1.0. I plan to flash and re-test with NVM23 later tonight. Will get a chance to check USB too.

Edited September 8, 2020 by meina222

[meina222]

@fabiosun, @iGPU - something interesting I noticed earlier and just confirmed it's a real phenomenon.

 

When I flash the TR card and I immediately read the flashed ROM afterwards, and compare the SHA against the .bin file I just flashed I get a match - gives me a peace of mind flash was successful (even though it's checked by the flashrom software anyways).

 

But when I then install the card in the PCIE slot and run with MacOS and then take it out to read the ROM agin the SHA's no longer match! How is this possible? Does the OS modify the firmware somehow? Happened twice (with NVM23 and NVM33 images alike).

 

About to re-flash backt to NVM23 to test if this fixes my display not using full TB3 capability.

 

Edit: Maybe when I specify a unique UID in the SSDT's ROM section that gets written in the chip and alters its contents.

Edited September 9, 2020 by meina222

[meina222]

@iGPU,

 

Attached is ACPI folder of my debug OC. I've also attached the log for completeness.

ACPI.zip opencore-2020-09-10-002201.txt.zip

[iGPU]

On 9/8/2020 at 8:35 AM, meina222 said:

Something I missed - It seems that reason for my display only hitting 30 hz at 4K is that it runs at legacy TB speeds (TB gen 1.0) and not 3.0.

 

I wonder if the reason is that Z390 firmware doesn't support displays well according to:

 

https://www.reddit.com/r/Thunderbolt/comments/g5as92/apple_thunderbolt_display_titan_ridge_aic_z390_a/

 

I may need to flash NVM23 after all, as I don't like this crippled display limitation.

 

 

 

 

 

The 40G/s only seems to appear with a proper SSDT, and only if the following is present under "Device (NHI0)":

 

                               "linkDetails", 
                                Buffer (0x08)
                                {
                                     0x08, 0x00, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00   // ........
                                }, 
 

If you're not certain about how to fix, upload your SSDT and I'll take a look.

[iGPU]

On 9/8/2020 at 8:23 PM, meina222 said:

@fabiosun, @iGPU - something interesting I noticed earlier and just confirmed it's a real phenomenon.

 

When I flash the TR card and I immediately read the flashed ROM afterwards, and compare the SHA against the .bin file I just flashed I get a match - gives me a peace of mind flash was successful (even though it's checked by the flashrom software anyways).

 

But when I then install the card in the PCIE slot and run with MacOS and then take it out to read the ROM agin the SHA's no longer match! How is this possible? Does the OS modify the firmware somehow? Happened twice (with NVM23 and NVM33 images alike).

 

About to re-flash backt to NVM23 to test if this fixes my display not using full TB3 capability.

 

Edit: Maybe when I specify a unique UID in the SSDT's ROM section that gets written in the chip and alters its contents.

 

I think it's possible that ROM is being fiddled with. (Oh, there's pun about 'fiddling while ROM burned' hiding in there.😉)

 

A similar thing happens with Aquantia ethernet ports. If a mobo with Aquantia is run under Windows and Windows is used to update drivers for Aquantia, those Aquantia ports then won't work under macOS. So OS's do play with the ROM.

Edited September 11, 2020 by iGPU

[fabiosun]

if I remember well also Asus TB 3 Ex card have had to be activated in windows before could be used in OSX

 

for ethernet card, maybe, it could be that an update changed ethernet card ID

 

 

[meina222]

The good news is that NVM23 works well in latest Ubuntu kernel 5.8. I have mirrored displays now and can choose DP passthrough or DP via the TB cable.

 

It's a minor annoyance I guess that under MacOS the display only connects at 10gb/s. It's as if the Titan RIdge controller in the display does not play nice with the flashed firmware. I'll play around with it a bit more, but happy that I can try passing through leaf devices now under Proxmox, as this was the main reason I wanted the TB card.

[iGPU]

I checked out all of the files inside the SysReport on the Designare mobo where USB works on TB (thanks for sharing meina222), and found nothing useful. Sigh.

 

I think the next step is probably looking at BIOS.

[fabiosun]

1 hour ago, iGPU said:

I checked out all of the files inside the SysReport on the Designare mobo where USB works on TB (thanks for sharing meina222), and found nothing useful. Sigh.

 

I think the next step is probably looking at BIOS.

I think for us problems are due about tb header and usb header connection

I would like to see an IOREG from @meina222without those cable connected

 

[iGPU]

6 hours ago, fabiosun said:

I think for us problems are due about tb header and usb header connection

I would like to see an IOREG from @meina222without those cable connected

 

 

That's a possibility.

 

Pursuing the BIOS approach, the BIOS for MSI Creator has a number of sections devoted to TB, found when doing a search for "thunderbolt" or "TBT". This is typical of AMI bios. (Note, I'm not probing the active BIOS on the mobo, but simply looking at a copy of the downloaded BIOS ROM file.)

 

Unfortunately, the tool crashes when trying to extract data running under Big Sur. I'll need to transfer things over to another machine running Mojave to sort it out.

 

 

 

Edited September 11, 2020 by iGPU

[meina222]

@fabiosun, I'll attach my IOReg in a few hours as I am logged into a different computer for work at the moment.

 

On an unrelated topic (but related to TB SSDT) - I noticed that the TB SSDT depends on "DTGP". Does anyone know what "DTGP" is used for and what is its relation to power management and is it compatible with Pavo's AGPMInjector? When should I use the former or the latter?

[fabiosun]

@iGPUwhat are you trying to extract exactly? in high Sierra works..but I find many Thunderbolt referrals..

 

----

Unicode text "Thunderbolt" in 900901B9-41CD-4D54-A842-4D559980960B/UI section at header-offset 10h

Unicode text "AmdCpmThunderbolt" in 900901B9-41CD-4D54-A842-4D559980960B/UI section at header-offset 04h

 

[iGPU]

13 hours ago, meina222 said:

@fabiosun, I'll attach my IOReg in a few hours as I am logged into a different computer for work at the moment.

 

On an unrelated topic (but related to TB SSDT) - I noticed that the TB SSDT depends on "DTGP". Does anyone know what "DTGP" is used for and what is its relation to power management and is it compatible with Pavo's AGPMInjector? When should I use the former or the latter?

 

It's simply a sub-routine often called by SSDTs when injecting properties. It is externalized to reduce SSDT size as the single, external DTGP can be called by many SSDTs. DTGP could be could be placed into every SSDT that calls it and you could do away with the external DTGP SSDT file.

 

Initially, I only called it from TB-SSDT. (KGP, the retired X299 guru, used to call it from almost every SSDT.) I'm presently calling DTGP for two SSDTs: FireWire and TB.

 

It's typically called when injecting properties :

 

           Method (_DSM, 4, NotSerialized)  // _DSM: Device-Specific Method
            {
                Local0 = Package (0x06)
                       {

                             { Inject #1 }

                             { Inject #2 }

                             { Inject #3 }

                       }
                DTGP (Arg0, Arg1, Arg2, Arg3, RefOf (Local0))
                Return (Local0)
            }

 

You can avoid by re-structuring the SSDT; the following is same as above without a DTGP call:

 

            Method (_DSM, 4, Serialized)  // _DSM: Device-Specific Method
            {

                   Return (Package (0x06)
                       {

                             { Inject #1 }

                             { Inject #2 }

                             { Inject #3 }
                       })

             }

 

 

[iGPU]

13 hours ago, meina222 said:

@fabiosun, I'll attach my IOReg in a few hours as I am logged into a different computer for work at the moment.

 

On an unrelated topic (but related to TB SSDT) - I noticed that the TB SSDT depends on "DTGP". Does anyone know what "DTGP" is used for and what is its relation to power management and is it compatible with Pavo's AGPMInjector? When should I use the former or the latter?

 

There are a couple I'm interested in. I moved the process over to my laptop that runs Mojave. I was able to extract two, turning them into *.bin files, but then I could not convert to text files using the "./ifrextract file.bin file.txt" call. I was busy at work today and didn't have time to pursue more. I don't know if the *.bin files are corrupted or the text conversion routine is broken.

 

I will work on more this weekend.

 

***

 

On another TB front, I had the idea of connecting the TB port to a TB dock. The dock lit up, but did not connect either a USB-C drive nor the TB-drive. This is strange. I must also investigate this more.

 

BTW, last weekend, I moved the GB TR AIC from the TRX40 mobo and swapped it with an identical GB TR AIC in the X299 build. Both worked in the X299, giving correct TB and USB trees. Both AICs are flashed with same firmware:, they are identical. So this means both cards are working well and that the problem is in how the AIC works (or doesn't work) in the MSI TRX40 mobo. 

[fabiosun]

@iGPU i did same test in x299 some message above

i cant re done the same test but i would like to know if in your test you have used tb header and usb connected to your x299 motherboard or used it as we have to use in trx40 without a tb header

i would like to see if usb tree appears also without usb and tb header connection on x299

[iGPU]

On 9/12/2020 at 3:25 AM, fabiosun said:

@iGPU i did same test in x299 some message above

i cant re done the same test but i would like to know if in your test you have used tb header and usb connected to your x299 motherboard or used it as we have to use in trx40 without a tb header

i would like to see if usb tree appears also without usb and tb header connection on x299

 

I have successfully used flashed GB TR AICs in Z390, X570 and X299 without TB headers, as long as the pins are rear of card are shorted. So it should be working in TRX40.

 

There was one post from 2 years ago on another forum where the poster had same issues we have  (here). I've tried to nmano, asking if or how the issue was resolved, but have not yet gotten a reply.