PC Processors

[cat2devnull]

I have searched high and low and cannot find a straight answer to this question...

Can you enable PCIe bifurcation 4x4 with a G series CPU?

I gather there were issues in the older models where the GPU was connected via PCIe so some lanes were reserved and they had less PCIe lanes to begin with. But my understanding is that with the 5000 series, the GPU is on the infinity fabric so in theory all x16 lanes are available to be bifurcated into 4x4.

Has anyone here got the definitive answer?
Has anyone got 4x4 working with a 5600g/5700g or similar APU?

[rutilumorph]

Goddamnit, I've been googling out of my mind these past few days to answer this very question. Asked on LTT, Reddit, looked for AMD datasheets (cause specs on official product info don't mention bifurcatio), and no luck. Then I tried to look for answers here, and a thread without answers is the first thing I find... smh

What I know so far is that my motherboard, ASRock X570D4I-2T, supports 4x4 bifurcation, but with a G-series GPU it's not available, not even on desktop ASRock mobos. This was tested by an ASRock representative with a RYZEN 7 PRO 5750GE, same CPU as mine. That person said "seems to be an AMD thing" :D. I've seen one other post on a different forum complaining of the very same problem with a 4750G CPU.

Now, it might be ASRock's BIOS at fault, but I think we could know more if your motherboard is different. Because if it claims to support 4x4 bifurcation, but it doesn't with a G-series CPU, then it must definitely be an AMD thing, which will beg the question whether a microcode update can change this.

What's your motherboard? 

[misterj]

cat2devnull, I am not familiar with APUs but have you looked in your BIOS? My 3970X on a Gigabyte MB has this:

If you still have questions I recommend you contact your MB vendor. Enjoy, John.

[rutilumorph]

Thanks for keeping the thread alive. Do you know whether the CPU has control over bifurcation options? I already contacted my MB vendor, and they imply that it's the CPU that restricts the 4x4 option.

Also, what's the Gigabyte MB model from the screenshot? I think I might be able to ask a local PC service to provide a MB such as yours to test my CPU on, and if 4x4 works on that MB, then I'd know whether the cause lies in the BIOS or CPU.

[misterj]

You are welcome, rutilumorph. I have a Gigabyte TRX40 DESIGNARE, but it is for Threadrippers and has a different socket. I do not know how bifurcation works for APUs so cannot answer your question. Did you look in your BIOS for the equivalent of my SS? Please post a SS here. Here is a SS from the your MB manual:

But this is for the PCIe slot and what you are looking for is internal to the processor. Confusing to me since I have never had an APU. I still think your MB support should know the answer, but do open a ticket with AMD here. What/where are you being told 4x4 is not supported? Please post a SS. Thanks and enjoy, John.

[rutilumorph]

Thanks for the info.

I already went through the MB's manual, and I contacted ASRock's support. The phrasing from the manual is not very grammatically correct:

---

PCIe/GFX Lanes Confguration
Confgure J10 & J3600 Slot PCIe Lanes. Auto - If J3600 Slot is connected to a device, J10
and J3600 both are x8, otherwise, J10 is x16; x8x4x4 - J10: x8, J3600: 4x4; x4x4x4x4 - J10:
x4x4x4x4 (J3600 Slot can't connect to any device).

---

I didn't find a J3600 slot on the motherboard. And even when the J3600 is completely disabled from BIOS, and when there are only HDDs connected via SATA and a USB to boot from, 4x4 bifurcation doesn't work. So I'm confident that supposed J3600 slot has nothing to do with the iGPU's presence.

This is what I see in my BIOS, and what I sent to ASRock (note the various Redriver settings related to the J3600 slot have no effect on bifurcation options):

An ASRock representative confirmed the 4x4 disappears with a 5750GE APU, but is present with a 5750X; we're both on the latest BIOS version:

I wouldn't qualify the test on an ASRock desktop MB as a definitive conclusion - what if there is some overlap between BIOS codebases? As a software engineer, I would expect this to be the case, even when coding firmware.

ServeTheHome have a review of the board here:

https://www.servethehome.com/asrock-rack-x570d4i-2t-amd-ryzen-server-in-mitx/

They use a Ryzen 9 3950X CPU - no iGPU - and they see even more bifurcation options, but they still boil down to the same set ASRock support and I see (except for the 4x4 one).

I'll write an email to AMD and post the resolution here once I have it.

[elstaci]

just adding my findings.

According to this LinusTechTips Thread one user stated that AMD APUs uses 8 PCIe lanes for its IGPU and the other for the Motherboard. So the best if it supports bifurcation is 4x4: https://linustechtips.com/topic/1295848-bifurcation-support/

I don't know how accurate the above User's statement is but it does seem to make sense concerning that half of the processor's PCIe lanes are used by the IGPU.

This Reddit thread claims the same thing about the AMD APUs only having 8 PCIe lanes instead of 16 PCIe lanes: https://www.reddit.com/r/Amd/comments/h0t3jk/pcie_bifurcation_and_ryzen_g_pro/

Does Apu use PCIe lanes?

APU paired with a GPU

Most lower-end APUs have fewer PCIe lanes than CPUs. If we take AMD's Ryzen 3 2200g or a Ryzen 5 2400g with Vega graphics as an example, you'll notice that they only have 12 PCIe lanes. 8 of which are reserved for the graphics card, while the remaining 4 are for storage (NVMe SSD).Dec 5, 2021

and

How many PCIe lanes does Ryzen 5700g have?

As such, the chip has 20 lanes of PCIe 3.0 connectivity (16 for graphics, four for the chipset, and four for storage) compared to 24 lanes of PCIe 4.0 found on the Ryzen 5000 models for the desktop PC.Jun 25, 2022

Found this Anatech article that shows all the PCIe lanes in all 5000 Series APUs: https://www.anandtech.com/show/16824/amd-ryzen-7-5700g-and-ryzen-5-5600g-apu-review

I guess this probably shows why you can't enable 4x4x4x4 Bifurcation on a motherboard with a AMD APU.

[elstaci]

Asus has an extensive charts of all its Asus Motherboards and what Bifurcation it supports: https://www.asus.com/us/support/FAQ/1037507/

Okay not really sure if my last two replies are helpful or not. But you might be able to understand Asus charts better than me.

[rutilumorph]

I don't think the implication that all APUs use 8 lanes for their iGPU is accurate.

OP mentioned the 5000 series APUs may no longer reserve a set of PCIe lanes for their iGPU.

Also, my mobo's specs page explicitly mentions the x16 slot's performance depends on the CPU: for 3000 G-series, it's 8 lanes wide (as you discovered on your own), but for the next-gen 4000 and 5000 it's 16 lanes. So the 8 lanes penalty may not even hold true for the 4000 G-series.

But whatever the case may be, I think ultimately the proof is in the pudding, so I attached my old GeForce 780 GTX GPU to the single x16 slot of the mobo, and I executed lspci -vv which showed the following:

(TL;DR: lspci reports the GPU is attached to a slot with x16 capability, has made use of all 16 lanes; but it's running at a PCIe Gen. 1 x1 speed - presumably because it's a Linux server with a console output only, and no NVidia drivers installed)

---

10:00.0 VGA compatible controller: NVIDIA Corporation GK110 [GeForce GTX 780] (rev a1) (prog-if 00 [VGA controller])
Subsystem: Gigabyte Technology Co., Ltd GK110 [GeForce GTX 780]
Control: I/O+ Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Latency: 0, Cache Line Size: 64 bytes
Interrupt: pin A routed to IRQ 143
IOMMU group: 9
Region 0: Memory at fb000000 (32-bit, non-prefetchable) [size=16M]
Region 1: Memory at d8000000 (64-bit, prefetchable) [size=128M]
Region 3: Memory at e0000000 (64-bit, prefetchable) [size=32M]
Region 5: I/O ports at f000 [size=128]
Expansion ROM at fc000000 [disabled] [size=512K]
Capabilities: [60] Power Management version 3
Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)
Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME-
Capabilities: [68] MSI: Enable+ Count=1/1 Maskable- 64bit+
Address: 00000000fee00000 Data: 0000
Capabilities: [78] Express (v2) Endpoint, MSI 00
DevCap: MaxPayload 256 bytes, PhantFunc 0, Latency L0s unlimited, L1 <64us
ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 75.000W
DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq-
RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+
MaxPayload 256 bytes, MaxReadReq 512 bytes
DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend-
LnkCap: Port #0, Speed 8GT/s, Width x16, ASPM L0s L1, Exit Latency L0s <512ns, L1 <4us
ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+
LnkCtl: ASPM L1 Enabled; RCB 64 bytes, Disabled- CommClk+
ExtSynch- ClockPM+ AutWidDis- BWInt- AutBWInt-
LnkSta: Speed 2.5GT/s (downgraded), Width x16 (ok)
TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-
DevCap2: Completion Timeout: Range AB, TimeoutDis+ NROPrPrP- LTR-
10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt- EETLPPrefix-
EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit-
FRS- TPHComp- ExtTPHComp-
AtomicOpsCap: 32bit- 64bit- 128bitCAS-
DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- OBFF Disabled,
AtomicOpsCtl: ReqEn-
LnkCap2: Supported Link Speeds: 2.5-8GT/s, Crosslink- Retimer- 2Retimers- DRS-
LnkCtl2: Target Link Speed: 8GT/s, EnterCompliance- SpeedDis-
Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS-
Compliance De-emphasis: -6dB
LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete+ EqualizationPhase1+
EqualizationPhase2+ EqualizationPhase3+ LinkEqualizationRequest-
Retimer- 2Retimers- CrosslinkRes: unsupported
Capabilities: [100 v1] Virtual Channel
Caps: LPEVC=0 RefClk=100ns PATEntryBits=1
Arb: Fixed- WRR32- WRR64- WRR128-
Ctrl: ArbSelect=Fixed
Status: InProgress-
VC0: Caps: PATOffset=00 MaxTimeSlots=1 RejSnoopTrans-
Arb: Fixed- WRR32- WRR64- WRR128- TWRR128- WRR256-
Ctrl: Enable+ ID=0 ArbSelect=Fixed TC/VC=01
Status: NegoPending- InProgress-
Capabilities: [128 v1] Power Budgeting <?>
Capabilities: [420 v2] Advanced Error Reporting
UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol-
UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol-
CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn-
MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap-
HeaderLog: 00000000 00000000 00000000 00000000
Capabilities: [600 v1] Vendor Specific Information: ID=0001 Rev=1 Len=024 <?>
Capabilities: [900 v1] Secondary PCI Express
LnkCtl3: LnkEquIntrruptEn- PerformEqu-
LaneErrStat: 0
Kernel driver in use: nouveau
Kernel modules: nouveau

---

But the Asus charts you posted do seem to imply that 5000 series APUs support a 3-way bifurcation of the x16 slot at most (which corresponds to x8x4x4), so it might indeed be an AMD thing . Well, the ball is in AMD's court now, since I did actually raise a support ticket with them. They haven't responded yet, but it's the weekend, so perfectly understandable.

[elstaci]

So you are running a Linux OS and not a Windows OS?

If it is Linux I can't help since I am not very familiar with that OS.

Also if this concerns a Server environment you might want to post your thread at AMD FORUM's SERVER GURU and see if you get any answers from there: https://community.amd.com/t5/server-gurus/ct-p/amd-server-gurus'

I though this was a regular PC with a Windows OS involved.

[cat2devnull]

Hi all, sorry for asking the question and then going quiet. I didn't realise that the forum wouldn't notify me of updates to the thread. I've spent a bit of time looking into this over the last couple of weeks and there is a lot of misinformation on various forums that even I fell for (see my original post).

One of the biggest misconception in the community that the Vega GPU connects to the CPU via PCIe. This has never been the case. Since the introduction of the first APU based on the Zen architecture (Raven Ridge), the GPU has always been connected to the CPU via the infinity fabric (not PCIe).
This probably came from the fact that they only had a total of 16 PCIe lanes. Every other model since has 24. I haven't found an definitive answer but a few tech pundits speculated it was to save die space for the added GPU.

I've made a quick table to help people see the differences. Now I wanted to explain PCIe lanes so the table makes sense. CPU PCIe lanes have 3 roles;
1) Connect to the first PCIe slot (or 2 slots) closest to the CPU, which are normally used for discrete GPU(s).
2) Connecting to the chipset. The chipset acts as a MUX and allows connectivity to more devices (NVMe, SATA, Ethernet, extra PCIe slots, etc) based on chipset specifications.
3) Storage connect to the CPU, normally for the primary NVMe but on older boards it was used for the SATA controller.
This is often written as 16xGFX + 4xChipset + 4xNVMe or shortened to 16+4+4 (as I have done below).

Codename	Series	Product	Model	PCIe Lanes
Summit Ridge	Zen	Desktop CPU	Ryzen 1000	PCIe 3.0 16+4+4
Raven Ridge	Zen	Desktop APU	Athlon 2000G	PCIe 3.0 8+4+4
Pinnacle Ridge	Zen+	Desktop CPU	Ryzen 2000	PCIe 3.0 16+4+4
Picasso	Zen+	Desktop APU	Athlon 3000G	PCIe 3.0 16+4+4
Matisse	Zen 2	Desktop CPU	Ryzen 3000	PCIe 4.0 16+4+4
Renoir	Zen 2	Desktop APU	Ryzen 4000G	PCIe 3.0 16+4+4
Vermeer	Zen 3	Desktop CPU	Ryzen 5000	PCIe 4.0 16+4+4
Cezanne	Zen 3	Desktop APU	Ryzen 5000G	PCIe 3.0 16+4+4

So every APU model other than the original Raven Ridge has x16 delivered to the first PCIe slot and in theory can be bifurcated to x8x8, x8x4x4 or x4x4x4x4.

This all started because I was looking at a Gigabyte B550 ITX motherboard with a 5600G and noticed this in the manual;
PCIEX16 Bifurcation: Allows you to determine how the bandwidth of the PCIEX16 slot is divided. Options: Auto, PCIE 2x8, PCIE 1x8/2x4, PCIE 2x4/1x8, PCIE 4x4 (Note). (Default: Auto)
(Note) This item is present only when you install a CPU that supports this feature.

I contacted Gigabyte and they responded with;
The CPU with integrated graphics doesn't support PCIEx4x4.
Why the onboard graphics disables 4x4? You may check with AMD. It's their CPU limitation.

So I contacted AMD and they said;
I would like to inform you that the PCIe bifurcation depends on the motherboard design and its up to the motherboard manufacturers how to implement it.

So after all that. I am still no wiser as to why APUs don't seem to support 4x4.

[rutilumorph]

I would like to inform you that the PCIe bifurcation depends on the motherboard design and its up to the motherboard manufacturers how to implement it.

---

Funny, I also contacted AMD, and this is what I got as an answer:

Thank you for the email.
I checked the information internally and got an update that 4x4 is not supported with AMD Ryzen 5750GE processor or APUs.
Thank you for contacting AMD.

---

I think it's safe to assume this also holds true for any 5600G/5700G APU.

Given the two contradicting answers, their support is obviously flawed. But I think a reasonable assumption is that the former support agent responded with common tech knowledge in mind - everyone here can agree the APUs should support 4x4 bifurcation - but the latter one checked more thoroughly, presumably because I bombarded him with information in my email.

Well, that's how it is at the moment. I didn't get an answer whether this can be fixed with a microcode update. But @cat2devnull if you'd like me to loop you in on the email thread with the hope that the pressure from 2 people with information from 3 motherboard manufacturers can get 4-wise bifurcation to work with a single microcode update , see whether you can send me a PM after adding me as a friend here, and I'll exchange email contacts.

[cat2devnull]

I'm sure you're right. I expect that the answer I got was made without any investigation.
So it looks safe to say that APUs don't support 4x4.
I doubt that this is a limitation they made by choice, there's undoubtedly some technical reason (even if it's an old architecture problem that may not be relevant anymore).
Maybe to do with the number of independent channels that can be controlled on the infinity fabric simultaneously. Just guessing.

[cat2devnull]

@rutilumorph  I never did work out how to PM you. Did you ever get any more information out of AMD?

[rutilumorph]

I didn't pursue the matter any further, because I didn't think I'd stand a chance alone 🤷‍. Also, I figured out how to make use of 4 NVMe SSDs on my motherboard, so I can live without 4x4 bifurcation.

However, 4x4 bifurcation will allow me to run 5 SSDs on the same motherboard .

If you're interested in attempting to take matters to AMD, I added you as a friend here in the forums, and I allowed my email to be seen by friends. So, drop me a line. My proposal would be to loop you in on an old support email thread with ASRock, explain we'd like to talk with someone from AMD who is not a 1st line support officer (because of the different replies the two of us got), and take it from there. Maybe we could ask ASRock and/or your motherboard's manufacturer to inquire about the lack of 4x4 bifurcation as well - obviously, it fits their needs and market segments which is why they produce motherboards that attempt to enable it.

[pani_alex]

can you please explain how you make x4*4 bifurcation work?

[rutilumorph]

Well... I didn't actually

I only figured out how to run 4 NVMe SSDs, not how to do 4x4 bifurcation. The MB and CPU still allow x8x4x4 bifurcation, and there's a x8x4x4 bifurcation card (https://c-payne.com/products/pcie-bifurcation-card-x8xm-2xm-2-low-profile-adapter?variant=3946671548...) that has two M.2 slots. Installing that card with one additional PCIe-to-M.2 adapter, the computer can have a total of four, completely independent M.2 slots.

There is a benefit to this setup - it allows at least 2 SSDs to be controlled from separate slots, which, when used in a mirror config, adds some extra redundancy in case either the x16 slot or the on-board M.2 one dies.

[cat2devnull]

@rutilumorph I still couldn't see your contact details. I haven't been pushing this because I ended up going with an Intel solution since I already had everything but a motherboard. The cost of switching to AMD would have only been justifiable if I could have run 4x4 with an APU.

The intel board had 2xM.2 and with a PCIe card I added a further 3 (x8x4x4) so 5 in total.

Also with TLC NVMe drive prices dropping at the rate they are, I will likely just replace my remaining 2TB drives with 4TB which will free up slots. I can't get over the fact that MP34 4TB TLC drive is under $200US now.

[johnnyenglish]

As stated above, its board dependent. 

Try checking board manual, some boards will split or do the bifurcation automatically, others won't and will disable m.2,pci slots or SATA ports.

Inside BIOS some should have a option to choose which Kind of bifurcation you want on the pcie slots if you wish do use expansion cards/crossfire, sacrificing GPU down to 8x+8x.

I would opt for a big storage instead of multiple nvme as you will always lose speed somewhere.

For the APU, the table above looks very helpful. 

[pani_alex]

thanks, my motherboard and cpu allow me x8x4x4 and in L1 forum someone point to me there is a x8x4x4 card with m.2 on the x4 lines and a x16 slot on the x8, so I can add on the cheap 2 nvme, very usefull

[geepeetee]

Came to this thread, and found what I needed on the total lanes Cezanne family specifically the ryzen 7 pro 5750G apu,  why AMD doesn't state this info on the product page of each CPU/APU is beyond me.   Had to search the internet for 3 hours and even asked chatGPT which told me it was a Zen 2 chip, so it's obviously wrong.  

Basically just wanted to chime in regarding contacting AMD, Gigabyte or any large tech other company for that matter.  Quite often you are contacting a call centre with employees that will know less than you on the subject, they are rarely knowledgeable and often google to get the answer.   

In this case, as the OP is mentioning 4x4 and Pcie, I think his replies from employees are getting confused.   They don't seem to understand that the OP is wanting to split the lanes.  I think it's beyond what those in a call centre can understand.  Because 4x4 can also mean Pcie 4.0 x 4  (which is used in the description of M.2 capabilities)  .  As the AM4 APUs all use pcie 3.0 the call centre staff will pass it to the manager who probably knows very little too, and the response will be it can't be done as they assume 4x4 means 4 lanes of pcie4.  

You will just get the standard reply to their best knowledge.  You will only get the answer from smaller tech companies or if you get lucky at a knowledgable retailer of the product.

But shame on AMD for not even publishing the number of PCIe lanes on there CPU/APU product pages.  And as for bifurcation and gigabyte's response, well that's just pathetic.  Huge companies need to be prepared to say "Were not sure, and sorry", if that's not acceptable for the company, then hire more specialized people who are not getting paid a slaves wage in a call centre, instead of sending the 100% incorrect response as they quite often do.    Same with all these companies, they hire glib slave masters as call centre managers, not technical experts.