If you have been following the AMD disclosures lately you probably remember Infinity Fabric. SemiAccurate first mentioned this when we talked about the new Instinct GPU compute cards but they play a part in Zen and Vega too.
On the surface it sounds like AMD has a new fabric to replace Hypertransport but that isn’t quite accurate. Infinity Fabric is not a single thing, it is a collection of busses, protocols, controllers, and all the rest of the bits. Infinity Fabric (IF) is based on Coherent Hypertransport “plus enhancements”, at a briefing one engineer referred to it as Hypertransport+ more than once. Think of CHT+ as the protocol that IF talks as a start.
Hypertransport lacked a lot of things that quite frankly were not on the roadmap when it was designed in the DEC Alpha days. It wasn’t really scalable as later Opterons brought into focus, had no real QoS capabilities, wasn’t very flexible, had no inherent security features, and lacked any authentication mechanism. If you throw in higher performance and greater efficiency you have the list of why AMD needed to redo their bus to get Infinity Fabric.
Simple but elegant is a good thing
Going down to the metal, or at least metal traces, there isn’t one fabric in IF but two. As you can see the control fabric is distinct from the data fabric which goes a long way towards enabling the scalable, secure, and authenticated goals. Control packets don’t play well with congested data links, and security tends to work better out-of-band too. QoS also play better if you can control it external to the data flows. So far IF seems to be aimed right.
Officially the control fabric controls power management, security, reset/initialization, and test functions. This is about what you would expect from a data plane/control plane structure like IF. Think of this fabric as doing the metadata work about the chip, it is probably fairly low-bandwidth but low latency, and very granular. It won’t be very wide unlike the data fabric.
Speaking of the data fabric it will be wide, fast, and scalable. AMD claims low latency links, high performance common bus structures, CHT+ protocols on this side, multi-die, and multi-socket capable links. This means it will also work on MCMs like Naples to connect the four dies on the massive package.
But here is where things start to get interesting. AMD claims Infinity Fabric is in Vega, Summit Ridge, and Raven Ridge, essentially all the products they make from here on out not counting retreads of older architectures. You will have one coherent data fabric and possibly the control fabric too across a single die, MCMs, sockets, PCIe links, CPUs, GPUs, APUs, and potentially new names made up to describe old ideas later on.
IF signaling does not constrain topologies either, it could be point to point, rings, busses, meshes, or 3D structures like a torus. Since Vega is using IF in a mesh topology it probably will start off on the complex end of that list. How Vega is using IF,whether for to and from die transmissions or does it subsume the on-die unit to unit communications too, is yet to be revealed, but there is a big hint in the granularity.
AMD went to great lengths to explain how granular IF is, it is one of the technology’s key selling points. Like all modern fabrics it has a firware driven microcontroller but as you would expect, that doesn’t scale well. IF has scalability as one of its pillars. Instead of a single controller AMD broke up the control tasks into many tiny sub-controllers, one per IP block. That IP block bit is a bit nebulous but given how the speakers were talking about granularity, think shader rather than GPU, it sounded like they made it granular enough to have thousands of sub-contollers across a die like Vega, not tens.
So that is the key to the new Infinity Fabric, the granularity, especially in mesh topologies it should allow bandwidth to scale with nodes. Topology is not protocol defined or restricted and the coherent links will work across sockets, CPUs, GPUs, and more. If the level of granularity is as fine as was intoned, it allows a CPU core to pass info to a shader ‘directly’ regardless of the two being on the same silicon or across a system. The separate control and data fabrics bring AMD up to modern SoC structures too, and in some ways beyond. Infinity Fabric is a really big deal, and at the risk of sounding like a broken record, it is going to be really interesting to see the details when AMD reveals them.S|A
Latest posts by Charlie Demerjian (see all)
- Raja Koduri and Dr. Randhir Thakur out at Intel - Mar 21, 2023
- What does Intel’s Emerald Rapids bring to the fight - Mar 21, 2023
- A new ARM code name for a new market pops up - Mar 20, 2023
- A big ARM server project shut down in silence - Mar 14, 2023
- A bit more on AMD’s Genoa memory issues - Mar 13, 2023