Since 2019, the semiconductor industry has been gradually shifting to a new chip design philosophy: the chiplet.On the surface, this seems like a fairly minor change, since all that's really happening is that chips are being broken up into smaller parts. Also, not every company is doing this, and even those that are haven't completely moved to smaller chips. is chiplet really that important?

Well, despite this, small chips are still very important to the semiconductor industry. Not only is it a counter to the woes that many companies have been experiencing lately, but now that AMD and Intel are having so much success with small chips, their competitors must surely be thinking about following their lead so as not to be at a disadvantage.

Chiplets are exactly what the name implies: small chips with only partial functionality. The whole idea of a chiplet is that instead of making a processor on a single piece of silicon (also known as a monolithic chip), you have multiple chips, each containing some part of the CPU. While it is natural to have a small chip for each function (e.g. one for core, one for connectivity, one for graphics, etc.), it is sometimes desirable to put many of the same chips into a single processor, which facilitates the addition of more core examples.

AMD is the company that really created (or at least popularized) and introduced the concept of small chips. It used a basic multi-chip modular design for the original Zen processors in 2017, where high-end models utilized multiple CPU chips to increase the number of cores rather than using a single, larger chip. But with 2019's Zen 2, AMD is splitting its CPUs in two: one for the smaller chip in the CPU core, and another for all the other features, such as PCIe lanes and RAM connectors.

Meanwhile, Intel has been struggling to catch up with its own small-chip implementation, which the company calls "tiles". Despite entering the space later than AMD, its first small-chip processor finally came out this year, and it's quite complex. the Ponte Vecchio datacenter GPUs have a couple of tiles filled with GPU cores, a couple of tiles for caching, one for HBM2 VRAM, and two more for connectivity. the Meteor Lake is the dominant four-block solution, though it's a laptop-specific one. solution, and while it's laptop-specific, its successor, Arrow Lake, is coming to desktops next year and is very similar.

Other companies, such as Fujitsu and Broadcom, have expressed interest in making processors with small chips, but so far AMD and Intel are the only companies to have launched small-chip-based products in volume production. However, especially for high-end computing, it seems that the move to smaller chips will be necessary to maintain a competitive edge.

What's fascinating about Chiplets is the variety of uses they can be put to; Chiplets aren't a one-trick pony like ray tracing or something extremely vague or non-specific like AI. Chiplets have distinct advantages that in many cases make monolithic processors completely obsolete.

One of the things AMD and Intel often talk about with smaller chips is how it's easier to provide more specific solutions for certain markets and customers. It is very simple to increase and decrease the number of cores, or to replace one small chip with another that is more suitable. For example, AMD's server CPUs not only have more CPU small chips than desktop models, but they also have bigger and better IO chips (for connectivity features.) AMD can also add another layer to consumer and server processors with its 3D V-Cache small chip, further providing buyers with more options.

You might also think that reusing small chips across generations is possible as long as they remain good enough, a key advantage of Intel's slicing system. whereas AMD has a CPU core and an IO small chip (plus a cache small chip), Intel's blocks include one for the core, one for the graphics, one for SOC functions, and one for the IO functions. While this is useful for providing multiple versions of these blocks, Intel's approach allows the company to not replace a block until absolutely necessary because it has more functionality spread across more blocks. For example, if Intel wanted to update its AI hardware, it could simply replace the SOC block.

While keeping an old tile longer is a way to save money, it's also easier to justify adding new features more incrementally than before. We've gotten used to doing generation upgrades every year or two and getting a bunch of stuff all at once; smaller chips can dramatically speed up the upgrade cycle.

However, these are just design considerations; we haven't even gotten to the manufacturing stage yet, and smaller chips are much cheaper to manufacture. This is because defects occur during processor production; in short, bigger chips are more prone to defects, which reduces production. For the same reason, smaller chips are less prone to defects, so much so that smaller chips can actually save considerable manufacturing costs. This effect is even more pronounced in entirely new process nodes with higher defect rates, which makes it almost impossible for large chips to be commercially viable.

But perhaps the most important thing about small chips in terms of manufacturing is Moore's Law, which predicts that the number of transistors in a chip will double every two years. What that actually means in the real world is a bit fuzzy, but it applies very well to high-end computing that breaks the record for the processor with the most transistors. If Moore's Law is still as true as it has been for 50 years, then in two years we should see a chip with twice as many transistors as today's biggest chip.

The debate among companies and analysts about whether Moore's Law is dead is intense, but there's no doubt that it's becoming increasingly difficult to improve process nodes, which largely contribute to Moore's Law by increasing transistor density. While increasing the number of transistors can also be accomplished by making physically larger processors, there is a practical limit to the size of a chip, and we've already reached that limit. So when TSMC's 3nm technology fails to increase cache density by even 1%, it's very bad news for the industry as a whole and signals the imminent demise, if not death, of Moore's Law.

Smaller chips can't increase density, but they can get around the size limit because no single chip can come close to that limit. In general, ~750mm² is the absolute maximum size of a chip on the latest processes, but for small chips the limit is really the size of the PCB. AMD's latest Zen 4 Genoa server CPUs are a whopping 1,271mm² for a model with a full 96 cores.

Speaking of the whole cache issue (which could become a problem for the entire industry), small chips can also alleviate this problem. Using cached small chips instead of adding more cache to the CPU or graphics small chips certainly has the benefit of making those core small chips smaller and cheaper, plus the overall specialization angle of technologies like 3D V-Cache, but there's a third benefit for manufacturing. If newer nodes can't really increase the cache density, then a cache chiplet can be made on an older and cheaper node without actually losing much (if any) performance.

While small chips are great, they may not be well suited for certain situations, such as very small processors like smartphone chipsets or intentionally simple chips (at least low-end ones) for microwave ovens and home assistants. The entire semiconductor world won't be built on small chips; many general-purpose but important chips are still made using some of the oldest and cheapest processes.

But for our laptops, desktops, servers, cars, and game consoles, small chips are increasingly likely to be the future. Of course, the opposite of small chips from the likes of Nvidia is a waste of time, because AI is so much better that doubling performance every two years is slow. As a result, Nvidia still makes GPUs (and now CPUs) the old-fashioned way, because Moore's Law doesn't matter anymore.

Nonetheless, as it stands, AI is underdeveloped as a technology, and if it fails or the space becomes extremely competitive, companies that use small chips will have an advantage over those that don't. Whatever the future holds for the semiconductor industry, it's hard to imagine that small chip technology won't be a part of it.