# Paradigm Shift in AI Scale Up Clusters using microLED based Interconnects -  Chris Pfistner

https://www.youtube.com/watch?v=GjWjYNVwrOs

[00:00] Well, now we have a a terrific sponsor
[00:03] talk coming up from Avisenna. Uh
[00:05] Avisenna is an amazing pioneer in
[00:08] actually uh an area that uh Peter U
[00:12] Vincer talked about yesterday for Nubis
[00:13] communications of going to surface mount
[00:17] um vertical clusters of optical IO to
[00:22] solve the the shoreline problem and
[00:24] Abisan has a unique way of doing that.
[00:26] Uh Dr. Dr. Chris Fner is head of sales
[00:28] and marketing at Avisena. I don't think
[00:31] I've ever met a company that had more
[00:32] PhDs in top positions from technical
[00:34] program managers, CTO, sales, marketing,
[00:38] and um uh and even operations. Um uh so
[00:44] it's just a pleasure to introduce Chris.
[00:45] I've known him for quite some time. And
[00:47] um uh as a sales and marketing guy, this
[00:49] guy is a deep technologist as well. So
[00:52] Chris, the floor is yours.
[00:54] Well, thank you, Dan. Thank you for the
[00:56] warm welcome. I assume everyone can hear
[00:59] me. Uh I'd also like uh Deepak for
[01:04] setting the stage here for us, kicking
[01:05] us off with a terrific uh keynote this
[01:07] morning. I'm going to share my slide.
[01:28] See, I should have presentation mode
[01:32] now. I'm going to turn on a laser
[01:33] pointer here.
[01:36] There you go.
[01:38] Are we good?
[01:40] We are good.
[01:41] Okay. Yeah. So, we're going to talk
[01:43] about this as Dan pointed out paradigm
[01:45] shift in AI clusters and this is now
[01:48] using microLED based interconnects uh
[01:51] specifically aimed at uh scale up
[01:55] I'm going to set the stage, but this is
[01:57] our version. There's many versions of
[01:58] these. It's basically these days in AI
[02:02] networks, it's all about energy
[02:03] efficiency because data centers are
[02:05] projected to possibly exceed 20% of
[02:08] global energy usage by uh 2030.
[02:12] And the other this is one representation
[02:15] of it. The other uh fact of life is that
[02:18] um the the process capacity has
[02:22] outstripped the interconnect capacity
[02:25] and were in need of a of a new
[02:28] technology to cope with that. Um in the
[02:34] data center architectures which were AI
[02:38] most of AI calculations are happening
[02:40] these days basically have three
[02:42] categories of of interconnects. We have
[02:45] the transport networks those have been
[02:47] optical for a long time. Uh intra data
[02:51] center we have scale out and optics has
[02:55] pretty much taken over this. So this is
[02:57] linking racks.
[02:59] These are traditional traditional
[03:01] transceivers. Nvidia is now proposing to
[03:04] introduce co-ackaged optics for this
[03:06] starting later this year. We'll see uh
[03:09] we're all looking forward to that. But
[03:11] the scale up portion of the network. So
[03:14] that's basically scaling the size of the
[03:16] individual computer clusters. That is
[03:19] still done in copper and it's rapidly
[03:21] coming to an end. Uh here is a picture
[03:24] of uh the Nvidia flagship the NVL72
[03:28] and you can see uh in the back plane.
[03:30] This is a massive uh cluster of that
[03:34] cables over 5,000 per rack but they're
[03:37] all limited basically to one meter
[03:38] reach. So um what was what's happening
[03:43] is a lot of talk about how do we get
[03:46] past that limit? How do we start to
[03:49] expand from one rack to multiple racks?
[03:52] There was a terrific workshop hosted by
[03:54] Nvidia not surprisingly at uh OFC where
[03:58] they summarized uh sort of five key
[04:00] aspects of what's needed in terms of
[04:03] parameters energy efficiency much below
[04:06] the five picoles that are considered
[04:08] kind of the the sound barrier currently
[04:10] for optical links um shoreline density a
[04:15] terabit or better per millimeter reach
[04:17] doesn't really need to be more than 10
[04:19] meters we now see there's another
[04:22] version of this um coming up and that's
[04:24] out of OAF. They're proposing to go
[04:27] extend that to 20 mters. High
[04:30] reliability, higher than other optical
[04:32] links and of course uh the cost target
[04:36] it can be in the range of current
[04:39] optical links. I'm just going to flash
[04:42] up what we how this would stack up with
[04:45] uh with LED links. um energy efficiency
[04:49] we can definely achieve picole per bit
[04:51] or less.
[04:53] Uh we can match the necessary bandwidth
[04:56] density because we're dealing with two
[04:58] the arrays. Um we can extend the reach
[05:01] to 20 meter as needed. And in terms of
[05:05] fit rates, well we don't have those
[05:08] those um new links are not deployed uh
[05:11] in large numbers yet. That's what will
[05:13] be needed to get a final fit number. But
[05:16] we're off to a good start with gallium
[05:18] nitride LEDs because they're generally
[05:20] very reliable. They're very temperature
[05:23] tolerant. Um we can we can easily do a
[05:26] milspec range. Uh we've actually shown a
[05:29] demonstration with our partner at
[05:31] University of North Carolina. They
[05:34] showed it operating all the way to 400°
[05:36] C. So lots of other things need to come
[05:38] into place for that. And they're very
[05:40] cost effective again because it's a
[05:43] technology that's getting optimized by
[05:46] micro display and all the whole
[05:48] um
[05:52] ecosystem that goes with it and those uh
[05:54] use millions of LEDs. We only need a few
[05:57] hundred typically per cluster.
[06:00] So how does the the whole thing look?
[06:02] It's a it's a modular architecture. You
[06:06] can see in the top right here, you
[06:09] basically use a cluster of LEDs. We're
[06:13] showing two clusters here, depending on
[06:15] how you architect it on a typically like
[06:18] a a kind of
[06:21] the 1200 meter process node. Uh next to
[06:25] it, you have the PDS. So those are your
[06:28] two transmit and receive front end chips
[06:30] if you like. uh vertical emission
[06:33] through a lens array
[06:36] going through your uh through your fiber
[06:39] coupler transport by fiber and then
[06:41] terminate on the other side on the LEDs
[06:44] depending on what architecture you use.
[06:46] If you use high-speed certities uh
[06:49] because fact of life today all scaleup
[06:51] networks still run in in 200 Gbit per
[06:55] lane. So that is that is an architecture
[06:58] that the ecosystem has chosen. In that
[07:01] case you'll have a high-speed sudis here
[07:04] that will add an energy tax but that's
[07:06] basically temporary out here. You have a
[07:09] die to die interface
[07:12] and then what you can do with this uh
[07:14] architecture is um you can use it uh on
[07:20] board so outside of the package because
[07:24] once you go inside the package which is
[07:26] definitely uh an architecture that we're
[07:29] also pursuing that will be co- package
[07:31] but that's a much bigger effort in terms
[07:33] of uh collaboration with your uh with
[07:36] your processor um design entity.
[07:42] Few more words on how the technology
[07:44] works. By the way, uh at 2 p.m. this
[07:47] afternoon, our founder and CEO Baria
[07:49] Beski, he'll give a talk uh where he
[07:52] dive much deeper into the technology.
[07:54] I'm going to focus more on the
[07:55] applications in this uh in this um uh
[07:58] talk here. But what we do is the the
[08:03] LEDs, they're grown on sapphire gallium
[08:06] nitride. They're then flip chip
[08:07] transferred using processes that have
[08:09] been been developed for the micro
[08:11] display industry, transferred onto CMOS,
[08:14] can be transferred onto any CMOS node.
[08:16] Uh you focus so so this is what the LED
[08:19] array looks like. You focus through a
[08:23] lens array. These look like these egg
[08:24] holders here. That's the array that you
[08:26] see here. couples into the uh fiber,
[08:30] transports across, comes out on the
[08:32] other side, gets again focused, passes
[08:34] through a micro lens array, and then
[08:37] gets terminated on PDS. Since we're
[08:40] operating in the blue by 425 nanometer
[08:44] silicon is actually an excellent
[08:46] detector. Uh you have an absorption
[08:48] length of less than a micron. So this is
[08:51] there's no exotic materials needed. So
[08:53] this thing is geared for simplicity and
[08:56] low cost.
[08:58] For those who like to think in terms of
[09:01] diagrams, this is a diagram that was
[09:03] originally generated by DARPA, but it
[09:06] shows it compares different interconnect
[09:08] types using the reach across the bottom.
[09:11] Here is one meter as you can see to the
[09:15] to the right. Um as you're going out,
[09:18] optical as always dominated. uh and for
[09:21] shorter reaches copper has always
[09:23] dominated and still does. And what you
[09:26] have on the y-axis is a figure of merit.
[09:29] It consists of bandwidth density. How
[09:32] much bandwidth can you get out on a of a
[09:35] millimeter of shoreline in a chip? And
[09:37] then energy efficiency. That's a big
[09:40] that's a big topic. This is expressed in
[09:42] pajles per bit. Um and so we're talking
[09:45] energy efficiency. So the bigger this
[09:47] num this figure of merit the better your
[09:49] interconnect. And basically here at this
[09:52] level this take this uh copper interface
[09:55] HBM that we know today goes roughly over
[09:58] a millimeter distance from a processor
[10:01] to an HBM stack.
[10:03] As you can tell the microLEDs can extend
[10:07] that efficiency out to about 10 mters.
[10:11] Um if you need to go further distance
[10:14] other technologies are better suited.
[10:16] Those are all laser based and they all
[10:18] have u higher energy consumption or
[10:21] lower efficiency. But that's where the
[10:24] that's the kind of the sweet spot and
[10:26] that's exactly the sweet spot that we
[10:28] are looking at for scale up.
[10:33] What do we do uh with this technology?
[10:36] We are coming out with a full demo kit
[10:39] later this year.
[10:41] Um the first product while not the ideal
[10:45] application but it is something that can
[10:47] still benefit from the energy efficiency
[10:49] is basically package it into an an AOC
[10:53] or what's now coming up uh a lot of
[10:56] different entities are proposing
[10:58] highdensity pluggable connectors because
[11:01] as was pointed out by Microsoft in the
[11:03] previous talk we're looking at uh
[11:05] interconnects starting around 6.4 4
[11:08] terabs but rapidly scaling to doubling
[11:10] that to 12.8 and then probably doubling
[11:12] that again to 25 terabs.
[11:17] And of course this is a more or less a
[11:20] copy exact of the 1.6 terra uh tiles if
[11:24] you like.
[11:26] This is still all outside of the
[11:28] package. The vision uh remains to go
[11:31] inside the package eventually be right
[11:34] on the interposer what a lot of people
[11:36] refer to as CPO and what we're also
[11:39] getting now and that's a little further
[11:40] out that's more in the 2030 time frame
[11:43] we're getting a lot of interest in uh
[11:46] optical memory interconnects and one
[11:49] driver uh has become these these GPUs
[11:54] next generation GPUs people are
[11:56] projecting up to 2,000 watt power
[11:59] consumption. So you need to get the
[12:01] memory away from the hot source. So
[12:04] that's that's what's driving this. This
[12:05] is not so much about memory pooling.
[12:07] That's another architecture that's also
[12:09] in discussion. But this is really just
[12:11] getting memory outside the package and
[12:13] spreading it out also make it accessible
[12:15] for a larger memory bank.
[12:18] Just a a couple of words again. There
[12:20] will be a talk this afternoon but in
[12:22] latest technical achievements we've
[12:24] while current um the current um
[12:27] incarnation of LEDs run at 4G per lane
[12:31] we've already shown 16 per lane
[12:33] performance now uh in our labs and we've
[12:36] also uh partnering with TSMC on the
[12:39] detectors we've also shown ultra low
[12:41] power links because you get uh you get
[12:44] much more efficient um better
[12:45] responsivity on your detectors you can
[12:48] uh then dial back the power on the
[12:50] transmitter. So we've achieved a 02
[12:52] picole bit per link. Uh as you can see
[12:55] here, this is an individual link and
[12:57] still the bit error rate of a little
[13:00] better than 10 the minus 7.
[13:03] With this I'll I'll summarize. I know
[13:05] I'm coming up on the the end point here.
[13:08] um
[13:10] microledd interconnects they're really
[13:13] well suited for scale up network scale
[13:16] up networks because of their energy
[13:18] efficiency their shoreline density think
[13:21] a terabit per millimeter the reach 10 to
[13:24] 20 mters so this is sweet spot if you
[13:26] need long distance this is not the
[13:28] optical technology but the 10 to 20 is
[13:31] really in the sweet spot for scale up
[13:33] and uh the operating temperature is very
[13:36] wide now in data centers That's usually
[13:39] not an issue for ambient temperature,
[13:41] but um with hot running processors, you
[13:44] do actually a lot of these processes run
[13:46] beyond 100° C. So, it is it is
[13:50] definitely good to know that we're
[13:51] covered on that front.
[13:53] And with this, I'll uh turn it over to
[13:55] Dan again and thank you very much for
[13:57] your attention.
[14:02] Thank you uh Chris. Um, fascinating
[14:06] developments and they continue to move
[14:07] along quickly. Um, we would love to have
[14:09] your slides available. Again, if you
[14:11] could remove the confidential markings
[14:12] then people won't feel conflicted about
[14:14] downloading them. So thank you Evasenna.