# Co-Package Optics at Scale: NVIDIA's Road to Sub-1 pJ/bit | Liron Gantz | Optica GAMA 2026

https://www.youtube.com/watch?v=Edqv8P05pt4

[00:02] [music]
[00:05] It was the dream of Olga when we set
[00:08] this group to make sure that the right
[00:11] companies in semiconductor manufacturing
[00:13] were here. Why? because everybody's
[00:14] talking about what a big company of 5
[00:18] trillion
[00:20] can do in copakage optics and they
[00:23] realize that there are many challenges
[00:25] whether it comes to bring lots of fibers
[00:26] into the CPO whether it comes to produce
[00:28] this in volume production when it comes
[00:30] to achieve the right yield working with
[00:32] different foundaries incorporating new
[00:34] technologies we wanted to have the right
[00:35] person at Nvidia who is looking at new
[00:38] technologies and incorporating them into
[00:40] the role of CPO and we couldn't find
[00:43] there is no better person than our
[00:45] keynote speaker today. Ladies and
[00:47] gentlemen, Leon Gans from Nvidia.
[00:53] [applause]
[00:53] Thank you.
[00:56] Thank you very much. I'm I'm honored to
[00:59] open this uh to be the first uh keynote
[01:02] speaker.
[01:04] Oh, okay. Yeah.
[01:06] Uh I'm hoping not to disappoint and if I
[01:09] do, please don't throw things at me.
[01:11] This is a brand new suit. So,
[01:15] uh, so with that we we'll start a a bit
[01:18] of caveat. Since I'm coming from a very
[01:19] big company, I'm going to talk to you
[01:21] about past a lot about the present. But
[01:25] regarding the future, I'm representing
[01:27] my group which is a research group. So,
[01:29] we are not committed to the uh uh to the
[01:32] road map. I can only comment on things
[01:34] that are uh existing and and have been
[01:37] declared. I cannot comment on something
[01:39] that is on the future. So again
[01:41] apologies but it is what it is. So with
[01:45] that let's start. So
[01:48] every I think AI related uh talk has to
[01:53] start with this with this graph. Uh this
[01:56] is uh this is actually the amount of
[01:59] compute that is required for each of the
[02:02] uh AI uh models. And uh as you can see
[02:06] we around 2018 we went from uh Moors law
[02:11] to a hyper Moors law or as people refer
[02:14] to it as Jensen law.
[02:17] And every two years we are a tfold there
[02:20] is a 10fold increase in the number of uh
[02:22] of flops or the amount of uh needed
[02:25] computing. But the good news is that we
[02:27] actually solve this problem and we solve
[02:29] this problem long ago. we switch to
[02:31] parallel parallel computing and that
[02:33] gives us the the amount of compute
[02:36] needed. Uh but in order to do that we
[02:39] have to use interconnect and that
[02:41] relates to the previous talk on how the
[02:43] infrastructure uh has to be optics and
[02:46] without this data center are just u a
[02:49] very big ovens where you can boil steaks
[02:52] on. Uh so when we're talking about
[02:55] interconnect there are actually three
[02:57] main domains that we are that we can
[02:59] divide it. So the first domain is scale
[03:01] up where we connect uh a few GPU to one
[03:06] uh super GPU and uh with that regard
[03:10] this is uh this this is how it looks
[03:12] like. This is like this is the NVL72
[03:15] where we connect 72 GPUs. Uh GB200 this
[03:19] is Grace Blackwall. So Grace is the CPU,
[03:22] Blackwall is the is the GPU. uh and as I
[03:26] said the scale up is a very very high
[03:28] bandwidth uh uh connection uh that
[03:30] connects a lot of GPUs to one big super
[03:33] GPU. The next domain is a scale out
[03:36] which regular networking uh this this is
[03:39] what we know as regular networking. You
[03:42] have the top of the REC switch and then
[03:44] you connect a few Rexs or across the
[03:46] data center. uh so from few meters to uh
[03:51] usually 2 kilometers that is the that is
[03:54] the scale out and here uh copackage
[03:57] optics come into play and I'll spend a
[03:59] lot of time talking about it the next uh
[04:02] domain is kind of new is the scale
[04:05] across where we connect a few data
[04:07] centers uh and we are start talking
[04:11] about how to do this and how to train uh
[04:14] models across different data centers uh
[04:17] So the scaleup uh network is uh governed
[04:20] by the uh copper cables. They are still
[04:23] the most efficient and the most um the
[04:26] most cheap. Uh scaleout networks is
[04:28] optic and of course also the scale
[04:29] across is optic. So hoping that this
[04:33] will be this will change in the future.
[04:35] Maybe at the end of my talk you'll see
[04:37] how that can happen.
[04:40] So yeah, so we need to as as you seen in
[04:44] the scale up, we need to we need to
[04:46] transfer the LV72 into this huge AI
[04:50] factory. Uh and just for you to to
[04:54] understand, this is the sizes of the of
[04:57] the new AI factories. They're a few
[04:58] kilometers across. I think there is a
[05:00] new one that uh I was in a conference a
[05:03] few uh a few months ago where someone
[05:05] showed the map of Manhattan and the new
[05:08] data center is is a big proportion of
[05:11] it. So data centers are becoming a huge
[05:14] and we are moving into the extreme scale
[05:16] out uh domain.
[05:19] The other thing I want to explain is
[05:21] that let's remember not every data data
[05:24] center is an AI factory. So and we have
[05:26] still the old-fashioned uh um data
[05:29] center which are the backbone of the
[05:31] internet. There are using Ethernet and
[05:34] they are um optimized for resiliency. Uh
[05:37] we have the high performance computing.
[05:39] They use infiniband that's what made uh
[05:41] Melanox very famous. Uh they use various
[05:44] topologies and they're optimized for uh
[05:47] performance at scale. And we have also
[05:49] the AI factories. They're using the envy
[05:52] links and for the scale out the infinity
[05:54] band and they're optimized for cl for uh
[05:57] cluster performance. I told you that by
[06:00] using parallelization we actually solve
[06:03] the problem of the amount of compute we
[06:05] need. The bad news is that the compute
[06:09] does not scale as memory and does not
[06:11] scale as interconnect. So we have to
[06:13] bridge this gap and we're breaching this
[06:16] gap using interconnects or increasing
[06:18] the number of interconnects
[06:19] exponentially
[06:21] and that's one of the reason power from
[06:24] interconnect is exploding and as you can
[06:25] see uh
[06:29] in regular data center this is what the
[06:32] transceiver power takes but this is an
[06:35] order of magnitude more is what is
[06:36] required for a AI uh factory. So optics
[06:41] is is is a main is a very big portion of
[06:44] the power consumption of an of a data
[06:46] center and data center at all just for
[06:49] you to to get a perspective I think that
[06:53] uh in the next in the last few years
[06:55] that was the power consumption of a data
[06:57] center were 4% of total power
[06:59] consumption around the world. So that's
[07:02] uh that's a huge portion.
[07:06] And of course uh one other thing to
[07:08] remember is that networking is actually
[07:10] a necessary evil. We want to compute. We
[07:12] don't want to we don't want to uh
[07:14] communicate. So every watt that goes for
[07:16] networking is not spent on compute.
[07:18] Having said that the bottleneck today is
[07:21] not is is actually moving the data
[07:23] around. So a lot of the time the GPUs
[07:25] are standing idle because they're
[07:26] waiting for uh for data to move from one
[07:29] place to another. So this is it's a
[07:31] necessary evil but it's an important uh
[07:33] it's an important part.
[07:37] Uh let's also talk about what makes uh
[07:42] uh what makes an AI or what makes
[07:44] interconnect uh efficient. So I want to
[07:48] start with the second uh second point
[07:50] here which is throughput that is
[07:53] actually understandable. You saw the the
[07:55] amount of compute we need. So I showed
[07:57] you that we need to increase the
[07:58] interconnect exponentially. So
[08:00] throughput is is logical and and
[08:02] understandable but I want to put an
[08:05] emphasis on the first uh the first uh
[08:09] article here and that is radics and that
[08:11] is often uh ignored I think and and I
[08:15] want to in every talk that I'm that I'm
[08:17] giving I'm I'm putting a light on this
[08:20] uh on this issue. So what is radics? ICS
[08:23] is basically the numbers of IO that you
[08:26] can out you can get out of a of a
[08:28] switch. The number of ports essentially
[08:31] now why is that important? I have a very
[08:33] simple uh example here. The higher the
[08:36] radics the flatter the network. So for
[08:39] example we have two uh two network here.
[08:42] This is with radics 2. So every switch
[08:44] is connected to two nodes. And here we
[08:46] have radics 4 which every switch
[08:49] connects into four nodes. So if I want
[08:51] to connect this nod to this to here in
[08:54] this kind of network I need to go one
[08:56] two three four hops and I connected this
[09:00] nod with this node with three switches
[09:03] in this network to connect these the
[09:05] same node I need one two hops and
[09:08] they're doing this with one switch. So
[09:11] every hop is latency. So when I increase
[09:14] the radics I reduce the latency but I
[09:17] also reduce the power because in this
[09:19] network I only use one switch and here I
[09:21] use three. So radic saves latency and in
[09:24] this kind of topology it also saves
[09:26] power. So that is the importance of
[09:28] radics. So that is something to bear in
[09:30] mind and it's going to connect
[09:32] beautifully to how we attach fibers into
[09:35] our chip.
[09:37] So another very uh important KPI of
[09:40] course is reach. That is why we cannot
[09:42] use copper for the scale out reach. The
[09:44] reach of copper is is three meters.
[09:47] Optics can go from millimeters to
[09:49] kilometers.
[09:51] Uh power efficiency of course the the
[09:53] the more efficient we are the less power
[09:55] we consume for the same uh for the same
[09:58] work. Latency as I uh as I mentioned you
[10:01] don't want to wait when you talk to CHP
[10:03] you don't want to wait 20 minutes before
[10:05] it responds. So that's that's directly
[10:08] related to latency
[10:10] and edge bandwidth density also known as
[10:12] shoreline density or beachfront. This is
[10:15] what connects to radics actually. Uh the
[10:18] higher the density the more uh IO's we
[10:21] can we can break out
[10:24] and of course we have the area bandwidth
[10:26] density and let's remember again that
[10:30] networking competing with compute. So we
[10:32] want we want to save we want high
[10:35] bandwid high area uh high area bandwidth
[10:39] in order to to achieve a lot of u a lot
[10:42] of connectivity. So let's briefly
[10:46] mention what happened in the past or
[10:47] even even in the in the present. So in
[10:51] the past and even in the present we are
[10:53] working in what I call the narrow and
[10:56] fast approach which means we are using
[10:58] one wavelength or four if it's FR4 but
[11:01] but we are increasing the rate each uh
[11:04] each generation and one time we switch
[11:07] from NRZ to PAM 4 which is a different
[11:09] kind of modulation and and we are doing
[11:11] this because we are what I call bump
[11:13] limited so the number of uh the number
[11:16] of bumps between the organic package and
[11:19] the printed circuit board is limited.
[11:22] These are C4 bump. So in order to
[11:24] increase the rate we have to aggregate
[11:26] data. So it's essentially we I'm saying
[11:30] every time that this is this is all
[11:32] about the plumbing and we are glorified
[11:34] plumbers. So if you have limited number
[11:36] of pipes then each pipe has to carry a
[11:38] lot of water and that's exactly what
[11:40] happens here. And since each pipe
[11:42] carries a lot of water or each uh each
[11:45] bump carries a lot of uh higher rate of
[11:48] uh of signals, then the problem is that
[11:51] in the pluggable area where the signal
[11:53] had to go all over the printed board, we
[11:56] had a lot of losses which made uh
[12:00] because of that we had to compensate. So
[12:02] so links were very expensive and until
[12:05] they reach the uh the front panel you
[12:08] had to resample them. So not only you
[12:10] need to aggregate a lot of data, you had
[12:12] to resample the uh the signal around
[12:14] here and that spends and this is caused
[12:17] to cause you to spend a lot of uh
[12:18] energy.
[12:20] Uh so and of course a lot of energy that
[12:24] goes uh all losses of the energy goes to
[12:28] heat. So so you need to so you need to
[12:30] uh evacuate it.
[12:33] So just by moving this connection to the
[12:36] organic package and that is essentially
[12:38] c- package optics we shortened the reach
[12:41] of this electrical signal on the expense
[12:43] of more uh more complex packaging.
[12:46] But we saved 40% of the power just by
[12:50] saving this this route that the signal
[12:53] has to go much much uh shorter route
[12:55] than than before. we have we are still
[12:58] bump limited and I'll I'll continue to
[13:01] talk about this later but
[13:04] by doing this we save uh 40% of the
[13:07] power and with Nvidia solution uh we use
[13:10] the TSMC coupe uh and I'm hoping this is
[13:16] not something you you are seeing at the
[13:17] first first time these are the two uh
[13:20] products that were uh announced by
[13:21] Nvidia so the first one is quantum for
[13:24] the uh infiniband and spectrum for the
[13:26] ethernet it and uh I will and I will
[13:31] talk about them in a moment but for now
[13:33] I will let Nvidia creative team uh
[13:36] introduce you to this uh to this product
[13:41] oh there is no music so um
[13:44] oh where
[13:50] >> [music]
[14:37] >> Heat. Heat.
[15:10] So, it probably won't surprise you to
[15:12] know that the music was generated by AI.
[15:15] So, and every time I'm hearing it, I I
[15:18] have the the the urgency to do like a
[15:21] like a conductor. And since AI is going
[15:24] to take all our job anyway, that's the
[15:25] only thing we can do. So,
[15:29] okay. So, let's dive in.
[15:31] Uh so this is uh so for those who didn't
[15:34] manage to catch all the details in the
[15:36] movie this is uh uh this is u this is
[15:38] the breakdown. So we have the futonic IC
[15:41] which is based on ring modulators. I
[15:43] think Nvidia is the only one that uses
[15:45] ring modulator at this point. All our
[15:47] competitor are using uh uh different
[15:49] modulators.
[15:51] Uh we have the electronic IC over the
[15:54] futonic IC. This is this is part of the
[15:56] coupe process.
[15:58] Uh and a part of the cool process we
[16:00] have a micro lens at the uh at the top
[16:04] of this uh of this assembly and we put
[16:07] an F fau fiber uh array unit and in
[16:12] quantum there are three of these units
[16:14] in each socket
[16:16] and this is uh this is the this is the
[16:18] optical the entire optical engine and
[16:20] the ASIC
[16:22] and we are feeding it with ELS external
[16:25] laser model which is a high power model
[16:28] uh which feeds few of these uh few of
[16:31] these channels and I like to think about
[16:32] it as the optical battery like we have
[16:35] uh like uh capacitors are the battery of
[16:37] electronics lasers are the battery for
[16:40] photonics two main differences between
[16:43] the two these two products so the
[16:45] quantum is has a socket base uh assembly
[16:49] uh the spectrum will have detachable
[16:51] connectors and I can't emphasize enough
[16:53] how detachable connectors are very very
[16:55] important to scale the CP CPO solution
[16:58] the CPOS as you can understand the CPO
[17:00] the CPO assembly is very very expensive
[17:03] the packaging is very very expensive and
[17:06] there is what we call the blast radius
[17:07] so what happens if something goes wrong
[17:10] and of course and we if we don't have
[17:11] detachable connectors the blast radius
[17:13] is is pretty big if we have detachable
[17:15] connectors it's becoming uh a bit
[17:17] smaller and of course uh we have the
[17:20] problem of alignment so we did a very
[17:23] simple calculation uh it's enough to to
[17:26] deviate made by fraction of degrees and
[17:29] it will have considerable loss in the in
[17:31] in the channel. So by using u detachable
[17:35] connectors you can you can even you can
[17:37] correct if if something if something
[17:39] goes wrong and of course we are using
[17:42] surface coupling because uh edge
[17:45] coupling limits the edge bandwidth
[17:47] density. uh now I know that there are a
[17:49] few works on doing some 2D array of uh
[17:52] edge coupling but I think it's much
[17:54] harder to do surface coupling because
[17:56] surface coupling you can do easily you
[17:58] can do 2D array and increase the edge
[18:00] bandwidth density and this is of course
[18:03] the LS uh since we are working at very
[18:06] very high rate this is a 200G per lane
[18:08] so there our receiver has our receiver
[18:11] is not that or receiver at that rate are
[18:15] not very sensitive so we have to use a
[18:17] very very high power laser and it also
[18:19] feeds a few uh a few lanes. So uh our uh
[18:23] our partners developed uh a very nice uh
[18:26] very strong uh DFB laser uh with low
[18:30] green. Low green is very important
[18:31] because we are modulating at pump 4. So
[18:34] there are four levels and green is
[18:37] really destructive in that kind of
[18:39] modulation and of course very narrow
[18:40] line with because we are using ring
[18:42] modulators and they require a narrower
[18:44] line with other than other modulators.
[18:47] But this is essentially the the ELS.
[18:51] Now when I started working on uh on the
[18:54] CPO back in 2018, there was no uh wafer
[18:58] level testing machine for futonics.
[19:00] These are these are new developments in
[19:02] in recent years or at least there wasn't
[19:04] for uh for product. There was research
[19:07] oriented one. But gladly uh with our
[19:10] partners we developed uh wafer level
[19:12] testers. uh but I can tell you that more
[19:15] of them uh are needed.
[19:18] Uh and of course at coupe level this is
[19:22] even more complex because if you come if
[19:24] you think about it we have the you have
[19:26] the optical IC and the and the
[19:28] electrical IC over that. So you have to
[19:30] probe the the the optics from here but
[19:34] the pumps are over here. So the
[19:36] electronics you have to probe from here.
[19:37] So we need to develop these kind of
[19:39] machines that you can probe from both
[19:41] sides which are not trivial at all. Uh
[19:44] and these machines are again as I said
[19:46] with with the help of our partners we we
[19:49] developed these but but these this is
[19:51] this is uh uh we need more of them. So
[19:55] these are solution that are still
[19:56] waiting. So let's talk also on the
[19:58] assembly. So the assembly had a
[20:00] potential to be very very complex but we
[20:02] actually found a way to do this in a
[20:03] modular way. So we are assembling each
[20:06] model and then like Lego just uh just
[20:08] connecting them. And if you look at uh
[20:12] an old pluggable box and you look at a
[20:15] co-ackage box, I think the co- package
[20:17] looks a lot a lot nicer and a lot
[20:18] neater. And actually uh it's also saving
[20:22] time. So to do an assembly of a CPO box
[20:26] is six times more efficient than to do
[20:29] an assembly on an old box uh uh using uh
[20:32] using pluggable modules. So even with
[20:34] that we we save time and I think I I
[20:37] don't know how many how many of you were
[20:39] at ECO this year but actually Facebook
[20:42] presented uh very nice work that showed
[20:44] that CPO is actually more reliable than
[20:47] pluggables and it makes sense because
[20:49] there are less movable parts. So the
[20:52] potential to be more reliable is is
[20:54] already there.
[20:57] Okay. So this is this was the present.
[20:59] Let's talk about the future. And here
[21:00] things are becoming a bit more
[21:02] speculative but I'll try to give both uh
[21:05] uh both aspects.
[21:08] So I call this how do we how what is the
[21:10] road to uh subpo bit links.
[21:14] So what's our scaling options? So until
[21:16] today because we were bump limited we
[21:19] had only two options of of scaling
[21:21] actually three but but two are are the
[21:24] main the main two ways is so one of them
[21:27] is time domain. So let's increase the
[21:29] rate from 100G to 200G to 400G and so
[21:32] on.
[21:34] The other option is to use higher uh
[21:37] higher schemes of modulation. So a few
[21:39] years ago we switched from NRZ to PAL 4.
[21:42] We can switch to PAL 6, PAL 8 or
[21:44] coherent. But notice that I draw a brick
[21:48] wall at each one of them. So when we
[21:51] increase the time domain, we are getting
[21:53] diminishing energy efficiency. And the
[21:56] reason is that it doesn't that the
[21:57] energy doesn't scale linearly. So you
[21:59] increase the rate you are paying uh
[22:03] double or triple the price because you
[22:05] need much stronger devices. These
[22:07] services that aggregate the data and
[22:10] these services are layers of layers of
[22:11] maxes and the and uh front layer maxes
[22:15] are uh consuming much more energy than
[22:18] than the back layer. The second thing is
[22:22] that we need a lot of DSP. we need
[22:25] resembling and the DSP is very very
[22:27] power hungry. The other thing is we use
[22:30] fck so we don't get the bit error rates
[22:32] that we used to get at NRZ. Uh it's not
[22:35] it's not error-free. So in order to
[22:37] correct it we need to use forward error
[22:39] correction that that is fck and there
[22:41] are and these devices are very very
[22:43] power hungry. So when you summarize all
[22:47] of this you can see that the the the
[22:49] energy does not scale linearly with
[22:52] regard to higher amplitude uh we get
[22:55] diminishing uh return regarding signal
[22:57] to noise. So and and this is very
[23:00] obvious because let's say you had an
[23:03] amplitude in NRZ. So it's just an on-off
[23:05] key. So this is the amount of of uh of
[23:08] amplitude that you have now you are
[23:10] doing pal 4 you are dividing this amount
[23:12] by four times. So the signal to noise is
[23:14] is uh much worse. And if you go to pal
[23:17] six or pal 8 it's it's going to be even
[23:20] worse. So so these are the diminishing
[23:23] returns that you get from uh signal to
[23:25] noise. Uh the other option is
[23:27] polarization but that only gives you
[23:29] twice uh twice the rate and maybe there
[23:33] are things that we can use polariz there
[23:35] are other problems that you can use
[23:37] polarization for. So let's let's save
[23:39] it. So this is what I refer as the
[23:41] narrow and fast. We use one wavelength
[23:44] and or one or four or uh and very very
[23:49] fast modulation. As I said, this this
[23:51] this generation is 200g, but the next
[23:54] one will probably be 400. So, this is so
[23:56] this is the narrow and fast. And again,
[23:58] I want to emphasize this is uh this is a
[24:02] constraint because we are bump limited.
[24:07] Okay. And so, which means that if we
[24:09] have x amount of data that we want to
[24:12] transmit and we have only n connection,
[24:15] so each connection is x divided by n.
[24:17] It's very very straightforward. And the
[24:20] solution is maxing. So as I said if we
[24:22] this is this is an example that I'm
[24:24] giving on the time domain but it's the
[24:25] same with the amplitude and because we
[24:28] are bound limited we have to aggregate
[24:30] the data and these are the devices of
[24:32] the services the serial the serial are
[24:34] the devices that are doing this maxing
[24:37] and as I said each each layer of moxing
[24:40] cost more than the than the uh previous
[24:43] layer
[24:45] the other thing is that since we are
[24:46] working at it's so such a high rate it's
[24:49] no it's not it's no longer SMOS uh
[24:52] devices are doing this is the CML uh
[24:54] logic that that is doing the driving
[24:58] uh and as long as we are bump limited we
[25:02] need to the as I said the only two ways
[25:05] we can increase is by rate so that's
[25:07] where new materials can come into play
[25:10] and in the in for in the optics
[25:12] community and so I asked perplexity to
[25:16] generate a a graph of what is the
[25:20] demonstrated rates that people showed uh
[25:23] and it's amazing. It used to take me
[25:25] like a day to do these kind of things
[25:27] because you have to read a lot of
[25:29] papers. So, Perplexity did it in 5
[25:31] minutes and that including the time that
[25:33] it took me to verify that the that the
[25:36] reference that it gives me is not is not
[25:38] made up. So, I highly recommend starting
[25:42] using that. uh but as you can see we
[25:45] have TFLN which has very nice bandwidth
[25:47] that can allow us to go to 400G uh and
[25:51] even and plasmonics are are amazing in
[25:54] that uh in that sense. So as long as we
[25:57] are bump limited, new materials are
[25:59] probably a must. But you need to take
[26:01] this with a grain of salt because even
[26:04] if I have an optical device that can go
[26:08] to that rate, I still need to drive it
[26:10] electrically. And the question is what
[26:13] kind of what kind of electronics will
[26:15] drive 200 uh 200 gigahertz? And that's
[26:20] the that's the real question we need to
[26:21] ask. So I know that people are in in in
[26:24] our community the optics community are
[26:25] running to demonstrate yeah we have we
[26:27] have uh devices that can go up to
[26:29] terahertz fine if I cannot drive it
[26:32] electrically it doesn't help me
[26:35] uh but if but if we uh for example for
[26:38] the for the next generation of 400G
[26:40] we'll we'll still we'll still drive it
[26:43] electronically and again people are
[26:46] saying that electronic is at the at the
[26:48] limit but they've been saying it for 10
[26:50] years so I'm Not just I'm not uh saying
[26:53] that they won't find a solution but
[26:55] there isn't a magic solution. The
[26:57] solution that they will find will will
[26:59] depend on uh uh inductive picking uh
[27:03] which is which means that we are adding
[27:05] inductors into our electronics IC and I
[27:09] don't know if you if you've seen
[27:10] inductors inductors are huge they're a
[27:12] huge part of the of the circuit. So we
[27:15] are so we are doing this on the expense
[27:17] of area bandwidth uh density.
[27:20] Uh the other thing is that we will use
[27:23] heavy DSP. We'll use all the tricks that
[27:25] we know heavy DSP, inductive peaking and
[27:28] we as I said we we won't get the bit
[27:31] error rate that we want. So we'll have
[27:32] to use very very strong facts. So again
[27:35] if we continue this way the power
[27:38] consumption is not growing linearly.
[27:40] It's going to grow much higher than uh
[27:42] than linear.
[27:44] But the other alternative is if somehow
[27:48] we can be not bump limited and that's
[27:51] where advanced packaging come comes into
[27:54] play and I want to emphasize something
[27:57] that I think that futonics becomes
[27:59] synonymous to advanced packaging. Uh
[28:02] these things are becoming so related to
[28:04] each other. So imagine that we have what
[28:07] we call 2.5D optics and again you don't
[28:10] have to imagine it's actually in the
[28:12] works
[28:14] in which we we which introduce silicon
[28:16] interposer and now we are not bump
[28:18] limited we are we are we have the the
[28:20] density of the bumps or now we can call
[28:22] them micro bumps is much higher. So
[28:25] there is no reason to increase the rate.
[28:28] We can incre we can increase the number
[28:29] of modulators
[28:31] and reduce the rate and this is what we
[28:34] called wide and slow
[28:36] and I will advocate that if we can go to
[28:39] wide and slow it's much better than
[28:41] going narrow and wide and you'll save a
[28:44] lot of energy. So these are the
[28:45] advantages. This is something that
[28:47] already been demonstrated in the in the
[28:49] literature. You can find it here. But of
[28:51] course there are challenges. What are
[28:53] the challenges? This is uh this is a
[28:56] schematic of of how the CPO looks today.
[28:58] This is how it looks like with the
[29:00] silicon interposer. So the main
[29:02] challenge is how we break out of this
[29:04] because we haven't increased the beach
[29:05] front. We actually decreased it. So
[29:07] that's the first challenge. The second
[29:08] challenge is to find modulators that are
[29:11] low and that are dense enough and and
[29:13] small enough. So but but that's not
[29:15] actually a challenge. We we I think we
[29:17] actually solved it because that's that's
[29:19] one of the reason that Nvidia used uh
[29:21] ring modulators. So ring modulators are
[29:23] easily scaled. Uh but I don't think that
[29:26] in this in this scheme we can use MCI
[29:29] anymore. So we have to go to either ring
[29:32] modulators or even uh electroabsorption
[29:35] modulators that are they're also small
[29:37] enough. So these are the main
[29:39] challenges. So now I'm going to wave my
[29:42] hand and explain to you why this is
[29:43] better and then I'm going to show you
[29:45] some numbers.
[29:47] So first of all, oh sorry uh
[29:51] with the with the wide and slow there
[29:53] are two more uh avenues that are open
[29:56] opening up for us. So one is the space
[29:58] domain
[30:00] uh we can use higher optical modes but
[30:03] again there is a there is a brick wall
[30:05] here because in the space domain we are
[30:08] limited by the size of the fiber. So the
[30:11] regular sizes are 127 micron. We can go
[30:14] maybe to 80 maybe 65 but that's uh uh
[30:18] that is a brick wall but I would call a
[30:20] soft brick wall because if any of you
[30:22] can come up with a way to break out of
[30:24] the ASIC and and fan out to a larger
[30:29] number then it's a valid solution that
[30:31] opens the way here. Uh, my preferred
[30:34] solution is actually the wavelength
[30:36] domain. And here I didn't I didn't draw
[30:39] a brick wall, but I'm actually cheating
[30:41] because I just dumped the problem on
[30:42] someone else. And which is okay in
[30:45] engineering if you can dump the problem
[30:47] on someone else if he can solve it
[30:49] better than you. And you can guess that
[30:52] the people I dump the problem on are the
[30:54] laser manufacturers.
[30:55] So we'll talk about this in in a few
[30:57] slides. But I would say that uh the
[31:00] wavel domain is actually natural to
[31:02] silicon phutonics. So we are we will
[31:04] take advantage of of the platform and I
[31:06] like to quote Richard Feman who said
[31:09] there is plenty of room in the fiber. Uh
[31:13] if you can break out of the ASIC. Now he
[31:15] didn't say that but if he was alive
[31:17] today I'm sure he would. So
[31:20] so okay. So now I'm going to wave my
[31:22] hand and show you why wide and slow is
[31:24] better than narrow and fast. So let's
[31:26] let's take an example of a 200 GB uh
[31:30] link.
[31:31] So if we go to the narrow and fast as I
[31:34] said our receivers are much less
[31:36] sensitive at higher rate. So we have to
[31:39] increase the power we have to and we
[31:41] have to use fck and we have to use DSP.
[31:44] Uh if we go to slow and wide we can go
[31:47] down to 32 gigabit and we can go back to
[31:50] NRZ. We don't have to do PAM 4 anymore.
[31:53] So our receiver is much more sensitive
[31:55] and we don't need fck or DSP because NRZ
[31:57] is is much easier way of modulation.
[32:02] The next thing is yeah with with regard
[32:03] to the link budget here we don't need to
[32:05] use moxing and here we do have to use
[32:07] moxing which can cause a bit more losses
[32:10] but we are improving our designs and I
[32:12] would advocate that some cases the
[32:14] moxing comes for free and I will touch
[32:16] upon this later uh circuitry. So I I
[32:20] told you that for the high uh bandwidth
[32:23] we need to use CMLS for low bandwidth we
[32:26] can go back to SMOS and it's not it's
[32:30] not very hard to do this we we we've
[32:32] been there before and the TS are not
[32:34] doesn't have to be very very strong
[32:37] with regard to the electrical uh
[32:39] interfaces here we need services we need
[32:41] to aggregate the data here maybe we need
[32:43] light services or no service at all so
[32:46] we are even saving in latency because
[32:48] every stage that you mox the data, you
[32:51] add latency. So these links don't only
[32:55] have the potential to save power, they
[32:56] have the potential to save latency.
[32:59] Uh with regard to clock distribution,
[33:01] clock distribution is much more easy at
[33:03] lower rate than higher rate. That's u
[33:06] it's understandable. Uh so the main
[33:09] thing and if there is one thing you take
[33:12] from my presentation this is the part
[33:15] that I want you to take when it comes to
[33:18] narrow and fast the burden is on the
[33:20] electronics not the futonics we have
[33:21] modulators that have a high bandwidth
[33:24] it's the electronics that we are pushing
[33:26] to the edge
[33:28] when we go to slow and wide the burden
[33:30] is on the optics not the electronics
[33:31] electronics is very easy and I would
[33:34] advocate that in optics we are
[33:35] scratching the surface there are so much
[33:37] more that we can do in optics and as as
[33:39] an optics guy it makes me very very
[33:41] happy and I can spend only two hours
[33:44] talking about how to improve this and
[33:47] and I'm sure and we can discuss this
[33:49] later uh over coffee
[33:52] and so as I said optics has plenty of
[33:54] ways to improve so now I'm going to show
[33:56] you this with numbers so we actually
[33:58] build a slow and wide transceiver and we
[34:01] presented it at the OFC
[34:04] and you can see here from the numbers uh
[34:07] we almost halfed the the power
[34:11] consumption and the area bandwidth
[34:13] density is much uh is much higher and
[34:16] even we save the laser efficiency. The
[34:18] laser efficiency is much better at it
[34:20] here.
[34:22] So this just to show you that I'm not
[34:23] bluffing this actually backed by with
[34:25] numbers.
[34:27] So now let's talk about the lasers the
[34:29] people that I dumped the problem on. So
[34:32] if we continue in the in the narrow and
[34:34] fast we are using single DFBs or array
[34:37] of DFBs and we already have this
[34:38] technology uh as as you've seen with the
[34:42] ELS case we have a very nice solution uh
[34:45] but the slow and wide is actually where
[34:48] optics can shine. So for 20 years I
[34:50] think comb blazers and modlock laser
[34:53] were uh business in the academy. So now
[34:56] they can really shine and become an
[34:58] actual product and I think that's also
[35:00] relate to what we talked about before
[35:02] that what what how can we invest in new
[35:04] companies that that are doing this
[35:06] wonderful technology because we will
[35:08] have to go to something that can give us
[35:11] a lot of wavelengths. Now I don't know
[35:13] what what is the exact number where comb
[35:16] lasers are much better than DFB. I don't
[35:18] know if it's 8 16 but it's obvious that
[35:22] it's somewhere around that. So
[35:26] comb lasers and modlock laser are become
[35:28] will become an essential part of of this
[35:30] technology.
[35:33] So I said we can improve uh optics uh a
[35:36] lot. Here are a few uh tricks that we
[35:38] can use. Again each one of them I can
[35:40] spend half an hour talking but
[35:44] uh uh but these are the main uh the main
[35:47] things. We can add an APD uh which give
[35:49] us gain. So the so the receiver is much
[35:52] more sensitive. So we can reduce the
[35:54] power. We can add gain material and then
[35:57] lower the power of the laser. uh we can
[36:01] use and this is what I'm also working on
[36:04] in in in my day-to-day is how to do
[36:06] better design how to do robust design
[36:08] that are robust to to variation in in
[36:11] process and by improving that uh and
[36:14] improving the thermal uh the thermal
[36:16] stability we can uh save on the thermal
[36:19] tuning which is a which is much high
[36:22] which has much higher cost and slow and
[36:24] wide than narrow and fast because the
[36:27] thermal tuning you can think about as as
[36:28] a DC power. Now, now you you divide this
[36:31] by the picoel by by by the number of
[36:33] beats. So when you go to narrow and
[36:36] fast, it takes a small percentage out of
[36:39] the entire power. If you go to slow and
[36:41] wide, it takes much more. So improving
[36:43] this is really really important for the
[36:45] slow and wide. Uh we can do we can go to
[36:48] process porting go from higher nodes uh
[36:51] 7 nanometer, 3 nanometers etc. and save
[36:54] power and the uh electricity there. And
[36:57] of course if we go low enough in in in
[37:00] frequency we don't have to use PLLs and
[37:03] we can improve the uh efficiency of the
[37:05] drivers. Uh and when we use all these
[37:09] sticks and tricks we can actually go
[37:11] below one picoel per bit and we actually
[37:14] published a very nice paper about this
[37:16] uh a few months ago. So why uh sub one
[37:20] picoel per bit? This is a psychological
[37:21] barrier. But at that at that point,
[37:25] optics becomes much more efficient than
[37:28] copper, which opens the way to not just
[37:31] using optics at the scale out, it's also
[37:35] opens the way to use optics in the scale
[37:37] up. And if you thought you had a problem
[37:41] uh with high volume manufacturing now
[37:44] wait until it gets to the scale up
[37:46] because that's the [laughter]
[37:47] that's the that's a huge volume. When we
[37:51] go to the scale up it and this is
[37:53] something that that's quite amazing for
[37:54] me. We will get the same kind of links
[37:57] from millimeter scale to kilometer
[37:59] scale. That's six orders of magnitude
[38:01] where optics can uh provide a solution
[38:05] which is mindboggling. And if you take
[38:07] into account that we also use optics in
[38:09] the scale out. So it's hundreds of
[38:11] kilometers. So it's even more. So again
[38:14] as an optics guys makes me very very
[38:16] very happy.
[38:18] So with that I would like to thank the
[38:20] members of my group. This is the what I
[38:22] show you is the culmination of all their
[38:24] work. Uh and with that thank you for
[38:27] your attention.
[38:28] >> Thank you very much. Thank you.
[38:29] [applause]
[38:34] >> How are we doing? No, I was I was
[38:36] checking the public while you were doing
[38:38] the presentation. Everybody was like
[38:40] boiling with energy. The laser companies
[38:42] here in the room, the companies in micro
[38:44] resonators, the companies doing micro
[38:46] optics, all of them got really excited.
[38:48] I must tell you, I've been following
[38:49] Nvidia presentations in the last three
[38:51] years. A lot of them, this one, in my
[38:54] opinion, was the best. Thank you very
[38:55] much.
[38:55] >> You've done a fantastic job. So
[38:57] [applause]
[39:03] >> everyone of course as well gets the same
[39:05] question. What can you do for them and
[39:07] what can they do for you?
[39:08] >> Okay, so what you can do for me? I think
[39:10] it was quite obvious from the from what
[39:12] I showed you and and this is another
[39:14] thing I want to emphasize. If I showed
[39:17] something that you disagree with, don't
[39:19] jump out of the window. Take it as a
[39:21] challenge. Prove me wrong. I'm I'm I
[39:23] promise you there is too much money on
[39:26] stake here for me to have an ego. So if
[39:29] you have a new idea that even
[39:31] contradicts this and you believe in that
[39:32] idea and you can prove me that you can
[39:35] prove to me that that is better by all
[39:37] means. So take it as a challenge
[39:40] >> all the way in the back from elytra.
[39:44] >> Um thank you for a very interesting
[39:46] presentation. My name is Maxim. uh we
[39:48] are developing comb blazers and actually
[39:50] one of the very interesting applications
[39:52] exactly a scale up um um application
[39:55] here for connecting the GPUs and one of
[39:58] my questions is um it's very important
[40:01] in this links to have a birectionality
[40:03] so essentially that the um the data
[40:06] could flow in one and another direction
[40:08] and I know that different um companies
[40:11] different hyperscalers consider
[40:12] different technologies for it so for
[40:14] example recently there was a new OCI MSA
[40:18] where um I think also ND is part of it
[40:20] uh where they consider different
[40:22] wavelengths. Um I also saw some other
[40:24] works where the publicization is using
[40:26] for the body dictionality. What
[40:28] >> what approach uh you consider is the
[40:30] best what are the pros and cons there?
[40:32] >> So I'm a bit biased but but again this
[40:35] this is my personal opinion because the
[40:36] work that you saw on polarization this
[40:38] is what I was alluding in the in the
[40:40] talk was our work. So so I think
[40:43] polarization is native to the problem of
[40:45] biirectionality. you can use each
[40:47] polarization in each direction and I
[40:49] think that the work that we saw was was
[40:52] was pretty good. The other approach is
[40:54] of course what light matter showed and
[40:56] they also have a wonderful solution
[40:58] dividing the wavelengths. Uh we are
[41:00] actually in the midst of of of examining
[41:03] these two solution and to see what is
[41:06] what is best. We are still debating uh
[41:09] and like many other things I think there
[41:12] isn't a clear winner. It's a it's a
[41:14] matter of trade-off.
[41:16] So but but but but thank you for raising
[41:18] this because because birectionality is
[41:21] another way to scale the uh the amount
[41:24] of uh it's another way to increase the
[41:25] radics and that's and that's definitely
[41:28] somewhere it's definitely a destination
[41:30] we need to we need to go.
[41:32] >> Thank you.
[41:34] >> Hello. Um so you showed quite a lot of
[41:36] packaging technologies. So DFB lasers
[41:38] FAU AS6 on interoses interoses on board.
[41:42] Which one do you see as the most
[41:43] critical part of the scale out and scale
[41:46] across?
[41:47] >> So, thank you. I maybe I I didn't
[41:50] mention this enough. Uh I think that the
[41:52] 2.5D integration that is the most
[41:55] critical part in this whole scheme. This
[41:57] whole scheme doesn't work if we don't go
[41:59] to higher and advanced packaging.
[42:02] >> And which part of that do you see as the
[42:03] most critical part? So the bonding of
[42:05] various different technologies with
[42:06] different bonding technologies or the
[42:08] accuracy of the interconnects? Uh so
[42:13] what do you mean by the accuracy of the
[42:14] interconnect?
[42:15] >> So the the density as things get smaller
[42:17] and more dense the accuracy of those
[42:19] bonds need to get higher.
[42:20] >> Yeah. But but you you we have the
[42:22] technology. So 3D technology is already
[42:25] already here. So you are doing a 3D
[42:27] stacking. Actually coupe is a 3D
[42:29] stacking. What I was alluding to is is
[42:32] what we refer to as coupon cost. C cost
[42:35] is the interposer that TSMC are using
[42:38] for uh uh for their 3D stacking. So that
[42:42] is the that is the uh that is the
[42:44] technology I'm referring to and I don't
[42:46] think there will be a problem of um of
[42:49] accuracy. The problems there are are are
[42:51] a bit different. Uh but so I don't see
[42:54] this as the uh as the bottleneck. the
[42:57] mere fact that you h you can put an
[42:59] optical engine on an interposer that's
[43:02] the uh that's the main issue.
[43:04] >> Okay. So you see coupe is the main focus
[43:06] point for the scaling. So 2.5
[43:08] >> I'm saying coupe because Nvidia is again
[43:11] it's not a secret that we're booking
[43:12] TSMC. So that's our our main technology.
[43:15] Uh it and as we go to slow and wide or
[43:18] in next generation the scheme might
[43:20] change a bit but uh again not from
[43:23] knowledge from just speculating. So
[43:25] >> thank you very much. It's always a great
[43:26] presentation from Nvidia. Thank you.
[43:28] >> Yeah, precise.
[43:29] >> Yep. Okay. So, I'm Yung from from Zeiss
[43:31] um technology manager. Um so, I have a
[43:34] question about the lasers for DWTM
[43:36] photonics. So, the for DFB um lasers
[43:40] used in um in the um CPO typically have
[43:44] uh several hundred mill.
[43:47] So if you switch to um comp solutions
[43:50] probably you would need a very powerful
[43:52] laser source to pump the um micro ring
[43:56] and what is the required u power and
[43:58] efficiency for for such. So that's
[44:00] actually depends on the number of lines
[44:02] and and uh detailed uh link budget that
[44:06] we'll do and that's of course also
[44:08] related to what is the rate that we're
[44:10] working on. But I want to comment on on
[44:12] what you said. The reason I really like
[44:14] the uh comb blazer idea is that you can
[44:19] integrate the essentially the com and
[44:21] especially if it's kiccom you can
[44:23] integrate it in the pick and then you
[44:24] can use the same scheme that we are
[44:26] using today. you just use this the ELS
[44:28] as a pump instead of the the actual
[44:30] source. You integrate the silicon nitro
[44:33] ring on the pick and you have maxing for
[44:36] free. And that's what I meant when I
[44:37] said maybe we can have the maxing for
[44:38] free. Having said that, people that are
[44:41] using um modlock laser with quantum dot
[44:43] don't jump out of the window. Your
[44:45] solution is also valid. We know how to
[44:47] do integration of that of that kind of
[44:49] material.
[44:49] >> Leon, we don't ask for power and
[44:52] efficiency. Let's give him a number. We
[44:54] measured power in mills. geeking a
[44:56] number. For example, if you imagine
[44:57] there 16.
[44:58] >> So again, no no but but but the number
[44:59] really depends on the on on the on the
[45:01] rate. So if you go to 32, this is
[45:04] there's a certain amount of power. Now
[45:06] our calculation with all the tricks that
[45:08] we've done and again I'm saying this
[45:10] carefully because we haven't published
[45:12] this and this is specy. We can go we
[45:14] believe to minus 10 dBm per line which
[45:18] is a lot less than what we have today.
[45:21] But again I'm saying this take this with
[45:23] a grain of salt. This is this is
[45:25] research still.
[45:26] >> That's a really great number. I'm
[45:28] actually
[45:28] >> I I I really hope that this will be the
[45:30] number.
[45:30] >> Could you do it with a pixel?
[45:32] >> Sorry.
[45:33] >> Could be could this be done with a pixel
[45:35] eventually?
[45:36] >> So vixels are a problem because and from
[45:39] a different point of view pixels are a
[45:41] problem from regarding the collection of
[45:43] of how do you how do you collect this a
[45:46] lot of a lot of vixel. So there are many
[45:49] ideas like uh multi-core fibers but
[45:51] multiore fibers are problematic. In this
[45:54] scheme we are staying with SMF fibers
[45:56] which are known technology uh [snorts]
[45:58] and very immature.
[46:00] >> Tobias from tech.
[46:02] >> Yes. Hello uh Tobias from we make um
[46:05] automation equipment for test and
[46:06] assembly and you've been highlighting
[46:08] one of the machines from our friends
[46:10] from Bremen. Um [snorts] and you know in
[46:13] our industry we are we are all trying
[46:16] like very hard to scale up our
[46:17] production capacities and and serve more
[46:20] um with more machines per week. Uh and
[46:23] question is are you defining standards
[46:26] um for maybe just your Nvidia production
[46:29] so more machine manufacturers can
[46:31] participate on the huge demand that you
[46:33] have or is it like that you're Yeah. So
[46:36] I guess if you talk to the guys in in
[46:38] production that that are in touch with
[46:40] with these companies, they will give you
[46:42] a spec.
[46:43] >> So I it's not I don't think this is
[46:45] something we we're hiding.
[46:46] >> Okay.
[46:47] >> So again, we have no interest of hiding
[46:49] that.
[46:50] >> Perfect.
[46:51] >> Leo, thank you very much for a great
[46:53] presentation. Thank you.
[46:55] [applause and music]
[47:01] >> [music]