# Trading at light speed: designing low latency systems in C++ - David Gross - Meeting C++ 2022

https://www.youtube.com/watch?v=8uAW5FQtcvE
Translation: zh-CN

[00:00] Today we're going to talk about low latency C++ trading systems.
  今天我们将讨论低延迟 C++ 交易系统。

[00:07] So first of all, what would you like to build such a system like an automated trading one?
  首先，你为什么要构建这样一个系统，比如自动化交易系统？

[00:14] Well, um, it's a heuristics game.
  嗯，这是一个启发式游戏。

[00:17] Everyone wants to crack the S&P 500 or, here in Germany, the DAX index or the Euro Stoxx.
  每个人都想破解标普 500 指数，或者在德国这里，破解 DAX 指数或欧洲斯托克指数。

[00:25] There is a lot to make or lose, and it's a hard problem.
  这里有很多获利或亏损的机会，而且这是一个难题。

[00:31] There is a lot of people on the market.
  市场上有很多参与者。

[00:35] So here on the bottom of that slide you can see a graph of the average daily traded volume in the past 10 years.
  所以在这里，在那张幻灯片的底部，你可以看到过去 10 年平均每日交易量的图表。

[00:43] And so this is on derivatives, but it doesn't really matter too much.
  这是关于衍生品的，但这其实并不太重要。

[00:47] The general information to take from this graph is that more and more people trade on financial markets, right?
  从这张图表中得出的总体信息是，越来越多的人在金融市场上进行交易，对吧？

[00:52] So compared to like 10 years ago, on stock options we have three times more volume traded.
  所以与 10 年前相比，我们在股票期权上的交易量增加了三倍。

[00:59] Why that? It's mostly retail flow as we...
  为什么会这样？这主要是因为散户流量，正如我们...

[01:03] Said so like people like you and me, we have a smartphone and we have an application where we can buy and sell stocks.
  就像你我这样的人，我们有智能手机，也有一个可以买卖股票的应用程序。

[01:08] And you know, it's just much easier than before to access markets.
  你知道，现在进入市场比以前容易多了。

[01:13] So that means that a lot of people are there trading.
  这意味着很多人都在那里进行交易。

[01:15] And that's why it's such a hard problem because it's both a trading problem and also a technological one, right?
  这就是为什么这是一个如此棘手的问题，因为它既是一个交易问题，也是一个技术问题，对吧？

[01:23] You need to be at the same time smart to come up with a price at which you're happy to sell, because you want to make good trades.
  你既需要足够聪明，能想出一个让你满意的卖出价格，因为你想进行好的交易。

[01:32] But there is a big technological aspect to it, the need of latency.
  但其中有一个很大的技术层面，那就是对延迟的需求。

[01:40] So now, why do we need such latency?
  那么现在，我们为什么需要这样的延迟呢？

[01:43] Um, well, so I work for Optimum and we are a market maker.
  嗯，好吧，我在 Optimum 工作，我们是一家做市商。

[01:50] And in our case, when you take to the application of a smartphone, you know, to buy or sell, if you want to buy a stock, we will be there on the other side, you know, to sell and vice versa.
  在我们的案例中，当你使用智能手机应用程序进行买卖时，如果你想买入一只股票，我们会在另一端进行卖出，反之亦然。

[01:59] To have to imagine that in our case we have a few.
  你得想象一下，在我们的案例中，我们有几个。

[02:04] Hundred thousand orders out there at any given time.
  在任何给定时间内，都有十万个订单在那里。

[02:09] Um, a bit more, I would say probably more like a million.
  嗯，多一点，我想大概更像是一百万个。

[02:10] But anyway, let's say 100,000 is already a lot on any instrument, you know, to facilitate, um, you know, like the trading with other people.
  但无论如何，假设十万个订单在任何工具上都已经很多了，你知道，为了促进，嗯，你知道，比如与其他人进行交易。

[02:20] And so let's imagine that you have this order out there and you have a headline like this.
  所以让我们想象一下，你有这个订单在那里，并且你看到了这样的头条新闻。

[02:28] Um, so you, you get, you know, you want to do a few things when you have such a headline.
  嗯，所以你，你明白，当你看到这样的头条新闻时，你想做几件事。

[02:33] Um, but there is one thing that you should always do, so what is it?
  嗯，但有一件事你应该总是去做，那是什么呢？

[02:40] Yeah, you are indeed, I mean, you could do that.
  是的，你确实是，我是说，你可以那样做。

[02:44] But the first thing you want to do, regardless of you being a market maker, a retail investor, or any, you know, hedge fund or any kind of like a trading, um, you know, actor, a financial actor, is that you want to cancel your buy orders.
  但无论你是做市商、散户投资者，还是任何，你知道，对冲基金或任何类型的交易，嗯，你知道，参与者，金融参与者，你要做的第一件事就是取消你的买单。

[02:58] If you had buy orders anywhere for a certain price, you know, now this information is obsolete and you...
  如果你在任何地方有特定价格的买单，你知道，现在这个信息已经过时了，而你……

[03:04] Immediately want to cancel this.
  立即想要取消这个。

[03:06] That's the first thing you want to do.
  那是你首先要做的事情。

[03:08] After that you're going to do other things.
  在那之后你还要做其他事情。

[03:10] But so that is of course just one example of where you need this like very like low latency in trading is uh for headlines.
  但这当然只是你需要这种极低延迟交易的一个例子，比如用于新闻标题。

[03:18] There are many many others of course, but that's um that's one.
  当然还有很多很多其他的例子，但这只是其中之一。

[03:24] So today we are going to um you know we we want to we would like to buy um build sorry together um such low latency automated trading system.
  所以今天我们想要，你知道，我们想要，我们想一起构建这样一个低延迟自动交易系统。

[03:37] And uh well you know it's it's it's hard to cover everything in an hour talk so I had to pick a few things.
  嗯，你知道，在一小时的讲座中涵盖所有内容很难，所以我不得不挑选一些重点。

[03:44] And I would like to cover with you four um topics.
  我想和你们讨论四个主题。

[03:46] The first one would be data model access.
  第一个是数据模型访问。

[03:52] Then system tuning.
  然后是系统调优。

[03:54] And eventually we would like to talk about how to measure performance.
  最后我们想谈谈如何衡量性能。

[04:00] And we are going to focus on some like really building you know like this this.
  我们将专注于一些真正构建的内容，你知道，就像这样。

[04:04] Building blocks that you can use on that effective you're being used in any low latency system.
  你可以使用的构建块，在任何低延迟系统中都能发挥作用。

[04:10] It's also time to say a bit more about me.
  现在也该多介绍一下我自己了。

[04:13] To introduce myself, I'm David.
  自我介绍一下，我是大卫。

[04:13] I've been working on automated trading systems for almost 10 years, so Optivia market maker.
  我从事自动化交易系统工作已经快10年了，所以是Optivia的做市商。

[04:22] Before that, I was working on high throughput systems, so quite some overlap with the low latency side of things, but in a very different industry in defense.
  在此之前，我从事高吞吐量系统工作，所以与低延迟方面有很多重叠，但在国防领域，这是一个非常不同的行业。

[04:35] So first of all, designed for performance, that's what we're going to talk about today.
  首先，为性能而设计，这就是我们今天要讨论的内容。

[04:40] Is it actually a good idea?
  这真的是个好主意吗？

[04:43] Do you know what Knute said, premature optimization is the root of all evil.
  你知道高德纳（Knute）说过什么吗？过早优化是万恶之源。

[04:47] Also a friend saying, you know, we will make it fast if it needs to be, but that sounds reasonable.
  还有个朋友说，你知道，如果需要的话我们会让它变快，这听起来很合理。

[04:51] You know, myself, I consider myself a pragmatic engineer, you know, only solving problems when they happen.
  你知道，我自己认为我是一个务实的工程师，只在问题发生时才去解决它们。

[05:00] Um, so, so I like that, I agree with that.
  嗯，所以，我喜欢这一点，我同意这一点。

[05:03] Um, but what I also say.
  嗯，但我还要说的是。

[05:05] Um, and actually, I heard it also yesterday at a meeting.
  嗯，其实我昨天在开会时也听到了。

[05:07] CBP was doing another talk on the deadline for the real-time system.
  CBP 正在就实时系统的截止日期进行另一场演讲。

[05:13] Uh, Matya said, you know, performance should be considered at the beginning when you design a system.
  呃，Matya 说，你知道的，当你设计一个系统时，应该在最开始就考虑性能。

[05:19] So yeah, performance can really be enough default.
  所以是的，性能确实可以作为默认考量。

[05:21] And these two quotes, you know, they seem to contradict each other.
  而这两句引言，你知道的，它们似乎相互矛盾。

[05:28] So, who agrees with Knute?
  那么，谁同意 Knute 的观点？

[05:33] I've got some people; I would expect more people to agree.
  有一些人举手了；我本以为会有更多人同意。

[05:35] But uh, who agrees with me saying it can rarely be an afterthought?
  但是，呃，谁同意我说的“它很少能被视为事后补充”的观点？

[05:41] Oh, okay, you agree more with me; that's very sweet of you.
  哦，好的，你更同意我的观点；你真是太贴心了。

[05:46] Well, effectively, um, I think both statements are correct, and they don't really contradict each other.
  嗯，实际上，我认为这两个陈述都是正确的，而且它们并没有真正的矛盾。

[05:52] The reason is that there are two things to look at when we consider not just performance, but in general, systems strategy and tactics.
  原因在于，当我们考虑的不仅仅是性能，而是通用的系统策略和战术时，有两件事需要关注。

[06:02] The strategy is your, um, you know, is your...
  策略是你的，嗯，你知道的，是你的……

[06:05] Is the approach like the overall overarching approach, you know, to meet a goal?
  这种方法是否就像是实现目标的整体统筹方法？

[06:11] So let's, for example, take as a goal, um, I'd like to handle a million events per second.
  举个例子，假设我们的目标是每秒处理一百万个事件。

[06:16] Or I would like to react off a certain event with a certain latency distribution, like average less than 100 nanoseconds, standard deviation less than 10 nanoseconds.
  或者我希望以特定的延迟分布对某个事件做出反应，比如平均延迟低于100纳秒，标准差低于10纳秒。

[06:27] There are requirements that are there, there are goals that you have.
  这些就是现有的需求，也是你所设定的目标。

[06:30] And so the strategy is like your overall approach to, you know, to solve that problem.
  所以策略就像是你解决那个问题的整体方法。

[06:36] Another, tactics, they are your individual actions.
  而战术则是你采取的各项具体行动。

[06:41] And so if I translate that to like an engineering approach, or just when we develop, it would be like the implementation details.
  如果将其转化为工程方法，或者仅仅是在我们进行开发时，它就相当于具体的实现细节。

[06:47] Like, hey, should I call reserve on the vector, which are the hash maps the most efficient, and so on and so forth.
  比如，嘿，我是否应该在向量上调用reserve，哪种哈希映射最高效，诸如此类。

[06:56] And so you need both.
  所以这两者你都需要。

[06:58] And now back to the previous slide with, like, Knuth saying a lot about premature optimization, is definitely talking about tactics.
  现在回到上一张幻灯片，高德纳关于过早优化的论述，绝对是在谈论战术。

[07:05] We shouldn't immediately try to, you know.
  我们不应该立即尝试去，你知道的。

[07:08] Do some local optimization everywhere.
  在各处进行一些局部优化。

[07:11] Because in general our intuition is wrong and we would just be complicating our software for probably nothing.
  因为总的来说，我们的直觉是错误的，我们很可能只是在徒劳地让软件变得复杂。

[07:19] So he's talking about tactics and then when I said you shouldn't be an after food talking about strategy, so today we are going to look at a bit of both.
  所以他在谈论战术，而当我提到你不应该在饭后谈论战略时，今天我们将两者都看一看。

[07:27] Again, unlike the photo picks that I mentioned previously, try to give you like a general strategy when it comes to what a very concrete topic of building a trading system and then look at some details still.
  同样，与我之前提到的照片精选不同，我尝试在构建交易系统这一非常具体的课题上，为你提供一个通用的策略，然后再看看其中的一些细节。

[07:44] So if you want to have a strategy on how to build such system, we need, you know, a goal.
  所以，如果你想制定一个关于如何构建此类系统的策略，我们需要一个目标。

[07:50] So what is our goal?
  那么我们的目标是什么？

[07:50] How fast is fast?
  多快才算快？

[07:53] Five years ago, one of my colleagues from Optical presented the talk at meeting CVP here and then CVP gone and here the similar slide.
  五年前，我的一位来自 Optical 的同事在 CVP 会议上发表了演讲，然后 CVP 结束了，这里是类似的幻灯片。

[08:03] And so you know if you can see on the slide that we are talking about 2.5 microseconds, you know, software.
  所以你知道，如果你能在幻灯片上看到，我们谈论的是 2.5 微秒的软件。

[08:10] Turning system that's roughly very well or what we're talking about.
  转换系统大致上非常好，或者说这就是我们正在讨论的内容。

[08:17] So where are we nowadays?
  那么我们现在处于什么阶段？

[08:21] Um, you know, things change a little bit.
  嗯，你知道，情况发生了一些变化。

[08:23] So the exchange drawers very quickly, um, on financial markets.
  所以交易所的格局在金融市场上变化得非常快。

[08:27] So what you can see on that slide on the right is what we call a trigger to target latency, okay?
  所以你在右侧幻灯片上看到的是我们所说的“触发至目标延迟”，明白吗？

[08:34] What is this?
  这是什么？

[08:36] A bit of a technical term that I will explain, um, in a bit.
  这是一个稍微有点技术性的术语，我稍后会解释。

[08:38] It's on Eurex, it's the biggest European exchange; it's actually in Germany, in Frankfurt.
  这是关于欧洲最大的交易所 Eurex 的，它实际上位于德国法兰克福。

[08:45] And so what is this trigger to target latency?
  那么什么是“触发至目标延迟”呢？

[08:49] Um, a trigger is when the exchange sends an event to the market participant.
  嗯，触发是指交易所向市场参与者发送事件。

[08:58] So the event could be a trade or a price change on a stock, for example.
  例如，该事件可以是交易或股票价格变动。

[09:00] So these events I will send in a very, you know, fair way so that all the market participants receive them at the same time.
  所以这些事件我会以一种非常公平的方式发送，以便所有市场参与者能同时收到它们。

[09:09] And so that's, uh, that's what we...
  所以这就是，呃，这就是我们……

[09:11] Call a trigger and then the target is when any market participants send an order back.
  调用一个触发器，然后目标是当任何市场参与者发送回订单时。

[09:17] So we have this two timestamp being taken by like a level one switch, like a very accurate device.
  所以我们有两个由一级交换机之类设备获取的时间戳，这是一种非常精确的设备。

[09:23] I mean a device that can very accurately timestamp packet in and out.
  我的意思是，一种能够非常精确地为数据包进出打上时间戳的设备。

[09:27] And if you measure the difference between the two, we see on that heat map that we approach or that we already are around 10 dollars.
  如果你测量两者之间的差异，我们在热力图上看到我们接近或已经处于 10 美元左右。

[09:38] Okay, that's it's very fast.
  好的，这非常快。

[09:41] So what kind of C plus plus can achieve that?
  那么什么样的 C++ 能够实现这一点呢？

[09:45] [Music]
  [音乐]

[09:48] By the way, taking the energy again of the speed of light, it is roughly, you know, the time that it takes for the light to travel between, you know, me and I think you.
  顺便说一下，再次利用光速的能量，这大约是你我之间光传播所需的时间。

[09:57] I think just the very first, whoa, so it's not a lot.
  我想仅仅是最初的，哇，所以这并不多。

[10:04] Not sure what kind of logic we can have that down.
  不确定我们能用什么样的逻辑来实现它。

[10:07] Um so yeah, no super C plus plus can achieve this, even the new coming PCI Express.
  嗯，所以是的，没有超级 C++ 能实现这一点，即使是即将推出的 PCI Express 也不行。

[10:12] Standard is, you know, already too slow.
  标准，你知道的，已经太慢了。

[10:15] Way too slow for land.
  对于陆地来说太慢了。

[10:17] So what people are doing is that they have FPGAs, right?
  所以人们正在做的是使用 FPGA，对吧？

[10:19] So they have FPGA VHGL, sorry, a flash on the FPGA that's a custom board, and that's how they achieve the latency.
  所以他们有 FPGA VHGL，抱歉，是 FPGA 上的闪存，那是一个定制板，这就是他们实现延迟目标的方式。

[10:27] So what are we going to talk about today is C++ obsolete?
  那么我们今天要讨论的是 C++ 是否过时了？

[10:29] Um, well, not really, effectively not at all.
  嗯，其实并没有，实际上完全没有。

[10:39] Um, for at least three reasons.
  嗯，至少有三个原因。

[10:43] So hey, it's like a simplified but realistic diagram of a trading system.
  嘿，这就像是一个简化但真实的交易系统图表。

[10:51] And so the first reason that low latency super space is still very much a thing is that what we saw on the police line is just the tip of the iceberg.
  所以低延迟超级空间仍然非常重要的第一个原因是，我们在警察线（此处指前文提到的基准线）上看到的只是冰山一角。

[11:00] And so people like to talk about these numbers because while they're impressive, that's true, but there is a lot going on in a trading system aside of that.
  人们喜欢谈论这些数字，因为虽然它们确实令人印象深刻，但交易系统中除了这些之外还有很多其他的事情。

[11:12] And so the first reason you need low latency is, well, if you have this amazing FPGA that can send a hundred thousand orders in a second, let's say, you probably want to have some low-rate C code around it, you know, to receive the notification that you sent something in order to do some post-checks.
  因此，你需要低延迟的第一个原因是，假设你拥有这种能够每秒发送十万个订单的强大 FPGA，你可能需要在它周围配备一些低速率的 C 代码，以便接收你发送了某些内容的通知，从而进行一些事后检查。

[11:28] Let's say it's not just the safety, like a risk mechanism.
  比方说，这不仅仅是为了安全，比如风险控制机制。

[11:33] The second reason is, um, if that FPGA is able to send an order within 10 nanoseconds, it is then probably, uh, you know, very, very simple.
  第二个原因是，嗯，如果那个 FPGA 能够在 10 纳秒内发送订单，那么它可能，呃，你知道的，非常非常简单。

[11:45] It's not easy; it's a lot of engineering going on in there, but it's simple as like it sees bits, compares bits.
  这并不容易；其中涉及大量的工程工作，但它很简单，就像它只是看到位、比较位一样。

[11:52] And bits doesn't really understand much about, you know, how to price a derivative instrument or anything.
  而位并不能真正理解如何为衍生金融工具定价之类的东西。

[12:00] It's just not possible because that's always, it's a trade-off.
  这根本不可能，因为这始终是一种权衡。

[12:04] So what it means is that you see, and there is this thing that we call in industry trigger price, which is a, you know, at what point do I want to buy or sell, how much do I think.
  所以这意味着，你看，行业内有一种东西我们称之为触发价格，也就是，你知道的，我在什么点位想要买入或卖出，以及我认为是多少。

[12:13] That this stock is worth.
  这只股票的价值。

[12:15] And so this is, uh, very much an input of this FPGA.
  所以这在很大程度上是这个 FPGA 的输入。

[12:19] And again, thinking about the headline that we saw before, if our friend Elon, you know, tweet, that can imagine that our opinion about the Tesla stock is going to change relatively quickly.
  再次思考我们之前看到的头条新闻，如果我们的朋友埃隆发了推文，你可以想象我们对特斯拉股票的看法会很快发生变化。

[12:30] It's also something there is some latency sensitivity in there.
  这也是其中存在一些延迟敏感性的地方。

[12:32] So that's again something that's, you know, all the strategy part of it.
  所以这又是关于策略的部分。

[12:36] We need C++ again, low latency.
  我们再次需要 C++，低延迟。

[12:41] And the final reason is that as for anything, um, it's just a trade-off.
  最后一个原因是，对于任何事情，嗯，这只是一个权衡。

[12:46] Um, like there are some and just a trade-off on like engineering requirements.
  嗯，就像在工程需求上存在一些权衡。

[12:50] There are some strategies where you really want to aim for, uh, yeah, blazing fast standard second latency.
  有些策略是你真正想要追求的，嗯，是的，极快的标准秒级延迟。

[12:56] Sure, then you're for, you know, really complicated hardware on VHDL, on you go for it.
  当然，那么你就可以去使用非常复杂的 VHDL 硬件，尽管去用吧。

[13:01] But there is a lot, um, for, you know, trading strategy where you absolutely do not need such a latency.
  但是，嗯，有很多交易策略是你绝对不需要这种延迟的。

[13:09] And then you use C++ like reasonably fast, like, you know, two.
  然后你使用 C++，速度相当快，就像，你知道的，二。

[13:15] Micro second, one micro second or even a bit below, and that's great.
  微秒，一微秒甚至更短，这太棒了。

[13:21] Ah, so we still have some things to talk about today, that's nice.
  啊，所以我们今天还有一些事情要谈，这很好。

[13:28] Um, now I'm going to share a story on the, you know, to come to our first, you know, strategy, um, about data modeling.
  嗯，现在我要分享一个故事，关于我们第一个策略，也就是关于数据建模的故事。

[13:36] A story that when I joined, uh, when I joined up TV almost 10 years ago, um, there was an algorithm turning study that was really smart, um, that wanted to be fast.
  这个故事是关于我大约10年前加入TV时，有一个非常聪明的算法研究项目，它想要实现高速运行。

[13:47] Now you probably already encountered that, um, in your career.
  现在你可能已经在你的职业生涯中遇到过这种情况了。

[13:50] You know, there is, there is a software A, and at some point some engineers start working on software B.
  你知道，有一个软件A，在某个时候，一些工程师开始开发软件B。

[13:57] And at the beginning, B, um, doesn't really have a, the goal of B is not per se to replace A.
  起初，B并没有，B的目标本身并不是要取代A。

[14:04] But at some point, B becomes so nice that everyone wants to work on B, and then it becomes a goal to replace A.
  但在某个时刻，B变得非常好用，以至于每个人都想在B上工作，然后取代A就成了目标。

[14:11] Right, you probably recognize that, it's a typical engineering thing.
  对吧，你可能认得出来，这是典型的工程现象。

[14:14] Well, in that case, the problem was that our...
  嗯，在这种情况下，问题在于我们的……

[14:17] Algorithm A was quite fast, not blazing, but relatively fast.
  算法 A 相当快，虽然算不上极快，但相对来说还是很快的。

[14:23] And B was a software that was then designed to be much smarter, but not perceived faster.
  而 B 是一款当时被设计得更智能的软件，但感知上并没有更快。

[14:29] And I witnessed brilliant engineers working on making this algorithm build that was very smart, trying, you know, as fast as a month of engineering time poured into the effort.
  我目睹了优秀的工程师们致力于构建这种非常智能的算法，你知道，他们投入了一个月的工程时间来努力实现它。

[14:45] And eventually I also tried; I was quite new in there, but you know, I also gave it a shot, and it didn't work.
  最终我也尝试了；当时我刚加入不久，但也试了一下，结果没成功。

[14:54] So I was quite surprised.
  所以我感到非常惊讶。

[14:55] I was like, oh, all these amazing tactics and everything, and like all this engineer trying to do that, but why didn't it work?
  我当时想，噢，所有这些了不起的策略和一切，还有这么多工程师试图实现它，但为什么它行不通呢？

[15:03] Well, back to what we said about the designing for performance, it was not designed for it.
  嗯，回到我们之前关于性能设计所说的话题，它本身就不是为此而设计的。

[15:10] And again, there are many reasons that a certain software wouldn't be able to be fast, but the main one that I want to share with you today...
  再说一次，某种软件无法变快的原因有很多，但我今天想与大家分享的主要原因是……

[15:19] Um, is about data.
  嗯，是关于数据的。

[15:21] Is that if you don't really design your software properly, it's just gonna be slow everywhere.
  那就是如果你没有真正地设计好你的软件，它在任何地方都会很慢。

[15:28] So what's on this graph?
  那么这张图表上是什么？

[15:32] This is like the result of a profiler, a profiling result.
  这就像是一个分析器的结果，一个性能分析结果。

[15:37] Um, it's an in-house profiler that we built on the top of a Clang X-Ray.
  嗯，这是一个我们基于 Clang X-Ray 构建的内部分析器。

[15:42] So we instrument the code and we add some like time stamping.
  所以我们对代码进行了插桩，并添加了一些类似时间戳的功能。

[15:47] On the right you see some function names, and on the left, not sure if you can read it, it's a bit small, but you have some time.
  在右边你可以看到一些函数名，而在左边，不确定你是否能看清，它有点小，但你有一些时间数据。

[15:54] And then there is this graph.
  然后就是这张图表。

[15:56] And so the take on this thing is that, you know, what you see in textbook when it comes to profiling and make things fast, that usually eighty percent of your CPU time is going to be in twenty percent of your code.
  所以关于这件事的结论是，你知道，当谈到性能分析和加速时，教科书上通常说你百分之八十的 CPU 时间会花在百分之二十的代码上。

[16:10] But it's awesome then you can just make that fast and you optimize everything.
  但这很棒，因为你可以只把那部分变快，然后你就优化了一切。

[16:13] Well, that's actually in the textbook.
  嗯，那实际上是在教科书里的。

[16:16] Reality is, um, if you don't model properly your data.
  现实是，嗯，如果你没有正确地建模你的数据。

[16:20] You will see something like that.
  你会看到类似那样的东西。

[16:23] And that's very underwhelming because there is no kind of hotspot.
  这非常令人失望，因为没有任何热点。

[16:27] There is nothing to see, it's just flow everywhere.
  没什么可看的，到处都是数据流。

[16:29] Okay, why is it slow everywhere?
  好吧，为什么到处都很慢？

[16:30] Because it cannot fit in cache.
  因为它无法放入缓存中。

[16:36] So the first strategy is going to be about the size and your locality of the data.
  所以第一个策略是关于数据的大小和局部性。

[16:44] And here I put an example of some size of cache just to get an idea of what we have on modern processors, on the different level of cache.
  在这里，我举了一个缓存大小的例子，只是为了让我们了解现代处理器在不同缓存级别上拥有什么。

[16:57] You know, up front when you design your system, how much data you need.
  你知道，在设计系统时，你需要预先考虑需要多少数据。

[17:03] Look, in trading, we know roughly how many stocks, for example, we are going to trade in an algo.
  看，在交易中，我们大致知道例如我们在算法中要交易多少只股票。

[17:08] We know how much data we manipulate.
  我们知道我们处理多少数据。

[17:11] We know if it's gonna fit in this cache or depending on the data structure that we use.
  我们知道它是否能放入这个缓存中，或者取决于我们使用的数据结构。

[17:17] We already know upfront, or we should know, we should have an idea if it.
  我们预先就已经知道，或者我们应该知道，我们应该对此有个概念。

[17:21] Fits or not, if it doesn't, then you can choose to have more cash.
  是否合适，如果不合适，那么你可以选择拥有更多的缓存。

[17:24] You can split your application across several cores or you want to call several servers, but you need to take that into account in your design.
  你可以将应用程序拆分到多个核心上，或者想要调用多台服务器，但你需要在设计中考虑到这一点。

[17:33] And something that is also often overlooked is that we think a lot about our code because we love our code, but we don't really think about our colleagues.
  还有一点经常被忽视的是，我们因为热爱自己的代码而对其思考良多，但我们并没有真正考虑到我们的同事。

[17:44] The thing is, you need to think about the system as well.
  问题在于，你也需要考虑系统。

[17:46] So like when you actually have your application on a core, you effectively share your last level cache with, you know, excluding you know that is not there already for a few years.
  所以，当你实际上在某个核心上运行应用程序时，你实际上是在与——你知道的，排除掉那些已经不存在几年的代码——共享你的末级缓存。

[17:56] His code is still running on that core next to yours, and it can affect a lot the performance of your application.
  他的代码仍然在你旁边的那个核心上运行，这可能会很大程度上影响你应用程序的性能。

[18:07] And so again, data model issues are, from my experience, the main caveats that engineers fall into when it comes to designing a system.
  所以再说一次，根据我的经验，数据模型问题是工程师在设计系统时容易陷入的主要陷阱。

[18:20] So this is like a war story.
  所以这就像是一个战斗故事。

[18:22] From just three weeks ago, so I thought I would include it in today's presentation.
  就在三周前，所以我认为我应该把它包含在今天的演示中。

[18:26] So what you have on the x-axis is to have two days.
  所以你在横轴上看到的是两天的时间。

[18:33] And then the second day we updated the binary around, you know, 20 past two or so.
  然后在第二天，我们在两点二十分左右更新了二进制文件。

[18:38] And so the y-axis is the latency of the system.
  纵轴是系统的延迟。

[18:42] And you have just drastic change.
  你会看到非常剧烈的变化。

[18:45] And the change was mostly about moving data around and making sure that it could fit internally in the cache, but just at least it would fit better.
  这种变化主要是关于移动数据，并确保它能适应内部缓存，或者至少能更好地适应。

[18:56] So it's not really, only if you would see the code change, it was not really anything fancy.
  所以这并不是，如果你看到代码变更的话，其实并没有什么特别花哨的东西。

[18:59] It's just moving things around or removing things so they're more packed and together.
  只是移动或移除一些东西，使它们更紧凑、更集中。

[19:05] So let's take an example.
  让我们举个例子。

[19:09] A core element of a training system is what we call an instrument store.
  训练系统的核心要素是我们所说的工具存储。

[19:14] We're not going to, you know, explain what is an instrument exactly, but you can imagine it's a stock or ETF, something like that.
  我们不需要解释工具到底是什么，但你可以想象它是一只股票或ETF之类的东西。

[19:22] It's a relatively small data structure.
  这是一个相对较小的数据结构。

[19:23] It's less than a kilobyte in general.
  它通常不到一千字节。

[19:26] And we have, you know, not so many below.
  而且我们知道，下面的数量并不多。

[19:29] So not so sure between one and ten thousands of them in, uh, in an algo.
  所以不太确定在算法中它们是一万到十万个之间。

[19:36] You want to do a lookup from time to time.
  你想要时不时地进行一次查找。

[19:38] You receive an ID from an external, um, you know, process.
  你从一个外部的，嗯，你知道的，进程中接收到一个ID。

[19:41] You receive an ID, so you want to do a lookup, but it's not...
  你接收到一个ID，所以你想进行查找，但它不是……

[19:46] The latency of this lookup is not too critical.
  这种查找的延迟并不是太关键。

[19:48] You want to be reasonably fast.
  你希望它足够快。

[19:51] And last requirement is that most of the code wants to store reference to this instrument.
  最后一个要求是，大部分代码想要存储对这个工具的引用。

[19:55] We don't want to do, um, you know, within the system lookups all the time.
  我们不想，嗯，你知道的，一直都在系统内进行查找。

[19:58] We want to keep reference because that's, that's just nice.
  我们想要保留引用，因为那样，那样很好。

[20:01] It's a nice characteristic.
  这是一个很好的特性。

[20:03] So the store, the instrument store, you know, it owns all these things.
  所以存储，这个工具存储，你知道，它拥有所有这些东西。

[20:06] So what data structure should we use with search requirements?
  那么对于这些搜索要求，我们应该使用什么数据结构呢？

[20:12] Well, from my experience, what I see most people using is the standard auto map.
  嗯，根据我的经验，我看到大多数人使用的是标准自动映射。

[20:18] And, um, I like to know the map.
  而且，嗯，我想了解这个映射。

[20:21] I think it does great characteristic, you know, it's.
  我认为它有很好的特性，你知道，它是。

[20:23] reasonably fast

[20:25] um it does actually stable reference on

[20:27] the riyash so that's that's nice

[20:30] um but now the big downside of another

[20:33] map

[20:34] is that

[20:35] um all the elements the value of the

[20:37] hash map will be all over your Hammer

[20:42] like all over your memory so it's like a

[20:44] random walk through your home

[20:46] so often

[20:48] um you know if you if you if you watch

[20:49] talk or from previous in a meeting CBP

[20:51] or cbbcon about like performance or cash

[20:53] friendliness you see all this Benchmark

[20:54] about stood list and stood vector

[20:57] and and I think sometimes people take a

[21:00] bit to

[21:01] um you know the first degree the fact

[21:02] that you know we look at stood vectors

[21:04] to at least

[21:05] and we say students is bad it's not

[21:08] really about to at least it's about the

[21:10] memory model of stood list

[21:12] if you want to be fast the memory moral

[21:14] of students itself is bad and stood

[21:16] another map of the same memory model

[21:19] um it is a node container which is each

[21:22] element is going to be allocated in

[21:23] separate node

[21:24] and so that's the main the main problem

[21:26] there

[21:29] so like already a better solution or

[21:32] something that is a nicer is uh to use a

[21:35] vector close them up

[21:39] but then this doesn't only satisfy our

[21:41] requirements anymore because stood

[21:42] Vector on the resides we're going to you

[21:45] know not have a value reference anymore

[21:47] so that that's not great

[21:51] so my like my favorite solution for that

[21:53] is to use like a container called stable

[21:55] Vector I'm going to show you like the

[21:57] the current next line

[21:59] it's a very very simple container

[22:02] and it is not boost stable Vector that

[22:05] is actually quite a bit different base

[22:06] uh boost table Vector is again a not

[22:08] container

[22:09] which was the stable Vector as the same

[22:13] property of a vector continuous memory

[22:15] stable pointer

[22:17] um this one actually Vector didn't have

[22:19] it but it actually had the stability to

[22:21] that vector

[22:22] so what is it

[22:26] so it's a stood Vector of

[22:29] static Vector so I'm not sure if you're

[22:31] familiar with Boost static Vector it's

[22:34] it's roughly a stood array

[22:36] with like also a size so that you know

[22:38] how many elements are in the right

[22:41] pre-allocated a vector and so we have a

[22:44] vector of the static vector and we can

[22:46] define a chunk size

[22:47] so can you recognize uh so this this

[22:51] container is very close to

[22:53] a STL container do you recognize it

[22:56] yes that was fast yeah stood deck so

[22:59] indeed it's it's to Deck the big

[23:01] difference to deck is that we can

[23:03] um yeah we can customize this chunk size

[23:04] we might say oh that's just a small

[23:06] difference effectively it's big because

[23:07] for our locality it actually matters and

[23:10] stood deck has a very poor locality

[23:11] because of that while the implementation

[23:14] is compiler dependent but most compiler

[23:16] will take a relatively small chunk size

[23:19] and as I said the downside is that you

[23:22] pay a little bit of overhead

[23:23] um when you call the you know the

[23:25] autopartum Square Market or a few

[23:27] iterators that's what I put on the right

[23:29] if you provide it you will see you can

[23:30] spend a lot of time shifting bits you

[23:32] know to calculate the index

[23:34] because yeah you don't do modulo or

[23:35] division because you take a power of two

[23:38] but that's you know if you have slightly

[23:40] different requirements if you want a

[23:41] good walk through this content of us you

[23:43] can actually do a better implementation

[23:44] I mean different implementation than

[23:45] that it's just a trade-off so this is

[23:48] one of my favorite container because

[23:50] it's simple simple things are fast and

[23:52] you know you keep things together

[23:56] so that's our strategy you know we want

[23:58] to keep things local and and small and a

[24:01] great measure you know to check that

[24:03] you're doing that well is the WSS the

[24:05] working Set Side so not to be

[24:08] um confused with RSS so the RSS is

[24:11] usually what we allocate and touch

[24:14] this is

[24:15] um the size of your workload

[24:18] and in general we can measure it in

[24:20] Cache lines

[24:22] um that effectively there is not really

[24:24] any any tool to measure it in Cache

[24:26] lines so Brandon Greggs your performance

[24:30] engineer from Netflix and developed a

[24:32] small and very nice tool you know to

[24:34] measure it but in pages

[24:37] so you might say oh that's not so nice

[24:39] because you know pages on Cache lines

[24:40] are actually quite different but um

[24:42] let me prove you that it's actually

[24:44] really really nice so we take a

[24:47] benchmark like this

[24:51] um

[24:52] to be clear we do not really care in

[24:55] that Benchmark about the speed of

[24:56] execution we we're not interested in

[24:58] that we just use this code you know to

[25:01] calculate the WSS

[25:03] so we declare a stable Vector we put

[25:06] some integer in there and then we do a

[25:07] sum on the sum you're just going to

[25:09] touch all the elements so we can

[25:10] calculate or see you know the WSS on

[25:12] this load we do similarly on the two

[25:14] dollar map

[25:16] small detail actually important detail

[25:19] you see that there is a stoot list there

[25:21] called TMP

[25:22] where we push back a bunch of elements

[25:24] in between our employees

[25:27] we do this and that's that's very

[25:29] important when you when you do such

[25:31] micro benchmark

[25:33] um it's a it's a it's in order to

[25:34] randomize the Heap

[25:36] otherwise the result you will have will

[25:38] be so different than what you see in

[25:40] production code that you need to do this

[25:44] in in other words

[25:46] um if if you if you don't do that for

[25:48] example then another map you know all

[25:49] your allocation will be one of these

[25:51] other and depending that's the problem

[25:52] depending on the Heap the malloc

[25:54] implementation you're using your system

[25:56] you might have contiguous allocation

[25:58] which will change a lot to results and

[26:01] you might have that but you might also

[26:02] know that at all so the idea is just to

[26:04] um to remove this this pattern that you

[26:06] have when doing micro benchmarking

[26:08] specifically

[26:10] and so this is what we get

[26:15] um

[26:16] one megabytes for the you know the

[26:18] stable vector

[26:21] so we have a hundred thousand in 32 to

[26:24] 400 000 uh bytes and then we have 400

[26:27] megabytes of another map

[26:29] of WSS

[26:31] so you might say okay this tool is just

[26:34] not working and you know this is just

[26:35] absolutely not not the truth you know

[26:37] why would why would we look at that

[26:39] and that's why I say that it's it's

[26:40] actually quite a nice thing that it

[26:42] measure in pages

[26:43] because yeah it is what it is it

[26:45] measuring pages and therefore you get a

[26:48] result that is you might say exaggerated

[26:51] but it gives you a sense of the locality

[26:53] of your data

[26:54] in this Benchmark every single in 32 is

[26:58] on a page

[27:00] so when you think about it you're

[27:01] wasting actually quite some memory right

[27:03] so even if even if you know if you

[27:07] measure in Cache line the result will be

[27:08] much closer the point is you're wasting

[27:10] so many pages that you will suffer

[27:13] of course from like many at least the

[27:15] main thing you will suffer will be tlb

[27:17] Miss because you will just do page

[27:19] lookup all the time this is just with

[27:20] one container

[27:23] so again strategy size and locality you

[27:26] can look at WSS

[27:28] but this is why it matters so much to

[27:30] keep things together

[27:34] so now that we talked about data model

[27:37] let's look at you know how to share this

[27:40] data because it's one thing to have

[27:43] a process that is fast you know that

[27:46] stay in cash

[27:47] usually I mean on any modern processor

[27:49] we have many many cores and we want to

[27:51] share data we're going to communicate

[27:52] between this application

[27:56] so

[27:58] one strategy so what we want to do here

[28:01] um is we receive this this Market data

[28:03] 10 gigabit per second so that's on the

[28:05] on the top in this market data receiver

[28:08] and we want to that's very typical of

[28:10] trading uh system we want to Fan out all

[28:13] this data

[28:14] to many different applications

[28:16] usually you have like 30 to call on that

[28:18] server you have like yeah roughly 30

[28:20] application running on that server and

[28:23] you want all of them to receive the same

[28:25] Market data

[28:27] right so what should be your strategy

[28:29] there

[28:31] um we want low latency right so average

[28:35] less than 100 nanoseconds

[28:38] um also like not so much cheetah

[28:41] and um and our strategy should Target

[28:43] many consumers

[28:45] but only one producer

[28:48] and the key element is that the consumer

[28:51] should not affect the producer why that

[28:54] because the producer in drain is getting

[28:57] linerates data

[28:59] if any of the consumer would block it

[29:01] you know we would start dropping packets

[29:03] you know and then then you know the old

[29:04] system just falls apart they should not

[29:07] affect it

[29:09] and so in order to do that we have um

[29:12] you know we make the decision between

[29:13] two two things here we want to have

[29:15] events

[29:17] that we want to send

[29:18] and state so

[29:20] what do I mean exactly by that

[29:23] okay first of all how much how much data

[29:25] are we exactly talking about

[29:28] um this is on you know one one multicast

[29:31] channel just to get an ID

[29:32] um it's it's on the on the U.S market

[29:34] SMP future

[29:37] we received per month for at least 300

[29:39] million message

[29:41] all right these are the Lord is that um

[29:45] effectively not so much you know if this

[29:48] is really like what one application

[29:49] should receive it's manageable the

[29:51] problem is

[29:52] it's very busy again think about the

[29:54] headliner we saw before

[29:56] can be very quiet during the day and

[29:58] then there is a headline and and this is

[30:01] it everyone reacts so you get a million

[30:02] message

[30:04] um you get a burst of like a minute

[30:05] message in less than a second

[30:09] back to what we were saying about Evans

[30:11] and States

[30:12] um

[30:13] so what you want to send between this

[30:15] application is for example this order

[30:17] book so think about another book as you

[30:19] know a few Pisces you know that we want

[30:21] to bury or sell

[30:23] and you have that instrument we want to

[30:26] we want to share this

[30:27] we have two ways in um in our system to

[30:31] to share this data the first one is to

[30:33] send an event

[30:36] um so think about it as like every

[30:37] single um you know new insertion or you

[30:41] know deletions like amendment of this uh

[30:44] of this other book we wanna send

[30:47] and the second one is well we just want

[30:49] to share the the old thing we don't want

[30:51] to share the state so in one you receive

[30:53] you receive a notification as well and

[30:55] the other one you don't

[30:57] so they are quite quite different

[31:01] and we want to use both in our system

[31:03] right so like the strategy ABC the one I

[31:06] react of this event in order to be able

[31:07] to send an order so they need to receive

[31:10] this event in a low latency way but then

[31:12] if I take the example of uh you know the

[31:14] at the bottom you see this trading GUI

[31:16] if you just you know send ties to a GUI

[31:19] it doesn't really matter you don't want

[31:21] to push 10 gigabyte 10 gigabyte bits of

[31:23] of price per second to agree you know

[31:25] you're not gonna never going to be able

[31:27] to to see that you just want to

[31:29] to have a state where you can as a

[31:32] consumer just

[31:33] pull the data you know from time to time

[31:37] so it's a push versus full approach here

[31:39] we want to look at both of them so let's

[31:41] start

[31:42] first of all state

[31:45] um we are definitely in the domain of

[31:47] like you know like clock data structure

[31:50] and so we have we have a lot of options

[31:53] there so today I'm just going to talk

[31:56] about one because I think this is this

[31:58] is like the you know like the maybe not

[32:00] the perfect fit but like one that is

[32:02] definitely underrated

[32:05] um not so many people talk about it and

[32:07] it's actually a great fit

[32:09] um for trading system or effectively

[32:11] it's used also in you know in chromium

[32:13] but also

[32:14] in the corner because it was originally

[32:17] introduced into the Linux kernel so

[32:18] that's they were called seclog

[32:21] originally fast read write lock

[32:23] and so the problem that the kernel

[32:26] developers had at the time is between

[32:28] user user land and on Canal land like as

[32:31] a user code if you would do a for Loop

[32:33] we'd get time of the day

[32:36] um the time that you get you know when

[32:38] you recall the C function get time of

[32:39] the day is um I mean it's updated by the

[32:43] canner

[32:44] and before using this lock

[32:46] um it was just a basic log that was

[32:49] there so the the reader the the code in

[32:52] New Zealand could do a Daniel of service

[32:54] where like the kernel wouldn't be able

[32:55] to get the log so I wanted to like

[32:57] change that

[32:58] and I also wanted to be a high

[32:59] performance so they come up with this

[33:01] SEC clock so what is it exactly

[33:05] so let's look at let's look at code I

[33:07] think it will be simple so this is by

[33:09] the way simplified version of the code

[33:12] um it's not a talk about atomics further

[33:14] in my slides the reference in case you

[33:17] know you want to find the full series

[33:19] code

[33:20] so let's not be distracted by atomics

[33:23] but also let's not use this code

[33:25] directly in uh you know in production

[33:27] because I think it compiles

[33:29] so we have an atomic

[33:31] version here

[33:33] and then we have our data

[33:36] and when the writer wants to publish

[33:39] something it's going to do a plus one on

[33:41] the version

[33:42] copy the data

[33:44] and press one on the version again

[33:48] right some

[33:50] on the right I have the assembly node of

[33:52] this version but of the let's say the

[33:54] the version you can find The Links at

[33:56] the end of my slides and it is you know

[33:58] us you know as fast as you can get you

[34:00] don't have actually even any any locked

[34:02] well this is on Intel of course on 86

[34:04] you don't have any prefix locked

[34:06] instruction

[34:08] um so it's it's fast

[34:10] so you might be wondering where is the

[34:12] trick well the trick is in the reader

[34:15] the reader is a bit special that's where

[34:17] like the smartness of that slot come

[34:18] come from

[34:19] as a reader you read the version

[34:23] if the lowest bit is set to one It it

[34:26] means that there is a right in progress

[34:30] and in that case you're going to return

[34:32] you failed reading the data otherwise

[34:35] you copy the data

[34:38] and then you check you need to recheck

[34:40] the version because while you're

[34:41] equipping the data Maybe the writer you

[34:43] know wrote something so it could be

[34:45] corrupted and therefore you check the

[34:46] version again if the version number

[34:48] match

[34:49] awesome it means that there was no right

[34:52] of course uh you know in that case T

[34:55] needs to be trivial equilibrium using

[34:56] student copy

[34:58] [Music]

[34:58] um

[34:59] that's um that's all good

[35:02] so this is quite special

[35:04] um and that are very interesting

[35:07] properties of that log that's you know

[35:09] in in trading system are amazing first

[35:13] of all you see that it's based on

[35:14] something that we call Optimist

[35:15] optimistic looking

[35:17] right why are we optimistic well because

[35:20] we we start copying the data thinking

[35:22] that it's not going to be changed at the

[35:25] same time as we read

[35:26] so if you have a lot of contention that

[35:28] log is going to be quite bad but if in

[35:31] general you don't then you go as fast as

[35:33] you can

[35:36] it fits a particular pattern when you

[35:38] will have only a few

[35:40] um producer in our case you only have

[35:42] one and many consumer in our case I mean

[35:45] on the diagram I showed you know we have

[35:46] like four or five in real trading

[35:50] application it's more like between 10 or

[35:51] 100. so that's that's quite nice and

[35:54] then the last characteristic is

[35:56] the producer is weight free the producer

[35:59] always get the right you know you always

[36:01] get the look no matter what the reader

[36:03] are doing and that that's also really

[36:04] nice

[36:07] so again for like um you know a system

[36:09] where you have a hundred thousand of

[36:11] this very small elements to share

[36:13] um the chance of contention are very low

[36:16] and that's a great property that the

[36:17] more elements you have

[36:19] um the better its case uh engineering

[36:22] system it's like it's it's inverse

[36:24] and in general yeah the data we share is

[36:27] quite small you know we are talking

[36:28] about a few double or floats you know we

[36:31] want to have a few prices there or

[36:32] volume you know system so it's um it

[36:35] works quite well

[36:36] so that's that's one of the you know

[36:39] like first Saturday you don't know to

[36:40] share data this is um this is really

[36:43] nice it's um so the latency you know we

[36:45] could measure it but it's it's just the

[36:47] latency of mem copy so I mean I could I

[36:49] could Benchmark it but like it's very

[36:51] much just benchmarking a mem copy and

[36:53] then of course on the reading side yeah

[36:55] it all depends about like you know the

[36:57] main variable will be the number of

[36:58] producer and also the frequency rates of

[37:01] your of the producer as well president

[37:03] or filament so

[37:05] it's actually yeah The Benchmark didn't

[37:07] really feel like adding one because it's

[37:09] uh it just depends on so many variables

[37:10] that depends on your system

[37:13] but yeah so with this you are you know

[37:15] as fast as you can get

[37:18] um when it comes to like sharing data so

[37:20] the second one

[37:21] that we want to look at is events

[37:25] you want to send now events with

[37:27] notification

[37:29] um not just sharing a state let's say on

[37:31] shell memory

[37:34] so in order to do that you have um again

[37:36] you know a few few options as I said you

[37:38] know our strategy is to have very low

[37:40] Jitter

[37:41] and you're aiming for like you know less

[37:43] than 100 nanoseconds

[37:45] so that discards quite a few options TCP

[37:49] well anyway not really your natural fit

[37:51] because we want to Fan out so it doesn't

[37:54] scale very well

[37:57] um UDP why not

[37:59] but you need to use or space it right if

[38:02] you you cannot go to Kernel you cannot

[38:03] afford going to Kernel for this event

[38:05] because of our requirements

[38:09] so you could use a space it but in my

[38:11] opinion you know a lot of code

[38:13] um to manage and quite some complexity

[38:15] while actually we we don't really want

[38:17] to send anything between server here our

[38:19] problem is just on one server we have 32

[38:22] core we wanna you know like have some

[38:24] communication between this course

[38:26] so what a proposition is a queue like

[38:31] very basic ring buffer on share memory

[38:35] so let's look at you know what kind of

[38:37] queue can we can we use here

[38:39] the first one is um not yet

[38:43] multi-consumer

[38:46] so this is a typical single producer

[38:48] single consumer queue it's um it's the

[38:51] one that's used in the kernel again if

[38:53] you look at uh you know the the are you

[38:56] are you urine

[38:58] um there is a queue it looks like that

[39:01] um you have two indices one for the

[39:03] rider one for the reader

[39:06] dial aligned uh okay this is again x86

[39:09] you know 64 so that you avoid any any

[39:11] sharing and that's why and so it's

[39:14] called

[39:15] um you know collaborative queue because

[39:18] the writer can you know the writer knows

[39:20] when when the queue is full and

[39:22] therefore you can push back

[39:25] we're looking at quite a different Beast

[39:27] here right we want multi multi-consumer

[39:30] and so this read index do we really want

[39:33] to have between 10 or 100 read index

[39:35] well actually not really that's gonna

[39:37] scale relatively poorly

[39:39] and at the same time this pushback

[39:42] mechanism do we actually want it

[39:44] no we said that we don't want a reader

[39:47] to compromise the writer right so in a

[39:49] system again when you have many many

[39:51] readers I don't want uh the reader to

[39:54] influence and to compromise the slow

[39:57] reader to compromise the inter trading

[39:59] system

[40:00] that that would be really bad if if a

[40:02] reader is slow I just want the hero to

[40:04] die right at some point the writer is

[40:06] just going to overrun it and then we

[40:09] will let it die

[40:11] so that's good news because you know

[40:13] relaxing

[40:14] um that constraint and effectively

[40:16] that's what we want

[40:17] we can think of something

[40:20] um

[40:20] that can scale better

[40:23] so this is the first first queue that I

[40:25] propose

[40:28] it uses two indices

[40:30] but both will be updated by the writer

[40:33] you know reader now we have many of them

[40:35] you can imagine that they are a bit all

[40:37] over you know like the queue reading we

[40:39] the writer doesn't really know where

[40:40] they are

[40:43] and so what's um what's happening there

[40:46] is that the

[40:47] the writer is going to update this

[40:50] pending index then it's going to copy

[40:52] the data and then update index

[40:54] and that's kind of like how we we will

[40:56] manage like uh you know like how we will

[40:58] detect that there is an overflow

[41:02] again we align this index yeah 64 you

[41:05] know 64 bytes okay this is again very

[41:08] x86 oriented

[41:12] and so simplified code again um you know

[41:14] the full code wouldn't really fit on the

[41:16] on the slide because we need to you know

[41:18] manage the whopping of the queue this

[41:19] kind of things but this is how it looks

[41:21] like

[41:26] and so why do we have these two indices

[41:30] um is again so the the logic

[41:33] kind of like the smile that is in a way

[41:35] is in the reader

[41:37] it's because you need you need the

[41:38] spending index on this index you need

[41:40] that to to check for overflow if you

[41:42] think that if let's imagine you will

[41:44] only have index so only one one of this

[41:47] Atomic Barber uh you would have to

[41:49] choose if you updated before or after

[41:52] your mem copy

[41:53] and there wouldn't be a possibility for

[41:55] the reader to detect if there is

[41:56] actually an overall so again very

[41:58] simplified code absolutely not complete

[42:00] but this is roughly how it looks like

[42:04] so what kind of performance do we get

[42:07] with uh with such code

[42:11] so this is where we are

[42:13] in Orange

[42:16] so I Benchmark against you know the

[42:19] famous Java descriptor because that's

[42:22] like the reference in terms of like

[42:23] blazing fast Q

[42:25] also doing you know fan out

[42:28] it's fast you know it's um and then two

[42:31] two other comparison uh I must say you

[42:33] know not fair comparison boost lock 3Q

[42:36] and Moody camera one not fair why that

[42:38] because uh there are although the SPM

[42:41] cqs they are doing unicast and not

[42:44] multicast so like each each of the

[42:46] consumer is going to you know pick pick

[42:48] on one of the elements it's kind of like

[42:50] a load balancer like a queue that we use

[42:51] now so I put them here but I must say

[42:54] you know not not fair to compare

[42:55] directly these numbers

[42:57] okay so now how do we go faster because

[43:00] you know that's that's nice but uh it's

[43:03] um it's always interesting to get a

[43:05] little bit faster effectively when when

[43:07] we have a queue in in Java that is in

[43:09] front of us

[43:11] um so

[43:14] the idea is to have less contention

[43:16] right so the reason that we are a bit

[43:20] limited is that here

[43:23] these two indices index and pending

[43:25] index

[43:27] um every single push and pop that we do

[43:30] is going to touch this Index right the

[43:33] writer is going to update them the

[43:34] reader is going to read them and that is

[43:36] going to create to cause a lot of cash

[43:39] currency traffic

[43:41] um

[43:42] so modern CPU all you know use different

[43:44] cash currency protocol but in short the

[43:47] cache line uh will never be in an

[43:50] executive state

[43:51] and that's that's the main bottleneck

[43:53] here

[43:55] so the idea that we want to try is to

[43:57] you know spread out this Atomic counter

[43:59] over

[44:00] you know like all over the place in our

[44:03] queue you know you kind of like a

[44:04] concept of elements in the queue

[44:07] and so we still need some atomics of

[44:09] course I mean some some uh some things

[44:11] in there you know to detect the Overflow

[44:12] so we need one bit you know to see if

[44:14] the elements is being written and we

[44:16] need a version to see if the if the

[44:18] writer didn't you know

[44:20] um didn't catch up with the reader so

[44:22] this is what it looks like we start at

[44:23] zero and then we publish a few elements

[44:27] the the the fifth one uh not sorry the

[44:30] sixth one is in is in progress

[44:32] um isn't being written

[44:34] and then the writer continued like that

[44:39] so this is now how it looks like we have

[44:41] a header with still a atomic viable

[44:44] but now the big difference is that this

[44:46] header is only used when when the reader

[44:48] joined the queue

[44:49] the rest of the time they are going to

[44:51] use the one per block

[44:56] and so the code looks like this

[44:58] again simplified version

[45:00] um

[45:01] didn't want to go through all the

[45:02] atomics because again it's a bit it's a

[45:04] bit tricky you need a you actually need

[45:06] fence

[45:07] to implement this but

[45:10] so this looks familiar what is it

[45:15] it it's uh it's a it's a seclog that we

[45:18] saw just a few slides before so what we

[45:21] actually did is we we have like this

[45:24] ring buffer it's an array of Vlog and we

[45:27] just have all our second there and again

[45:28] why does it work

[45:31] um because the secular properties are

[45:32] you know optimistic looking the more we

[45:34] have the better it gets

[45:36] um and also they work greatly you know

[45:39] as we saw before with just a few

[45:41] producer in our case one and many

[45:43] readers

[45:44] okay so that looks quite promising

[45:47] and it is

[45:49] so we we get a you know decent like

[45:52] performance

[45:53] um again you know like uh comparing to

[45:56] the other queues uh okay one reader you

[45:59] know you are like more than 60 million

[46:01] messages per second that's that's you

[46:03] know that's really a lot I'm not sure

[46:04] what we would do with that exactly but

[46:06] the nice property of that queue is that

[46:08] you know even at 10 or 100 readers we we

[46:10] stay relatively constant and we are

[46:12] actually like um we stay on like

[46:15] reasonable numbers you know we're able

[46:16] to dispatch

[46:18] um I mean in that case for 10 readers

[46:19] more than 20 million message per second

[46:23] so that's nice

[46:24] um

[46:26] finish on this concurrent access you

[46:27] know our strategy you know low latency

[46:29] low cheetah it doesn't actually also

[46:32] compromise the the producer that's

[46:34] that's what we have here to dispatch our

[46:36] event

[46:38] so now let's zoom out again a little bit

[46:40] so you know we looked at first you know

[46:42] optimizing on a data model in a process

[46:44] then we see okay how to share data

[46:46] between different applications

[46:47] and now the last you know building block

[46:50] or the last piece of knowledge that you

[46:52] need

[46:52] um you know to to build such low latency

[46:55] system is um is to look at the entire

[46:57] server serve on its own

[47:02] so you can run you know a quick test

[47:05] to see if your server is properly tuned

[47:09] or not

[47:11] this graph

[47:13] is a graph of

[47:15] each point is consecutive calls of Rd

[47:20] TSC rdts season 86 instruction

[47:24] it's uh it's reading the TSC TSC counter

[47:27] is um the the wall clock that we have in

[47:31] in the in the CPU itself

[47:34] so we are reading this clock

[47:36] you know 200 times or so I mean 256 here

[47:40] and then we are we are plotting the

[47:43] distribution where we are plotting uh

[47:45] the the latency

[47:47] um and also the standard deviation so

[47:48] what we see in the graph

[47:51] is that we have um roughly five or like

[47:55] six buckets of latency

[47:57] quite quite a high standard deviation

[48:01] and so you know why would we get that so

[48:03] this is what you should get if your

[48:05] system is properly

[48:07] um

[48:08] well configure or tune for low latency

[48:10] application

[48:12] okay what is what is the difference

[48:17] oh this is code actually just for

[48:18] reference so again you know calling GSM

[48:21] um using using some boost accumulator to

[48:24] not do the you know any mistakes when

[48:26] calculating the statistics

[48:29] but so the main difference between these

[48:30] two

[48:31] um graphs are C States and P States

[48:34] so your your CPU is uh optimized for

[48:38] power consumption

[48:40] and um and the first one is then P

[48:43] States uh you know this laptop now that

[48:46] I'm using just to show the slides well

[48:48] we're not doing high performance

[48:49] Computing with it at the moment it's

[48:51] just you know displaying the slides

[48:52] therefore it's probably running at 400

[48:54] megahertz

[48:55] it can run on a much higher uh voltage

[48:58] and frequency up to three gigahertz but

[49:00] at the moment it's at 400 megahertz

[49:03] and that's that's great that's also like

[49:04] why your battery can last so long

[49:07] um the problem is uh if you have a lot

[49:11] of like

[49:12] if you have a lot of violence in like

[49:14] the input you get in your system back to

[49:16] financial markets you can have a very

[49:18] very quiet day and then there is a

[49:20] headline or something is happening and

[49:21] you get a burst of a million of

[49:23] messenger

[49:24] it's problematic that we are at 400

[49:27] megahertz when we start receiving a

[49:29] burst of a million message you actually

[49:31] want to wanna you you can't really uh

[49:34] afford that even worse than that

[49:37] um if the system is really not doing

[49:38] much you might and I mean not you but

[49:41] like your CPU might and

[49:43] assistate on the C state are effectively

[49:46] idle States so ECB is sleeping

[49:48] so imagine as if you were sleeping in a

[49:50] receive a million message

[49:52] well that's that's not gonna fly

[49:53] especially when the CPU enters sea State

[49:55] depending on the C State without going

[49:57] into the details you're going to start

[49:59] flushing all the different levels of

[50:00] cash

[50:01] so not just your awake uh you're all of

[50:05] a sudden being wake up

[50:07] um but also all your cash are empty so

[50:10] you're going to be relatively slow

[50:12] by processing your data

[50:16] so yeah what do you want to what you

[50:17] want to do if you already aim and that's

[50:19] not like for every single server that

[50:21] you have of course but like for the one

[50:22] that needs to process at line rate

[50:25] you need to tune that you need to like

[50:27] disable this again not on all your

[50:29] server but just on the one that like

[50:31] need like to process 10 gigabits per

[50:33] second

[50:37] another common optimization is like

[50:39] shared

[50:40] um you know L3

[50:42] and so LLC is like the the L3 last level

[50:45] of cache it doesn't have to always be as

[50:47] driven in general days and so here this

[50:50] is like an optimization that I did a few

[50:51] years ago we were changing a single line

[50:53] of code like no code change

[50:55] just reorganizing application

[50:58] you know moving them from One Core to

[51:01] another

[51:01] and so this is what we

[51:04] um briefly you know talked about earlier

[51:06] which is like

[51:07] you need to consider a system of the

[51:09] whole some simple some some code not

[51:11] just even zippers first but like any

[51:12] application can run on different core or

[51:14] news more or less memory

[51:16] and can do what we call you know L3

[51:18] thrashing

[51:20] um and so by by moving application

[51:22] between server or on different cores and

[51:26] making sure that all the low latency

[51:28] ones are together and because in general

[51:30] they shouldn't use so much memory

[51:33] um you you just you achieve great great

[51:35] results like that we were changing any

[51:37] any code

[51:42] you have much more

[51:44] um you know CPU Co isolation in

[51:47] interrupt Affinity

[51:50] um between servers you definitely you

[51:52] know we mostly Focus today about sharing

[51:54] data on the same server so we're talking

[51:56] about shared memory but between server

[51:59] definitely yeah you need to use a

[52:00] networking stack there is not so much

[52:01] option but you need to to use a space

[52:04] itself

[52:06] um and then pneuma aware code

[52:09] um you know if you allocate your memory

[52:11] on the far new pneuma node and you

[52:14] access it you're just gonna pay for for

[52:16] nothing

[52:18] um and the last one not least like huge

[52:20] Pages

[52:21] um yeah you can get great great benefit

[52:23] from from huge Pages

[52:26] um and again you you can enable them

[52:28] with um what's called on Linux

[52:30] transparent huge pages but there's also

[52:32] have a little bit of a cost actually a

[52:35] quite a quite a quite an expensive cost

[52:37] so in general again your code if if it

[52:41] really matters needs to be aware about

[52:43] huge pages and you you should use

[52:46] allocator Finance

[52:50] um further in the slides again on

[52:51] reference I put some links if you want

[52:54] to go

[52:55] further on this

[52:59] so now that the last last section of

[53:01] this talk

[53:02] again zooming out now you have you know

[53:06] reasonably fast application

[53:09] um you have also reasonably fast

[53:11] communication between this application

[53:13] and you also did all the system tuning

[53:16] so the system is on everything is fast

[53:18] awesome

[53:21] um one thing that you want is to scale

[53:23] right you you're going to have one

[53:25] trading server and then you have a

[53:27] second one and you know maybe at some

[53:28] point you have a thousand of them and um

[53:30] and even if you have only ten you still

[53:32] want to make sure that all the things

[53:34] that you measure day Zero because it's

[53:36] always the same you know days yo we're

[53:37] all very excited so all the eyes are on

[53:40] the new system

[53:42] um but how about six months later or a

[53:44] year later or even five years later you

[53:46] actually need to make sure that you stay

[53:48] fast

[53:51] and yeah no secret you know it's um it's

[53:55] not really

[53:56] um

[53:57] uh a breakthrough here and I'm going to

[53:58] say but like you just need to

[53:59] continuously measure

[54:01] so the same as like you would do a

[54:03] Google Benchmark to make sure that you

[54:04] got this fast Day Zero you need to keep

[54:06] on measuring in pod always

[54:09] so here this is you know just a graph

[54:12] and a dashboard

[54:13] I think it just illustrates so well you

[54:15] know why you should measure in

[54:16] production constantly because looking at

[54:18] this graph immediately you have a few

[54:20] questions that comes to mind so these

[54:22] what what is it this is just a period

[54:24] like a few hours where the same

[54:26] application running on different

[54:27] exchanges or products are running and

[54:30] why access is the latency of a

[54:32] processing

[54:34] but even without knowing

[54:36] um what is this application exactly

[54:37] doing

[54:39] because it's the same application

[54:41] I can immediately you know ask myself

[54:43] hey why why is this top application like

[54:46] this instance that is in blue

[54:48] why is the latency of this one like what

[54:51] is it two times

[54:52] yeah actually a four times higher than

[54:55] the second one

[54:57] um and then why do I have a difference

[54:59] of 10 times uh you know with the third

[55:01] one what's happening there so you need

[55:03] you need like this very like high level

[55:06] um dashboards on Matrix in order to yeah

[55:09] have this like

[55:10] View and then you can dig in into like

[55:13] uh a more like you know system level

[55:15] metrics or something that is a bit more

[55:16] low level

[55:20] so how does it work how do you do it

[55:22] again you need to do that when you

[55:25] design your application

[55:26] you can't just uh you know go on every

[55:29] single server and run perf or something

[55:30] that's uh you should do that as well if

[55:33] there is a poem you know if but it's a

[55:35] later stage first you want to have you

[55:37] want to collect all these metrics and

[55:39] therefore it needs to be built in your

[55:41] application

[55:42] and it's not really hard to to do it

[55:46] just needs to be usually done at the

[55:48] beginning of the you know when when you

[55:49] design that software

[55:51] in short what you want to do is you just

[55:54] want to have like you know time stamping

[55:56] uh all your you know not just not

[55:59] functions let's say but like critical

[56:01] part of your system like you know how

[56:02] long does it take to process iOS

[56:05] um so CPU time unlike for example you

[56:09] took you know earlier about cues

[56:12] that's awesome all these queues but all

[56:14] the skus that are fixed size

[56:16] okay you probably you probably are very

[56:17] interested about like the the capacity

[56:20] of the Queue at a given interval you

[56:22] know am I 10 behind or am I 90 behind

[56:27] because even 90 behind it means that you

[56:29] know if it's a bit busier then then the

[56:32] consumer will be out

[56:33] so you're interested in all this

[56:35] characteristic of your system

[56:37] and then

[56:39] um even you know better than you know

[56:41] looking at them because you can only

[56:43] it also doesn't really scale right I

[56:45] mean you can look at this graph every

[56:46] day when you wake up with a bit of

[56:47] coffee but um what's even better is to

[56:50] have some code I actually check that

[56:53] you know the median of the average or

[56:55] you know you name it the statistic uh

[56:58] that you that you pick for your system

[56:59] actually match and the expectation much

[57:01] and much higherity

[57:05] and so yeah how does it work concretely

[57:07] back back to your system

[57:10] um again cues

[57:13] um this metrics you know you're going to

[57:15] have like a timestamp

[57:17] on a x86 you know we always use rdtsc

[57:21] because that's like as fast as you can

[57:23] get you know in just a few cycle you get

[57:26] you get a nice wall clock

[57:27] and and then you just you know you have

[57:30] a you have statistics on the difference

[57:31] between rdtsc and you push that to a

[57:34] queue and that's just you know spacq

[57:37] um and then you have a another process

[57:40] or thread

[57:41] feeding this Matrix and then just

[57:43] publishing that to database

[57:46] and then you can visualize it in in the

[57:48] previous slide we're using a grafana to

[57:50] visualize that

[57:55] come to the conclusion

[57:57] um

[57:58] we you know we we saw a few things today

[58:02] but there is of course a lot more to say

[58:04] about low latency systems also about

[58:07] trading system and ours of course you

[58:10] know way way too short

[58:13] um but I hope that you know today's

[58:14] strategies and tactics die enough or

[58:17] maybe not to build a full trading system

[58:19] of course but uh there are definitely

[58:21] the building blocks that that we use on

[58:22] that you know most of the companies in

[58:24] the industry use so that you can be you

[58:28] know robust

[58:30] um which means no matter how busy things

[58:33] get you can keep trading and then also

[58:35] fast

[58:36] um back to this latency on like you know

[58:38] why you need it

[58:41] um a few reference uh you know if you

[58:43] wanna go further in in these topics

[58:47] um I always put up the first reference

[58:48] you know this uh this famous paper from

[58:50] ulrish for his maintenance of gdpc

[58:53] because it's very much when it comes to

[58:55] data model uh it is the reference

[58:57] relatively old now but still very much

[59:00] up to date and then you know some others

[59:04] um I always like Mike Acton talk or TV

[59:07] Beacon already eight years ago about

[59:09] data and the design I think it's great

[59:10] talk and um on the paper but seclog you

[59:13] know if you're interested uh it's as I

[59:15] said it's quite tricky to do in C plus

[59:17] plus so you know there is a full paper

[59:19] about you know how to do that correctly

[59:21] in uh in C plus plus

[59:24] so I am thank you very much

[59:28] thank you

[59:34] I see that it's exactly 2 p.m

[59:36] um yeah I will stay around if you have a

[59:38] if you have questions thank you

[59:42] foreign

[59:43] [Applause]