# When Nanoseconds Matter: Ultrafast Trading Systems in C++ - David Gross - CppCon 2024

https://www.youtube.com/watch?v=sX2nF1fW7kI
Translation: zh-CN

[00:09] Thank you all for being here.
  感谢大家的光临。

[00:11] Very excited to be yeah here with all of you.
  非常激动能在这里和大家在一起。

[00:13] In Denver, Colorado.
  在科罗拉多州的丹佛。

[00:13] Thank you.
  谢谢你。

[00:15] So that we're going to watch this video later online.
  所以我们稍后将在网上观看这段视频。

[00:17] Today we're going to talk about low agency trading system in C++.
  今天我们将讨论C++中的低延迟交易系统。

[00:23] So I'm David.
  我是大卫。

[00:23] I've been working in trading in the industry for about 10 years.
  我在交易行业工作了大约10年。

[00:28] I always adopt Market maker.
  我一直采用做市商的模式。

[00:32] Before that, I was working on relatively similar systems in a very different industry in defense.
  在此之前，我在一个非常不同的行业——国防领域，从事相对相似的系统工作。

[00:37] And today we are going to talk about engineering low latency systems.
  今天我们将讨论工程化低延迟系统。

[00:43] I like to put the emphasis on engineering because it's not going to be a very theoretical talk.
  我喜欢强调工程化，因为它不会是一个非常理论化的演讲。

[00:48] And we're going to be uh looking at actual problems that we want to solve.
  我们将着眼于我们想要解决的实际问题。

[00:53] And throughout this St we're going to look at some principles along the way.
  在整个过程中，我们将探讨一些原则。

[00:57] Some some of them that I you know collected through my career that I found are important and I think that could help you.
  其中一些是我在职业生涯中收集到的，我认为它们很重要，并且可以帮助到你。

[01:04] And also some profiling techniques.
  还有一些性能分析技术。

[01:06] But before going.
  但在开始之前。

[01:09] into the technical details um let's backtrack a little bit.
  进入技术细节，嗯，让我们稍微回顾一下。

[01:15] by around two two and a half thousand years where most things started at least in the Western World.
  大约在两千两千五百年前，在西方世界，大多数事物都开始了。

[01:22] which is the Antiquity we start with the Roman Empire and um there are few I mean many things that are actually fascinating about the Roman Empire.
  那就是古代，我们从罗马帝国开始，嗯，有很多关于罗马帝国真正令人着迷的事情。

[01:31] and but at least you know two of them are which is you know it lasted for a very long period of time and it was already vast you know it went from like the Hadrian worlds in modern UK to um modern Iran which is Persia.
  但至少你知道其中两点是，你知道它持续了很长一段时间，而且它已经非常庞大了，你知道它从现代英国的哈德良长城一直延伸到现代伊朗，也就是波斯。

[01:45] and um you know like historians in general do not agree on a lot of things about Romans for example the reason of their decline but they all agree on one thing which is the key to their success.
  嗯，你知道，总的来说，历史学家们在很多关于罗马人的事情上并不一致，例如他们衰落的原因，但他们都同意一件事，那就是他们成功的关键。

[01:58] uh is that they they were very good at planning things they had a really good infrastructure and organized they were very disciplined.
  呃，就是他们非常擅长规划事物，他们拥有非常好的基础设施，并且组织良好，他们非常有纪律。

[02:06] now you know what did the Romans plan well a few.
  现在你知道罗马人规划得很好的是什么，一些。

[02:10] things here have some example Urban food.
  这里有一些城市食物的例子。

[02:12] supply military campaigns they already.
  供应军事战役，他们已经。

[02:15] famous for the military achievements now.
  以军事成就而闻名。

[02:18] something a little bit more fun to talk about than military achievements is festivals.
  比军事成就更有趣的是节日。

[02:22] here you have a pictures of circus Maximus which you can still see the rest in uh in Rome in Italy.
  这里有罗马圆形剧场的图片，你仍然可以看到它在意大利罗马的遗迹。

[02:31] and it has an estimated capacity of 200,000, 300,000 people.
  它的估计容量为 20 万至 30 万人。

[02:34] so they were organizing small parties now.
  所以他们现在正在组织小型聚会。

[02:37] and if you want to you know if if you're oring such large events you have an interest into yeah planning things for your economy.
  如果你想，你知道，如果你正在组织如此大型的活动，你就会对规划经济事务感兴趣。

[02:48] so what the Romans actually figured out and what they did is that they would actually go to you know Farmers you know wine maker etc.
  所以罗马人实际上想出了什么，他们做了什么，就是他们会去，你知道，农民，你知道，酿酒师等等。

[02:56] you know they needed some grain some meat all kind of things for the events.
  你知道，他们需要一些谷物，一些肉类，各种各样的东西来举办活动。

[03:00] and they would ask them a year from now, you know what price would be would you be willing to sell you know that grain.
  他们会问他们，一年后，你知道，你愿意以什么价格出售，你知道，那些谷物。

[03:09] and by doing so the Romans.
  通过这样做，罗马人。

[03:11] Invented very much, you know, like early derivative trading.
  发明了很多，你知道，就像早期的衍生品交易。

[03:14] It was actually called Roman future contracts.
  它实际上被称为罗马期货合约。

[03:16] And so what the Romans figured out is that like the world are a certain uncertainty.
  所以罗马人发现的是，世界存在着一定的不确定性。

[03:21] And so when I was preparing this, I actually learned about this index called the World Uncertainty Index.
  所以，在我准备这个的时候，我actually了解到了一个叫做世界不确定性指数的指数。

[03:32] Now, what you can see on that, you know, that blue line, you mostly see relatively, let's say, negative events.
  现在，你可以在那上面看到，你知道，那条蓝线，你主要看到相对来说，比如说，负面事件。

[03:38] It doesn't mean it's all bad, but it's mostly that us humans are psychological creatures and we attach more weight to potential loss than potential gains.
  这并不意味着一切都糟，但它主要是说我们人类是心理生物，我们更看重潜在的损失而不是潜在的收益。

[03:53] I think that makes sense, usually if you cannot sleep at night, it's mostly because you can actually lose something more.
  我认为这很有道理，通常如果你晚上睡不着觉，那主要是因为你实际上可能会失去更多。

[03:57] Actually, if you can make some some profit, right, you can relate to that.
  实际上，如果你能赚到一些利润，对吧，你能理解的。

[04:01] And so back to the Romans, they indeed found a solution to, you know, reducing the world uncertainty by doing derivative trading.
  所以回到罗马人，他们确实找到了一个方法，你知道，通过进行衍生品交易来减少世界的不确定性。

[04:06] One thing that they
  他们的一件事

[04:12] didn't actually truly solve is uh.
  实际上并没有真正解决的是嗯。

[04:14] usually when things are quite uncertain.
  通常当事情非常不确定时。

[04:17] H you have also potentially a problem of liquidity.
  H您也可能面临流动性问题。

[04:20] and I think makes sense to think that if there is a certain economic turmoil and there is a lot of uncertainty in the world at the same time.
  而且我认为有道理的是，如果存在一定的经济动荡，并且同时世界上存在很多不确定性。

[04:27] people would be less likely to actually agree on a price a year or even two years from now.
  人们不太可能同意一年甚至两年后的价格。

[04:32] because you know who knows what can happen.
  因为你知道谁知道会发生什么。

[04:37] now going into you know the modern world.
  现在进入，你知道，现代世界。

[04:41] I mean that's why we've got market makers.
  我的意思是，这就是为什么我们有做市商。

[04:43] and I like to yeah ground that token to the reality and so least my reality.
  而且我喜欢，是的，将那个代币锚定到现实，至少是我的现实。

[04:50] I've been working for 10 years for a market maker.
  我为一家做市商工作了10年。

[04:52] so that's that's why we've got that.
  所以这就是为什么我们有它。

[04:53] there are many different actors of course on the financial markets.
  当然，金融市场上有很多不同的参与者。

[04:57] and today we're going to talk about low latency and many of these different actors of different requirements in in low latency.
  今天我们将讨论低延迟，以及许多具有不同低延迟要求的不同参与者。

[05:03] we'll come to that in a bit.
  我们稍后会谈到这一点。

[05:07] um we say that market making is a losers game.
  嗯，我们说做市是一个输家的游戏。

[05:10] and so what we mean by that.
  所以我们的意思是。

[05:16] is that effectively you need to be cons.
  那就是你实际上需要做到好。

[05:20] consistently good at pretty much everything right.
  在几乎所有事情上都持续优秀，对吧？

[05:22] it's not about this one Silver Bullet that's going to allow you to to to beat the market.
  这不是关于一个能让你打败市场的“银弹”。

[05:30] you need to be constantly good at everything.
  你需要持续在所有事情上都做得好。

[05:31] and so effectively um what market makers you know how it works is that you you you make some small profit.
  所以有效地，嗯，做市商，你知道它是如何运作的，就是你，你，你赚取一些小利润。

[05:38] um and the small profits are from well back to the Romans you know we you agree on a price a year from now.
  嗯，小利润来自于，嗯，回到罗马人那里，你知道，我们，你同意一年后的价格。

[05:44] and um you know most of the time there is actually you actually got to pay for this small you know optionality.
  嗯，你知道，大多数时候，你实际上要为这种小的，你知道的，期权支付费用。

[05:53] can't be free because you have some risk you know prices might go up and down and you want to avoid big losses.
  不能免费，因为你承担一些风险，你知道，价格可能会上涨或下跌，你想避免大的损失。

[05:59] now why why would you actually lose a lot of money.
  现在，为什么你会损失很多钱呢？

[06:04] you're providing prices at any point of time of you know 100 thousands of instrument.
  你在任何时候都提供，你知道的，成千上万种工具的价格。

[06:13] and um there is a news I mean.
  嗯，有新闻，我的意思是。

[06:16] News come out all the time, but it can be a big news that come out.
  新闻层出不穷，但有时会爆出大新闻。

[06:21] And, um, you know, what would happen if you just leave all your prices and change what you would have actually, you know, stale prices and you would do actually a lot of bad trades?
  而且，嗯，你知道，如果你只保留你所有的价格，并改变你实际上会有的，你知道，过时的价格，你实际上会做很多糟糕的交易，会发生什么？

[06:34] So that's that's what we mean by like losers game consistently good at everything.
  所以这就是我们所说的像输家一样，在一切事情上都持续表现出色。

[06:38] Now, elaborating a little bit more about, you know, why do we need low latency programming here?
  现在，再详细阐述一下，你知道，为什么我们需要低延迟编程？

[06:42] Um, breaking this down into two categories.
  嗯，将此分为两类。

[06:46] I think the first one is relatively well known.
  我认为第一个相对来说是众所周知的。

[06:49] I think it's the most intuitive one, which is we want to, you know, we have we have a need for like reacting fast to uncertain event.
  我认为这是最直观的一个，那就是我们想要，你知道，我们有，我们需要快速响应不确定的事件。

[06:58] That news that I just mentioned, that news that comes out, we need to react, we need to effectively update our price or maybe just cancel our.
  我刚才提到的那个新闻，那个新闻出来了，我们需要做出反应，我们需要有效地更新我们的价格，或者也许只是取消我们的。

[07:08] The second one is a little bit maybe less straightforward, which I'm going to explain in the next line, which is you.
  第二个可能有点不那么直接，我将在下一行解释，那就是你。

[07:16] also need to be smart now like being smart is a vague concept um so you need good models and so on and so forth but you also need to be accurate and again accuracy is inly related to latency again for the same idea of like you've got an information information flow into your system and you want to yeah you want actually to have like accurate prices you want to ingest this information so on this slide here um you can see um what I would call relatively standard modern trading system and the reason I put that slide here is like one of the question and I get most of the time um when I go to conference or like in general is you know hey we've got fpgas they are already fast so why do we still care about software you know why do we still care about lcy C++ in that case and so there are two answers to
  现在也需要聪明，比如聪明是一个模糊的概念，所以你需要好的模型等等，但你也需要准确，再次，准确性仅与延迟有关，再次，对于同样的想法，就像你的系统中有信息流，你想，是的，你实际上想要有准确的价格，你想摄取这些信息，所以在这个幻灯片上，你可以看到，我称之为相对标准的现代交易系统，我放这张幻灯片的原因是，我最常被问到的问题之一是，当我参加会议或一般情况时，你知道，嘿，我们有FPGA，它们已经很快了，那我们为什么还要关心软件呢？你知道，为什么我们还要关心C++呢？所以有两个答案

[08:17] that question so on that graph you see the exchanges on the right price are disseminated into the system they flow into the system and so indeed you you've got that blazing fast fbga here going to say a bit more about it in a bit.
  这个问题，所以在那张图上，你看到右边的交易所价格被传播到系统中，它们流入系统中，所以确实你有了这个闪电般快速的fbga在这里，我将在稍后多说一点。

[08:35] why do we still need you know like to send orders even with software it's just what I would say is cost benefit.
  为什么我们还需要，你知道，像发送订单，即使有软件，我只能说这是成本效益。

[08:42] fpj is expensive not the card itself but just the engineering time or just like operationally it's something more complicated to manage than software.
  fpj很贵，不是卡本身，而是工程时间，或者说在运营上比软件更复杂。

[08:52] software is more flexible so it's always going to be there it's kind of hand to hand it's just an you know engineering you system properly on finding the right solution for the right problem so that's on the right.
  软件更灵活，所以它会一直在那里，它是手把手的，你只是你知道，工程，你正确地系统化，找到正确问题的正确解决方案，所以这就是右边的。

[09:04] now there is something a little bit deeper on why we need low latency a software which is you see the yellow boxes on the left the strategies what the strategies are doing is that they send so-called we
  现在，关于为什么我们需要低延迟软件，还有更深层的原因，你看到左边的黄色框，策略，策略正在做的是它们发送所谓的我们

[09:20] call them rules to the fbga the idea is
  将它们称为 fbga 的规则，其思想是

[09:23] that for the fpga to be blazing fast it has to be really simple
  为了让 fpga 运行得飞快，它必须非常简单

[09:30] I don't mean that it's a simple things to engineer it's actually quite complicated but functionally speaking it's very simple
  我并不是说工程设计很简单，实际上它相当复杂，但从功能上讲，它非常简单

[09:36] it sees bits compare bits send bits you could actually make it more complicated but it's just a trade-off then it would be slower
  它看到比特，比较比特，发送比特，你实际上可以使其更复杂，但这只是一个权衡，然后它会变慢

[09:42] and so the strategy send this rule so what is the rule very simple
  所以策略发送这个规则，那么规则是什么呢？非常简单

[09:48] if price greater than 10 update my price of conso and my order and so now if you think about it the strategies themselves they have already low latency requirement
  如果价格大于 10，则更新我的 conso 价格和我的订单，所以现在如果你仔细想想，策略本身已经有低延迟的要求了

[09:57] because if for example we would only update this rules every minute even every second with all the information that flows on the world on markets we would have again you know stale prices stale information
  因为，例如，如果我们每分钟甚至每秒才更新一次这些规则，并且考虑到流入世界市场的全部信息，我们就会再次出现，你知道的，过时的价格，过时的信息

[10:14] lots of people would be really happy to trade against you because you do mostly you know you do
  很多人会非常乐意与你交易，因为你主要做的，你知道的，你做的

[10:21] trade on like old information now let's get into you know some more technical content one of the first data structure you know I thinking when preparing this talk on like what would be something nice to talk about like a data stru that is that is actually very interesting is a order book now we're going to Define it in a bit but why it's interesting is because no matter you know like the trading system that you're working on could be algorithmic manual pretty much anything you always have this core component you know you always ingest price from The Exchange and you want to know what they are so let's quickly Define what it is on the left you see the bids that are the prices at which people are willing to buy you see $92 is what we call Our Best bid why is this interesting is because if you want

[11:21] To sell this is actually the price that you know you're going to sell it there.
  卖出这个实际上是你知道的，你将要卖出的价格。

[11:25] Is no reason for you to sell at 90 or $85 a year.
  没有理由让你一年卖出90美元或85美元。

[11:31] There is also you see 50 the 50 stocks.
  还有你看，50，50只股票。

[11:34] This is actually the total amount that people are actually willing to buy at this level.
  这实际上是人们愿意在这个价位上购买的总金额。

[11:36] So there might be one or multiple uh investor ready to buy at this price.
  所以可能有一位或多位投资者准备以这个价格购买。

[11:41] Um similarly you've got asks same same story there.
  嗯，同样，你也有卖出价，那里的故事也一样。

[11:46] Some people are willing to sell this is you know you've got a best ask $95 that's your the best price at which you can.
  有些人愿意卖出，你知道的，你有最好的卖出价95美元，那是你可以达到的最高价格。

[11:52] Bu why why is this interesting uh in um you know today's talk that is mostly going to be about low latency programming.
  为什么，为什么这很有趣，嗯，你知道的，今天的谈话主要将是关于低延迟编程。

[11:59] Is because you actually do have a lot of low latency constraint and requirements and these data structure at the very least two.
  是因为你实际上有很多低延迟的约束和要求，以及这些数据结构，至少有两个。

[12:07] I mean these are the two main ones.
  我的意思是，这是两个主要的。

[12:10] If you do algorithmic trading again back to what we said before.
  如果你进行算法交易，再说一遍，回到我们之前说的。

[12:14] You want to be as accurate and fast as you can so you inest this.
  你想尽可能准确和快速，所以你投资这个。

[12:22] prices and so you want a really fast data structure this is your your first one.
  价格，所以你想要一个非常快速的数据结构，这是你的第一个。

[12:26] if me want it to be fast and the second one is a little bit different.
  如果我想要它快速，第二个就有点不同了。

[12:31] it's more like I would say a system constraint is that you have a network card.
  我更倾向于说这是一个系统约束，那就是你有一个网卡。

[12:35] the network card as pretty much any you physical device has a finite number of buffers.
  网卡几乎和任何物理设备一样，都有有限数量的缓冲区。

[12:40] and if your application is not fast enough reading from you know these buffers well I mean you're just going to the network just going to drop.
  如果你的应用程序不够快，无法从你知道的这些缓冲区读取，那么你就会直接进入网络，就会丢包。

[12:51] I mean you're going to miss some of them which will cause an outage.
  我的意思是你会错过一些，这将导致停机。

[12:57] now going through some properties of this of this order book and we have two sequences that are ordered.
  现在我们来看一下这个订单簿的一些属性，我们有两个有序的序列。

[13:03] and you could you could you could argue you know we could have them unordered but I try to put some emphasis on why it's important that there that effectively this this prices stay ordered.
  你可以争辩说，我们可以让它们无序，但我试图强调为什么它们有效地保持有序很重要。

[13:14] that's what we need um concept of price level when we talked about it.
  这就是我们需要的东西，嗯，当我们谈论它时，价格水平的概念。

[13:19] you have a price the volume each order that is going to come.
  你有一个价格，一个数量，每一个即将到来的订单。

[13:25] Into this order book has an id64.
  这个订单簿有一个id64。

[13:30] And an important property is that a typical stock order book today.
  一个重要的属性是，今天的典型股票订单簿。

[13:33] We're going to look at stocks.
  我们将看看股票。

[13:36] Um has around thousand levels per side.
  嗯，每边大约有千个级别。

[13:44] Looking at our API, it's relatively straightforward.
  看看我们的API，它相对直接。

[13:47] Now, every exchange that are many, many exchanges around the world, they all offer slightly different messages.
  现在，每个交易所，世界上有很多很多交易所，它们都提供略有不同的消息。

[13:53] But in general, we've got an API that looks like this.
  但总的来说，我们的API看起来是这样的。

[13:56] This is not too you're not too crazy, it's relatively standard.
  这并不太疯狂，它是相对标准的。

[14:01] So we've got like an hard order operation, a new order comes in at the price volume modify and delete.
  所以我们有一个硬订单操作，一个新的订单以价格、数量修改和删除的形式进来。

[14:12] So let's go through some example.
  所以让我们来看一些例子。

[14:16] We want to buy at $92 for like 25 25 stocks.
  我们想以92美元的价格购买25股股票。

[14:21] So we're going to join that level.
  所以我们将加入那个级别。

[14:24] We go from 50 to
  我们从50到

[14:26] 75 then we add an order on a level that doesn't exist you know $110 so this is it's inserted.
  75 然后我们在一个不存在的级别上添加一个订单，你知道 110 美元，所以这是插入的。

[14:34] We modify our first order, we only interested to buy 15 stocks now not 25 anymore so it's a reduction of 10.
  我们修改我们的第一个订单，我们现在只对购买 15 只股票感兴趣，而不是 25 只了，所以减少了 10 只。

[14:42] And then we can also delete an initial observation that we have is you know looking at this API especially the the delete operation.
  然后我们也可以删除我们最初的一个观察，你知道，看看这个 API，特别是删除操作。

[14:53] I mean just as an example but modify as well is that no matter what data structure uh we're going to you know use for this for this order book we need a hashmap right.
  我的意思是，这只是一个例子，但修改也是如此，那就是无论我们打算你知道用什么数据结构来处理这个订单簿，我们都需要一个哈希表，对吧。

[15:03] We need a hash map around it for the reason that yeah again the D we only have the ID so we need to retrieve the price volume sign.
  我们需要一个哈希表围绕它，原因是因为，是的，再次，我们只有 ID，所以我们需要检索价格数量符号。

[15:11] Now today we are not going to talk about hashmap much main reason I didn't really want to talk about hashmap is that there are many many talks about it it's very very well documented engineering problem.
  现在今天我们不打算多谈哈希表，主要原因是我真的不想谈哈希表是因为有很多关于它的讨论，它是一个非常非常完善的文档工程问题。

[15:24] So I'm not going to not going to talk about that.
  所以我不打算不打算谈论那个。

[15:30] The most natural data structure that you can use for this order book is two map.
  你可以为这个订单簿使用的最自然的数据结构是两个映射。

[15:34] First, we're going to go through a bit of code, and then I will explain why this is the most obvious, natural implementation that you can do for this data structure.
  首先，我们将看一些代码，然后我将解释为什么这是你可以为这个数据结构做的最明显、最自然的实现。

[15:43] Right, you've got two St two St map, one for your beads, one for the as because they're ordered differently.
  对，你有两个 St 两个 St 映射，一个用于你的珠子，一个用于 as，因为它们的顺序不同。

[15:51] So the implementation is relatively straightforward.
  所以实现相对直接。

[15:56] In the OD order, we try to imp place.
  在 OD 顺序中，我们尝试 imp place。

[15:59] If the Imp Place succeeds, awesome, that means that the level didn't exist, we inserted it, nothing else to do.
  如果 Imp Place 成功，太棒了，这意味着该级别不存在，我们已插入它，无需做其他事情。

[16:08] Otherwise, we update it, we add our volume.
  否则，我们更新它，我们添加我们的数量。

[16:12] The D is also quite straightforward.
  D 也相当直接。

[16:15] The main thing to see here is that we take as a parameter the iterator.
  这里要看的主要内容是我们将迭代器作为参数。

[16:20] And you see that the add order effectively returns the iterator.
  你看到 add 顺序有效地返回了迭代器。

[16:24] The idea here is that for the operations we are doing, St map um as a great advantage that the
  这里的想法是，对于我们正在做的操作，St map um 具有一个巨大的优势，即

[16:33] iterators stay valid Le for the operations we're doing and so that's awesome because we can start the iterator in the hashmap we're already doing the hashmap lookup so might as well use that and so this is why what I meant with like this is the most obvious natural data structure for this problem because complexity wise it's it's it's actually really good um I actually think that this is like the best the best complexity that you can get for this problem if you find something better can can email me but it's pretty good it's you know the ad is is log in otherwise amorti constant now this is what we get so we're going to look at today at you know various measurements but here when we look at a data structure like this I find it important to look at the interior latency distribution for the reason if you just be looking at you know a few percentile or median average min you'd be missing some
  迭代器对于我们正在进行的操作保持有效，所以这很棒，因为我们可以在哈希表中启动迭代器，我们已经在进行哈希表查找，所以最好使用它，这就是我所说的，这是这个问题的最明显、最自然的数据结构，因为从复杂性来看，它实际上非常好，嗯，我认为这是你能为这个问题获得的最佳复杂度，如果你找到更好的，可以给我发邮件，但它相当不错，你知道，添加是 O(log n)，否则是摊销常数，现在我们得到了这个，所以我们将要看今天，你知道，各种测量，但在这里，当我们看这样的数据结构时，我认为重要的是要看内部延迟分布，因为如果你只看，你知道，几个百分位数或中位数、平均值、最小值，你就会错过一些东西。

[17:34] Information, this latency distribution is.
  信息，这个延迟分布是。

[17:37] It's not a micro benchmark, it's also not from production.
  它不是一个微基准测试，也不是来自生产环境。

[17:38] Uh, when I was preparing this stock, I, I gathered a week.
  呃，当我在准备这个股票时，我，我收集了一周的数据。

[17:41] It's a full week of market data on dozens of stocks, big tech stocks like Microsoft, Nvidia, Tesla, and so on.
  这是一整周的市场数据，涉及数十只股票，像微软、英伟达、特斯拉等大型科技股。

[17:51] And so yeah, I said, not production, but also not like a, um, a micro benchmark somewhere.
  所以是的，我说，不是生产环境，但也不是像，嗯，某个地方的微基准测试。

[17:57] Somewhere in between, and so this is what we, what we've got here.
  介于两者之间，所以这就是我们，我们在这里得到的结果。

[18:01] We see a distribution, not too surprising.
  我们看到一个分布，并不太令人惊讶。

[18:03] We see two peaks, one on the very left, which is the case for the modify delete when we only do a hashmap lookup.
  我们看到两个峰值，一个在最左边，这是我们在只进行哈希映射查找时的修改删除情况。

[18:10] And then you a bit more to the right when we go through binary tree.
  然后当你往右边一点，当我们通过二叉树时。

[18:20] Now, effectively, this first distribution, as you call it, it's a bit of a line.
  现在，有效地，这个你称之为的第一个分布，有点像一条线。

[18:24] And so this is, this is the true, you know, what I call the true story, the real distribution that you for this data.
  所以这是，这是真正的，你知道的，我称之为真实的故事，你对这些数据的真实分布。

[18:34] structure and so what I did here is between between the AUD operation I just uh added some memory locations not really to really do much with them but just to randomize the the Heap um and this is mostly to um this is mostly because the most modern Malo implementation um they have like this thing that is at the same time great and also not so great when you measure that when you call Malo several times you get a continuous block um of memory and so yeah that that's great but it's also not so great if you actually want to know the performance of this data structure how it's going to behave in production and so very much here we are we are we are messing a little bit we we're kind of like we we're checking and measuring how you know the the cach locality of sud map and we know that it's it's going to be relatively poor.

[19:35] right it's a not container so in production it's very rare that there is no look at quite some trading system and it's very rare that there are zero Dynamic memo location we all know they on great but there always some around and so this is replicating this a pattern and so I said that we you know we will go through some principle along the way so this is our first principle most of the time you do not want node containers right so the enter family stood map set unordered map unordered set mean stood list of course but this one is is really obvious you know you put that on the side in general you you don't touch them and that reminds me a your talk of Sean par at cbon around like 10 10 years ago maybe 15 years ago he were saying roughly the same thing at Adobe in Photoshop 995 90% of the time they only use Vector not because it's simple not because they
  对了，它不是容器，所以在生产环境中，很少会没有，看看一些交易系统，很少会有零动态内存分配，我们都知道它们很棒，但总会有些。所以这是在复制这种模式，所以我说，你知道，我们会一路学习一些原则，所以这是我们的第一个原则，大多数时候你不想使用节点容器，对吧？所以整个家族，std::map, set, unordered_map, unordered_set，当然还有std::list，但这个真的很明显，你知道，你把它放在一边，总的来说，你不会碰它们，这让我想起你10年前在cbon上关于Sean Par的演讲，也许15年前，他在Adobe的Photoshop中也说了大致相同的话，99.5%的时间他们只使用Vector，不是因为它简单，不是因为它

[20:37] don't need other data structure but just

[20:39] because there is a performance aspect to

[20:42] it and of course closed hash map right

[20:45] hashmap backed by arrays of vector and

[20:49] so here you know like you see this you

[20:51] know on the picture you see a developer

[20:53] that

[20:54] is this land of this spair trying to

[20:57] fight it's kind of ghost of like the N

[21:01] containers and it's going to be really

[21:03] hard because it's everywhere on nowhere

[21:05] right so that's going to be that's going

[21:06] to be a tough fight this can remind you

[21:09] maybe some stories I witnessed a few of

[21:11] the stories several times in my career

[21:13] where there is a you know team of

[21:14] Engineers that have as a mission they go

[21:17] on a crusade to

[21:20] um improve the performance or make fast

[21:23] an application that hasn't been designed

[21:25] you know to be fast and therefore using

[21:27] this kind of content and data structure

[21:29] all over the

[21:30] place usually you know when you you know

[21:33] when this happens you you you look at

[21:35] them from far and you say good luck

[21:37] because it's going to be really hard

[21:39] most of the time you actually need to go

[21:41] back all the way to drawing board

[21:43] develop it from scratch which is which

[21:46] is quite bad actually should never do

[21:49] that so moving on what are we doing yeah

[21:52] we use St vctor lower

[21:54] bound it's great you know it's like cash

[21:57] locality is as good as you can get um

[22:01] C++ 23 brings the support by the way of

[22:04] St flat map um I don't think the support

[22:07] is there at least not in not in the

[22:09] version of GCC or Clank that I'm using

[22:11] so this is what we what we use

[22:15] today the complexity is really different

[22:18] than what we got before

[22:21] um we now have you know a logarithmic

[22:24] complexity for our order but if we need

[22:26] to insert a level I mean Vector insert

[22:29] is no secret it's it's linear and then

[22:32] we we cannot store the iterator in our

[22:34] Ash map anymore right that doesn't work

[22:36] because we are modifying our structure

[22:38] and the iterators are not

[22:41] valid so complexity is much

[22:44] worse implementation wise relatively

[22:49] straightforward we do lower bound if our

[22:52] level exists we adjust it otherwise we

[22:55] insert it

[22:58] and so this is what we what we

[23:01] get sit in green we've got this new

[23:05] latency distribution for the

[23:08] vector and so I would say that this

[23:10] latency distribution is

[23:13] fine does anyone have an idea or want to

[23:16] say you know why it is just fine and not

[23:18] so

[23:22] good yes there is a really big fat tail

[23:27] which is absolutely not what we want on

[23:30] such system you want something really

[23:32] narrow

[23:33] and so what does this tail come

[23:37] from to answer that question we need to

[23:39] look at the

[23:41] data because here we are actually not

[23:43] you know this data structure is actually

[23:45] quite specific here

[23:48] so looking at data so what does this

[23:50] graph actually shows so I picked one of

[23:53] the stocks that I had Nvidia and so we

[23:56] look at the distribution of the updated

[23:58] levels

[23:59] so what that means is so you remember

[24:02] before we were talking about best price

[24:04] or top price this is our index zero and

[24:05] then when we go deeper into the book one

[24:07] two three and so on so forth and so here

[24:10] this graph is by the way in L

[24:12] scale and there are two ways of saying

[24:14] that scientific ways the updates are

[24:17] exponentially distributed over price the

[24:20] simplest way of saying it is the action

[24:22] is happening on the top of your book

[24:24] right makes sense you have best best

[24:25] pric is where people want to buy and

[24:27] sell so it's constantly disappearing or

[24:30] improving and so on so forth now what

[24:33] that means for data

[24:35] structure our Vector effectively you

[24:38] know we're just humans so the most

[24:41] intuitive ordering for this Vector was

[24:43] to have our best price at the beginning

[24:46] index zero right so this $92 best bid

[24:49] was in index zero so we just follow

[24:52] intuition there the problem is that with

[24:55] what we just saw on the previous slide

[24:57] we're going to shift continuously our

[24:59] levels in memory all the time shift or

[25:02] elements in the

[25:05] vector and so good news then you know

[25:08] the solution is relatively simple we can

[25:10] just reverse our Vector it's a

[25:14] relatively small code change bit less

[25:17] intuitive but now we minimizing the

[25:19] amount of copies I mean our moves me the

[25:22] same like with this uh with this types

[25:24] that we've got

[25:28] and so we have a much you know much

[25:31] nicer latency distribution here that

[25:33] tail is just completely gone so that's

[25:36] awesome that bring us to our second

[25:40] principle which is kind of like you know

[25:43] like a a different way of saying this

[25:45] old code which is like a a problem well

[25:49] stated half solved so yeah you need to

[25:53] understand your problem this is very

[25:54] much an engineering problem if you just

[25:56] goe down into the data structure not

[25:57] looking at

[25:59] you know what's around it very much like

[26:01] the business domain you will uh you will

[26:04] be missing

[26:06] out and a third principle is also really

[26:12] important this is a very specific

[26:15] problem that we're going to solve here

[26:17] and we need to leverage these properties

[26:21] this very specific properties in order

[26:23] to get performance otherwise it's going

[26:26] to be very hard right we um we can't

[26:28] really get anything much faster if we

[26:30] don't leverage some of these underlying

[26:33] properties and of course to leverage

[26:34] them we need to find them and so here

[26:37] like you know in the picture you see on

[26:38] the left we've got this very down to

[26:40] earth engineer pragmatic engineer and

[26:43] then on the right you know more kind of

[26:45] like in the in the sky of the IDS thing

[26:49] and we need both but very much we are

[26:51] now in this stock on on the left side I

[26:54] mean actually not just in this stock but

[26:56] when solving a specific problem

[26:59] now you know how do we go faster from

[27:05] there we're using p so here I'm just

[27:08] going to show a little bit how I I'm

[27:11] usually doing things

[27:13] um this is not going to be extremely

[27:16] accurate so it's not going to work or

[27:18] should I say it's not going to work for

[27:20] a micro Benchmark or if your benchmark

[27:21] is really short this is definitely not

[27:23] going to work but in in in my case in

[27:26] general my Benchmark and not B micro

[27:28] Benchmark that take seconds or minutes

[27:30] so effectively just forking running per

[27:32] is awesome the idea here is just that

[27:35] you don't want to measure anything about

[27:37] the initialization phase because there

[27:39] you have actually a lot of stuff going

[27:40] on and you don't want to measure that

[27:42] it's not

[27:45] interesting and uh the first measurement

[27:47] that you want to do and this is actually

[27:49] really important because I see these

[27:51] mistakes this m actually being done over

[27:54] and over in general Engineers um all of

[27:57] us are getting quite excited when you

[27:59] get to that stage and so we start

[28:00] measuring you know everything and

[28:02] nothing we just start I'm going to

[28:04] measure the number of cashm for like the

[28:06] level one data Cas awesome you know why

[28:09] is that yeah intuition well in general

[28:12] intuition is wrong so here we need to be

[28:14] a bit you know thorough disciplined like

[28:17] like the Romans were um and uh you know

[28:20] we need to be a bit

[28:22] scientific and so you might recognize

[28:24] there different you know four categories

[28:27] I didn't come up with them they are from

[28:29] the Intel

[28:30] teum um which is the top down

[28:33] microarchitecture analysis

[28:35] method and why this categories are

[28:37] already nice is because there is very

[28:40] little overlap between them and you're

[28:43] also not missing

[28:45] anything right so these four categories

[28:47] retiring bad speculation front and bound

[28:49] back and bound they you you're not

[28:50] missing anything on what your CPU is

[28:54] doing now elaborate a little bit on what

[28:56] they mean

[28:59] in the best case every single of your

[29:02] you know instruction per cycle is going

[29:03] to be retired so you have 100% there

[29:06] effectively this never happens you

[29:08] always have bot neck but theoretically

[29:12] on a theoretical workload of course

[29:14] depending on your CPU microarchitecture

[29:18] can only only talk about x86 you know

[29:21] you've got a

[29:22] maximum again depend on CPU let's say

[29:24] seven instruction per

[29:26] cycle and so this IP of seven would mean

[29:29] that your yeah seven instruction per

[29:31] cycle will be all retired and if not

[29:33] then they would be you know um they

[29:37] would

[29:38] be not executed because of bad

[29:42] speculation not going to go into details

[29:44] there it could actually be an talk um

[29:47] but the main idea is there is usually

[29:48] it's caused by Branch misprediction or

[29:51] front and bound usually on decoding

[29:53] instruction or back and back and bound

[29:55] their own

[29:56] memory so that are the results that we

[29:58] that we've got for the you know for the

[30:01] code that we have you know the one that

[30:03] we looked at with our St vector and so

[30:06] if you've never seen if you didn't

[30:08] really measure or or looked at these

[30:10] categories before this is going to be

[30:12] maybe a little bit hard to get a grasp

[30:14] on if this is actually good or bad my

[30:17] best advice would be start measuring you

[30:19] know if you care about performance you

[30:21] got to measure so start measuring pretty

[30:23] much anything you know your workload

[30:25] some things that are more HPC low

[30:27] latency using data and so very quickly

[30:29] you will have a good gut feeling of what

[30:31] this numbers should be and so here I can

[30:34] tell you that the 25% of bad speculation

[30:36] is very high so let's you know dive into

[30:40] why is

[30:42] that per record is awesome it's a

[30:44] sampling profiler so very different than

[30:47] startat and it's going to give us a

[30:50] really quick you know answer to that

[30:52] question so what we see so what is this

[30:54] assembly code this assembly code is

[30:55] still the same it's so out order delete

[30:59] order if you remember this code is

[31:01] really short it's very much just lower

[31:03] bound stood lower

[31:06] bound and so what we see what's nice

[31:08] with this output is that we very quickly

[31:11] spot that 30% more than 30% of our CPU

[31:14] time is spent on two conditional jump

[31:18] and this two conditional jump are yeah

[31:21] in St L on it's a binary

[31:23] search although the CPU predictor is

[31:27] really good

[31:28] it is still going to struggle with the

[31:30] updates that we have right we cannot yet

[31:33] predict the markets through our CPU

[31:35] predictor that would that would be

[31:37] awesome um so that's our problem so how

[31:40] do we how do we fix it how do we solve

[31:44] it but we do branchless binary research

[31:48] right we just if we just remove the

[31:51] branch

[31:52] now there is something really important

[31:55] about branchless miny research

[31:58] so we shouldn't get too excited too

[32:00] early because the fundamental difference

[32:03] between this algorithm and lower bound

[32:05] the traditional binary sear is that

[32:06] there is no early xity anymore so we're

[32:09] going to go through the entire

[32:11] collection no matter what touching more

[32:16] memorium still it it is going to be you

[32:20] know we got a nice speed up

[32:22] here we also see that we have two peaks

[32:25] in our distribution again

[32:28] can speculate that you know it's it's

[32:30] due to this effect that the branchless B

[32:32] is touching more memory a lot of it is

[32:34] going to be really warm and some of it

[32:37] less less

[32:43] warm now if you want to go a little bit

[32:45] deeper here so just in the last tool

[32:48] effectively not the last one but another

[32:50] tool that you can use is that you can

[32:52] programmatically access Hardware

[32:53] counters to be even more accurate

[32:58] and so that's quite simple you can

[32:59] actually just put it around your code

[33:01] measure disc counters and so validate

[33:04] what your branches B research is doing

[33:07] so you cut down the branch misses by two

[33:09] awesome IPC going from 1.4 to 1.6 it's

[33:13] decent you're never going to get you

[33:16] know the two times two times on your IPC

[33:19] that would be huge so it's a decent

[33:20] optimization and then you can also see

[33:22] that indeed we are executing more

[33:25] instruction because there is no early

[33:27] exit anymore

[33:32] so another question is again you know

[33:33] how do we go faster from

[33:36] there well at this point we need to take

[33:39] a step

[33:40] back and think about the operations

[33:43] we're doing this is what binary search

[33:45] is doing it's you know we start our

[33:47] first pivot in the middle of the

[33:49] collection we go left right and so on so

[33:51] forth the heat map

[33:53] here well effectively doesn't really

[33:55] represent the heat map for this Ben mark

[33:58] this is assuming a more you know

[34:00] uniformly distributed uh

[34:03] updates but still it give us an idea

[34:05] about you know the you know how warm or

[34:07] cold will be the element in our

[34:10] collection at this point you might think

[34:12] that you know you you you might want to

[34:14] use for example an eight singer

[34:17] array because it's nice for cach

[34:19] locality and and also the access of the

[34:21] elements will be better for the

[34:23] prefecture now downside is of the iing

[34:26] array is it's actually

[34:28] there is a cost to build it also the

[34:31] ordering would be different so that

[34:33] actually doesn't really suit our

[34:37] needs and from there on effectively when

[34:40] when I prepared this St I already did my

[34:42] my work of I think almost archaeologist

[34:44] or historian right going through like

[34:46] all the possible order book that I could

[34:48] find adop also online think I measured

[34:52] like 30 different implementations more

[34:55] or less complicated and so we can go

[34:58] really complicated I've seen really

[34:59] things that like I was like wow this is

[35:02] this should be really fast usually it

[35:04] was actually not so

[35:06] much and if you if you really think

[35:08] about the hardware you know and how it

[35:10] works

[35:12] the the best implementation we can have

[35:15] we can find and the fastest is linear

[35:19] search it's blazing fast it's very like

[35:22] distribution is really narrow it's nice

[35:24] there is no

[35:26] tail um and so at this point you might

[35:29] be actually

[35:30] disappointed and you should not you

[35:33] should really actually embrace the

[35:35] Simplicity of this solution and how fast

[35:37] it is so this our fourth

[35:40] principle which is this is when you know

[35:44] that you know you've done your your your

[35:46] job well as an engineer when when it's

[35:49] fast and it's extremely simple as well

[35:51] at this point actually you can stop it's

[35:54] it's awesome

[35:58] and um and we also have a fifth

[36:00] principle

[36:01] here mechanical sympathy so what I mean

[36:05] here

[36:06] is you want algorithm that are in

[36:10] harmony with your

[36:12] Hardware which is what linear search is

[36:15] doing

[36:16] perfectly right so it's great for of

[36:20] course your cash

[36:21] locality it's great for the prefecture

[36:24] way the way you access memory it check

[36:26] all the boxes branches as well

[36:31] now there are certain things along the

[36:32] way that I didn't mention but are also

[36:35] important they are harder to

[36:38] measure mostly because they are around

[36:40] instruction

[36:42] cache you can definitely measure them

[36:44] but it's always going to be on your ENT

[36:47] workload right you you need your ENT

[36:49] application there you need all the

[36:50] instruction executed so you can't really

[36:53] you can definitely measure some effects

[36:55] in Benchmark but it's going to be harder

[36:57] to reason

[36:59] about and so one of them is yeah likely

[37:03] unlikely your compiler cannot know about

[37:07] your data right cannot cannot take a

[37:10] guess effectively we WR this keyword

[37:12] here on that

[37:14] Branch both GCC and clang decide to move

[37:18] this ad instruction at the very end of

[37:20] your

[37:22] function which is I mean it's just you

[37:24] know it's a reasonable approach um but

[37:28] what you want is actually to to keep

[37:30] this add instruction very warm and pack

[37:33] packed actually with the with the

[37:35] instructions because that branch is

[37:38] actually taken more often than the

[37:41] others a second one is I Fair important

[37:45] to include it as a proxy on um I guess

[37:49] another

[37:50] point which is there actually a lot of

[37:53] talks and a lot of you know things

[37:55] online um people really like inlining

[37:59] things and so there is a lot to be said

[38:02] about inlining and performance and and

[38:04] it's

[38:06] true but there is also a lot to be said

[38:08] about not inlining things and you know

[38:11] how that affect your your performance

[38:14] the performance of your instruction

[38:15] cache in that case and so you might have

[38:18] seen you know this expect on some of the

[38:21] previous slides it's just an asset that

[38:25] you have in relase build right so when

[38:27] you hit that asset condition is false

[38:30] you just you just die just go down and

[38:33] so effectively what you want there is

[38:34] you want to keep that code really far

[38:37] from from actually the code that's

[38:39] supposed to be

[38:40] warm and so that's one way of doing it

[38:43] right you really want that Co that code

[38:45] far into a cold section of your binary

[38:48] and also not in

[38:54] lined and so this is what the assembly

[38:56] looks like and that's great

[38:58] you um you have it really far there and

[39:00] it's not messing with your instruction

[39:05] cash last point also something that I

[39:07] actually I've seen you I mean I do see

[39:11] quite

[39:12] often we did use lambdas on F

[39:15] functors and they are awesome because

[39:18] the compiler knows the type and so he

[39:20] can actually really go far into the

[39:22] optimization if you were in our previous

[39:24] example if you were to use a St fun

[39:27] function here for you know for some

[39:30] reason if you were to like you know pass

[39:31] it in your Constructor or like just I

[39:33] don't know sometime matter of style

[39:35] people comment you know this Lambda

[39:37] could be

[39:38] refactored the consequence would

[39:40] actually be huge for the performance of

[39:42] this data structure right because you

[39:44] you lose I mean it's you lose type

[39:47] information stot function is using type

[39:49] eraser and so the code generated would

[39:52] be well very different I actually tried

[39:55] to put the assembly on the slide he

[39:58] wouldn't fit but the the performance

[40:01] would be would be would be terrible if

[40:03] to

[40:05] function so now that we optimize this

[40:08] data structure now that we've got this

[40:09] aut book you know nice and it's simple

[40:12] simple and

[40:13] fast the next thing we want to do is to

[40:16] send it around so we're going to talk a

[40:17] little bit about networking and

[40:19] concurrency the networking part will be

[40:21] relatively short because we still at CBP

[40:24] con um but I still want to say a word

[40:26] about it

[40:28] the general pattern is going to be to

[40:31] bypass

[40:33] Kel for your low latency um connections

[40:38] could be receiving or sending and once

[40:40] you're on a once you're on a server once

[40:42] you're on your box

[40:44] locally you would just use share memory

[40:46] to F out this information to all the

[40:48] different processors that you

[40:51] have and so here again in this talk not

[40:54] that I want to do any advertisement but

[40:56] I'm only talking about technology that

[40:57] I'm using solar flare I think is a

[41:00] relatively you know wellknown industry

[41:01] standard for low latency

[41:06] Nick and in general it could be solar

[41:08] flare other network cards that always

[41:09] come with like

[41:11] tools and tools or Library should I call

[41:14] them you usually have like um it's

[41:17] called open unload in that

[41:19] case it's it's just a wrapper on your

[41:22] binary using LD preload that is going to

[41:25] hook your BSD sockets and that's awesome

[41:28] because you can do user space networking

[41:30] without changing any of your

[41:34] code if you if you want to if you want

[41:36] to go even you know lower in terms of

[41:39] latency and if you want TCP stack which

[41:43] is actually quite nice to have can use

[41:45] TCP direct and if you really want to

[41:48] squeeze the last Nan on your operation

[41:52] on your networking operation you would

[41:53] use a layer two API it's called efvi or

[41:57] the equivalent would be dpdk from

[41:59] Intel and so that's you know as low as

[42:02] you can get you need to do all the

[42:03] buffer management

[42:05] yourself but it's the fastest so this is

[42:09] um sub measurements I didn't make the

[42:11] measurements that are actually from

[42:13] AMD again not to focus on the AMD part

[42:17] that are the same with like most low

[42:20] latency Nick but the idea here is to

[42:23] see um you know the difference between

[42:25] the kernel and then onload

[42:27] and then ebii so this is for UDP

[42:31] bucket and you see that you're on three

[42:34] mics for like your minimal size of UDP

[42:37] packet and then you know lowest L you

[42:40] can get it around

[42:42] 700 700 NS that's where you are when

[42:46] when using layer 2

[42:49] API which bring us to our six

[42:53] principle which is kind of similar to

[42:55] the to the principle four but simply

[42:57] it it's like the efficient you know if

[43:00] you want to be fast and

[43:02] efficient you should be mindful of what

[43:04] you're using and so here you see an

[43:07] engineer you're looking at this

[43:09] beautiful you know big machinery that's

[43:12] on the left which is the Linux Conner

[43:15] I'm not saying that in a bad way it's

[43:17] actually a beautiful Machinery it's

[43:19] awesome and but you should only use it

[43:22] you know if you actually really need

[43:24] it and in our case here it's it's not a

[43:27] perfect fit for our needs so you you

[43:30] want to bypass

[43:33] it connecting the dots so back to our

[43:36] trading

[43:38] system we've got again you know exchange

[43:40] sending prices now we have this purple

[43:42] box on the left where we have our fast

[43:45] Autobook we bypass the kernel and then

[43:48] we want to disseminate this information

[43:50] into all the process on that box we've

[43:52] got 50 strategy let's say and they all

[43:55] want the same order book

[43:58] and so we will put it in Shen memory in

[44:01] cues and on the right you see you know

[44:03] similarly if you want to send

[44:05] orders same story you know canot bypass

[44:08] reading from the

[44:10] cues now let's talk a little bit about

[44:12] shared

[44:14] memory um when I was also preparing the

[44:18] talk I was actually asking I asked on

[44:20] Reddit what people were interested you

[44:22] know to hear and uh and Shan memory

[44:24] actually came up a few times realized

[44:26] that yeah it's an industry St we use

[44:28] share memory a lot but it's not always

[44:30] very clear on like how or even why we're

[44:32] using it I wanted to say a word about

[44:36] that the reason is um it's principle six

[44:39] again if you're locally on a

[44:43] server you just don't need sockets so

[44:46] why would you use

[44:48] them it's not because they are slow it's

[44:50] just that you you don't really need that

[44:53] effectively if you would do user space

[44:54] networking on on the host you know the

[44:57] be kind of kind of kind of weird so you

[44:59] just want to use memory it's awesome

[45:02] it's as fast as it can get you don't

[45:03] have the kernel involved only when you

[45:06] map it and I think the reason that share

[45:08] memory is also quite quite popular on

[45:10] used is because you very much need it

[45:12] when you do multiprocess right if your

[45:15] architecture isn't really multi-

[45:16] threaded right you can just share data

[45:18] structure directly but there is also an

[45:21] upside of doing multiprocess mostly for

[45:24] operational

[45:25] reasons for not having something too

[45:27] monolithic that if one strategy goes

[45:30] down everything goes down right kind of

[45:32] separation of

[45:35] concerns what works well in memory

[45:37] continuous arrays awesome that's

[45:40] actually what we want to use that's also

[45:41] what's

[45:43] fast and concurrency here can be yeah a

[45:47] little bit tricky so in general my

[45:50] advice would be you know for for one

[45:52] shade memory

[45:53] file keep it to like one writer one

[45:56] producer

[46:00] in practice how it works is very much

[46:02] you know using using the cap

[46:05] API and then we have a header that looks

[46:09] like like this

[46:11] one this very much needed you need to

[46:13] describe a little bit your protocol so

[46:16] we've got your protocol name magic

[46:18] number it's very much needed because if

[46:20] you have 50 different protocols you got

[46:22] to be a little bit careful if at some

[46:24] point you open the wrong shared memory

[46:25] file you might you you might interpret

[46:27] the bytes the wrong

[46:29] way and then you have minor major

[46:33] version also very much needed it's

[46:36] always really easy to engineer a

[46:38] protocol day one it's much harder to

[46:40] have that protocol for years in

[46:42] production and then you still need to

[46:44] make updates to that system so you got a

[46:46] version of that

[46:49] protocol and then in our case we

[46:51] interested to send still you know this a

[46:52] books so sh memory so we have we have

[46:55] cues and so we got to describe this cues

[46:57] how big are they and from there on when

[46:59] you have when you have this um once you

[47:00] have this

[47:02] header you can you know do a little bit

[47:04] of pointer arithmetic and you have the

[47:06] full view of your sh memory so there is

[47:10] a bit of lowlevel thing but you can hide

[47:12] that around nice layer of nice layers of

[47:16] abstraction sorry in

[47:21] C++ now congruent cues

[47:26] the one we're going to look at today is

[47:29] going to

[47:30] be

[47:32] bounded so it's simple simple on fast we

[47:35] don't want to do any

[47:37] resizing it's not going to block this is

[47:39] really

[47:41] important again thinking of our trading

[47:43] system we have 50 strategies there if

[47:46] one strategy is too slow or as a bug we

[47:49] do not want to affect the writer because

[47:53] that would affect then all the

[47:54] strategies again which would be

[47:57] problematic we've got many

[47:59] consumers we want to support message

[48:02] variable length

[48:05] message reason here is um it's nice to

[48:09] support message that are actually um not

[48:12] just we don't want to just send pointers

[48:14] around for the reason that if you have a

[48:16] you know an architecture with multiple

[48:19] processes like at describe you you

[48:21] actually want to send also data also

[48:23] copying data is is fast you know copy is

[48:27] fast sending pointers around can lead to

[48:31] other

[48:32] problems needs to do dynamic memory

[48:34] location can lead to Heap fragmentation

[48:37] send pointers between different

[48:40] threads we want to dispatch this

[48:42] information so it's very much a fun out

[48:44] so all the consumer get the same data

[48:46] it's not a load balancer and we support

[48:49] pods so earlier this week there was

[48:51] actually another talk about concurent

[48:53] cues so at the end of this week you're

[48:55] all going to be expert in concent cues

[48:58] um the good news is this is actually a

[49:00] very different one than the one

[49:01] presented earlier this week right so

[49:04] that's

[49:06] good principle

[49:08] seven yeah you have a lot of different

[49:11] things that you can choose

[49:13] from and yeah you got to you got to you

[49:15] got to use the right tool for the right

[49:17] task right just presented a a few

[49:19] different things few different CES and

[49:20] you need to pick the right

[49:24] one now this is the queue we're going to

[49:26] look at for the next five five minutes

[49:29] or

[49:30] so we've got two

[49:33] counters right counter and read

[49:35] counter these two counters are both

[49:39] Modified by the

[49:41] producer right the producer has no idea

[49:44] if there is a consumer or not there can

[49:46] be zero there can be 50 producer doesn't

[49:49] know the consumer all read this counters

[49:52] they don't touch them they don't modify

[49:55] them this coun have the same value they

[49:57] point to the same element when there is

[49:59] no right

[50:02] operation when a right operation is

[50:04] happening the right counter is first

[50:06] Advance you advance the right counter

[50:09] then you copy your

[50:11] data and then you advance your read

[50:13] counter right so they have the same same

[50:15] point the same element Advance copy

[50:18] Advance the other one so very much the

[50:21] the enti Q header is just these two

[50:23] atomics each of them have their own cach

[50:25] line that's very much to false sharing

[50:28] and there are

[50:29] u64 they grow from zero to the infinite

[50:32] so it's very much the number of bytes

[50:34] that we write to this

[50:36] quebe and our API is relatively

[50:42] straightforward yeah as I said we write

[50:44] bytes we read bytes the read might

[50:46] actually return zero and so in the API

[50:49] for the reader we we we have to pass you

[50:52] know the buffer where we want to cop

[50:54] your data

[50:58] now this is simplified code but not too

[51:00] far from the um the interent code the

[51:03] only thing that's not there is the the

[51:05] case to actually wrap around the

[51:07] queue um I think it's quite simple and

[51:10] it didn't really fit on that slide so

[51:13] what you do here for the right operation

[51:15] add

[51:16] set you you know you calculate the the

[51:20] size of your payloads that's going to be

[51:23] so we do viable length message so we end

[51:25] cut the size so in that case four bytes

[51:28] plus your buffer you advance the right

[51:31] counter copy your data you advance your

[51:34] read

[51:37] counter the reader is a little bit more

[51:40] complicated but still relatively

[51:43] simple so local counter here is just a

[51:46] local you know it's a variable of our of

[51:48] our class it's not on sh memory we first

[51:51] check if there is anything to read then

[51:53] we read the

[51:55] size before using that science it's

[51:58] really important to check for a right

[52:00] counter because there could have been an

[52:03] overflow so we check for the right

[52:05] counter then we can use the size and

[52:08] copal data and then we need to check the

[52:10] right counter

[52:13] again now what's important to note here

[52:15] is

[52:17] that although this algorithm is

[52:21] correct from a language point of view

[52:24] we've got a problem here there is there

[52:25] is a data right I mean this St M

[52:29] Copy U we we are copying data while

[52:32] there is actually concurent we have

[52:34] concurrent access uh on nonatomic

[52:38] variables

[52:39] there and and by the way there is

[52:41] actually something that I forgot to

[52:43] mention which is also important is that

[52:45] some you the reason we do the St St M

[52:48] Copy on the size and not just you know

[52:50] using the equal operator is to avoid any

[52:53] strict alizing or alignment issue right

[52:57] so you actually have to use stent map

[52:58] copy is actually really

[53:01] important so here we've got a data um

[53:04] you know what to do about

[53:05] it um yeah in our case as I said the

[53:10] algorithm is correct when we detect the

[53:13] data R effectively we go down we

[53:15] die so I don't find it too problematic

[53:18] but still from a long rage perspective

[53:20] it's it's it's not you know we've got a

[53:22] data R here a solution is in this um

[53:25] proposal by atomic M Copy you can

[53:29] actually implement it already now with

[53:31] um the recent Atomic

[53:34] C but you're going to pay for this lency

[53:37] right it's it's not it's only it only

[53:40] supports um you know one B at a time so

[53:43] going to pay actually a lot of

[53:45] performance for

[53:46] this and uh you actually have the same

[53:49] problem with um SE loocks actually got a

[53:52] talk meeting CBP two years ago where I

[53:54] was using SE loocks and yes SE Tech

[53:56] loocks are also a real problem you

[53:58] actually cannot Implement them could be

[54:00] in C or

[54:02] C++ um in a very you know correct way

[54:06] from a language

[54:12] perspective performance measurements um

[54:15] we're using an AMD Z4 zen4 architecture

[54:20] it's tuned for low latency Co isolated

[54:23] its me its message is 73 bytes why 73

[54:26] bytes it's not too long not too short so

[54:29] I thought it was a nice

[54:31] number and we compare it against two

[54:34] libraries

[54:36] dtor because it's you know like the most

[54:38] famous ring buffer out there and then

[54:41] iron which is relatively well-known IPC

[54:44] Library used in U trading uh and other

[54:48] industry as

[54:50] well and Iron by the way is is is kind

[54:54] of baked into some more Java stuff

[54:57] around so that actually I didn't didn't

[54:59] use I just I just picked the two C++

[55:02] header file from the repo otherwise you

[55:04] need to instantiate some Javas to use

[55:06] some cues not sure that shouldn't

[55:09] influence influence anything

[55:12] now oh yeah and another queue we're

[55:14] looking at this queue that I just

[55:15] mentioned um which is actually very

[55:17] different design uh it doesn't have a

[55:19] header that it's using SE loocks I'm not

[55:23] going to talk about this one today you

[55:25] you know you can can just watch watch

[55:27] the to online it's a very different

[55:29] design and so our Baseline for

[55:32] this queue that we're looking at today

[55:34] is I would say it's good but it's not

[55:39] outstanding so we're going to look at a

[55:41] few things on how to make it

[55:46] fast and so the idea here is that very

[55:48] much you know we have we've got these

[55:49] two atomics and um in order to make this

[55:52] fast it's all about going to be the

[55:54] contention and the operation we are

[55:56] doing around this

[55:58] atomics and by far the biggest

[56:00] optimization you can do is

[56:03] to not touch the right counter on every

[56:07] message so what we were doing

[56:09] is we effectively on every single

[56:11] message we had this you know moving the

[56:13] right counter copying moving the right

[56:15] counter what you can do is you can just

[56:18] say well I'm going to reserve 100

[56:20] kilobytes I'm going to move the right

[56:22] counter the 100 kilobytes in my queue

[56:26] it will mean that like readers will have

[56:28] 100 kilobytes less of data available to

[56:30] read but it shouldn't be too much of a

[56:32] problem if you have 8 megabytes I mean

[56:33] you can choose another number the idea

[56:36] is that if you have 100 bytes message

[56:38] and you reserve 100 kilobytes it mean

[56:39] that you're only touching this Atomic

[56:41] one every thousand message which is

[56:47] huge and so that's going to be really

[56:49] big because the readers right the

[56:50] readers are reading this right G all the

[56:52] time

[56:57] a much more obvious uh optimization I

[57:00] still wanted to mention it for a reason

[57:01] that usually we we align we like to

[57:04] align things on a cach line

[57:05] size effectively there is no need to

[57:08] align things on the cach line size here

[57:09] like the on x86 effectively the best

[57:12] alignment you have for your data is

[57:13] eight

[57:14] bytes you actually do not want to align

[57:17] that on the on the cach line because it

[57:19] affects actually uh the locality of your

[57:22] element and uh and there is not really a

[57:24] reason to to align it on a cach

[57:27] line last optimization is um also quite

[57:32] straightforward when you read the r

[57:34] counter if from your previous from this

[57:37] previous read operation you know you you

[57:40] read that there is one megabyte of data

[57:42] to be read if you only read a kilobyte

[57:46] you don't need to to touch it you don't

[57:47] need to read it

[57:49] again these three things together

[57:51] effectively brings a really

[57:54] decent performance so we're in green

[57:58] here and

[57:59] so it's really

[58:02] good right so compared to Iron

[58:04] distributor compared to this to this c q

[58:07] so this Q is interesting it's actually

[58:09] faster in some case slower in some other

[58:11] case effectively the Cy Q is faster

[58:14] between I think four to 10

[58:18] consumers um but in general we' we've

[58:20] got something again quite simple and and

[58:24] fast here so back to our princip for and

[58:28] here I don't mean that like you know we

[58:29] should all get get out of um I mean I

[58:32] don't want people to give the impression

[58:33] to people that you know like what I'm

[58:35] saying that we should all now develop

[58:38] our own cues um because still

[58:40] concurrency is hard but what I mean here

[58:43] is that like when I was looking and

[58:45] doing a lot of benchmarks on looking at

[58:48] cues I still often get a feeling that

[58:50] like there is a lot of complexity that

[58:51] is not

[58:52] needed

[58:54] right not not trying to to to rent here

[58:57] or anything but why would I need 100,000

[58:59] lines of code with some some you know

[59:02] like really complicated system on

[59:03] framework while I actually just want AQ

[59:06] on sh memory that should be you know 100

[59:10] maybe 500 lines of cod Max effectively

[59:12] this one is

[59:16] 150 if you want to go further often what

[59:20] I see in

[59:22] apis is that it's it's easy to get it

[59:25] wrong so why this

[59:27] API wrong or why is it a bit

[59:29] disappointing here is that we need to we

[59:32] need to pass to the Q a buffer that we

[59:35] that we're going to write to it and

[59:38] what's a bit sad about this API is that

[59:41] it forces the application to serialize

[59:44] into this buffer and then we're going to

[59:46] copy this buffer into the queue well

[59:49] effectively what you want to do is you

[59:51] want to you want to be able to serialize

[59:53] directly into the cube this actually can

[59:56] give you a depending on your calization

[59:58] of course plenty of different calization

[01:00:00] libraries out there but depending on the

[01:00:03] way you cize this can give you easily a

[01:00:05] two two times speed up just with a

[01:00:08] simple API change on just opening

[01:00:11] opening your the the the way that you

[01:00:14] write to that queue and then you can go

[01:00:16] even further I'm not going to elaborate

[01:00:18] here too much are just IDs the

[01:00:20] effectivity things that um yeah we we

[01:00:23] we're doing and there is actually more

[01:00:25] bulk writing yeah bu cing is really is

[01:00:29] really big here because again you can

[01:00:30] avoid touching even more your counters

[01:00:33] and then you can have a queue that is

[01:00:35] more Numa whereare if you have like

[01:00:37] consumer and producer that are well

[01:00:40] sorry if if your consumers are

[01:00:41] effectively um on different

[01:00:44] panod could be different between the

[01:00:47] consumers um not sorry not between the

[01:00:49] consumers I mean different than actually

[01:00:51] the the num man of your

[01:00:53] producer there is also something

[01:00:55] something nice you can do there by

[01:00:57] duplicating for example the the header

[01:00:59] with your with these two

[01:01:04] atomics so now we looked at some data

[01:01:06] structure looked at some

[01:01:07] concurrency we use per you know we

[01:01:10] looked at Hardware

[01:01:12] counters something I also wanted to

[01:01:14] share

[01:01:15] today was specific measurements for low

[01:01:19] lency systems especially the one that I

[01:01:23] um EV driven

[01:01:27] and so here the idea is this is going to

[01:01:29] be me of course it's very simplified

[01:01:31] code it's just an idea this is this is

[01:01:33] how your trading system looks

[01:01:36] like there is a there is a y Loop so

[01:01:39] this is your main event

[01:01:41] Loop and you you know you pulled your

[01:01:43] network card you're waiting for some

[01:01:47] event and every packet goes into this

[01:01:50] easy interesting function you can

[01:01:51] imagine this is kind of your strategy

[01:01:53] and then you know if it's interesting

[01:01:54] you send an order now the tricky part

[01:01:57] when you measure performance is that you

[01:01:58] know with all the tools that I mentioned

[01:02:01] in this last hour is that they're going

[01:02:04] to have a very hard time to give you a

[01:02:06] good idea of what's going on in these

[01:02:09] functions for the reason that you know

[01:02:11] we looked at per stat per stat is a very

[01:02:15] Co you know measurement that are just

[01:02:17] counters being incremented so this

[01:02:19] functions SC are going to be you know in

[01:02:21] the middle of you know like pulling

[01:02:24] network cards and most of the time not

[01:02:26] doing

[01:02:27] much per uh record is a sampling

[01:02:31] profiler it's just going to sample I

[01:02:34] mean you can actually set the sampling

[01:02:36] frequency let's say a thousand times a

[01:02:38] second most of the time you're not going

[01:02:40] to get into these functions or if you

[01:02:41] get into them you're you know you're

[01:02:43] lucky but it's not going to be very

[01:02:44] accurate because you don't really

[01:02:46] capture the ENT function so what you got

[01:02:48] to do here and I don't think there

[01:02:50] actually plenty of different solution to

[01:02:52] this problem if you want to be as

[01:02:54] accurate as possible and very low

[01:02:56] overhead you do something like this so

[01:02:58] you do what we call you know intrusive

[01:02:59] profiling you modify your code you have

[01:03:02] this little object here you save some

[01:03:05] metadata and in the Constructor you

[01:03:08] start reading the TSC okay now this is

[01:03:10] again quite ex6

[01:03:13] specific start reading your TSC in a

[01:03:16] Constructor um in the destructor you

[01:03:19] read the TSC again and you have your

[01:03:21] time

[01:03:23] interval and and then you can imagine

[01:03:25] you know that you again you know right

[01:03:27] to a que that's why we looking at Q

[01:03:29] because they're a little bit

[01:03:31] everywhere now that's that's great but I

[01:03:34] don't think it's really it's really

[01:03:39] new and the main problem of this

[01:03:41] approach is

[01:03:43] that you you know you you're not going

[01:03:47] to like add this little object that we

[01:03:50] just looked at everywhere in your code

[01:03:52] because then you

[01:03:53] know it's not it's not great you know

[01:03:56] you have that just laying around um it's

[01:03:58] hard to maintain it also it has a cost I

[01:04:00] mentioned it it's low overhead it's low

[01:04:02] overhead if you just have this two TSC

[01:04:04] but if you start having this into every

[01:04:06] single of your function yeah at some

[01:04:07] point you're going to use half of your

[01:04:09] Co period time just you know in TSC

[01:04:11] instructions um so you don't want that

[01:04:13] and the challenge here um as an engineer

[01:04:16] is to is that you do not know where the

[01:04:19] next bottom neck is going to be and so

[01:04:21] at some point you have a bottom neck you

[01:04:22] need to kind of go back into your code

[01:04:24] add this you know add this kind of

[01:04:26] instructions again recompile your binary

[01:04:29] ship it to production or run some

[01:04:31] simulation and so on so forth and so

[01:04:33] this is actually not a great it's not a

[01:04:35] great

[01:04:36] workflow and so the nice thing here is

[01:04:39] actually to use x-ray from clang and so

[01:04:43] what what's clang

[01:04:44] x-ray is you know you're going to

[01:04:47] compile your binary with a special

[01:04:50] flag I think it's called x-ray

[01:04:51] instrument and so this is like an

[01:04:53] instrumentation Library provided by

[01:04:56] clang and so that's

[01:04:57] awesome how it works in under the wood

[01:05:00] is that it's going to add knobs at the

[01:05:04] beginning and at the end of your

[01:05:05] function and so by default it's not

[01:05:08] doing anything you just execute a few

[01:05:11] knobs at the beginning of your function

[01:05:13] at the end so it's very low

[01:05:16] overhead but when you want to actually

[01:05:19] profile you can patch your

[01:05:22] binary and you can replace this knobs

[01:05:25] with some fun

[01:05:27] calls so you can you can have a really

[01:05:30] nice profiler that is as accurate as you

[01:05:35] can low overhead and at the same time

[01:05:38] you know you avoid recompiling your

[01:05:40] binary and going through that workflow

[01:05:42] that I

[01:05:45] describe and so that that's awesome

[01:05:47] because it really kind of like like

[01:05:50] gives you like The Best of Both Worlds

[01:05:52] where you can really have like a yeah

[01:05:54] nice workflow as an engineer measure

[01:05:56] things could be in production or not

[01:05:59] without having to constantly recompile

[01:06:00] your

[01:06:04] code and then on a on a on a on a

[01:06:08] similar topic is that you know once you

[01:06:11] fixed you know all your

[01:06:14] latency bottlenecks Etc you've got a

[01:06:16] graph like this I'm actually not saying

[01:06:18] you know what kind of latency dis is it

[01:06:20] actually doesn't matter too

[01:06:22] much but the idea here is once you

[01:06:25] actually have you know a latency that

[01:06:27] you're happy

[01:06:28] with you still have a lot of work to do

[01:06:31] usually that work is a little bit less

[01:06:32] fun but it's really

[01:06:34] important you know you got to you got to

[01:06:37] measure a lot in your system

[01:06:40] and often say that like it's really nice

[01:06:43] to actually send to a database millions

[01:06:45] of numbers per second so all your

[01:06:47] measurements it's that's that's nice but

[01:06:49] what's really important what actually

[01:06:51] really matters is to have alerts is to

[01:06:54] have actually the audits on this numbers

[01:06:56] right we can always pump a lot of

[01:06:57] numbers to database and look at them

[01:06:58] with nice graph if you don't have actual

[01:07:01] audits on

[01:07:03] expectation like things checking this

[01:07:06] numbers it is not going to be very

[01:07:08] useful because it's always the same when

[01:07:10] you roll out something new there all the

[01:07:12] eyes on it and then a year later no one

[01:07:14] is really looking at it so you you got

[01:07:16] to have this this checks here and so

[01:07:18] that bring us to our principle 8 it's

[01:07:21] nice to be fast you know what's really

[01:07:24] hard here like what takes a lot of

[01:07:26] engineering time is actually to to stay

[01:07:35] fast we're slowly approaching the end of

[01:07:38] this talk and

[01:07:39] so as an outro I have you know

[01:07:43] one last ID that I am that I find really

[01:07:47] important something that

[01:07:50] I that kind of like kept coming back you

[01:07:53] know throughout my careers when I would

[01:07:55] look at latency of this

[01:07:58] system and I call it you're not alone so

[01:08:01] in that picture you see an engineer lot

[01:08:04] of screens lots of code it's really easy

[01:08:07] to forget you know that yeah you're

[01:08:09] you're just not here on your own

[01:08:11] developing code so let me let me explain

[01:08:14] a little bit what I mean with

[01:08:16] um some last snippet of

[01:08:19] code so this is a relatively simple code

[01:08:23] so what are we doing here we are

[01:08:25] generating a vector of Shuffle

[01:08:29] indices

[01:08:31] and to we generate a vector we actually

[01:08:34] want to run this Benchmark for different

[01:08:36] size so we care about the working set

[01:08:39] size and then once we once we've got

[01:08:41] that Vector we are going to Summit in

[01:08:44] what I would call the worst possible way

[01:08:47] you know which is you follow all this

[01:08:49] you know you submit following actually

[01:08:52] the order of this indices this is kind

[01:08:54] of going to simulate

[01:08:57] a you know an almost perfect you know

[01:08:59] random walk into memory and then you

[01:09:02] measure the

[01:09:06] throughput and so the result that you're

[01:09:08] going to get is something like

[01:09:10] this so let me explain because there are

[01:09:13] a few things few things going on on that

[01:09:15] graph the Blue Line called single

[01:09:20] worker is going to be a single instance

[01:09:22] of this application on a single CPU

[01:09:25] nothing nothing else running on that

[01:09:28] server and so what you see when you

[01:09:30] measure you know the

[01:09:32] throughput that you clearly see our

[01:09:34] three level of cash you see we we start

[01:09:37] we already you know high throughput and

[01:09:40] then we pass our first level first level

[01:09:43] of cash 30 32 kilobytes on that Ser drop

[01:09:46] L2 and then we reach L3 drop again and

[01:09:49] then we in

[01:09:51] Ram and then you've got six workers

[01:09:55] the six workers each of them are their

[01:09:57] own

[01:09:58] CPU and so you instantiate six you know

[01:10:01] instance of this

[01:10:02] application and you measure the

[01:10:04] throughput again and you see that it's

[01:10:06] almost the same as the single worker

[01:10:10] except around our L3 cache effectively

[01:10:13] for the integrality of the L3 cache

[01:10:16] right so on This Server it's between uh

[01:10:19] you know it's it's around where we start

[01:10:22] there like a little bit around um 6 six

[01:10:25] me megab 8

[01:10:27] megabytes and then up until like

[01:10:31] 664 on this area of the L3

[01:10:34] cach the performance this is the scaling

[01:10:37] Factor if we take the sum of the six

[01:10:41] workers and we divide it by the single

[01:10:43] worker we've got a scaling Factor right

[01:10:45] how much speed up do we get from having

[01:10:47] six CPUs against

[01:10:49] one and we're not sharing any data right

[01:10:52] in this card what we see here is we

[01:10:54] almost go to

[01:10:57] one and we almost go to one for

[01:10:59] integrality of is stre

[01:11:03] cach and so why why is this really you

[01:11:06] know why does this matter why you know

[01:11:08] what the point that I'm trying to make

[01:11:10] here is that effectively for the vast

[01:11:13] majority of the system that I've been

[01:11:15] working on and I think most trading

[01:11:18] system aren't going they're not going to

[01:11:21] fit into your L1

[01:11:22] cach if you're trading very few

[01:11:26] instruments and depending on your

[01:11:28] strategy you might be able to fit some

[01:11:30] things into the L2 but most of the time

[01:11:32] from experience you in this you're

[01:11:35] effectiv in this a in your L3 I

[01:11:40] mean and the point here is that you need

[01:11:44] to think about the system as a all

[01:11:45] that's what I mean with like you know

[01:11:46] you're not

[01:11:48] alone you really need to like not just

[01:11:51] think about your application or you know

[01:11:54] an application A and B Etc you really

[01:11:57] need to look at the entire

[01:12:00] server and you know if this entire

[01:12:02] server with all the applications that

[01:12:03] are on it make sense otherwise there

[01:12:06] will be no way to actually have

[01:12:08] something performant and low latency so

[01:12:10] the point is that you know today we

[01:12:12] looked at a lot of examples optimization

[01:12:16] data structure Etc you can do all that

[01:12:19] and still your system will be relatively

[01:12:22] slow if not everything on that same

[01:12:25] server if not all your colleagues did

[01:12:28] the same and so this is our last

[01:12:31] principle here which is I'd say you know

[01:12:33] less lesson of empathy which is you

[01:12:37] shouldn't just care about the

[01:12:38] performance of your Cod yeah it all

[01:12:41] depends also on the performance of the

[01:12:43] code of the CPU things that are running

[01:12:46] on the same same

[01:12:51] servers final

[01:12:54] thoughts so we started you know looking

[01:12:57] at um I was saying at the beginning of

[01:12:59] this talk Market making losers game you

[01:13:02] you need to be consistently good at at

[01:13:05] everything there is no silver

[01:13:07] ballet well it's pretty much the same

[01:13:10] when it comes to low lency programming

[01:13:13] there is no Silver Bullet we would all

[01:13:14] love that there is one but there is none

[01:13:17] so you need to be

[01:13:19] disciplined keep things simple because

[01:13:22] as we thought simple things are fast

[01:13:26] they're not just fast they're also

[01:13:27] simple to understand well for you for

[01:13:31] your colleagues if they're simpler to

[01:13:33] understand you can actually build a

[01:13:35] simpler system that which you know on

[01:13:38] its own is going to be again faster

[01:13:41] that's our last point that we looked

[01:13:43] at and um I probably today you know

[01:13:47] mentioned many many times the word

[01:13:50] latency there is actually one latency

[01:13:52] that I didn't mention which I also

[01:13:53] wanted to say that it matters that it's

[01:13:56] important which is like time to Market

[01:14:00] effectively like if if I actually also

[01:14:02] today put an emphasis on like Simplicity

[01:14:05] it's also because it it does matter

[01:14:07] really you know how how fast actually

[01:14:09] can you can you ship your code to

[01:14:11] production so that's a latency as

[01:14:15] well now going through some credits just

[01:14:17] want to say thanks to some C some people

[01:14:20] that you know help me build that talk

[01:14:22] some people that inspired me some people

[01:14:24] that I can bounce IDE of so thanks to

[01:14:26] them and then I always like to give

[01:14:29] references you know to go further there

[01:14:32] are plenty of plenty of interesting

[01:14:34] things out there and this is just pretty

[01:14:36] much one slide of what you know things

[01:14:39] that I found interesting there are you

[01:14:42] know some talks there that I really

[01:14:44] appreciate it from fedo from Mike Acton

[01:14:47] some of them are actually quite quite

[01:14:48] old but still extremely relevant for

[01:14:52] today so that conclude our talks and we

[01:14:55] still have yeah solid 10 minutes for

[01:14:58] questions yeah thank

[01:15:00] [Applause]

[01:15:18] you thank you for the talk um so about

[01:15:22] the sequential search and binary search

[01:15:25] that you had yeah um did you try like

[01:15:28] partitioning the thing and doing

[01:15:30] sequential search for the first few and

[01:15:33] then doing binary search for the next

[01:15:36] say say that again so on the the on the

[01:15:38] binary search did I try to batch things

[01:15:40] yeah like first few you could do like

[01:15:43] linear search yeah and then if you can't

[01:15:46] find then you can switch to Binary yeah

[01:15:49] I actually did try that yeah in general

[01:15:52] like I I did try a lot of things that

[01:15:55] I did try some meta parameters you know

[01:15:57] indeed going between I think actually

[01:16:00] Andre presented that in in in one of his

[01:16:03] St so like you know kind of meta

[01:16:04] parameter between I think it was about

[01:16:06] sorting more than searching I tried that

[01:16:09] uh

[01:16:10] effectively and it might just be like

[01:16:12] you know a property of like the things

[01:16:14] that we looking at yeah exactly like I

[01:16:17] think the idea is just that the

[01:16:18] collections are not big

[01:16:20] enough okay so in general linear search

[01:16:23] it's good enough is is is going to just

[01:16:26] be the the best there no matter what I

[01:16:28] mean maybe there is an instrument or

[01:16:30] something out there with like I don't

[01:16:32] know you know 100,000 levels and where

[01:16:36] indeed linear search would then be

[01:16:39] slower and couldn't find it okay thank

[01:16:42] you thank

[01:16:45] you hi David uh thanks for the thanks

[01:16:49] for the talk and for building the toy

[01:16:51] example of the all the books for the

[01:16:54] talk so pretty much fallowing on the

[01:16:56] previous question um well naturally like

[01:17:00] the book updates are heavily skewed

[01:17:02] towards the top of the book right so did

[01:17:04] you out of curiosity try two ideas so

[01:17:07] one idea is to split the prices with the

[01:17:12] rest of the data to compress your vector

[01:17:14] yeah and the second one if simd

[01:17:17] instructions for the like first few

[01:17:19] levels your second second point was AVX

[01:17:22] or AVX yeah they

[01:17:25] x552 to slow your frequency but so I did

[01:17:30] I did try that effectively effectively

[01:17:32] they go Hand by they go they go together

[01:17:35] because in the in the code example

[01:17:37] actually like the we we have pairs of

[01:17:40] elements you have pairs of price and

[01:17:42] volume and effectively the

[01:17:44] compiler um I mean maybe one day it will

[01:17:47] be able to do it but at least in the

[01:17:49] state of things the compiler is not

[01:17:51] going to generate any AVX instruction

[01:17:53] with that and so you your first question

[01:17:55] was you know did I try to split vectors

[01:17:57] yes I did it have a slight positive

[01:18:00] effect but the nice effect about

[01:18:01] splitting the vectors into two and just

[01:18:04] having your price and your volume is

[01:18:05] that once you do your linear search I

[01:18:08] mean very much defin if on your vector

[01:18:11] now actually you get the AVX generation

[01:18:14] for free from the

[01:18:16] compiler because because you have all

[01:18:19] your prices you just have a vector of un

[01:18:22] 64 and then the compiler is you're going

[01:18:25] to generate

[01:18:26] AVX the interesting things is so you

[01:18:28] actually do get yeah the answer is yes

[01:18:30] you have like a slight performance gain

[01:18:33] of it um I'm not sure if it's true for

[01:18:36] you know like for every single case

[01:18:39] that's also why I actually didn't like

[01:18:41] mention it but but you do get some some

[01:18:44] performance out of it because yes indeed

[01:18:46] it's more packed and AVX effectively

[01:18:49] results are a little bit mixed which is

[01:18:52] if you have um if you have effectively

[01:18:54] just a few level something quite

[01:18:56] thin uh the cost the initial the initial

[01:18:59] cost of latency of your AV instruction

[01:19:01] are going to make it

[01:19:03] slower so yeah it's it's a little bit

[01:19:07] mixed I see thank you but thank

[01:19:10] you uh hey hi uh that was a great talk

[01:19:14] uh the fast queue which you explained

[01:19:16] was a inmemory queue so have you ever uh

[01:19:18] encountered a case where you want to

[01:19:20] store the state of the events which are

[01:19:22] going through the queue the state of the

[01:19:25] events that are going through the queue

[01:19:27] yeah let's say if my process crashes and

[01:19:30] I want to build the state of the

[01:19:33] messages which were passing through that

[01:19:35] queue yeah so yes how how does it impact

[01:19:39] your like first thing how do you do it

[01:19:41] and how does it impact the latency you

[01:19:45] you you should look at um for example

[01:19:47] postgress

[01:19:48] internals um might be familiar with like

[01:19:52] how database work they use like a SoCal

[01:19:55] worldall so

[01:19:57] w you can look at this data structure

[01:19:59] and this is what you want so you

[01:20:01] basically store it in the database

[01:20:03] that's what you're saying no sorry no no

[01:20:05] I didn't mean to use the database I

[01:20:06] meant to look at the data structure you

[01:20:07] use in in a database like pogress there

[01:20:10] is this thing called wall

[01:20:13] W which is what you want so I don't mean

[01:20:15] to use the database but just this data

[01:20:17] structure can actually do what you want

[01:20:19] here which is in case there is actually

[01:20:21] a writer or you know that crash you

[01:20:24] actually assist on disk and you avoid uh

[01:20:28] any loss of data I mean the que that I

[01:20:30] presented here clearly doesn't do it but

[01:20:33] it is an element a building block of

[01:20:37] such a such structure so essentially if

[01:20:40] my process uh multiple processes are on

[01:20:42] the same machine and I Implement right

[01:20:44] ahead lock which you talked about right

[01:20:46] now along with the fastq implementation

[01:20:49] that might give me a better performance

[01:20:51] than rabbit mq which

[01:20:53] is across the different

[01:20:58] networks sorry can you say that again

[01:21:00] actually I'm not sure if I you mentioned

[01:21:02] a few different things here multiple

[01:21:03] producers yeah so what what I'm saying

[01:21:05] is uh if we are able to implement the uh

[01:21:09] wall the right ahead lock along with the

[01:21:11] fastq implementation which you explained

[01:21:14] right I think fast fastq implementation

[01:21:16] doesn't have the wall implemented in it

[01:21:18] right no no absolutely not okay so that

[01:21:22] can can can it be a better solution that

[01:21:24] using rabbit mq like Technologies right

[01:21:27] I I do not know that this is too

[01:21:29] specific I know rabbit mq never used it

[01:21:33] um my good feeling is yes but I don't

[01:21:36] want to say yes too quickly so I mean

[01:21:38] you got to just look at it cool measure

[01:21:42] thanks thanks for yeah for sure thank

[01:21:47] you hi uh thanks for talk I think uh I

[01:21:51] have a question about the security so uh

[01:21:54] in your implementation you mention we

[01:21:56] can use a share libr sorry share memory

[01:21:59] and we can uh also avoid to copy data to

[01:22:03] improve the performance yeah uh but in

[01:22:06] previous talk uh I heard something like

[01:22:09] to tou T to check T to use that is

[01:22:12] attacker may use if we use share share

[01:22:16] memory attacker May uh change the input

[01:22:19] data after we ready that it so how do we

[01:22:24] think if we use share memory uh how we

[01:22:27] can prevent this kind of

[01:22:30] attack what so what kind of attack

[01:22:32] exactly is this uh the name is a to tou

[01:22:36] time to check time to

[01:22:38] use right yeah and can you be like so

[01:22:41] what uh basically it's a kind of tack

[01:22:45] like uh after our program validate the

[01:22:48] input is uh Leger and uh after that

[01:22:54] because Al we use sham memory yeah so

[01:22:57] attacker can change our input you're

[01:23:00] talking about very much like an attacker

[01:23:01] like you mean like you're talking about

[01:23:03] like vulnerabilities security Etc right

[01:23:06] you mean like someone that would exploit

[01:23:08] yeah wow that's that's great question

[01:23:11] actually I actually have to pass on this

[01:23:13] one I I don't have um yeah I don't have

[01:23:17] much opinion there mostly because I mean

[01:23:19] I guess I do but I'm definitely not an

[01:23:21] expert there um yeah I really can't say

[01:23:25] much the the thing with like the area of

[01:23:28] like you know I mean trading system is

[01:23:29] that our processes are always on servers

[01:23:32] that we own now you know part of our

[01:23:34] servers our a so effectively yeah here

[01:23:38] this is actually not I mean definitely

[01:23:40] security is a thing right on a company

[01:23:42] level but when it comes to like such

[01:23:45] data structure on the share memory I'm

[01:23:46] actually not too worried about attackers

[01:23:50] you know like looking at my arm that's

[01:23:53] yeah I got a pass on this one

[01:23:57] okay uh hello uh I'm also from another

[01:24:00] trading phone so like it's very

[01:24:02] interesting talk about like low lat in

[01:24:05] JD system so I I I want to ask one

[01:24:09] question regarding like the use of

[01:24:13] socalled uh stru of arrays like you

[01:24:16] demonstrated like this linear search I

[01:24:18] believe like uh Str like using stru of a

[01:24:21] race should uh reduce the number of like

[01:24:25] catch miss things like that I'm

[01:24:27] wondering like does uh optiva use a

[01:24:30] struct of arrays like technology

[01:24:33] internally and if uh you do do you have

[01:24:36] some kind of like framewor to make it

[01:24:40] easier yeah yeah absolutely um

[01:24:42] effectively that's why I put as a

[01:24:44] reference Mike Acton here so that this

[01:24:46] talk is about like you know what you

[01:24:48] just mentioned and effectively the

[01:24:49] previous question was also a little bit

[01:24:51] about that so we're definitely using it

[01:24:53] now

[01:24:55] we use it where it's needed so what what

[01:24:58] I mean is that it's not like something

[01:25:00] that like we follow on everything in the

[01:25:01] structure for right but I very much yes

[01:25:04] in some cases it does help and uh and so

[01:25:08] as I said actually in the previous for

[01:25:10] the for the very specific case of this

[01:25:12] Vector with like this pairs yeah you can

[01:25:13] split it and you do get actually like

[01:25:16] some performance gain out of it uh so

[01:25:19] you mean like uh it's mostly kind of

[01:25:22] manual like only if it's Leed you

[01:25:25] do some manual like refactoring to make

[01:25:28] but there no fun Work N no we we usually

[01:25:32] quite pragmatic about these things and

[01:25:33] like we like yeah we definitely use

[01:25:36] something when it's strictly needed and

[01:25:41] um and maybe it's a bit of a you know

[01:25:43] like at least I tend to I probably say

[01:25:46] that world too many times today

[01:25:48] like I have strong opinions about like

[01:25:51] Simplicity I like to keep things very

[01:25:52] simple I'm also like very like we about

[01:25:54] pulling dependency so Frameworks to do

[01:25:56] something like struct array yeah stru

[01:25:59] array is a simple thing um so you know I

[01:26:02] can do it I can just do it to it on my

[01:26:04] own reason about it I'm like sometimes

[01:26:07] worried about putting an inter framework

[01:26:08] to do this um sometimes the substraction

[01:26:12] layer I'm not saying they're not needed

[01:26:14] sometime they're great but sometime it's

[01:26:16] also like uh except if it's literally

[01:26:19] everywhere in your code base but that's

[01:26:20] actually also not not our case thanks

[01:26:24] yeah yeah thank

[01:26:25] you hello uh thank you for the talk I

[01:26:29] was um I was going to ask about the the

[01:26:32] Linux message cues we have a real-time

[01:26:35] application probably not quite as

[01:26:37] demanding as yours there's not as much

[01:26:38] money on the line uh but we are we're

[01:26:42] using we have multi multiple processes

[01:26:45] that need to communicate and we're using

[01:26:46] the Linux message cues we're using Sue

[01:26:50] Linux similar to The Red Hat Linux that

[01:26:52] you're using and I wonder if you could

[01:26:55] give me a feel for how the the

[01:26:57] underlying uh Q message CU

[01:27:00] implementation in in Linux kernel

[01:27:02] differs from the custom message cue that

[01:27:05] you're using or did you look at that

[01:27:08] it's great question actually actually

[01:27:10] didn't really look at like this the

[01:27:12] implementation of what like of the cues

[01:27:15] that you're referring which which one

[01:27:16] which one are you referring I believe

[01:27:17] that they are multiples do you have a

[01:27:20] specific name um I I'm not that familiar

[01:27:23] with the underlying Tech technology CU

[01:27:24] we have a rapper that we call system

[01:27:26] Services I believe it's calling down to

[01:27:28] the Linux uh messaging function I

[01:27:31] believe there's a level function there

[01:27:33] in general like I mean the Linux colel I

[01:27:35] mean like it's I I would I would think

[01:27:38] that like the Q implementation right I

[01:27:40] mean that they use there is not really

[01:27:42] the problem I mean it's probably going

[01:27:43] to be very fast I I can totally trust

[01:27:46] that the main problem with with with you

[01:27:49] know going to the kernel are going to be

[01:27:50] cises and so the viant that you get from

[01:27:53] CIS so that's a little bit the same as

[01:27:55] like Network you know user spacing or

[01:27:57] not which is like as soon as you go into

[01:27:59] the kernel the variance and Jeter that

[01:28:02] you have from like going into the from

[01:28:04] newand to canaland that is actually the

[01:28:06] main thing what we are that we want to

[01:28:08] prevent here but I I I really you know I

[01:28:10] can trust the external developers that

[01:28:12] probably have very fast data structure

[01:28:13] in that thanks that's a great answer I

[01:28:16] appreciate it thank you well I think we

[01:28:18] a lot of time I would still you know

[01:28:20] stick around for for more questions but

[01:28:23] yeah thank you