FFmpeg: The Incredible Technology Behind Video on the Internet | Lex Fridman Podcast #496

Full Transcript

https://www.youtube.com/watch?v=nepKKz-MzFM

[00:00] The important is, is your code good?
[00:03] The important is, is your code good?
[00:03] We care about excellent code.
[00:05] We don't care about excellent code.
[00:05] We don't care who you are.
[00:07] Like maybe you're a dog.
[00:07] I don't care, right?
[00:09] I don't care, right?
[00:09] I don't care where you come from.
[00:11] I need to look at your code.
[00:13] Oh yeah, but I'm engineer at this very large company in in Italy, in Germany, in the US.
[00:16] We don't care.
[00:18] We don't care.
[00:18] We care about the quality of your code
[00:20] because this is what defines our community.
[00:22] And which means that we have a lot of people who contribute who are from very different backgrounds and and and very introverts.
[00:26] Sure.
[00:28] But that's okay, right?
[00:30] >> FFmpeg is probably one of the biggest CPU users in the world.
[00:33] Everything we've just said in the past couple of minutes,
[00:35] every sentence is someone's lifetimes work.
[00:38] There are books about every sentence.
[00:40] So the level of complexity >> in many cases is inordinate.
[00:41] FFmpeg has 100,000 lines of assembly for all the codecs and just this one has 240,000.
[00:44] Every cycle matters.
[00:46] We are talking about probably 3 billion devices which are going to decode video non-stop
[01:01] are going to decode video non-stop because for example 30% of the video
[01:04] because for example 30% of the video from Netflix are now in everyone 50% of
[01:07] from Netflix are now in everyone 50% of YouTube.
[01:09] YouTube. This is what peak video codec should look like.
[01:14] should look like. 79.9% assembly 19.6%
[01:18] 6% C and uh 0.5% other.
[01:20] other. And what's incredible is with those tweets, which is factual, people
[01:23] those tweets, which is factual, people get
[01:25] get crazy.
[01:26] crazy. For the last two years, they go crazy.
[01:28] crazy. No, intrinsics is fine. The compiler,
[01:28] compiler, >> you can optimize your compiler autoacctorization.
[01:30] >> you can optimize your compiler autoacctorization. It's your fault. You don't understand.
[01:32] don't understand. And we've tried that forever, right?
[01:34] forever, right? For 2 years and two years later, showing hundreds of examples of of handwritten assembly.
[01:36] years later, showing hundreds of examples of of handwritten assembly. No,
[01:39] examples of of handwritten assembly. No, no, no. You're doing it wrong.
[01:41] No, no, no. You're doing it wrong. The compiler can do this. intelligence agencies tried to like say, "Can you put
[01:42] compiler can do this. intelligence agencies tried to like say, "Can you put a back door in VC?"
[01:44] agencies tried to like say, "Can you put a back door in VC?" >> Yes. Two of them.
[01:47] a back door in VC?" >> Yes. Two of them. >> What What do you say?
[01:48] >> Yes. Two of them. >> What What do you say? >> No. I was a lot less polite.
[01:50] What What do you say? >> No. I was a lot less polite. >> Basically saying, "Hell no."
[01:51] No. I was a lot less polite. >> Basically saying, "Hell no." >> Like if we had to compromise our software, we would shut it down.
[01:54] Basically saying, "Hell no." >> Like if we had to compromise our software, we would shut it down. This is clear.
[01:56] Like if we had to compromise our software, we would shut it down. This is clear. >> Any tweets uh Karen, you regret?
[01:57] software, we would shut it down. This is clear. >> Any tweets uh Karen, you regret?
[01:59] This is clear. >> Any tweets uh Karen, you regret?
[02:00] clear. >> Any tweets uh Karen, you regret?
[02:03] Any tweets uh Karen, you regret?
[02:03] No tweets of regret.
[02:05] No tweets of regret.
[02:07] Or is it like that?
[02:07] Well, how's the French song go?
[02:09] Regret nothing.
[02:09] Don't regret anything.
[02:12] No, it's because regrets are attacks on your mind.
[02:17] The following is a conversation all about FFmpeg and VLC with Jean Baptiste Kemp and Karen Kuna.
[02:27] FFmpeg is an open-source software system that is the invisible backbone behind YouTube, Netflix, Chrome, VC, Discord, and basically every platform that touches video or audio on the internet.
[02:42] It can decode, encode, transcode, stream, and play almost any video or audio format ever created.
[02:51] To me, it is one of the most incredible software systems ever developed.
[02:57] And it's all done by volunteers.
[02:59] VC is also a legendary piece of software.
[03:02] It is an open-source media
[03:04] software.
[03:06] It is an open-source media player that plays basically anything you player that plays basically anything you throw at it, any format, any platform.
[03:09] No ads, no tracking.
[03:12] It has been downloaded over 6 billion times.
[03:15] And again, for me, it has been one of my favorite pieces of software ever with
[03:20] the most legendary logo, which I of course had to honor in this conversation
[03:24] by wearing the VLC traffic cone hat the whole time.
[03:30] So again, above all else, thank you to the incredible volunteer engineers who put their heart and soul
[03:37] into this code that has been used and loved by billions of people. Thank you.
[03:43] and about the two great engineers and human beings I'm uh talking to in this episode.
[03:48] John Baptiste is the president of Video Land and is a key figure behind VLC and FFmpeg.
[03:55] Kieran is a longtime Kodak engineer, FFmpeg contributor, and the man behind the now infamous FFmpeg
[04:06] the man behind the now infamous FFmpeg account on Twitter X that I recommend everybody follow for the memes and for the unapologetic celebration of open source and great lowlevel software engineering.
[04:18] Let me also say that it's inspiring and humbling that so much of modern civilization rests on software built by people who are not chasing fame or money, but are obsessed with the craft of engineering.
[04:32] We live in a world where billions of people consume video every day without ever thinking about the invisible machinery underneath it.
[04:42] But that machinery matters.
[04:44] Open- source infrastructure matters.
[04:47] It is one of the great examples of human beings quietly collaborating across borders to build something useful, durable, and elegant for the rest of us.
[04:58] And so this conversation is not just about codecs and media pipelines.
[05:03] It is also about the deeper spirit of engineering and
[05:06] the deeper spirit of engineering and generosity that makes projects like FFmpeg possible.
[05:13] Again, I can never say it enough.
[05:13] Thank you.
[05:16] This is the Lex Freedman podcast.
[05:18] To support it, please check out our sponsors in the description where you can also find links to contact me, ask questions, give feedback, and so on.
[05:28] And now, dear friends, here's Jean Baptiste Kemp and Karen Kuna.
[05:35] So, the legend goes VLC can open everything.
[05:38] What's the weirdest thing that you know that it can open?
[05:43] You know there is a ton of people who are using VC to record VHS videos, right?
[05:48] Like it's just like you plug it with a capture card and you can basically record VHS video.
[05:52] Well, how does that work?
[05:55] Basically, it's you know those type of capture card where you can put a perel in or or RGA and you put that and actually VC can play those type of cards and there is a module which allows to control directly some those VCR
[06:06] control directly some those VCR camcorders.
[06:09] We support DVD audios camcorders.
[06:09] We support DVD audios lately, right?
[06:11] We spend the the summer working on DVD audio support and like there is no no one's making any DV audio support.
[06:18] There is a custom encryption schemes.
[06:18] What about Lucasfilm?
[06:20] Oh yeah.
[06:20] And there is of course all the weird codecs support game codec supported by by FFmpeg.
[06:25] The one Star Wars video game the first 10-second opening sequence.
[06:27] Someone has gone and implemented that and make sure that's bit exact on one disc that existed at one time of one little sequence in the game.
[06:35] And then finally there was a at one video land conference we made a competition to make make the weirdest and most horrible file ever and see if VC could play it.
[06:44] What did it end up being?
[06:46] What's the file?
[06:48] It was an MKV file made by Derek which each of the frame was changing resolution, aspect ratio, rotation and it was like did it work?
[06:58] Yes.
[06:58] And there was another one where the whole video was actually animated subtitles, right?
[07:07] was actually animated subtitles, right?
[07:07] SSA, right?
[07:10] So each this one was so each SSA, right?
[07:10] So each this one was so each each frame was a black frame, but on top of that there was a a subtitle that was animated for each frame.
[07:16] animated for each frame.
[07:16] There was a file that's a valid zip and a valid MP3 at the same time or something like that.
[07:20] So yeah, we made a competition of stupid files and it worked.
[07:23] It opened all of the stupid files.
[07:27] Yes.
[07:27] By the way, uh for people who are not familiar, I am wearing a hat.
[07:31] Would it be fair to say this is the best worst logo of all time, the cone?
[07:36] Yeah, by far.
[07:36] Right.
[07:36] The logo of VC is so iconic, right?
[07:39] Like we are a team with a small number of people and and the icon is known everywhere.
[07:43] I go to middle of nowhere in India or in China.
[07:46] People know the cone, right?
[07:48] And and 25% of the website traffic that comes to uh our main website is cone player, right?
[07:56] So, so many people don't know VC, right?
[07:58] They know the cone player.
[07:59] That's the thing that Google for is cone player.
[08:01] Yeah.
[08:01] They go on Google and they they put cone player and they download VC, right?
[08:05] So, that's iconic.
[08:05] And once we
[08:07] right? So, that's iconic.
[08:07] And once we tried to change it as a joke, right?
[08:09] We tried to change it as a joke, right?
[08:12] We said it was going to be a type of um uh Caterpillar construction and we said
[08:16] Caterpillar construction and we said that during April 1st and we had around
[08:18] 10,000 emails saying no, don't change
[08:20] the logo and so on. Right?
[08:23] So, it's so iconic, right?
[08:23] It's so distinctive, right?
[08:24] If you want to do a video player, you're going to put a play button on the
[08:26] TV, right?
[08:28] And that's a YouTube YouTube logo, right?
[08:30] It's an original. This one is orange, right?
[08:32] Very bright. Uh, and and it's weird
[08:35] and it's ridiculous and it's absurd and
[08:37] it's hilarious. It becomes meme and meme
[08:38] becomes culture
[08:40] and you keep it and you know about it
[08:41] and you know that in 20 years like you still have going to have the cones and
[08:43] remember, oh yeah, that was a video player.
[08:45] Yeah. And we'll talk about, you know, the the mission of FFmpeg being a kind
[08:47] of the archival aspect of it. So you can
[08:49] think about a thousand years from now, we'll have all these videos that only
[08:49] VLC can open. Humans, human civilization
[08:51] has already destroyed itself multiple times and the only thing that will
[08:53] remain is this like, you know, the
[09:08] remain is this like, you know, the cockroaches will be crawling around.
[09:10] cockroaches will be crawling around.
[09:10] It'll be the VC logo with our some of the archival footage that VLC can open
[09:14] the archival footage that VLC can open and the aliens will show up and they'll
[09:16] and the aliens will show up and they'll press play and they'll get to see.
[09:18] press play and they'll get to see.
[09:18] really hope so, right?
[09:20] really hope so, right? But there is also so many memes where people say, "Well,
[09:22] so many memes where people say, "Well, I'm sure I can put a pancake inside my
[09:23] I'm sure I can put a pancake inside my DVD drive and VC will play it."
[09:25] DVD drive and VC will play it." Can they?
[09:26] they? >> No, we tried. It doesn't.
[09:28] >> No, we tried. It doesn't. But we actually have a video of us trying that.
[09:30] actually have a video of us trying that. Uh, didn't work.
[09:31] Uh, didn't work. >> A codec for physical reality.
[09:33] >> A codec for physical reality. I don't know what that would even look like.
[09:34] know what that would even look like. >> There was a guy who did that, right?
[09:35] >> There was a guy who did that, right? He he he printed a small cone, right?
[09:38] he he printed a small cone, right? Like the the ones we we distribute as
[09:39] the the ones we we distribute as goodies. And inside he put an RFID chip
[09:43] goodies. And inside he put an RFID chip um which was his way of playing a movie,
[09:46] um which was his way of playing a movie, right? And so he put this on FID player
[09:49] right? And so he put this on FID player and when he put that it was playing like
[09:50] and when he put that it was playing like the last Star Wars and so on.
[09:52] the last Star Wars and so on. So instead of having like DVD boxes he had like VC
[09:55] of having like DVD boxes he had like VC cones all around and he plugged that and
[09:57] cones all around and he plugged that and there was like physical objects.
[10:01] there was like physical objects. So the thing that we're talking about is
[10:02] thing that we're talking about is everything around video codecs, video
[10:04] everything around video codecs, video encoding, video decoding, video
[10:06] encoding, video decoding, video streaming, video player client that I'm
[10:08] streaming, video player client that I'm wearing on my head, the entire ecosystem.
[10:11] wearing on my head, the entire ecosystem enabling free media.
[10:13] We'll talk about FFmpeg.
[10:17] talk about video land VC and all the other incredible video technology uh that is used probably by billions of people.
[10:23] So JB, you're the lead developer behind the legendary VLC player.
[10:27] Kieran, amongst many other things, you're lead developer behind the legendary FFmpeg handle on Twitter.
[10:34] And both of you have spicy opinions, I would say.
[10:37] So today I want to talk about FFmpeg and VC.
[10:45] Uh for context for people who are not aware and I'm sure basically everybody listening to this have used these two technologies probably regularly without knowing it.
[10:57] So FFmpeg underlies basically most video on the internet including YouTube, Netflix, Chrome, Firefox, of course VC and countless other video platforms.
[11:06] It is estimated
[11:09] other video platforms.
[11:11] It is estimated that over 90% of video processing workflows online and offline involve ffmpeg.
[11:15] ffmpeg VC has been downloaded at least 6.5 billion times but likely that number cuz it's impossible to really count the number uh is much higher than that.
[11:28] Virtually any operating system supports virtually any media format.
[11:35] The limitation being it can't open pancakes.
[11:37] So, uh, can we just lay out some of the basics to to help people understand what's involved in all of this?
[11:45] So, when we press play on a video player like VC, what happens?
[11:52] What how does it go from the the file or the stream to the pixels on the screen and the sound on the speaker?
[11:59] What are the big stages to be aware of?
[12:01] So, there are several stages, right?
[12:03] The first stage is to get from an address, right?
[12:06] Which is the type of URL to give
[12:09] right? Which is the type of URL to give you a bite of streams, right?
[12:11] So this would be for example HTTP file DVD, right?
[12:15] You give the path to the media and gives you a stream of data.
[12:19] The stream needs to be cut up by what's known as the container, the demultiplexer or demox.
[12:22] Um we'll try and keep the jargon light throughout this, but um it needs to go and start demarcating video and audio frames.
[12:28] So it just gets data from the operating system blocks at the time and needs to start cutting these frames up into compressed data.
[12:35] It then needs to start doing simple parsing of the video frames mainly to figure out whether that codec is GPU decodable or needs to fall back to software.
[12:47] We're very sort of used to assuming the GPU will play all of these things.
[12:50] There'll be hardware acceleration.
[12:51] I think it's up to 45% of files are not GPU decodable.
[12:55] So these need to be probed.
[12:57] They need to be detected.
[12:59] There can be variants of a given codec, some of which are decodable on the GPU.
[13:04] Different vendors of GPU might have different capabilities.
[13:05] So those need to be detected.
[13:07] So if if it's GPU capable,
[13:10] be detected.
[13:10] So if if it's GPU capable, you pass it through to the GPU blackbox.
[13:13] you pass it through to the GPU blackbox.
[13:13] So now if there's a software fallback,
[13:15] So now if there's a software fallback,
[13:15] that means in the beginning is is to first do deentropy coding.
[13:17] that means in the beginning is is to first do deentropy coding.
[13:17] So removing the mathematical coding of the bitstream.
[13:20] first do deentropy coding.
[13:20] So removing the mathematical coding of the bitstream.
[13:21] the mathematical coding of the bitstream.
[13:21] So this uses capabilities such as Hoffman coding or arithmetic coding to actually decompress the mathematical layer of the bitstream.
[13:24] such as Hoffman coding or arithmetic coding to actually decompress the mathematical layer of the bitstream.
[13:26] coding to actually decompress the mathematical layer of the bitstream.
[13:26] We then need to start reading the syntax elements for intra prediction.
[13:28] mathematical layer of the bitstream.
[13:28] We then need to start reading the syntax elements for intra prediction.
[13:31] then need to start reading the syntax elements for intra prediction.
[13:31] So intra prediction are like still images of the video.
[13:32] elements for intra prediction.
[13:32] So intra prediction are like still images of the video.
[13:34] So intra prediction are like still images of the video.
[13:34] So your frames.
[13:34] So this works and operates in the spatial domain.
[13:37] prediction are like still images of the video.
[13:37] So your frames.
[13:37] So this works and operates in the spatial domain.
[13:40] video.
[13:40] So your frames.
[13:40] So this works and operates in the spatial domain.
[13:42] operates in the spatial domain.
[13:42] So you do your intra prediction spatial domain.
[13:44] So you do your intra prediction spatial domain.
[13:44] You you have a residual because your prediction isn't quite matching that of reality.
[13:46] You you have a residual because your prediction isn't quite matching that of reality.
[13:46] So you've made a prediction but then there's a little bit left and that's what's known as the residual.
[13:48] prediction isn't quite matching that of reality.
[13:48] So you've made a prediction but then there's a little bit left and that's what's known as the residual.
[13:50] So you've made a prediction but then there's a little bit left and that's what's known as the residual.
[13:50] then there's a little bit left and that's what's known as the residual.
[13:52] then there's a little bit left and that's what's known as the residual.
[13:52] This is stored in the frequency domain and these are quantized to decompound their space.
[13:53] that's what's known as the residual.
[13:53] This is stored in the frequency domain and these are quantized to decompound their space.
[13:55] This is stored in the frequency domain and these are quantized to decompound their space.
[13:55] We then need to do the inverse transform to bring them back to the spatial domain and apply these residuals.
[13:57] and these are quantized to decompound their space.
[13:57] We then need to do the inverse transform to bring them back to the spatial domain and apply these residuals.
[14:00] We then need to do the inverse transform to bring them back to the spatial domain and apply these residuals.
[14:00] So a lot of the process of the decoding is this thing is compressed.
[14:02] inverse transform to bring them back to the spatial domain and apply these residuals.
[14:02] So a lot of the process of the decoding is this thing is compressed.
[14:05] the spatial domain and apply these residuals.
[14:05] So a lot of the process of the decoding is this thing is compressed.
[14:07] So a lot of the process of the decoding is this thing is compressed.
[14:07] Yes.
[14:09] the decoding is this thing is compressed.
[14:09] Yes.
[14:10] compressed.
[14:10] Yes.
[14:10] >> Yes.
[14:10] And you have to predict the highest
[14:12] Yes.
[14:12] And you have to predict the highest quality thing that's supposed to go there.
[14:17] I frame is the best representation you have spatially.
[14:21] And then there's a lot of temporal compression that can happen depending on the codec.
[14:26] And then you're predicting, you're predicting what the reality that was captured in this raw form.
[14:31] Yeah.
[14:31] Because what people don't realize is that the compression on video and audio is 100 times, right?
[14:38] Like people don't realize how compressed we we do, right?
[14:40] For audio you move you compress by when you go from normal audio to MP3 you compress by 10 times right when you move to video you need 100 time 200 times right so you need to remove all the details but that you don't care about.
[14:56] Because all the compressions that we do and that's very important people forget about that is to be viewed by humans right.
[15:01] So all the codecs either for audio mimic basically how your ear works right.
[15:07] And and a lot of things about like the the the response on the ear and same for for your eyes right and and so for
[15:14] for your eyes right and and so for example on video we don't work on RGB.
[15:17] example on video we don't work on RGB right.
[15:19] everyone expect to work in RGB we don't right.
[15:22] we move to YUV which is basically one is luminance brightness.
[15:25] and the other are colors and this matches your eyes where inside your eyes.
[15:26] you have the cones and the buttons right.
[15:28] some of them look on brightness and more on the other on colors right.
[15:30] so we need to compress a lot and so we need to degrade.
[15:34] but in order to degrade we need to match the human perception and this.
[15:39] is why it's so difficult and then we need to use the maximum power.
[15:43] mathematical power very complex technologies we move to the frequency.
[15:48] domain as Karan said we do a ton of the quantizing and and in order to get the.
[15:53] best compression but it still looks good.
[15:58] you're trying to compress in order to maximize the highest quality thing for human perception that is correct.
[16:05] and that is correct and this is very important right.
[16:07] compression is not like a zip Right?
[16:08] A zip you have data in, you get data out, right?
[16:12] And you try with
[16:14] get data out, right?
[16:16] And you try with all the the zip compression to arrive with uh the limit.
[16:19] Here we are degrading the signal, right?
[16:21] And so we need to degrade both the audio and the video signal in the best way possible.
[16:25] And we can do that, but it involves first a lot of theoretical knowledge about how it works, the eye works, but it a lot of mathematical uh change, a lot of mathematical tricks, right?
[16:35] For example, when you move to RGB and you do go to YUV, for example, what we do very often is that we scale down the resolution of the color compared to the brightness and most of the time and just this without compression it divides the size by two but most people don't see it right um
[16:57] and so on and so on right and then you go to very complex mathematical change so of course uh 4 years transformed which the factor are not for transform they have like um discrete continuous transform but that's the same idea.
[17:10] So frequency domain um we split the video by blocks right so that's why when it's
[17:15] by blocks right so that's why when it's wrongly decoded you see those blocks and
[17:17] wrongly decoded you see those blocks and badly encoded you see those blocks and
[17:20] badly encoded you see those blocks and so on to arrive to compression states
[17:22] so on to arrive to compression states that are insanely high right and each
[17:25] that are insanely high right and each generation of the codec is like 30% less
[17:28] generation of the codec is like 30% less Mhm. for the same quality, right? And
[17:30] Mhm. for the same quality, right? And this requires amount of power um of
[17:34] this requires amount of power um of computational power that are huge.
[17:35] computational power that are huge. >> No, you should you should elaborate.
[17:37] No, you should you should elaborate. It's 30% better, but an order of
[17:39] It's 30% better, but an order of magnitude perhaps perhaps even two
[17:41] magnitude perhaps perhaps even two orders of magnitude more compression
[17:43] orders of magnitude more compression power. That that's the big difference.
[17:45] power. That that's the big difference. >> What do you mean by compression power?
[17:47] What do you mean by compression power? >> Sorry, CPU power to achieve that level
[17:49] Sorry, CPU power to achieve that level of compression.
[17:49] of compression. >> Oh, yeah. So, you have to be able to
[17:51] Oh, yeah. So, you have to be able to leverage the CPU and sometimes GPU like
[17:53] leverage the CPU and sometimes GPU like you mentioned. And then we should
[17:55] you mentioned. And then we should mention that a lot of this programming
[17:58] mention that a lot of this programming uh is done at the lowest possible stack
[18:01] uh is done at the lowest possible stack whether it's C and of course as as a
[18:04] whether it's C and of course as as a legendary Twitter handle um reemphasizes
[18:08] legendary Twitter handle um reemphasizes over and over a lot of assembly. So what
[18:10] over and over a lot of assembly. So what happens is globally is that you have an
[18:12] happens is globally is that you have an address right which gives you uh with
[18:14] address right which gives you uh with the operating system a stream of bytes a
[18:16] the operating system a stream of bytes a stream of data right and this is the stream of data right and this is the first step and the second step arise with demoxing where you're going to separate audio video or subtitle in type of different tracks and then on each of those tracks you're going to decompress them decode them either audio with an audio codec video to video codec and subtitle to subtitle codec um and once you've decompressed those type of things you have raw images row and then you're going to talk with your uh graphic card in your screen and display that.
[18:43] And same for the audio, you're going to talk to your audio card, which then is going to go um in analog to to your audio speakers.
[18:50] And everything we've just said in the past couple of minutes, every sentence is someone's lifetime's work.
[18:54] There are books about every sentence.
[18:56] So, the level of complexity in many cases is inordinate.
[19:00] You know, it's it's every sentence has thousands of people working on this in in the industry as a whole.
[19:09] books written about it.
[19:12] So there's a lot of detail, there's a lot of subtleties, there's a lot of both academic and
[19:17] there's a lot of both academic and practical realities.
[19:19] Um, both of which matter.
[19:22] Uh, we mentioned Kodaks, but I don't think you mentioned containers.
[19:25] So what what's the actual containers for some of the stuff we're talking about?
[19:30] So people are familiar with MP4, uh, MOV, MKV.
[19:33] So anyway, what what are containers versus uh the thing that goes inside?
[19:41] >> So the container is what we call also the muxer right when I say demoxing it means decontainerizing right.
[19:45] So actually if you look Mox multiplexer and de multiplexer right MX and demox are those and same codec is actually coder decoder right.
[19:56] um and um so containers are this collection of multiple tracks right.
[20:03] so it's a what normal people call the file format but it's a bit more um subtle than that.
[20:09] but the most known one of course is MP4 but uh when I started it was AVI right as the the video format.
[20:17] it was AVI right as the the video format from from uh Microsoft and uh move which
[20:21] from from uh Microsoft and uh move which became MP4 was a format from Apple. Um
[20:24] became MP4 was a format from Apple. Um in the open source community one of the
[20:26] in the open source community one of the person that is still active on video
[20:27] person that is still active on video land is called Steve Lom and started
[20:29] land is called Steve Lom and started this Matroska format which is like a bit
[20:32] this Matroska format which is like a bit more complex and and more feature proof
[20:35] more complex and and more feature proof um and um there are so many others. So I
[20:39] um and um there are so many others. So I mean there's it's a pretty common thing
[20:40] mean there's it's a pretty common thing and maybe it'll even happen in this
[20:42] and maybe it'll even happen in this conversation that people confuse
[20:44] conversation that people confuse container and the codec right. So
[20:47] container and the codec right. So confuse MP4 and H.264 for example. Is
[20:50] confuse MP4 and H.264 for example. Is that a horrible violation?
[20:51] that a horrible violation? >> No it's not because technically the name
[20:54] >> No it's not because technically the name of H.264 264 is ME 4 part 10 because ME
[20:59] of H.264 264 is ME 4 part 10 because ME 4 is actually a meta specification which
[21:02] 4 is actually a meta specification which has several things in it right there is
[21:05] has several things in it right there is the part two uh so there is like audio
[21:08] the part two uh so there is like audio codecs right AEC is the factory is MP4
[21:11] codecs right AEC is the factory is MP4 audio something there is actually
[21:13] audio something there is actually several video codecs right inside the ME
[21:16] several video codecs right inside the ME 4 specification one of them is ME 4 part
[21:19] 4 specification one of them is ME 4 part 10 called also AVC called also H264
[21:23] 10 called also AVC called also H264 right So it's completely the fault of
[21:25] right So it's completely the fault of the industry to to to to make things
[21:28] the industry to to to to make things difficult to understand. So that's very
[21:29] difficult to understand. So that's very difficult so that people then don't
[21:32] difficult so that people then don't understand why sometimes you talk about
[21:34] understand why sometimes you talk about me 4 part 10 where you mean H.264 and
[21:37] me 4 part 10 where you mean H.264 and why it's not MP4. So you can technically
[21:39] why it's not MP4. So you can technically shove in all kinds of different codecs
[21:41] shove in all kinds of different codecs inside containers and horribly so. But
[21:44] inside containers and horribly so. But broadly speaking though MP4 is
[21:46] broadly speaking though MP4 is understood to generally be H.264 plus
[21:50] understood to generally be H.264 plus AAC audio. 99% of the time that's that
[21:53] AAC audio. 99% of the time that's that and that the the rest are dimminimous
[21:56] and that the the rest are dimminimous the small effects you know edge effects
[21:57] the small effects you know edge effects really compared to that so it's not the
[22:00] really compared to that so it's not the end of the world that there are people
[22:02] end of the world that there are people who do get annoyed by that but also in
[22:04] who do get annoyed by that but also in reality something like VC just to point
[22:06] reality something like VC just to point out the file may saympp4 but it may be
[22:09] out the file may saympp4 but it may be something completely different and
[22:10] something completely different and that's one of the challenges both ffmpeg
[22:12] that's one of the challenges both ffmpeg and VC have is the real world is a
[22:14] and VC have is the real world is a completely different place to a
[22:16] completely different place to a threeletter file format
[22:17] threeletter file format >> and this is very important to say right
[22:19] >> and this is very important to say right like for example in VC and ffmpeg We
[22:22] like for example in VC and ffmpeg We discard the file format, right? We we
[22:25] discard the file format, right? We we look into the file to understand what's
[22:27] look into the file to understand what's in it because so many people like they
[22:30] in it because so many people like they say, "Oh, it's a video. It must be MP4,
[22:32] say, "Oh, it's a video. It must be MP4, but technically it's an MOV or maybe
[22:34] but technically it's an MOV or maybe it's a MKV, right?" So, we analyze in
[22:37] it's a MKV, right?" So, we analyze in real time everything that we have and we
[22:40] real time everything that we have and we don't trust uh the the the format.
[22:42] don't trust uh the the the format. >> So, what information does the fact that
[22:44] >> So, what information does the fact that it's MP4 give you?
[22:45] it's MP4 give you? >> It helps, right? It gives you a hint,
[22:47] >> It helps, right? It gives you a hint, right? just like oh it it's finished by
[22:50] right? just like oh it it's finished by MP4 I will start first by opening
[22:54] MP4 I will start first by opening probing it with the MP4 container
[22:57] probing it with the MP4 container demoxer to see well it should be that
[22:59] demoxer to see well it should be that but I don't trust it and if I'm lost I
[23:01] but I don't trust it and if I'm lost I say okay maybe I'm going to try to so it
[23:03] say okay maybe I'm going to try to so it bumps the priority of the module so how
[23:07] bumps the priority of the module so how do you get to uh just to take a bit of a
[23:09] do you get to uh just to take a bit of a tangent there you know the dumb thing is
[23:13] tangent there you know the dumb thing is if you try MP4 but it turns out it's a
[23:16] if you try MP4 but it turns out it's a different codec than you would have
[23:17] different codec than you would have expected
[23:19] expected Uh most players just break there.
[23:21] Uh most players just break there. >> Yes.
[23:22] >> Yes. >> Yes.
[23:22] >> Yes. >> So how do you not break? There's just a
[23:24] >> So how do you not break? There's just a philosophically I'm sure there's a bunch
[23:27] philosophically I'm sure there's a bunch of stumbling blocks along the way where
[23:29] of stumbling blocks along the way where you it's easy to just break and stop,
[23:32] you it's easy to just break and stop, freak out. That's it. How does VC not?
[23:34] freak out. That's it. How does VC not? This is why VC is popular. Um but the
[23:37] This is why VC is popular. Um but the reason is because actually VC was is
[23:40] reason is because actually VC was is just a client of a streaming solution
[23:43] just a client of a streaming solution called videoland from from from very
[23:46] called videoland from from from very long time ago from the late '9s and when
[23:49] long time ago from the late '9s and when you're playing video which are on UDP
[23:52] you're playing video which are on UDP right in network they might be damaged
[23:54] right in network they might be damaged right so you don't trust your inputs and
[23:56] right so you don't trust your inputs and this is very important in today's
[23:57] this is very important in today's security is that you don't trust your
[23:59] security is that you don't trust your inputs so everything in VC is prepared
[24:03] inputs so everything in VC is prepared to um work with broken files. Mhm. And
[24:06] to um work with broken files. Mhm. And it's a philosophical idea from the
[24:09] it's a philosophical idea from the beginning and everything is engineered
[24:11] beginning and everything is engineered into that and it's a culture, right? And
[24:14] into that and it's a culture, right? And so for example, NVLC became very popular
[24:16] so for example, NVLC became very popular on that because a long time ago when
[24:18] on that because a long time ago when people were pirating content um which
[24:21] people were pirating content um which they do a lot less today um
[24:23] they do a lot less today um >> and none of us ever have
[24:24] >> and none of us ever have >> no um the metadata to play some files
[24:28] >> no um the metadata to play some files like AVI is at at the end of the file,
[24:32] like AVI is at at the end of the file, right? And when you're downloading you
[24:33] right? And when you're downloading you don't have that, right? So VC was just
[24:35] don't have that, right? So VC was just like, "Hey, this file is broken, but I'm
[24:37] like, "Hey, this file is broken, but I'm still going to try to interpret it." And
[24:39] still going to try to interpret it." And this was very useful. We hinted at the
[24:43] this was very useful. We hinted at the awesomeness of the various different
[24:44] awesomeness of the various different stages. We hinted at the awesomeness of
[24:47] stages. We hinted at the awesomeness of codecs, the depth and the richness and
[24:49] codecs, the depth and the richness and the complexity of everything involved
[24:50] the complexity of everything involved there. What let's try to define what is
[24:54] there. What let's try to define what is a video codec? What what's involved
[24:56] a video codec? What what's involved there? What what does it mean to
[24:57] there? What what does it mean to compress something? You already started
[24:58] compress something? You already started to hint at it, but can can we elaborate
[25:00] to hint at it, but can can we elaborate a little bit more? So there's a huge
[25:02] a little bit more? So there's a huge amount of redundancy in any video uh
[25:05] amount of redundancy in any video uh both spatial and temporal and the point
[25:07] both spatial and temporal and the point of any video codec is to remove this
[25:09] of any video codec is to remove this redundant data. Use mathematical
[25:11] redundant data. Use mathematical properties as part of this reduction
[25:13] properties as part of this reduction process. So more often than not using
[25:16] process. So more often than not using several orders of magnitude more compute
[25:17] several orders of magnitude more compute to compress because that's more costly
[25:20] to compress because that's more costly versus both costly both financially and
[25:22] versus both costly both financially and in CPU resources
[25:23] in CPU resources >> versus the decompression. So it's
[25:25] >> versus the decompression. So it's asymmetric in that respect. Often the
[25:28] asymmetric in that respect. Often the case because compression is done once
[25:29] case because compression is done once but there could be lots of viewers of
[25:31] but there could be lots of viewers of another file.
[25:33] another file. >> So to take that information and compress
[25:34] >> So to take that information and compress it by 100x 200x removing redundant
[25:38] it by 100x 200x removing redundant information and using mathematical
[25:40] information and using mathematical properties to make that small but also
[25:43] properties to make that small but also have properties such as error
[25:44] have properties such as error resilience. So as as JB suggested
[25:47] resilience. So as as JB suggested VC in the beginning was was used to play
[25:49] VC in the beginning was was used to play UDP network feeds and UDP network feeds
[25:52] UDP network feeds and UDP network feeds lose packets. And so some of the design
[25:54] lose packets. And so some of the design goals of a codec is also to be
[25:56] goals of a codec is also to be recoverable.
[25:57] recoverable. >> You you need to actually be able to join
[25:59] >> You you need to actually be able to join a stream. It's not necessarily a file.
[26:01] a stream. It's not necessarily a file. You need to join get on the decoding
[26:04] You need to join get on the decoding process and start decoding.
[26:06] process and start decoding. >> And and to give a more image to to to to
[26:10] >> And and to give a more image to to to to people who are not familiar, right? Like
[26:11] people who are not familiar, right? Like when you're going to see any type of
[26:13] when you're going to see any type of movie, right? You're going to see the
[26:15] movie, right? You're going to see the camera is going to pan, right? and and
[26:17] camera is going to pan, right? and and and travel and you realize that for
[26:19] and travel and you realize that for example all the background is the same
[26:21] example all the background is the same from for like a minute right or 30
[26:24] from for like a minute right or 30 seconds right so you can reuse the cloud
[26:25] seconds right so you can reuse the cloud that you see uh on the background you
[26:28] that you see uh on the background you can reuse that from a frame to another
[26:30] can reuse that from a frame to another right and so it's
[26:32] right and so it's >> gets the more the more memory you have
[26:36] >> gets the more the more memory you have the more power the more comparisons you
[26:38] the more power the more comparisons you can make right and so the more
[26:39] can make right and so the more compressed you can be and most of the
[26:42] compressed you can be and most of the modern codecs are basically doing that
[26:44] modern codecs are basically doing that so just to make it even more explicit.
[26:47] so just to make it even more explicit. So what is video? Video is a bunch of
[26:51] So what is video? Video is a bunch of pixels off an RGB
[26:54] pixels off an RGB have three values and you have a grid of
[26:57] have three values and you have a grid of pixels and you have let's say 24 or 30
[27:00] pixels and you have let's say 24 or 30 or 60 frames a second and you just have
[27:05] or 60 frames a second and you just have all these pixels repeating and showing
[27:07] all these pixels repeating and showing different stuff 30 times a second. And
[27:10] different stuff 30 times a second. And so the question, the philosophical, the
[27:13] so the question, the philosophical, the technical question is how can I compress
[27:16] technical question is how can I compress all of that, store all of that at 100x
[27:20] all of that, store all of that at 100x >> or 1,000x, right?
[27:22] >> or 1,000x, right? >> 1,000x.
[27:22] >> 1,000x. >> The target is 1,000x, right?
[27:24] >> The target is 1,000x, right? >> And the goal is when you say redundancy,
[27:28] >> And the goal is when you say redundancy, what is redundant? Meaning stuff at best
[27:32] what is redundant? Meaning stuff at best that humans wouldn't notice if it was
[27:34] that humans wouldn't notice if it was missing. So for example, you have a
[27:37] missing. So for example, you have a picture of a cloud, right? and from the
[27:39] picture of a cloud, right? and from the next frame there's still going to be the
[27:40] next frame there's still going to be the same cloud. So it's redundant. You could
[27:42] same cloud. So it's redundant. You could just put it once and not do it right or
[27:44] just put it once and not do it right or you have a a black background behind me
[27:47] you have a a black background behind me for example the black is the same on the
[27:48] for example the black is the same on the whole picture. Right? So you can say
[27:50] whole picture. Right? So you can say well you know in this picture take the
[27:52] well you know in this picture take the pixels that you have on the top left and
[27:54] pixels that you have on the top left and the one on the top right I'm not going
[27:56] the one on the top right I'm not going to give the value. I'm just going to
[27:57] to give the value. I'm just going to tell you it's the same at the top left.
[27:59] tell you it's the same at the top left. And then you can say for frame one um
[28:02] And then you can say for frame one um reuse something from the previous frame
[28:04] reuse something from the previous frame or the previous previous frame and so on
[28:06] or the previous previous frame and so on and so on. Right? So you could basically
[28:09] and so on. Right? So you could basically it's unlimited but then it's limited in
[28:12] it's unlimited but then it's limited in terms of memory or in terms of compute
[28:14] terms of memory or in terms of compute power because for example if you need to
[28:16] power because for example if you need to compare pixels on 200 frames in the past
[28:20] compare pixels on 200 frames in the past on 4K resolutions it's a huge amount of
[28:25] on 4K resolutions it's a huge amount of compute and then when you're showing it
[28:27] compute and then when you're showing it you have to do the decompress of all of
[28:29] you have to do the decompress of all of that. So is it the codec the has the
[28:33] that. So is it the codec the has the encoding and the decoding is is a
[28:35] encoding and the decoding is is a coupled process that you're developing.
[28:37] coupled process that you're developing. >> Exactly. Right. And those are two
[28:39] >> Exactly. Right. And those are two different um uh trade-offs. Right. Are
[28:42] different um uh trade-offs. Right. Are you going to compress more uh but then
[28:44] you going to compress more uh but then it might be more difficult to to to
[28:46] it might be more difficult to to to decode? Um are you going to comp to make
[28:49] decode? Um are you going to comp to make it a codec that is more complex to
[28:52] it a codec that is more complex to encode and easier to decode? Are you
[28:54] encode and easier to decode? Are you going to make a codec that is easier to
[28:55] going to make a codec that is easier to encode because you need to be fast. But
[28:57] encode because you need to be fast. But then the the client side the player is
[29:00] then the the client side the player is going to spend more time. That's why you
[29:01] going to spend more time. That's why you have so many different type of codecs is
[29:03] have so many different type of codecs is that it's not always easy. And to make
[29:06] that it's not always easy. And to make it even more complex, modern code decks
[29:09] it even more complex, modern code decks like AV1, AV2 or VVC are actually not
[29:12] like AV1, AV2 or VVC are actually not codecs. They are a collection of tools,
[29:14] codecs. They are a collection of tools, right? There are multiple tools,
[29:16] right? There are multiple tools, multiple codecs in the same codec to
[29:18] multiple codecs in the same codec to depending on the image get the more
[29:20] depending on the image get the more compression. So just to elaborate codecs
[29:24] compression. So just to elaborate codecs like AV1, VVC
[29:26] like AV1, VVC have a much wide have a wide audience.
[29:28] have a much wide have a wide audience. It could be a screen share content. It
[29:31] It could be a screen share content. It could be video, it could be animation.
[29:33] could be video, it could be animation. All of these require different coding
[29:36] All of these require different coding tools. So what happens these days is a
[29:40] tools. So what happens these days is a collection of tools are put in and
[29:41] collection of tools are put in and called AV1, called AV2, called DVC to
[29:45] called AV1, called AV2, called DVC to allow for different use cases. So you
[29:47] allow for different use cases. So you may be on Zoom and sharing your
[29:49] may be on Zoom and sharing your PowerPoint and then you need to show the
[29:50] PowerPoint and then you need to show the audience a video. That codec needs to
[29:53] audience a video. That codec needs to start changing its tool set depending on
[29:56] start changing its tool set depending on the content to compress in a different
[29:58] the content to compress in a different way.
[29:59] way. >> And like you said, there's a bunch of
[30:01] >> And like you said, there's a bunch of incredible engineers behind each part of
[30:03] incredible engineers behind each part of that, each part of the tools that make
[30:04] that, each part of the tools that make up AV1, for example.
[30:05] up AV1, for example. >> Sure. Uh so we've uh kind of danced
[30:08] >> Sure. Uh so we've uh kind of danced around it. We talked about VC, the logo,
[30:11] around it. We talked about VC, the logo, the hat. Uh let's talk about FFmpeg.
[30:14] the hat. Uh let's talk about FFmpeg. What what is ffmpeg exactly?
[30:17] What what is ffmpeg exactly? >> FFmpeg is basically the low-level
[30:22] >> FFmpeg is basically the low-level libraries for codec. So compressions and
[30:26] libraries for codec. So compressions and decompression mixes and demoxes and
[30:29] decompression mixes and demoxes and filters. It's the core is this and then
[30:32] filters. It's the core is this and then you have a several tool which allow you
[30:33] you have a several tool which allow you to create a type of pipeline to process
[30:36] to create a type of pipeline to process any type of video files. uh and it's
[30:38] any type of video files. uh and it's used as a library absolutely inside
[30:41] used as a library absolutely inside everything from VC to Chrome to your
[30:44] everything from VC to Chrome to your smart TVs to basically any video that
[30:47] smart TVs to basically any video that you see online. um you usually use
[30:49] you see online. um you usually use ffmpeg and ffmpeg in it has all those
[30:53] ffmpeg and ffmpeg in it has all those type of tools and sometimes depend on
[30:56] type of tools and sometimes depend on other libraries like x264 uh lib vpx and
[31:00] other libraries like x264 uh lib vpx and others right so so it's really now the
[31:02] others right so so it's really now the de facto um tool to process images from
[31:06] de facto um tool to process images from a philosophical level I think it's
[31:08] a philosophical level I think it's incredible that your home videos your
[31:11] incredible that your home videos your your grandmother's home videos and
[31:13] your grandmother's home videos and trillion dollar corporations
[31:15] trillion dollar corporations effectively are on a level playing field
[31:17] effectively are on a level playing field using the technology stack. Uh
[31:20] using the technology stack. Uh >> it's it wouldn't be a surprise, you
[31:22] >> it's it wouldn't be a surprise, you know, these big companies just have
[31:24] know, these big companies just have 3,000 line ffmpeg commands. There are
[31:27] 3,000 line ffmpeg commands. There are some that use the API, but there are
[31:29] some that use the API, but there are some that just have long command lines.
[31:31] some that just have long command lines. So yeah, there's a bunch of tools like
[31:34] So yeah, there's a bunch of tools like literally command line tool, ffmpeg of
[31:36] literally command line tool, ffmpeg of course, ff probe, there's libraries,
[31:39] course, ff probe, there's libraries, libby vc codec, lib format, libby v fil
[31:42] libby vc codec, lib format, libby v fil filter, but the ffmpeg
[31:46] filter, but the ffmpeg on the command line is like legendary
[31:49] on the command line is like legendary because you can cut there's so many
[31:51] because you can cut there's so many parameters you can customize everything
[31:53] parameters you can customize everything to help.
[31:53] to help. >> It's a language. It's an actual
[31:54] >> It's a language. It's an actual language.
[31:55] language. >> It's an actual Yeah, you could think of
[31:56] >> It's an actual Yeah, you could think of it as a programming language.
[31:57] it as a programming language. >> Yeah, of course. I'm sure because so
[31:59] >> Yeah, of course. I'm sure because so most of people they're going to take
[32:01] most of people they're going to take ffmpeg file in file out and specify the
[32:04] ffmpeg file in file out and specify the format right but you can we've seen
[32:07] format right but you can we've seen thousands of characters and we've seen
[32:09] thousands of characters and we've seen also like people like doing um
[32:11] also like people like doing um programming um uh generation of common
[32:14] programming um uh generation of common lines to to to make ffmpeg there is a
[32:16] lines to to to make ffmpeg there is a ton of people who are using AI to
[32:18] ton of people who are using AI to generate common lines for ffmpeg because
[32:20] generate common lines for ffmpeg because you have no idea what it is but you can
[32:22] you have no idea what it is but you can do specify so many filters right uh on
[32:24] do specify so many filters right uh on common line right so um ffmpeg is this
[32:27] common line right so um ffmpeg is this collection of toolbox for multimedia
[32:30] collection of toolbox for multimedia processing that everyone everyone uses
[32:34] processing that everyone everyone uses and everyone that is watching your
[32:36] and everyone that is watching your videos are also using right you you're
[32:38] videos are also using right you you're on YouTube well it's ffmpeg on on the
[32:40] on YouTube well it's ffmpeg on on the client side well the your server side on
[32:42] client side well the your server side on the server side the client side is
[32:44] the server side the client side is polychrome well you're using ffmpeg also
[32:46] polychrome well you're using ffmpeg also and and you're using OBS to record well
[32:48] and and you're using OBS to record well it's ffmpeg right you're using a ton of
[32:51] it's ffmpeg right you're using a ton of important like big box professional
[32:53] important like big box professional boxes well it's very possible that
[32:55] boxes well it's very possible that inside some part of ffmpeg is I mean
[32:58] inside some part of ffmpeg is I mean there there's like so many just to give
[32:59] there there's like so many just to give people idea like I I use FFmpeg a lot on
[33:03] people idea like I I use FFmpeg a lot on on everything just trivial stuff like uh
[33:06] on everything just trivial stuff like uh take a video add an intro video and an
[33:09] take a video add an intro video and an outro video and fade one into the other
[33:12] outro video and fade one into the other like uh what is it called dip to black
[33:14] like uh what is it called dip to black like where it it dips and then shows the
[33:17] like where it it dips and then shows the next video and does the same thing with
[33:19] next video and does the same thing with the audio. Uh there's like a cross
[33:22] the audio. Uh there's like a cross dissolve of the audio. It's a quiet it
[33:24] dissolve of the audio. It's a quiet it quiets the audio and makes it loud
[33:26] quiets the audio and makes it loud again. And there's a bunch of stuff like
[33:28] again. And there's a bunch of stuff like uh showing the captions on screen hard
[33:32] uh showing the captions on screen hard like baking the captions in. You can
[33:34] like baking the captions in. You can customize the font. You can do all kinds
[33:38] customize the font. You can do all kinds of layering of audio and video. There's
[33:40] of layering of audio and video. There's a million things. And of course, all of
[33:43] a million things. And of course, all of that works
[33:45] that works like magically with basically any codec.
[33:49] like magically with basically any codec. Like anything you could shove in on the
[33:51] Like anything you could shove in on the audio and the video side, it works. But
[33:53] audio and the video side, it works. But it's like if you if you look like for
[33:55] it's like if you if you look like for example you can do things that you would
[33:57] example you can do things that you would do with adob after effects
[33:59] do with adob after effects >> in command line on ffmpeg right it's and
[34:02] >> in command line on ffmpeg right it's and it's very interesting because for
[34:03] it's very interesting because for example for for for images there is not
[34:06] example for for for images there is not such tool there is a few tools but not
[34:09] such tool there is a few tools but not with the breath of ffmpeg
[34:10] with the breath of ffmpeg >> so image magic has a similar kind of
[34:13] >> so image magic has a similar kind of >> yes but we not do some filters complex
[34:16] >> yes but we not do some filters complex filters you don't have the equivalent of
[34:18] filters you don't have the equivalent of photoshop in common line right but you
[34:20] photoshop in common line right but you for video you have ffmpeg in common
[34:22] for video you have ffmpeg in common Yeah, it's incredible. I mean, it's like
[34:24] Yeah, it's incredible. I mean, it's like it's an example of a thing when a bunch
[34:26] it's an example of a thing when a bunch of great people get together and they
[34:28] of great people get together and they get a vision and they stick by that
[34:31] get a vision and they stick by that vision for many years, which is
[34:33] vision for many years, which is incredible. And the vision behind and
[34:35] incredible. And the vision behind and the same for VC and FFmpeg is that we
[34:38] the same for VC and FFmpeg is that we make every everything that is very
[34:41] make every everything that is very complex easy to use for the normal
[34:44] complex easy to use for the normal people, for everyone, right? Our goal is
[34:46] people, for everyone, right? Our goal is to make something that is insanely
[34:48] to make something that is insanely complex technically and make it easy to
[34:50] complex technically and make it easy to use, right? and and people they use VC
[34:53] use, right? and and people they use VC they drop a file they don't realize how
[34:55] they drop a file they don't realize how complex the file is but they they play
[34:57] complex the file is but they they play it or or people put any type of thing
[34:59] it or or people put any type of thing inside ffmpeg with complex filters and
[35:02] inside ffmpeg with complex filters and it just works like magically right and
[35:04] it just works like magically right and people
[35:05] people >> and this is our mission right make very
[35:07] >> and this is our mission right make very complex things we wouldn't be here and
[35:09] complex things we wouldn't be here and you wouldn't be here if this required
[35:12] you wouldn't be here if this required you know a traditional television studio
[35:14] you know a traditional television studio setup
[35:16] setup >> it's tools like ffmpeg that democratize
[35:18] >> it's tools like ffmpeg that democratize this the the podcast and streaming
[35:20] this the the podcast and streaming revolution The YouTube revolution
[35:22] revolution The YouTube revolution >> was caused, you know, FFmpeg was a big
[35:24] >> was caused, you know, FFmpeg was a big player in that because it it
[35:26] player in that because it it democratized this technology that was
[35:28] democratized this technology that was once in the in the '90s, for example,
[35:30] once in the in the '90s, for example, you needed equipment that cost hundreds
[35:33] you needed equipment that cost hundreds of thousands of dollars to do
[35:34] of thousands of dollars to do compression. It was the size of a car.
[35:37] compression. It was the size of a car. And now everybody has that at almost an
[35:39] And now everybody has that at almost an exact level playing field. And that's
[35:41] exact level playing field. And that's something that's so remarkable. It gave
[35:43] something that's so remarkable. It gave voice to a lot of people. And we just to
[35:45] voice to a lot of people. And we just to clarify, we say you you wouldn't be
[35:47] clarify, we say you you wouldn't be here, not the human, but the podcast.
[35:49] here, not the human, but the podcast. >> The podcast. Sorry. you you as a sorry
[35:51] >> The podcast. Sorry. you you as a sorry VLC did not have anything to do on a
[35:53] VLC did not have anything to do on a biological leveling
[35:55] biological leveling me as a human
[35:56] me as a human >> you realize also everything move from
[35:59] >> you realize also everything move from text to images and images to video right
[36:02] text to images and images to video right look look at social networks video is
[36:04] look look at social networks video is everywhere it's the most powerful um
[36:07] everywhere it's the most powerful um medium there is right and and when you
[36:09] medium there is right and and when you see shorts and and in reals and and Tik
[36:12] see shorts and and in reals and and Tik Tok right it's amazingly powerful to to
[36:15] Tok right it's amazingly powerful to to to give video is amazing for that right
[36:18] to give video is amazing for that right but the complexity is important.
[36:20] but the complexity is important. >> This what people don't realize. I mean
[36:22] >> This what people don't realize. I mean this is really
[36:24] this is really it gave power to the individual all
[36:27] it gave power to the individual all across the world. It's real freedom and
[36:29] across the world. It's real freedom and I think I can't believe it but we still
[36:31] I think I can't believe it but we still haven't mentioned the actual obvious
[36:32] haven't mentioned the actual obvious thing for people who are not familiar
[36:35] thing for people who are not familiar which it's opensource and there's a
[36:39] which it's opensource and there's a open- source community of users and
[36:42] open- source community of users and developers behind it. So it's really
[36:44] developers behind it. So it's really it's a movement. So like we'll talk a
[36:47] it's a movement. So like we'll talk a bunch in a bunch of different ways about
[36:48] bunch in a bunch of different ways about the community behind it but uh can you
[36:50] the community behind it but uh can you speak to the open source element? So
[36:52] speak to the open source element? So when we say what is ffmpeg it's an
[36:55] when we say what is ffmpeg it's an opensource project. Yeah. So ffmpeg VC
[36:59] opensource project. Yeah. So ffmpeg VC x264 video everything we do is fully
[37:02] x264 video everything we do is fully open source. And for the people who
[37:03] open source. And for the people who don't understand how open source is my
[37:05] don't understand how open source is my usual analogy is about a chocolate
[37:07] usual analogy is about a chocolate cheesecake right. Um usually for your
[37:10] cheesecake right. Um usually for your when you want to buy your cheesecake you
[37:11] when you want to buy your cheesecake you go to a bakery they give you the
[37:12] go to a bakery they give you the cheesecake. Um, the other one way of
[37:15] cheesecake. Um, the other one way of having a cheesecake is have your grandma
[37:16] having a cheesecake is have your grandma give you a recipe of how to make that.
[37:19] give you a recipe of how to make that. When we do open source, we give you the
[37:21] When we do open source, we give you the chocolate cake and we give you the
[37:23] chocolate cake and we give you the recipe to actually remake the same cake,
[37:26] recipe to actually remake the same cake, but at the same times tell you how to
[37:28] but at the same times tell you how to build the oven and also how you're
[37:31] build the oven and also how you're allowed to modify the recipe and resell
[37:34] allowed to modify the recipe and resell it to someone else. And this is because
[37:36] it to someone else. And this is because software is just a very long recipe of
[37:38] software is just a very long recipe of small instruction. Computers are not
[37:41] small instruction. Computers are not very clever. they go very very fast. So
[37:43] very clever. they go very very fast. So a normal program has tens of billions of
[37:46] a normal program has tens of billions of instruction instead of the tens when you
[37:48] instruction instead of the tens when you have your chocolate recipe. So a lot of
[37:51] have your chocolate recipe. So a lot of the software industry was about selling
[37:53] the software industry was about selling software like like where you just have
[37:55] software like like where you just have like the final cheesecake in open
[37:57] like the final cheesecake in open source. We make we give you everything
[38:00] source. We make we give you everything and that managed to get a lot of people
[38:03] and that managed to get a lot of people work together, right? because then you
[38:05] work together, right? because then you decide that you're going to make the
[38:06] decide that you're going to make the best program, the best recipe for video
[38:09] best program, the best recipe for video and you create communities in um FFmpeg
[38:12] and you create communities in um FFmpeg since the beginning of FFmpeg probably
[38:15] since the beginning of FFmpeg probably >> 2,000 to 3,000 people have contributed
[38:18] >> 2,000 to 3,000 people have contributed from the beginning, right? And then it's
[38:20] from the beginning, right? And then it's exactly like the Linux kernel, right?
[38:21] exactly like the Linux kernel, right? The Linux kernel has probably 10,000
[38:24] The Linux kernel has probably 10,000 people contributing everywhere and they
[38:27] people contributing everywhere and they get together well mostly online, right?
[38:29] get together well mostly online, right? So they virtually get together to create
[38:32] So they virtually get together to create the best tool for something and on
[38:35] the best tool for something and on ffmpeg and VC is just like
[38:38] ffmpeg and VC is just like well this codec doesn't work so I'm
[38:40] well this codec doesn't work so I'm going to work on the codec and I'm going
[38:42] going to work on the codec and I'm going to add the support for this file inside
[38:44] to add the support for this file inside ffmpeg so it will be beneficial to
[38:47] ffmpeg so it will be beneficial to everyone because again we work for the
[38:49] everyone because again we work for the greater good we work for everyone and
[38:52] greater good we work for everyone and that is what open source is
[38:54] that is what open source is >> and we should mention depending on the
[38:56] >> and we should mention depending on the licensing
[38:58] licensing uh you could probably build a billion
[39:00] uh you could probably build a billion dollar maybe even a trillion dollar
[39:02] dollar maybe even a trillion dollar company around basic as a rapper.
[39:07] company around basic as a rapper. >> Well, yes, people do. People do, right?
[39:09] >> Well, yes, people do. People do, right? Uh there was a lot of uh um problems
[39:12] Uh there was a lot of uh um problems with mostly uh cloud providers who are
[39:15] with mostly uh cloud providers who are basically running some open- source
[39:16] basically running some open- source tools um in in the cloud and just give
[39:19] tools um in in the cloud and just give you uh the API to access to that. And
[39:22] you uh the API to access to that. And there was a lot of uh um databases like
[39:26] there was a lot of uh um databases like or Elastic who change their
[39:28] or Elastic who change their license in order to avoid those type of
[39:30] license in order to avoid those type of uh of scenarios. This is a question we
[39:32] uh of scenarios. This is a question we get a lot in FFmpeg is why don't you do
[39:34] get a lot in FFmpeg is why don't you do that and you can't we have we have
[39:36] that and you can't we have we have thousands of contributors some of whom
[39:38] thousands of contributors some of whom aren't even alive anymore.
[39:40] aren't even alive anymore. >> You would need all of their agreement to
[39:42] >> You would need all of their agreement to do that and and JB will go maybe a bit
[39:44] do that and and JB will go maybe a bit later and talk about how challenging
[39:45] later and talk about how challenging that process was in VC to do the
[39:48] that process was in VC to do the relicensing. The license is a social
[39:51] relicensing. The license is a social contract in the terms of who the facto
[39:54] contract in the terms of who the facto of the community. The community does not
[39:57] of the community. The community does not agree on much beside the license. people
[40:00] agree on much beside the license. people go around discuss around because of the
[40:04] go around discuss around because of the license and that also allow those
[40:05] license and that also allow those license fork right sometime the
[40:07] license fork right sometime the community splits but it's possible
[40:09] community splits but it's possible because of the license then to merge
[40:11] because of the license then to merge back and we've seen that so many times
[40:13] back and we've seen that so many times right GCC and GC um and EGCC in the past
[40:18] right GCC and GC um and EGCC in the past we have seen for example all the web
[40:19] we have seen for example all the web browsers right they started as web KHTML
[40:22] browsers right they started as web KHTML which becames webkit and then which
[40:24] which becames webkit and then which becames blink right so open-source
[40:26] becames blink right so open-source license is like the the core of the
[40:30] license is like the the core of the community and um people are coming from
[40:33] community and um people are coming from all around the world, very different
[40:35] all around the world, very different type of uh religion, um political
[40:38] type of uh religion, um political borders. They they work in the same way
[40:42] borders. They they work in the same way on a project to solve a specific problem
[40:45] on a project to solve a specific problem and the specific problem we're working
[40:47] and the specific problem we're working on is to make multimedia easy for
[40:49] on is to make multimedia easy for everyone. Uh looking up on perplexity
[40:51] everyone. Uh looking up on perplexity here, looking at the different open
[40:52] here, looking at the different open source licenses, most major open source
[40:55] source licenses, most major open source licenses fall into two buckets.
[40:58] licenses fall into two buckets. permissive very few conditions and copy
[41:01] permissive very few conditions and copy left share alike requirements for
[41:03] left share alike requirements for derivatives below is a brief practical
[41:05] derivatives below is a brief practical summary of the main ones you'll see in
[41:07] summary of the main ones you'll see in the wild MIT license BSD
[41:11] the wild MIT license BSD ISC
[41:13] ISC uh Apache
[41:15] uh Apache uh GNU GPL
[41:17] uh GNU GPL GNU AGPL
[41:20] GNU AGPL where's LGPL yeah LGPL
[41:23] where's LGPL yeah LGPL let's see there's the the Mozilla public
[41:25] let's see there's the the Mozilla public license there's Eclipse public license.
[41:27] license there's Eclipse public license. It goes on. There's a lot of variety. I
[41:30] It goes on. There's a lot of variety. I mean, I think I think the really popular
[41:31] mean, I think I think the really popular ones is MIT, GPL, LGPL,
[41:34] ones is MIT, GPL, LGPL, >> NBSD,
[41:35] >> NBSD, >> NBSD, Apache. Sometimes you
[41:39] >> NBSD, Apache. Sometimes you >> license, that's an option attempts to
[41:40] >> license, that's an option attempts to dedicate code to the public domain with
[41:42] dedicate code to the public domain with a fallback permissive license.
[41:43] a fallback permissive license. >> There are many licenses for many
[41:45] >> There are many licenses for many different thing. What people don't
[41:47] different thing. What people don't understand that public domain is
[41:48] understand that public domain is something that doesn't exist worldwide,
[41:51] something that doesn't exist worldwide, right? So it's um all the open source
[41:54] right? So it's um all the open source licensing use the copyright law right
[41:57] licensing use the copyright law right the international copyright law in order
[41:59] the international copyright law in order to give rights on how you use the
[42:02] to give rights on how you use the software or how you modify it's the
[42:04] software or how you modify it's the facto a a copyright license contract
[42:07] facto a a copyright license contract that you give to the end user or to the
[42:09] that you give to the end user or to the developer and so you have like the first
[42:11] developer and so you have like the first one which are basically very permissive
[42:13] one which are basically very permissive MITBSD you give the code and basically
[42:16] MITBSD you give the code and basically you do whatever you want right you take
[42:19] you do whatever you want right you take it you want you modify you do what you
[42:21] it you want you modify you do what you want. Um and this is popular for uh uh
[42:25] want. Um and this is popular for uh uh JavaScript and the type of BSD operating
[42:28] JavaScript and the type of BSD operating system.
[42:29] system. >> So some of them one one of the
[42:30] >> So some of them one one of the parameters is whether they require
[42:32] parameters is whether they require attribution meaning if you use the code
[42:34] attribution meaning if you use the code you have to say
[42:35] you have to say >> yes. So in those type of permissive
[42:38] >> yes. So in those type of permissive license some you need to say if you use
[42:40] license some you need to say if you use it which is called attribution and some
[42:41] it which is called attribution and some you don't and then there is a the other
[42:44] you don't and then there is a the other part of license which are copy left
[42:46] part of license which are copy left where you need to give back to the
[42:49] where you need to give back to the community your modifications and with
[42:52] community your modifications and with different string attached um some weak
[42:55] different string attached um some weak uh copy left license like um the
[42:57] uh copy left license like um the modzilla public license to some which
[42:59] modzilla public license to some which are a bit stronger uh like GPL or even
[43:02] are a bit stronger uh like GPL or even very strong like AGPL. So all of those
[43:05] very strong like AGPL. So all of those are different um type of licensing that
[43:08] are different um type of licensing that depends on what your goal are and how
[43:10] depends on what your goal are and how you want to structure your community
[43:11] you want to structure your community which is why I spoke about social
[43:13] which is why I spoke about social contract because this is very important
[43:15] contract because this is very important to understand. Um ffmpeg and VC are
[43:18] to understand. Um ffmpeg and VC are mostly GPL or LGPL. Uh the Linux kernel
[43:22] mostly GPL or LGPL. Uh the Linux kernel is GPL. Uh but Android is Apache. Uh a
[43:27] is GPL. Uh but Android is Apache. Uh a ton of uh JavaScript framework that are
[43:29] ton of uh JavaScript framework that are using mostly MIT. um all the BSD kernels
[43:34] using mostly MIT. um all the BSD kernels open BSD, NetBSD are of course BSD um
[43:37] open BSD, NetBSD are of course BSD um and and so the it's philosoph
[43:39] and and so the it's philosoph philosophical change on how you want
[43:42] philosophical change on how you want people to contribute back basically.
[43:44] people to contribute back basically. >> So there's uh I think you talked about
[43:46] >> So there's uh I think you talked about that
[43:48] that uh you've moved at one point from GPL to
[43:51] uh you've moved at one point from GPL to LGPL on certain parts of the project.
[43:54] LGPL on certain parts of the project. What what can you describe the
[43:55] What what can you describe the difference between the two and what does
[43:56] difference between the two and what does it take to move to I guess a more
[43:59] it take to move to I guess a more permissive so that direction is more
[44:01] permissive so that direction is more permissive LGPL is more permissive than
[44:03] permissive LGPL is more permissive than GPL. Yeah. So you have to realize that
[44:05] GPL. Yeah. So you have to realize that you can always go from more permissive
[44:08] you can always go from more permissive to less permissive right because of
[44:11] to less permissive right because of course those license are basically
[44:14] course those license are basically statement. So if you restrict you can
[44:16] statement. So if you restrict you can always restrict more right. So in a GPL
[44:19] always restrict more right. So in a GPL project you can take MIT code but you
[44:22] project you can take MIT code but you cannot do the opposite right because
[44:23] cannot do the opposite right because they are more constrained to to match.
[44:26] they are more constrained to to match. Um indeed in fact I changed the core of
[44:30] Um indeed in fact I changed the core of libc which is the engine of
[44:34] libc which is the engine of >> from um GPL to LGPL.
[44:37] >> from um GPL to LGPL. >> Uh and there were two reasons to do
[44:39] >> Uh and there were two reasons to do that. The first one is that so people
[44:41] that. The first one is that so people can use um the VC engine lib VC into
[44:45] can use um the VC engine lib VC into third party application. So a lot of
[44:47] third party application. So a lot of applications which are playing video on
[44:49] applications which are playing video on your phone or on your tablet are
[44:50] your phone or on your tablet are actually VC engine in it which is
[44:53] actually VC engine in it which is calling ffmpeg in it.
[44:54] calling ffmpeg in it. >> Yeah. Yeah. Um so that was one of the
[44:58] >> Yeah. Yeah. Um so that was one of the way to create uh one of the company I
[45:00] way to create uh one of the company I created which is doing consulting and
[45:02] created which is doing consulting and integration of those application where
[45:04] integration of those application where you integrate VC into third party
[45:06] you integrate VC into third party solutions um like inside game engines or
[45:09] solutions um like inside game engines or stuff like that. With DPL you couldn't
[45:11] stuff like that. With DPL you couldn't do that because that means you needed to
[45:13] do that because that means you needed to open source everything and those are for
[45:15] open source everything and those are for a lot of like commercial companies who
[45:17] a lot of like commercial companies who don't want that. So you can create a
[45:18] don't want that. So you can create a company with LGBL you can create a
[45:21] company with LGBL you can create a company around it. You can do a
[45:22] company around it. You can do a commercial thing. You don't have to open
[45:24] commercial thing. You don't have to open source it. So that's a big big leap.
[45:25] source it. So that's a big big leap. >> So you can play video in your game.
[45:27] >> So you can play video in your game. >> Yes.
[45:27] >> Yes. >> The the problem is I'm a game developer
[45:30] >> The the problem is I'm a game developer and I want to play some videos.
[45:32] and I want to play some videos. >> Mhm.
[45:32] >> Mhm. >> And I don't want to be forced to open
[45:33] >> And I don't want to be forced to open source the entire game just to play
[45:35] source the entire game just to play those videos. So that's where the the
[45:36] those videos. So that's where the the consulting business the libc LGPL
[45:40] consulting business the libc LGPL >> allows you to do that. The LGPL the
[45:42] >> allows you to do that. The LGPL the library GPL as it used to be known
[45:43] library GPL as it used to be known allows you to do that.
[45:44] allows you to do that. >> And FFmpeg is exactly the same. It forc
[45:48] >> And FFmpeg is exactly the same. It forc LGPL forces you to give back what you
[45:50] LGPL forces you to give back what you change on this component this library
[45:52] change on this component this library which is why it's library GPL and so you
[45:56] which is why it's library GPL and so you can use FFmpeg as LGPL into like any
[45:59] can use FFmpeg as LGPL into like any type of application even non-open source
[46:01] type of application even non-open source but you need to give back the
[46:03] but you need to give back the modification you did on FFmpeg same on
[46:05] modification you did on FFmpeg same on lib VLC is it limiting from an open
[46:08] lib VLC is it limiting from an open source perspective to go GPL because if
[46:11] source perspective to go GPL because if you if your uh library if your code is
[46:15] you if your uh library if your code is GPL it means means you're not you you're
[46:18] GPL it means means you're not you you're basically discouraging companies from
[46:21] basically discouraging companies from building a business around it right is
[46:22] building a business around it right is that is that fair to say
[46:23] that is that fair to say >> it depends on it depends on the company
[46:25] >> it depends on it depends on the company but the company whose business model
[46:27] but the company whose business model requires the source the application to
[46:29] requires the source the application to be closed source yes it's limited so
[46:32] be closed source yes it's limited so that's why for example I moved to LGB
[46:34] that's why for example I moved to LGB the second reason um is a bit more
[46:36] the second reason um is a bit more obscure is that the terms of condition
[46:38] obscure is that the terms of condition of the uh app store the Apple app store
[46:42] of the uh app store the Apple app store for iOS makes it very complex to have
[46:44] for iOS makes it very complex to have GPL application on it while it's easier
[46:47] GPL application on it while it's easier to have LGPL applications on it. So VC
[46:50] to have LGPL applications on it. So VC on Windows and on Mac uh and on Linux is
[46:53] on Windows and on Mac uh and on Linux is GPL, the core is LGPL, but on iOS uh the
[46:57] GPL, the core is LGPL, but on iOS uh the iPhone version and the Apple TV version
[46:59] iPhone version and the Apple TV version is a type of different license called
[47:02] is a type of different license called the MPL. Um and yes, I I went and
[47:05] the MPL. Um and yes, I I went and changed the license and it was a long
[47:07] changed the license and it was a long story.
[47:08] story. >> Yeah. So I I think basically to change
[47:10] >> Yeah. So I I think basically to change the license, you have to contact all the
[47:12] the license, you have to contact all the contributors. Yes, it's very important
[47:14] contributors. Yes, it's very important to understand that open-source projects
[47:16] to understand that open-source projects are what we call in the US copyright uh
[47:19] are what we call in the US copyright uh law joint work or in um civil law um
[47:23] law joint work or in um civil law um collective works or collaborative works
[47:25] collective works or collaborative works is that you work all together in terms
[47:28] is that you work all together in terms of the same goal and then it creates one
[47:30] of the same goal and then it creates one software which is one release um but the
[47:33] software which is one release um but the copyright is kept by all the
[47:35] copyright is kept by all the individuals. Some open source projects
[47:37] individuals. Some open source projects don't do that they force copyright
[47:38] don't do that they force copyright assignment but this is not what do we do
[47:40] assignment but this is not what do we do with communities. So everyone has
[47:43] with communities. So everyone has basically copyright on what they
[47:44] basically copyright on what they changed. Um and this copyright stays
[47:47] changed. Um and this copyright stays even if at the end your your
[47:50] even if at the end your your contribution was deleted because the new
[47:51] contribution was deleted because the new contribution was based on your previous
[47:53] contribution was based on your previous one. Right? So if you want to properly
[47:55] one. Right? So if you want to properly relic you need to find all the
[47:57] relic you need to find all the contributors and at that time I had to
[48:00] contributors and at that time I had to contact more than 350 people and
[48:03] contact more than 350 people and sometimes well they are just an email
[48:05] sometimes well they are just an email right so it's you need to actually track
[48:06] right so it's you need to actually track down I actually like travel to some
[48:09] down I actually like travel to some place to go somewhere that I was like
[48:11] place to go somewhere that I was like sorry that had found online to see to go
[48:14] sorry that had found online to see to go to their job and say well you licensed
[48:17] to their job and say well you licensed that can you do you want to change from
[48:19] that can you do you want to change from GPL to LGL most of the times they don't
[48:21] GPL to LGL most of the times they don't even care they wanted to help PLC but
[48:24] even care they wanted to help PLC but also So it brought me to very complex
[48:26] also So it brought me to very complex situation. I arrived to the work of a
[48:28] situation. I arrived to the work of a person who was a factory worker. Um and
[48:31] person who was a factory worker. Um and I said well I need to you to sign that
[48:35] I said well I need to you to sign that because it was his son who died who
[48:38] because it was his son who died who actually wrote the code right so I had
[48:40] actually wrote the code right so I had to explain all those type of open-source
[48:43] to explain all those type of open-source meaning and no I was not a company
[48:45] meaning and no I was not a company trying to rip out the two lines or five
[48:47] trying to rip out the two lines or five lines that that guy did but was useful
[48:50] lines that that guy did but was useful and the whole community agreed on that
[48:52] and the whole community agreed on that and he had no idea I was a factory
[48:54] and he had no idea I was a factory worker this and I was a lot younger
[48:56] worker this and I was a lot younger right like it was 14 years ago and like
[49:00] right like it was 14 years ago and like like I was almost in tears right it's we
[49:02] like I was almost in tears right it's we very difficult right we are talking
[49:04] very difficult right we are talking about lives of people and he explaining
[49:06] about lives of people and he explaining and we went to talk about the photo of
[49:08] and we went to talk about the photo of this guy right so it's important to do
[49:11] this guy right so it's important to do it right and to do it correctly um but
[49:14] it right and to do it correctly um but yes that means tracking down everything
[49:16] yes that means tracking down everything because every contribution works
[49:18] because every contribution works >> there are some projects who don't
[49:19] >> there are some projects who don't respect that and we do relicensing a bit
[49:22] respect that and we do relicensing a bit like aggressively
[49:23] like aggressively >> but as I said it destroy the whole heart
[49:25] >> but as I said it destroy the whole heart of the community because it's
[49:27] of the community because it's >> we only agree on this on the license so
[49:29] >> we only agree on this on the license so That's important.
[49:31] That's important. >> I would emphasize the community is such
[49:33] >> I would emphasize the community is such a wide ranging group of people. There's
[49:36] a wide ranging group of people. There's people in a Syrian war zone with
[49:38] people in a Syrian war zone with electricity part-time. There's there's
[49:42] electricity part-time. There's there's all people from all walks of life, rich,
[49:44] all people from all walks of life, rich, poor, young, old. Um, so it's quite
[49:48] poor, young, old. Um, so it's quite remarkable to get, you know, a group of
[49:51] remarkable to get, you know, a group of people aligned on something and now
[49:54] people aligned on something and now that's an achievement in itself.
[49:55] that's an achievement in itself. >> Yeah, it's incredible. And a lot of them
[49:57] >> Yeah, it's incredible. And a lot of them are introverts. So you coming coming to
[50:00] are introverts. So you coming coming to find them and getting them and getting
[50:02] find them and getting them and getting them to answer an email might be quite
[50:04] them to answer an email might be quite quite difficult.
[50:05] quite difficult. >> Most of us are introverts, right? You
[50:07] >> Most of us are introverts, right? You you need to be more precise. You are
[50:09] you need to be more precise. You are extremely introverts, extremely
[50:10] extremely introverts, extremely extremely introverts and introverts,
[50:12] extremely introverts and introverts, right? It's just like a whole spectrum
[50:14] right? It's just like a whole spectrum of different people. It doesn't matter.
[50:16] of different people. It doesn't matter. >> The important is is your code good? Is
[50:19] >> The important is is your code good? Is your code great? Is your technology
[50:21] your code great? Is your technology great? We care about excellent code. We
[50:24] great? We care about excellent code. We don't care who you are. Sorry. It's just
[50:26] don't care who you are. Sorry. It's just like we have no idea to check. We cannot
[50:28] like we have no idea to check. We cannot check, right? Like maybe you're a dog. I
[50:30] check, right? Like maybe you're a dog. I don't care, right? I don't care where
[50:32] don't care, right? I don't care where you come from. I need to look at your
[50:34] you come from. I need to look at your code. And this is important because
[50:36] code. And this is important because people don't understand that. And they
[50:38] people don't understand that. And they come to the community and send them some
[50:39] come to the community and send them some patches and they get rejected and they
[50:42] patches and they get rejected and they don't like that because I mean just like
[50:45] don't like that because I mean just like sorry it's not up to our standards. Oh
[50:47] sorry it's not up to our standards. Oh yeah, but I'm engineer at this very
[50:49] yeah, but I'm engineer at this very large company in in Italy, in Germany,
[50:52] large company in in Italy, in Germany, in the US. We don't care. We care about
[50:55] in the US. We don't care. We care about the quality of your code because this is
[50:57] the quality of your code because this is what defines our community and which
[50:59] what defines our community and which means that we have a lot of people who
[51:01] means that we have a lot of people who contribute who are some very different
[51:03] contribute who are some very different backgrounds and and and very introverts.
[51:05] backgrounds and and and very introverts. Sure. But that's okay, right? So one one
[51:08] Sure. But that's okay, right? So one one of the legends of the community is uh of
[51:11] of the legends of the community is uh of course Lionus Torvalds who uh created
[51:14] course Lionus Torvalds who uh created Linux and is a longtime maintainer of
[51:17] Linux and is a longtime maintainer of the Linux kernel. As the legend goes, he
[51:20] the Linux kernel. As the legend goes, he can be pretty harsh on this meritocratic
[51:22] can be pretty harsh on this meritocratic process of reviewing the code and saying
[51:24] process of reviewing the code and saying it's not good enough. Can can you just
[51:26] it's not good enough. Can can you just speak to to the legend of Listo Wallet?
[51:29] speak to to the legend of Listo Wallet? Linux is one of a kind, right? And and I
[51:34] Linux is one of a kind, right? And and I would even go and say that what he did
[51:35] would even go and say that what he did on Git is more interesting than what he
[51:37] on Git is more interesting than what he did on the Linux kernel.
[51:39] did on the Linux kernel. >> He's very harsh, but what people don't
[51:42] >> He's very harsh, but what people don't see is usually when he's harsh to it's
[51:44] see is usually when he's harsh to it's people who are maintainer of part of the
[51:47] people who are maintainer of part of the kernel, right? So they know him, right?
[51:49] kernel, right? So they know him, right? So he's not very harsh like that to
[51:51] So he's not very harsh like that to everyone. The thing is what he created
[51:53] everyone. The thing is what he created in his room is basically powering every
[51:57] in his room is basically powering every server online, right? Even at Microsoft
[51:59] server online, right? Even at Microsoft cloud called Asia, I'm quite sure 70 80%
[52:03] cloud called Asia, I'm quite sure 70 80% of the servers are running Linux. All
[52:05] of the servers are running Linux. All your Android phones are running Linux.
[52:07] your Android phones are running Linux. What he did um with the power of a of a
[52:10] What he did um with the power of a of a boot sure is amazing. Um and yes the
[52:14] boot sure is amazing. Um and yes the quality of the Linux kernel is very high
[52:17] quality of the Linux kernel is very high and yes it's difficult but we cannot
[52:21] and yes it's difficult but we cannot compromise on that we cannot compromise
[52:23] compromise on that we cannot compromise on quality because in the end and you
[52:26] on quality because in the end and you have to understand that is the core
[52:28] have to understand that is the core community of VC is five people the core
[52:31] community of VC is five people the core community of ffmpeg is 10 to 15 and we
[52:35] community of ffmpeg is 10 to 15 and we are the ones who are going to maintain
[52:36] are the ones who are going to maintain your code right because 1,000
[52:39] your code right because 1,000 contributors in the timeline and just 10
[52:41] contributors in the timeline and just 10 staying it's 1% chance that someone
[52:44] staying it's 1% chance that someone comes and stays 1%. So you will have
[52:48] comes and stays 1%. So you will have change of job, change of wives, you have
[52:49] change of job, change of wives, you have children, you have accident in life,
[52:51] children, you have accident in life, you're going to change jobs, whatever.
[52:53] you're going to change jobs, whatever. You're not going to come back. It's most
[52:56] You're not going to come back. It's most likely. So we are the one going to m
[52:59] likely. So we are the one going to m maintain your code. It needs to be
[53:01] maintain your code. It needs to be maintainable. It needs to be excellent.
[53:05] maintainable. It needs to be excellent. And yes, sometimes that means that you
[53:06] And yes, sometimes that means that you need to rework your work because it was
[53:08] need to rework your work because it was good, but it's not excellent. And we
[53:10] good, but it's not excellent. And we need excellence because we have very few
[53:13] need excellence because we have very few to maintain something that is critical
[53:14] to maintain something that is critical for the whole.
[53:15] for the whole. >> But we should also mention that there is
[53:17] >> But we should also mention that there is some some spiciness, some harshness to
[53:19] some some spiciness, some harshness to the language that's sometimes used when
[53:22] the language that's sometimes used when you're uh keeping this high bar of
[53:24] you're uh keeping this high bar of excellence. Uh is there something to say
[53:26] excellence. Uh is there something to say to that?
[53:27] to that? >> It's it's true, right? Um it's also the
[53:29] >> It's it's true, right? Um it's also the fact that for example what we're doing
[53:31] fact that for example what we're doing is lowlevel. It's extremely technical.
[53:33] is lowlevel. It's extremely technical. uh you get into this community. the tone
[53:36] uh you get into this community. the tone gets um very like a type of it's a
[53:40] gets um very like a type of it's a subculture right so people who arrive
[53:42] subculture right so people who arrive from the external are basically not
[53:44] from the external are basically not known to the subculture most of those
[53:47] known to the subculture most of those people around fmp and VC we do videoand
[53:49] people around fmp and VC we do videoand dev days v every every year they are so
[53:52] dev days v every every year they are so fun in real life and they love it but
[53:54] fun in real life and they love it but it's true that you're online and
[53:56] it's true that you're online and sometimes like the the tone you don't
[53:58] sometimes like the the tone you don't realize how it is uh but that's okay
[54:01] realize how it is uh but that's okay it's a culture I mean you get this in
[54:02] it's a culture I mean you get this in the gaming culture there's pretty harsh
[54:04] the gaming culture there's pretty harsh intense It's the way people communicate
[54:07] intense It's the way people communicate and it's everyone understands that the
[54:09] and it's everyone understands that the way you show love and respect just looks
[54:11] way you show love and respect just looks different in different communities.
[54:12] different in different communities. Sometimes people it depends if it's a
[54:15] Sometimes people it depends if it's a book club usually people are going to be
[54:18] book club usually people are going to be much sweeter. if it's a open source
[54:21] much sweeter. if it's a open source project that's very high stakes and used
[54:23] project that's very high stakes and used by millions of people
[54:24] by millions of people >> but it's very not often insults that you
[54:27] >> but it's very not often insults that you see for example in the gaming right so
[54:29] see for example in the gaming right so Linux tone is a bit unusual even for the
[54:32] Linux tone is a bit unusual even for the open source community it's more like
[54:34] open source community it's more like it's more harsh on the result saying no
[54:36] it's more harsh on the result saying no this is not good this is crap those type
[54:38] this is not good this is crap those type of things that you will see try not to
[54:40] of things that you will see try not to make it about the person make it about
[54:42] make it about the person make it about the code yes
[54:43] the code yes >> it's very very matter of fact and I
[54:44] >> it's very very matter of fact and I think you've got to look at it in terms
[54:46] think you've got to look at it in terms of you know the famous ffmpeg is
[54:48] of you know the famous ffmpeg is developed almost entirely by volunteers.
[54:50] developed almost entirely by volunteers. And that's true. And you've got to
[54:51] And that's true. And you've got to imagine someone's done a hard day's work
[54:53] imagine someone's done a hard day's work at their day job. They come home,
[54:56] at their day job. They come home, you know, tursess might be a thing, you
[54:58] you know, tursess might be a thing, you know, and that's not something to take
[55:00] know, and that's not something to take personally.
[55:01] personally. You're tired, you're busy, but you still
[55:03] You're tired, you're busy, but you still care about this open source stuff,
[55:05] care about this open source stuff, >> but you may not be able to explain and
[55:07] >> but you may not be able to explain and and handhold someone on every subtle
[55:09] and handhold someone on every subtle detail.
[55:10] detail. >> And also, you have to realize that most
[55:13] >> And also, you have to realize that most people don't speak English as native
[55:15] people don't speak English as native language. And and this is especially for
[55:18] language. And and this is especially for for open source projects like FFmpeg and
[55:20] for open source projects like FFmpeg and VC which are mostly centered out of
[55:23] VC which are mostly centered out of Europe sometimes like um people who are
[55:27] Europe sometimes like um people who are from the US or or just like are very not
[55:30] from the US or or just like are very not happy about the tone but most of the
[55:32] happy about the tone but most of the time it's also like they don't know
[55:33] time it's also like they don't know better right it's difficult language is
[55:35] better right it's difficult language is English is a difficult language there is
[55:37] English is a difficult language there is so many subtilities and tones and so on
[55:39] so many subtilities and tones and so on that you don't have right so um often
[55:41] that you don't have right so um often it's also difficult in those type of
[55:43] it's also difficult in those type of community about like different cultures
[55:45] community about like different cultures and languages
[55:46] and languages So, as the legend goes, JB, you uh
[55:49] So, as the legend goes, JB, you uh repeatedly turned down millions of
[55:51] repeatedly turned down millions of dollars to uh keep VC open source, free
[55:56] dollars to uh keep VC open source, free for everyone without ads.
[56:00] for everyone without ads. So, take me through the reasoning behind
[56:02] So, take me through the reasoning behind that decision of leaving millions of
[56:05] that decision of leaving millions of dollars on the table.
[56:06] dollars on the table. >> Yeah, that's like almost a meme, right?
[56:08] >> Yeah, that's like almost a meme, right? On on Reddit,
[56:09] On on Reddit, >> there literally is a meme on Reddit.
[56:11] >> there literally is a meme on Reddit. >> Nine gag and Yeah. Yeah. I see there's
[56:14] >> Nine gag and Yeah. Yeah. I see there's you
[56:16] you looking like a wizard in the in the VC
[56:18] looking like a wizard in the in the VC hat on Reddit. This is uh JB the creator
[56:22] hat on Reddit. This is uh JB the creator VLC media player. He refused tens of
[56:24] VLC media player. He refused tens of millions of dollars in order to keep VCs
[56:27] millions of dollars in order to keep VCs free. Thanks, Jean Baptisk. You can even
[56:32] free. Thanks, Jean Baptisk. You can even summon him on Reddit.
[56:33] summon him on Reddit. >> Yeah. And usually if you see right, it's
[56:35] >> Yeah. And usually if you see right, it's usually like people tag me, right? And
[56:37] usually like people tag me, right? And and then there is me and like I say,
[56:39] and then there is me and like I say, good morning. I got 24K up votes, which
[56:41] good morning. I got 24K up votes, which is great, right? My karma on Reddit is
[56:43] is great, right? My karma on Reddit is amazing at least on that account. Um, so
[56:47] amazing at least on that account. Um, so the question is needs to be answered
[56:50] the question is needs to be answered first. What is a story about VC? Right.
[56:52] first. What is a story about VC? Right. Um, because yes, this is true. I refuse
[56:57] Um, because yes, this is true. I refuse dozens of millions of dollars. Yes,
[56:59] dozens of millions of dollars. Yes, several time. Yes, I could be a
[57:00] several time. Yes, I could be a multi-millionaire and be somewhere on
[57:02] multi-millionaire and be somewhere on the beach. Um, but I did not do it
[57:05] the beach. Um, but I did not do it because I thought it was not moral and
[57:08] because I thought it was not moral and it was not the right thing to do. And
[57:10] it was not the right thing to do. And this is very important for myself is to
[57:12] this is very important for myself is to be like I work for the greater good. I
[57:15] be like I work for the greater good. I work for people and I don't want it's
[57:17] work for people and I don't want it's not just by my by myself. But the reason
[57:20] not just by my by myself. But the reason is also because I I did not feel that
[57:22] is also because I I did not feel that I'm completely legitimate to do that.
[57:24] I'm completely legitimate to do that. And let me explain you why. VC story is
[57:27] And let me explain you why. VC story is a very weird story. In France we have
[57:31] a very weird story. In France we have university and we have a type of top
[57:33] university and we have a type of top colleges. Um and those top of excellency
[57:36] colleges. Um and those top of excellency schools are engineering school, business
[57:39] schools are engineering school, business schools and basically lawyers and
[57:42] schools and basically lawyers and medical, right? But they're outside of
[57:44] medical, right? But they're outside of university. And in order to enter those,
[57:46] university. And in order to enter those, you spend two years working like crazy
[57:50] you spend two years working like crazy mass physics to enter those best
[57:51] mass physics to enter those best engineering school. One of the school is
[57:54] engineering school. One of the school is called the EKL Central Par has changed
[57:56] called the EKL Central Par has changed name since, but it was called the EK
[57:57] name since, but it was called the EK Central Par. And because it was central,
[58:00] Central Par. And because it was central, they had to move it because it was too
[58:02] they had to move it because it was too small after the World War II. And and
[58:04] small after the World War II. And and they moved it. They wanted to move it to
[58:06] they moved it. They wanted to move it to the central of France in a place called
[58:08] the central of France in a place called Clare Ferron. And the alumni decided
[58:10] Clare Ferron. And the alumni decided that this was not okay, right? It is a
[58:13] that this was not okay, right? It is a the school that Eel, right, the one who
[58:16] the school that Eel, right, the one who did the Eel Tower attended to, right? So
[58:18] did the Eel Tower attended to, right? So they said, "No, no, we are amazing great
[58:20] they said, "No, no, we are amazing great school. We cannot do that." And so they
[58:21] school. We cannot do that." And so they bought a piece of land south of Paris,
[58:23] bought a piece of land south of Paris, uh, very near Paris. Um and it was a
[58:27] uh, very near Paris. Um and it was a campus managed by a nonprofit of the
[58:30] campus managed by a nonprofit of the alumni. Okay.
[58:32] alumni. Okay. Because of that everything on the campus
[58:34] Because of that everything on the campus was managed by students. The university
[58:36] was managed by students. The university did nothing right. So uh radio, uh TV,
[58:40] did nothing right. So uh radio, uh TV, uh supermarket, uh library, um defining
[58:44] uh supermarket, uh library, um defining who was going into which rooms. Um
[58:46] who was going into which rooms. Um everything was managed by the student.
[58:48] everything was managed by the student. >> That's amazing. That's an amazing
[58:50] >> That's amazing. That's an amazing experiment that it all all didn't go to
[58:52] experiment that it all all didn't go to hell quickly. it somehow flourished. It
[58:56] hell quickly. it somehow flourished. It worked great and I learned so much in my
[58:59] worked great and I learned so much in my life doing those side activities, right?
[59:01] life doing those side activities, right? Because you're you're 22 and you need to
[59:03] Because you're you're 22 and you need to run your compass else you don't have
[59:04] run your compass else you don't have electricity, right? So you care about
[59:06] electricity, right? So you care about that, right? But anyway, in the 80s um
[59:09] that, right? But anyway, in the 80s um they did a full experiment of deploying
[59:11] they did a full experiment of deploying a network uh mostly sponsored by IBM and
[59:14] a network uh mostly sponsored by IBM and 3COM which was a token ring network. So
[59:17] 3COM which was a token ring network. So token ring is something that probably
[59:20] token ring is something that probably almost no one knows about anymore. It's
[59:23] almost no one knows about anymore. It's a networking technology where you don't
[59:25] a networking technology where you don't have routters, right? Everyone is
[59:27] have routters, right? Everyone is linked. Um, it's like really a ring and
[59:30] linked. Um, it's like really a ring and when you want to send a message, you
[59:31] when you want to send a message, you talk to your neighbor who's going to put
[59:33] talk to your neighbor who's going to put the message to the next one who's going
[59:35] the message to the next one who's going to put the things to the next one. In
[59:37] to put the things to the next one. In terms of ring, the issue with token ring
[59:40] terms of ring, the issue with token ring is of course is that it's very slow
[59:42] is of course is that it's very slow because every computer on the network
[59:44] because every computer on the network needs to open the message, see if it's
[59:47] needs to open the message, see if it's okay. Is it for me? No, it's not. And
[59:48] okay. Is it for me? No, it's not. And then send it back like a token which is
[59:50] then send it back like a token which is traveling around the ring. In the 80s,
[59:54] traveling around the ring. In the 80s, you're doing some TNET and sending mails
[59:57] you're doing some TNET and sending mails as university. That's okay, right? But
[59:59] as university. That's okay, right? But starts the '9s in the '9s um and start
[01:00:03] starts the '9s in the '9s um and start video games. And when you have high
[01:00:05] video games. And when you have high latency in video games, basically you
[01:00:07] latency in video games, basically you die, right? So in 1994, 1995 around Doom
[01:00:10] die, right? So in 1994, 1995 around Doom and Duke Nukem coming around, they want
[01:00:12] and Duke Nukem coming around, they want a faster network. So the students go and
[01:00:14] a faster network. So the students go and see the university and say, you know
[01:00:16] see the university and say, you know what, we want a faster network. We need
[01:00:18] what, we want a faster network. We need to work also play video games. Um, and
[01:00:22] to work also play video games. Um, and the university tells them that
[01:00:24] the university tells them that basically, oh, I'm sorry, we cannot help
[01:00:26] basically, oh, I'm sorry, we cannot help you because you understand the compass
[01:00:28] you because you understand the compass is not ours. You manage it. So do
[01:00:31] is not ours. You manage it. So do something and you should see some
[01:00:33] something and you should see some basically partners of this university
[01:00:36] basically partners of this university and basically go away and they go and
[01:00:39] and basically go away and they go and they actually go and see um the the the
[01:00:41] they actually go and see um the the the CIO of uh Buig which is a large uh
[01:00:44] CIO of uh Buig which is a large uh French company and who's doing some TVs
[01:00:47] French company and who's doing some TVs in France and he says well you know what
[01:00:50] in France and he says well you know what the future of video is satellite well
[01:00:52] the future of video is satellite well today we know it's not but at least it
[01:00:54] today we know it's not but at least it was a good idea in 1995 the first of
[01:00:57] was a good idea in 1995 the first of satellite dish and he says that instead
[01:00:59] satellite dish and he says that instead of having like one satellite dish and a
[01:01:01] of having like one satellite dish and a big decoder for each of the students
[01:01:04] big decoder for each of the students which are 1,500. What about you build
[01:01:07] which are 1,500. What about you build like you put an enormous dish and only
[01:01:11] like you put an enormous dish and only one decoder and you send the video
[01:01:13] one decoder and you send the video directly on the network.
[01:01:15] directly on the network. >> Mhm.
[01:01:15] >> Mhm. >> And that required a very fast network.
[01:01:17] >> And that required a very fast network. Today it's obvious but at the time was
[01:01:20] Today it's obvious but at the time was like the first to do video streaming. So
[01:01:22] like the first to do video streaming. So they built this project which was called
[01:01:24] they built this project which was called network 2000.
[01:01:26] network 2000. Of course, right? We are in the '9s,
[01:01:28] Of course, right? We are in the '9s, right? Everything is futuristic is
[01:01:30] right? Everything is futuristic is called 2000.
[01:01:31] called 2000. >> 2000. Yeah.
[01:01:32] >> 2000. Yeah. >> And so they do the network 2000 project.
[01:01:35] >> And so they do the network 2000 project. It's completely hacked. It crashes after
[01:01:37] It's completely hacked. It crashes after 45 seconds. That's okay. The demo is 40
[01:01:39] 45 seconds. That's okay. The demo is 40 seconds. It leaks memory. That's okay.
[01:01:41] seconds. It leaks memory. That's okay. They put 64 megabyte of RAM instead of
[01:01:43] They put 64 megabyte of RAM instead of the eight or 16 you have. And the demo
[01:01:46] the eight or 16 you have. And the demo should have stopped there. And that was
[01:01:47] should have stopped there. And that was the network 2000 project by the
[01:01:49] the network 2000 project by the students.
[01:01:49] students. >> What was the format of the video that
[01:01:51] >> What was the format of the video that they had to work with? Meek 2 because
[01:01:52] they had to work with? Meek 2 because satellite is meek 2 TS for transport,
[01:01:55] satellite is meek 2 TS for transport, meek 2 video and meek 2 audio at that
[01:01:59] meek 2 video and meek 2 audio at that time and the project should have stopped
[01:02:01] time and the project should have stopped there. Everyone was happy. They had like
[01:02:03] there. Everyone was happy. They had like amazing ATM network at 155 55 megabits
[01:02:07] amazing ATM network at 155 55 megabits pers. They had probably one of the best
[01:02:09] pers. They had probably one of the best network in Europe at that time and they
[01:02:11] network in Europe at that time and they stopped the project. Six months or a
[01:02:13] stopped the project. Six months or a year later, two students arrive and say,
[01:02:15] year later, two students arrive and say, "Well, you know what? Maybe other people
[01:02:17] "Well, you know what? Maybe other people care about video streamed on a local
[01:02:19] care about video streamed on a local network and they create the video
[01:02:22] network and they create the video project video." And one of them is
[01:02:25] project video." And one of them is called Kristoff Masio that is a good
[01:02:27] called Kristoff Masio that is a good friend of both Kieran and me and they
[01:02:29] friend of both Kieran and me and they start the project. It's not even open
[01:02:31] start the project. It's not even open source yet. and they spend around 3
[01:02:34] source yet. and they spend around 3 years to get the school to agree to make
[01:02:36] years to get the school to agree to make it open source because the university
[01:02:39] it open source because the university wanted to get because of the IP and
[01:02:42] wanted to get because of the IP and copyright of the student wanted to
[01:02:44] copyright of the student wanted to basically monetize these ME2 decoders.
[01:02:47] basically monetize these ME2 decoders. >> Just to be clear, so what was the main
[01:02:48] >> Just to be clear, so what was the main application streaming on a local
[01:02:50] application streaming on a local network?
[01:02:50] network? >> It was streaming on a local network.
[01:02:52] >> It was streaming on a local network. >> By the way, this is like to state the
[01:02:54] >> By the way, this is like to state the obvious. This is before YouTube. This is
[01:02:56] obvious. This is before YouTube. This is before
[01:02:56] before >> 10 years before YouTube. You have a
[01:02:58] >> 10 years before YouTube. You have a Pentium 60 or 75, right? you the main
[01:03:01] Pentium 60 or 75, right? you the main machine was 488 86 DX at 33 MHz, right?
[01:03:04] machine was 488 86 DX at 33 MHz, right? Bear in
[01:03:04] Bear in >> mind tele television was the main form
[01:03:06] >> mind tele television was the main form of video at the time. You could get new
[01:03:08] of video at the time. You could get new channels in the ' 90s. Having even one
[01:03:10] channels in the ' 90s. Having even one new channel when you you grew up with
[01:03:12] new channel when you you grew up with four channels, having a fifth or a sixth
[01:03:14] four channels, having a fifth or a sixth was a big deal. And so having this
[01:03:16] was a big deal. And so having this satellite service with, you know,
[01:03:19] satellite service with, you know, dozens, even hundreds of channels was so
[01:03:21] dozens, even hundreds of channels was so groundbreaking, especially because this
[01:03:22] groundbreaking, especially because this is university where you had ton of
[01:03:24] is university where you had ton of different nationalities, right? So there
[01:03:26] different nationalities, right? So there was a ton of people who wanted so they
[01:03:29] was a ton of people who wanted so they in the end they had like several dishes
[01:03:30] in the end they had like several dishes on different type of satellite right
[01:03:32] on different type of satellite right because for example a lot of people were
[01:03:34] because for example a lot of people were coming from the Maghreb or the Middle
[01:03:35] coming from the Maghreb or the Middle East and and they they so they went to
[01:03:38] East and and they they so they went to different type of uh uh of satellites.
[01:03:40] different type of uh uh of satellites. Anyway the solution worked great and
[01:03:43] Anyway the solution worked great and they started the video project. The
[01:03:45] they started the video project. The villain project has several and some are
[01:03:48] villain project has several and some are completely crazy solutions like one how
[01:03:51] completely crazy solutions like one how to create multiccast on an uniccast
[01:03:54] to create multiccast on an uniccast network but let's not come to that it's
[01:03:56] network but let's not come to that it's too too complex but video client part is
[01:03:59] too too complex but video client part is what became VC.
[01:04:01] what became VC. >> Mhm. actually they basically strong
[01:04:04] >> Mhm. actually they basically strong armed the the university to force it to
[01:04:06] armed the the university to force it to open source because the university did
[01:04:08] open source because the university did not understand that and in 2001 it's
[01:04:10] not understand that and in 2001 it's still early but basically yes the
[01:04:12] still early but basically yes the university agreed early 2001 to make it
[01:04:15] university agreed early 2001 to make it open source I joined the project in 2003
[01:04:18] open source I joined the project in 2003 because that's when I joined the
[01:04:19] because that's when I joined the university so the first thing is I'm not
[01:04:23] university so the first thing is I'm not the one who created VC because actually
[01:04:25] the one who created VC because actually no one did right just kind of naturally
[01:04:27] no one did right just kind of naturally emerged from the video land project and
[01:04:29] emerged from the video land project and we should mention that like again you
[01:04:31] we should mention that like again you you said it Just but to make it clear V
[01:04:35] you said it Just but to make it clear V is what it became was at the time is a
[01:04:37] is what it became was at the time is a set of technologies around video the VC
[01:04:41] set of technologies around video the VC what you call the client that's the
[01:04:43] what you call the client that's the thing that most normies
[01:04:47] thing that most normies uh
[01:04:47] uh >> that is correct think of like as the
[01:04:49] >> that is correct think of like as the thing which is like the thing that pops
[01:04:50] thing which is like the thing that pops up when you click on a video and you
[01:04:52] up when you click on a video and you play it. So I arrived in 2003 and then I
[01:04:55] play it. So I arrived in 2003 and then I will create u the open-source um
[01:04:59] will create u the open-source um nonprofit organization called video and
[01:05:01] nonprofit organization called video and I took everything out of the university
[01:05:03] I took everything out of the university to created a a nonprofit project and
[01:05:06] to created a a nonprofit project and something sustainable. It's yes it's
[01:05:08] something sustainable. It's yes it's true that I spent more time than anyone
[01:05:10] true that I spent more time than anyone on VC and video land that is sure um but
[01:05:13] on VC and video land that is sure um but it's a continuity of a previous project
[01:05:16] it's a continuity of a previous project video land the student project which is
[01:05:18] video land the student project which is a competity of the network 2000 project
[01:05:20] a competity of the network 2000 project which is a continuity of that in that
[01:05:22] which is a continuity of that in that I'm sure there's moments along the way
[01:05:24] I'm sure there's moments along the way there you were thinking of like what is
[01:05:27] there you were thinking of like what is the future of this from an open source
[01:05:29] the future of this from an open source perspective cuz as as the internet is
[01:05:31] perspective cuz as as the internet is blowing up and there's companies I mean
[01:05:34] blowing up and there's companies I mean for people who don't remember like
[01:05:36] for people who don't remember like there's companies is making huge amounts
[01:05:39] there's companies is making huge amounts of money
[01:05:39] of money >> and I can tell you that in 2005 the
[01:05:42] >> and I can tell you that in 2005 the project should have died. Um and I made
[01:05:46] project should have died. Um and I made it to continue the project. At some
[01:05:49] it to continue the project. At some point we were only two active
[01:05:50] point we were only two active developers. Um and I thought it was
[01:05:54] developers. Um and I thought it was great technology and was useful and it
[01:05:56] great technology and was useful and it will be useful and and and I made that
[01:05:58] will be useful and and and I made that my my my life and my my time. Um and I
[01:06:02] my my my life and my my time. Um and I made that grow from a few hundreds of
[01:06:05] made that grow from a few hundreds of thousands of users, millions of users to
[01:06:07] thousands of users, millions of users to what we have now, which is probably
[01:06:09] what we have now, which is probably billions of uh version of VC around the
[01:06:12] billions of uh version of VC around the world and used everywhere. Um so that's
[01:06:16] world and used everywhere. Um so that's a bit the story of VC. There is ton of
[01:06:18] a bit the story of VC. There is ton of very funny story around that. Many
[01:06:21] very funny story around that. Many people from around the world working on
[01:06:24] people from around the world working on it like you said in Syria or middle of
[01:06:26] it like you said in Syria or middle of nowhere in India. But along the way, I
[01:06:28] nowhere in India. But along the way, I got several offers which were either to
[01:06:31] got several offers which were either to bundle uh toolbarss, right? You remember
[01:06:34] bundle uh toolbarss, right? You remember those horrible toolbars which were
[01:06:36] those horrible toolbars which were basically spyware or changing your web
[01:06:39] basically spyware or changing your web browser or your search engine or even
[01:06:43] browser or your search engine or even like advertisement inside VC and I
[01:06:46] like advertisement inside VC and I didn't like that. Right? I am and and
[01:06:48] didn't like that. Right? I am and and people don't understand that is not I'm
[01:06:50] people don't understand that is not I'm not against money, right? I'm very happy
[01:06:52] not against money, right? I'm very happy to make money. I created several
[01:06:54] to make money. I created several startups and one I hope that is going to
[01:06:57] startups and one I hope that is going to work very well. It's the fact that I
[01:06:59] work very well. It's the fact that I believe that you need to win money
[01:07:02] believe that you need to win money ethically. there is the right way of
[01:07:04] ethically. there is the right way of doing that and and doing sneaky
[01:07:06] doing that and and doing sneaky advertisement or stealing data is not
[01:07:09] advertisement or stealing data is not the correct way right for example if
[01:07:11] the correct way right for example if Netflix arrived at some point and say
[01:07:12] Netflix arrived at some point and say well we want to put Netflix inside VC
[01:07:15] well we want to put Netflix inside VC probably the story would have been
[01:07:16] probably the story would have been different right but they didn't the only
[01:07:18] different right but they didn't the only people who came to us were
[01:07:21] people who came to us were >> shady ads company and if I do that right
[01:07:23] >> shady ads company and if I do that right I would have a ton of money right and
[01:07:25] I would have a ton of money right and then 3 years later
[01:07:27] then 3 years later >> project is gone right someone forks it
[01:07:29] >> project is gone right someone forks it and and something else happens so it's
[01:07:32] and and something else happens so it's not even necessarily ads or any of that.
[01:07:34] not even necessarily ads or any of that. It's the shadiness of the dishonesty of
[01:07:36] It's the shadiness of the dishonesty of the So, you had a good radar. You had a
[01:07:39] the So, you had a good radar. You had a good threshold of like, no, this
[01:07:42] good threshold of like, no, this compromises the spirit of what this is
[01:07:44] compromises the spirit of what this is supposed to represent. But also, it's
[01:07:46] supposed to represent. But also, it's it's for me, right? Like very selfishly,
[01:07:49] it's for me, right? Like very selfishly, I need to go to bed at night and be
[01:07:51] I need to go to bed at night and be happy about what I've done, right? Maybe
[01:07:53] happy about what I've done, right? Maybe it's my upbringing. Um uh maybe it's my
[01:07:56] it's my upbringing. Um uh maybe it's my parents fault or or whatever, right? But
[01:07:57] parents fault or or whatever, right? But I believe there is right and wrong,
[01:08:00] I believe there is right and wrong, right? And this was the right decision
[01:08:04] right? And this was the right decision at the time. It still is. I want to be
[01:08:06] at the time. It still is. I want to be proud of what I've been doing. And
[01:08:10] proud of what I've been doing. And like if I had sold out, I would have
[01:08:12] like if I had sold out, I would have betrayed so many other people work.
[01:08:14] betrayed so many other people work. Yeah. Well, I should say me and uh most
[01:08:17] Yeah. Well, I should say me and uh most the internet, thank you for that
[01:08:18] the internet, thank you for that decision. It's inspiring.
[01:08:21] decision. It's inspiring. uh for others uh I think that um that
[01:08:25] uh for others uh I think that um that are pushing the open source movement
[01:08:27] are pushing the open source movement forward that it's okay to do these kinds
[01:08:30] forward that it's okay to do these kinds of huge sacrifices if you believe it's
[01:08:33] of huge sacrifices if you believe it's right and I think in that case it was
[01:08:35] right and I think in that case it was right and it was the reason that BLC
[01:08:36] right and it was the reason that BLC became as successful as it was cuz it it
[01:08:39] became as successful as it was cuz it it it's an embodiment it's a symbol of like
[01:08:43] it's an embodiment it's a symbol of like you know freedom and what the open
[01:08:44] you know freedom and what the open source community can create
[01:08:46] source community can create >> yeah and be a service for so many people
[01:08:48] >> yeah and be a service for so many people around the world and this is important
[01:08:50] around the world and this is important >> we should emphasize In the 2000s, it was
[01:08:52] >> we should emphasize In the 2000s, it was really normal to download a program and
[01:08:54] really normal to download a program and it secretly installed some spyware.
[01:08:56] it secretly installed some spyware. >> It was it was buried in very faint text
[01:08:59] >> It was it was buried in very faint text or in the license text box that nobody
[01:09:02] or in the license text box that nobody reads that at the bottom. Oh, I will be
[01:09:03] reads that at the bottom. Oh, I will be installing this toolbar and changing all
[01:09:05] installing this toolbar and changing all these things. And it was it was very
[01:09:06] these things. And it was it was very common to have to, you know, you install
[01:09:09] common to have to, you know, you install a program to do something at the time of
[01:09:10] a program to do something at the time of any sort
[01:09:11] any sort >> to put yourself in the mind of a
[01:09:12] >> to put yourself in the mind of a developer at that time.
[01:09:14] developer at that time. >> I think it's very easy to everybody
[01:09:17] >> I think it's very easy to everybody listening to this. It's very easy at
[01:09:19] listening to this. It's very easy at that time to convince yourself to take a
[01:09:22] that time to convince yourself to take a few thousand dollars, a few thousand
[01:09:24] few thousand dollars, a few thousand dollars to do it.
[01:09:26] dollars to do it. >> To to say no to much more money takes
[01:09:31] >> To to say no to much more money takes guts and takes vision. The last offer I
[01:09:35] guts and takes vision. The last offer I had was obscene. And they say, "Yeah,
[01:09:38] had was obscene. And they say, "Yeah, but imagine with all that money you
[01:09:40] but imagine with all that money you could build something new, open source,
[01:09:43] could build something new, open source, right?" It was like the mind trick was
[01:09:46] right?" It was like the mind trick was it was difficult but for me it was just
[01:09:48] it was difficult but for me it was just like no this doesn't work like that or
[01:09:50] like no this doesn't work like that or this is not the right thing so I don't
[01:09:52] this is not the right thing so I don't do it
[01:09:54] do it >> and again right it's not that I don't
[01:09:56] >> and again right it's not that I don't like money or or whatever it's just like
[01:09:58] like money or or whatever it's just like it wasn't right
[01:10:00] it wasn't right well once again thank you from me and
[01:10:02] well once again thank you from me and from the rest of the internet let me
[01:10:05] from the rest of the internet let me talk a little bit more about the open
[01:10:06] talk a little bit more about the open source movement about the fact that as
[01:10:09] source movement about the fact that as you say over and over and over and over
[01:10:11] you say over and over and over and over ffmpeg
[01:10:13] ffmpeg uh is and many open source projects are
[01:10:16] uh is and many open source projects are built by volunteers. So uh there's a bit
[01:10:19] built by volunteers. So uh there's a bit of drama recently uh Karen on the
[01:10:22] of drama recently uh Karen on the interwebs on Twitter. Uh you have a
[01:10:26] interwebs on Twitter. Uh you have a spicy style on Twitter that I think
[01:10:28] spicy style on Twitter that I think articulates and celebrates all the
[01:10:30] articulates and celebrates all the incredible developers and uh development
[01:10:33] incredible developers and uh development and the uh the code especially assembly
[01:10:36] and the uh the code especially assembly that's involved in uh building some of
[01:10:38] that's involved in uh building some of these codecs and building some of this
[01:10:40] these codecs and building some of this incredible technology. But that brings
[01:10:42] incredible technology. But that brings us to the a bit of a debacle that
[01:10:44] us to the a bit of a debacle that happened. Tell me the full saga of what
[01:10:47] happened. Tell me the full saga of what happened with the Google security
[01:10:49] happened with the Google security engineers. Just to be clear, Google are
[01:10:52] engineers. Just to be clear, Google are one of the biggest supporters of open
[01:10:53] one of the biggest supporters of open source out there. They have been for a
[01:10:55] source out there. They have been for a long time. It's just I think some things
[01:10:57] long time. It's just I think some things kind of went a bit overboard this time.
[01:11:00] kind of went a bit overboard this time. So ffmpeg itself and and this is not
[01:11:02] So ffmpeg itself and and this is not like a secret. It's on the homepage. You
[01:11:04] like a secret. It's on the homepage. You know the it processes untrusted data.
[01:11:07] know the it processes untrusted data. There can be security issues when you
[01:11:09] There can be security issues when you pass untrusted data. That's very normal.
[01:11:11] pass untrusted data. That's very normal. But recently what changed was Google
[01:11:13] But recently what changed was Google started using AI to create security
[01:11:15] started using AI to create security reports on an open source project.
[01:11:18] reports on an open source project. Volunteers had to deal with that. They
[01:11:21] Volunteers had to deal with that. They did they provided very limited funding
[01:11:23] did they provided very limited funding and they even went to the media first
[01:11:25] and they even went to the media first announcing how good their AI was before
[01:11:27] announcing how good their AI was before the issues could be fixed.
[01:11:29] the issues could be fixed. >> And this is in the public forum.
[01:11:31] >> And this is in the public forum. >> Yeah. So report reporting an issue using
[01:11:34] >> Yeah. So report reporting an issue using AI to find an issue in the code which is
[01:11:36] AI to find an issue in the code which is a security vulnerability and then
[01:11:38] a security vulnerability and then reporting that publicly before you're
[01:11:39] reporting that publicly before you're able to fix it.
[01:11:40] able to fix it. >> Yeah, it's announcing how good their AI
[01:11:42] >> Yeah, it's announcing how good their AI is that they provided a standard 90day
[01:11:45] is that they provided a standard 90day industry deadline um without potent
[01:11:49] industry deadline um without potent without really understanding the nature
[01:11:50] without really understanding the nature of volunteerdriven development. In
[01:11:53] of volunteerdriven development. In addition, this vulnerability was on an
[01:11:55] addition, this vulnerability was on an obscure 1990s game codec.
[01:11:59] obscure 1990s game codec. um
[01:12:00] um the way and and let's look at it from
[01:12:02] the way and and let's look at it from their standpoint to begin with. Let's um
[01:12:04] their standpoint to begin with. Let's um you know
[01:12:04] you know >> Yeah. Can you steal me on their case?
[01:12:06] >> Yeah. Can you steal me on their case? >> Yeah, sure. They have substantial
[01:12:08] >> Yeah, sure. They have substantial resources working on the security of
[01:12:09] resources working on the security of open source projects that you know are
[01:12:13] open source projects that you know are ubiquitous and they've used you know a
[01:12:15] ubiquitous and they've used you know a lot of compute to do that and very
[01:12:17] lot of compute to do that and very expensive and very capable security
[01:12:18] expensive and very capable security researchers um to do that and that
[01:12:20] researchers um to do that and that that's their viewpoint is they are
[01:12:22] that's their viewpoint is they are contributing by doing that but I think
[01:12:25] contributing by doing that but I think that's where
[01:12:28] that's where opinions differ um it opened up a lot of
[01:12:32] opinions differ um it opened up a lot of interesting fishes uh I would say Um
[01:12:36] interesting fishes uh I would say Um it does seem that there's a portion of
[01:12:38] it does seem that there's a portion of the security community that um look at
[01:12:41] the security community that um look at themselves a bit like building
[01:12:42] themselves a bit like building architects that never have to go to
[01:12:44] architects that never have to go to site. You know, going to site is
[01:12:46] site. You know, going to site is something that is a little bit beneath
[01:12:47] something that is a little bit beneath them, the actual day-to-day
[01:12:49] them, the actual day-to-day construction. They're there to do their
[01:12:50] construction. They're there to do their security things and it's someone else's
[01:12:52] security things and it's someone else's problem.
[01:12:54] problem. The security industry also kind of has
[01:12:57] The security industry also kind of has a very aggressive tone towards things.
[01:12:59] a very aggressive tone towards things. The the language they use is extremely
[01:13:01] The the language they use is extremely aggressive. They use very strong
[01:13:02] aggressive. They use very strong language like you will get popped. Um
[01:13:05] language like you will get popped. Um and to to Joe public get popped, you
[01:13:07] and to to Joe public get popped, you know, it means something quite bad. Um
[01:13:09] know, it means something quite bad. Um for them it means to get hacked. The way
[01:13:11] for them it means to get hacked. The way I would look at it personally is a
[01:13:13] I would look at it personally is a little bit like the padlock on your
[01:13:14] little bit like the padlock on your home. Um
[01:13:17] home. Um not everyone that a padlock on your home
[01:13:19] not everyone that a padlock on your home or you know the lock on your home is
[01:13:21] or you know the lock on your home is there to to protect against the
[01:13:25] there to to protect against the capabilities of of what it's there to
[01:13:27] capabilities of of what it's there to protect. It's not there to protect
[01:13:29] protect. It's not there to protect nuclear secrets. It's not there to
[01:13:30] nuclear secrets. It's not there to protect Fort Knox. And it could be
[01:13:33] protect Fort Knox. And it could be looked at that they're using AI at a
[01:13:35] looked at that they're using AI at a level of scale to go and pick those
[01:13:38] level of scale to go and pick those locks and then say, "Hey, your locks
[01:13:41] locks and then say, "Hey, your locks your locks not secure. You need to deal
[01:13:43] your locks not secure. You need to deal with this." Whereas actually they're the
[01:13:44] with this." Whereas actually they're the ones with resources to be able to to fix
[01:13:48] ones with resources to be able to to fix this. But that seems to not be something
[01:13:50] this. But that seems to not be something either they'll contribute to in terms of
[01:13:52] either they'll contribute to in terms of patches or in terms of financially. And
[01:13:55] patches or in terms of financially. And the scale of AI is kind of the issue
[01:13:56] the scale of AI is kind of the issue that the the bug reports are very wordy.
[01:14:00] that the the bug reports are very wordy. They're very very it's almost a denial
[01:14:02] They're very very it's almost a denial of service by AI generated bug reports
[01:14:05] of service by AI generated bug reports on very niche codecs. Um and the other
[01:14:08] on very niche codecs. Um and the other issue the security community has is
[01:14:10] issue the security community has is everything is marked high priority.
[01:14:12] everything is marked high priority. You're going to you know this is the
[01:14:13] You're going to you know this is the most important thing in the world and
[01:14:14] most important thing in the world and you need to deal with this high high
[01:14:16] you need to deal with this high high vulnerable scary scary on a game codec
[01:14:20] vulnerable scary scary on a game codec used on one disc in 1993.
[01:14:23] used on one disc in 1993. >> Yeah. And that that's where the
[01:14:25] >> Yeah. And that that's where the dichotomy lies going around telling
[01:14:28] dichotomy lies going around telling everyone that their padlocks not safe.
[01:14:30] everyone that their padlocks not safe. Well, that's a hobby project of somebody
[01:14:33] Well, that's a hobby project of somebody that the safety of that codec is
[01:14:35] that the safety of that codec is consumate to what that person thinks.
[01:14:37] consumate to what that person thinks. It's their hobby. It's good that they're
[01:14:38] It's their hobby. It's good that they're security analyzing it, but it doesn't
[01:14:40] security analyzing it, but it doesn't need a big scary warning. This is a
[01:14:42] need a big scary warning. This is a critical vulnerability. Um may also may
[01:14:45] critical vulnerability. Um may also may may recently also um see that um there
[01:14:47] may recently also um see that um there was another quote unquote vulnerability.
[01:14:49] was another quote unquote vulnerability. It wasn't a Google in this case, but um
[01:14:53] It wasn't a Google in this case, but um a filter could overflow and have an
[01:14:55] a filter could overflow and have an integer overflow and one of your pixels
[01:14:58] integer overflow and one of your pixels could be the wrong color. And this was
[01:15:01] could be the wrong color. And this was marked high 7.5 severity in red.
[01:15:05] marked high 7.5 severity in red. And at some point, the security industry
[01:15:07] And at some point, the security industry needs to realize you can't keep crying
[01:15:09] needs to realize you can't keep crying wolf like this because this just leads
[01:15:10] wolf like this because this just leads to people, you know, the equivalent
[01:15:12] to people, you know, the equivalent thereof of putting password stickers on
[01:15:14] thereof of putting password stickers on their PC. You know, you can't just keep
[01:15:16] their PC. You know, you can't just keep crying wolf every day. and and I
[01:15:18] crying wolf every day. and and I appreciate you know that's their modus
[01:15:20] appreciate you know that's their modus operandi is to create as much scared and
[01:15:24] operandi is to create as much scared and fear but from the Google standpoint at
[01:15:26] fear but from the Google standpoint at the end of the day
[01:15:28] the end of the day they need to contribute either
[01:15:30] they need to contribute either financially or with patches Google uses
[01:15:33] financially or with patches Google uses ffmpeg and a scale probably you or I
[01:15:36] ffmpeg and a scale probably you or I couldn't even contemplate millions of
[01:15:38] couldn't even contemplate millions of CPU cores and yes they contribute in
[01:15:41] CPU cores and yes they contribute in areas mostly regarding their own
[01:15:43] areas mostly regarding their own products so VP9 AV1 on but in a wider
[01:15:49] products so VP9 AV1 on but in a wider sense there's a disproportionate level
[01:15:52] sense there's a disproportionate level of contribution yes they fund students
[01:15:54] of contribution yes they fund students yes they fund summer of code
[01:15:57] yes they fund summer of code and I think uh so Alex Strange is a
[01:15:59] and I think uh so Alex Strange is a former ffmpeg developer um I think
[01:16:01] former ffmpeg developer um I think posting in a personal capacity so he uh
[01:16:03] posting in a personal capacity so he uh posted about security engineers on
[01:16:07] posted about security engineers on hacker news uh his post reads the
[01:16:09] hacker news uh his post reads the problem with security reports in general
[01:16:11] problem with security reports in general is security people are rampant
[01:16:13] is security people are rampant self-promoters in parenthe parenthesis.
[01:16:16] self-promoters in parenthe parenthesis. Lionus once called them something worse.
[01:16:20] Lionus once called them something worse. Imagine your humble volunteer open
[01:16:22] Imagine your humble volunteer open source developer. If a security
[01:16:24] source developer. If a security researcher finds a bug in your code,
[01:16:28] researcher finds a bug in your code, they're going to make up a cute name for
[01:16:29] they're going to make up a cute name for it, start a website with a logo, Google
[01:16:33] it, start a website with a logo, Google is going to give them a million dollar
[01:16:35] is going to give them a million dollar bounty. They're going to go to Defcon
[01:16:38] bounty. They're going to go to Defcon and get a prize and I assume go to some
[01:16:41] and get a prize and I assume go to some kind of secret security people orgy
[01:16:44] kind of secret security people orgy where everyone is dressed like they're
[01:16:45] where everyone is dressed like they're in the Matrix. Nobody is going to do any
[01:16:49] in the Matrix. Nobody is going to do any of this for you when you fix it. Uh
[01:16:54] of this for you when you fix it. Uh basically commenting on the sort of the
[01:16:56] basically commenting on the sort of the incentives
[01:16:58] incentives for the different uh people involved and
[01:17:00] for the different uh people involved and misaligned. The problem here is is a
[01:17:04] misaligned. The problem here is is a disproportion of means on discovery
[01:17:07] disproportion of means on discovery compared to patching it, right? And this
[01:17:10] compared to patching it, right? And this is the biggest issue, right? Um and
[01:17:12] is the biggest issue, right? Um and after that debacle, Google did some
[01:17:14] after that debacle, Google did some changes now starting to send patches
[01:17:16] changes now starting to send patches which is
[01:17:17] which is >> and they also now have rewards to for
[01:17:19] >> and they also now have rewards to for fixing issues. So it has changed a bit
[01:17:22] fixing issues. So it has changed a bit because of that uh debacle. So it's
[01:17:24] because of that uh debacle. So it's good, right? But we've seen and we
[01:17:26] good, right? But we've seen and we talked about Google, but we have seen
[01:17:28] talked about Google, but we have seen like some some other large companies
[01:17:30] like some some other large companies saying, "Oh, you need to fix this bug
[01:17:32] saying, "Oh, you need to fix this bug because it's critical in our product.
[01:17:33] because it's critical in our product. Can you explain XZ fiasco?" The FFmpeg
[01:17:37] Can you explain XZ fiasco?" The FFmpeg tweet reads,
[01:17:39] tweet reads, "The XZ fiasco has shown how a
[01:17:43] "The XZ fiasco has shown how a dependence on unpaid volunteers can
[01:17:45] dependence on unpaid volunteers can cause major problems. Trillion dollar
[01:17:47] cause major problems. Trillion dollar corporations expect free and urgent
[01:17:50] corporations expect free and urgent support from volunteers." Microsoft
[01:17:53] support from volunteers." Microsoft Microsoft Teams posted on a bug tracker
[01:17:56] Microsoft Teams posted on a bug tracker full of volunteers that their issue is
[01:17:59] full of volunteers that their issue is high priority.
[01:18:02] high priority. After politely requesting a support
[01:18:03] After politely requesting a support contract from Microsoft for long-term
[01:18:05] contract from Microsoft for long-term maintenance, they offered a one-time
[01:18:07] maintenance, they offered a one-time payment of a few thousand instead. This
[01:18:10] payment of a few thousand instead. This is unacceptable. We didn't make it up.
[01:18:12] is unacceptable. We didn't make it up. This is what Microsoft Microsoft Teams
[01:18:14] This is what Microsoft Microsoft Teams actually did. And then they uh you give
[01:18:17] actually did. And then they uh you give the image and the details and all that
[01:18:19] the image and the details and all that kind of stuff showing that these
[01:18:20] kind of stuff showing that these trillion dollar companies are not giving
[01:18:23] trillion dollar companies are not giving much money, not giving much support.
[01:18:24] much money, not giving much support. >> They think they think an open source
[01:18:26] >> They think they think an open source project is a traditional vendor that
[01:18:27] project is a traditional vendor that they have an SLA. They think a public
[01:18:30] they have an SLA. They think a public bug tracker is actually, you know, a
[01:18:32] bug tracker is actually, you know, a third party vendor's Jurro where you can
[01:18:34] third party vendor's Jurro where you can do all of these things. It's not. It's
[01:18:36] do all of these things. It's not. It's there to report bugs. I think the thing
[01:18:39] there to report bugs. I think the thing that made this particularly heinenous
[01:18:40] that made this particularly heinenous was the name dropping of Microsoft. the
[01:18:43] was the name dropping of Microsoft. the name dropping this is a visible product
[01:18:46] name dropping this is a visible product if this was if this was a just a general
[01:18:49] if this was if this was a just a general bug report I think that would have made
[01:18:50] bug report I think that would have made it a lot better.
[01:18:51] it a lot better. >> Yeah. So they literally said like this
[01:18:53] >> Yeah. So they literally said like this is a big deal because a lot of people
[01:18:55] is a big deal because a lot of people are using it in Microsoft. I wonder what
[01:18:59] are using it in Microsoft. I wonder what happens psychologically. So I think what
[01:19:01] happens psychologically. So I think what happens in these companies, maybe you
[01:19:03] happens in these companies, maybe you can correct me, is they, you're you're
[01:19:05] can correct me, is they, you're you're right, they just think of FFmpeg as like
[01:19:08] right, they just think of FFmpeg as like a vendor that Microsoft surely is paying
[01:19:12] a vendor that Microsoft surely is paying a huge amount of money to.
[01:19:14] a huge amount of money to. >> They kind of assume that in their
[01:19:17] >> They kind of assume that in their interaction and nobody anywhere on the
[01:19:19] interaction and nobody anywhere on the stack is going like, "Wait a minute,
[01:19:22] stack is going like, "Wait a minute, shouldn't we be giving like millions of
[01:19:24] shouldn't we be giving like millions of dollars to FFB?" And this is a very big
[01:19:26] dollars to FFB?" And this is a very big problem in large like we're talking
[01:19:28] problem in large like we're talking about some companies but it's the same
[01:19:30] about some companies but it's the same everywhere right um a lot of those
[01:19:33] everywhere right um a lot of those companies like the the when we talked to
[01:19:36] companies like the the when we talked to that person right he was just like a
[01:19:38] that person right he was just like a manager on one project in Microsoft
[01:19:40] manager on one project in Microsoft teams right he had like never really
[01:19:43] teams right he had like never really discussed with open source community he
[01:19:45] discussed with open source community he had no idea right it was like and but
[01:19:48] had no idea right it was like and but the problem is that usually there is
[01:19:50] the problem is that usually there is what we call opos right open source
[01:19:52] what we call opos right open source program offices in those type of
[01:19:54] program offices in those type of companies companies and they are the
[01:19:56] companies companies and they are the ones who are supposed to discuss with
[01:19:57] ones who are supposed to discuss with open source vendors um or or open source
[01:20:00] open source vendors um or or open source communities but like they often don't
[01:20:04] communities but like they often don't explain that correctly internally right
[01:20:05] explain that correctly internally right and here it's just like
[01:20:08] and here it's just like >> we are not your supplier if you want me
[01:20:10] >> we are not your supplier if you want me to be your supplier I'm very happy right
[01:20:13] to be your supplier I'm very happy right I will send you a contract and SLAs like
[01:20:15] I will send you a contract and SLAs like I created a five companies who are doing
[01:20:17] I created a five companies who are doing that around open source projects so
[01:20:19] that around open source projects so that's okay
[01:20:20] that's okay >> we should say that some some of the
[01:20:21] >> we should say that some some of the spicy tweets that Karen you're behind
[01:20:24] spicy tweets that Karen you're behind And some of the debacle
[01:20:27] And some of the debacle produced results, positive results.
[01:20:29] produced results, positive results. >> Donations have increased substantially.
[01:20:32] >> Donations have increased substantially. They're still not enough to cover even a
[01:20:34] They're still not enough to cover even a single full-time developer, but on both
[01:20:37] single full-time developer, but on both a, you know, awareness level and a
[01:20:39] a, you know, awareness level and a technical level. There's substantially
[01:20:41] technical level. There's substantially more technical awareness and sort of
[01:20:43] more technical awareness and sort of awareness of the importance of FFmpeg as
[01:20:46] awareness of the importance of FFmpeg as a result as a result of X and and what's
[01:20:49] a result as a result of X and and what's happened. I can say, you know, it it
[01:20:51] happened. I can say, you know, it it solved its purpose. people realize the
[01:20:54] solved its purpose. people realize the level of importance ffmpeg has
[01:20:55] level of importance ffmpeg has >> and on video it's the same right like
[01:20:57] >> and on video it's the same right like for example a very simple example
[01:21:01] for example a very simple example for more than a year we couldn't update
[01:21:04] for more than a year we couldn't update VC on Android because of a bug on the
[01:21:07] VC on Android because of a bug on the play store on Android play store right
[01:21:10] play store on Android play store right the only way we got someone to answer
[01:21:13] the only way we got someone to answer was to put an very spicy as you say um
[01:21:17] was to put an very spicy as you say um tweet saying that we were going to stop
[01:21:20] tweet saying that we were going to stop distributing VC for Android Right. And
[01:21:22] distributing VC for Android Right. And we have around 100 million people using
[01:21:25] we have around 100 million people using that. Um and now then someone from
[01:21:28] that. Um and now then someone from Android actually came and discussed to
[01:21:30] Android actually came and discussed to us. Right. Um we had the same issue with
[01:21:33] us. Right. Um we had the same issue with with Microsoft or or like saying that we
[01:21:36] with Microsoft or or like saying that we are going to stop distributing VC on the
[01:21:38] are going to stop distributing VC on the Windows store. And unfortunately we are
[01:21:41] Windows store. And unfortunately we are so small that the only very strong power
[01:21:45] so small that the only very strong power we have to solve those issues is blaming
[01:21:49] we have to solve those issues is blaming on social network because it's nos and
[01:21:52] on social network because it's nos and now they listen to us but those large
[01:21:54] now they listen to us but those large companies often have difficulty
[01:21:57] companies often have difficulty talking to us like for example VC right
[01:22:00] talking to us like for example VC right is probably one of the top 10 software
[01:22:02] is probably one of the top 10 software used on Windows I am not part of
[01:22:06] used on Windows I am not part of Microsoft ISV programs. Right? I don't
[01:22:10] Microsoft ISV programs. Right? I don't have a point of contact at Microsoft's.
[01:22:12] have a point of contact at Microsoft's. Right? Well, I'm sure any other
[01:22:14] Right? Well, I'm sure any other software, Adobe, Spotify has a a point
[01:22:17] software, Adobe, Spotify has a a point of contact. I don't have that. Right?
[01:22:18] of contact. I don't have that. Right? So,
[01:22:20] So, raising awareness works. It's sometimes
[01:22:23] raising awareness works. It's sometimes very spicy, lot of drama. Well, X and
[01:22:27] very spicy, lot of drama. Well, X and Twitter are okay for that, but it's
[01:22:28] Twitter are okay for that, but it's efficient.
[01:22:30] efficient. Uh, so everybody listening to this
[01:22:32] Uh, so everybody listening to this should go uh follow FFmpeg on Twitter on
[01:22:37] should go uh follow FFmpeg on Twitter on X. Follow Video Land on Twitter on X.
[01:22:41] X. Follow Video Land on Twitter on X. Uh, go donate donate to FFmpeg. And
[01:22:45] Uh, go donate donate to FFmpeg. And thank you, Lex. Over the years, several
[01:22:47] thank you, Lex. Over the years, several years, you've been a supporter of, you
[01:22:49] years, you've been a supporter of, you know, FFmpeg and Video Land OnX, you
[01:22:52] know, FFmpeg and Video Land OnX, you know, giving us shout outs,
[01:22:53] know, giving us shout outs, appreciating, you know, what we do.
[01:22:55] appreciating, you know, what we do. >> FFmpeg for life. And for example like um
[01:22:58] >> FFmpeg for life. And for example like um Tim Sweeney Carmarmac and a few others
[01:23:01] Tim Sweeney Carmarmac and a few others like very high level people have raised
[01:23:03] like very high level people have raised also the awareness on on our ex accounts
[01:23:07] also the awareness on on our ex accounts and that helped a lot also. Kopathi
[01:23:10] and that helped a lot also. Kopathi Kapati as well.
[01:23:11] Kapati as well. >> Yeah. I mean also, you know, outside of
[01:23:13] >> Yeah. I mean also, you know, outside of the fact that so many people use it,
[01:23:15] the fact that so many people use it, it's so impactful on the world, it's
[01:23:17] it's so impactful on the world, it's also a great representation of a great
[01:23:19] also a great representation of a great open source project. Like the value of
[01:23:23] open source project. Like the value of assembly and C and making sure that like
[01:23:26] assembly and C and making sure that like you take programming seriously for real
[01:23:29] you take programming seriously for real world systems. It's not just that. We
[01:23:31] world systems. It's not just that. We we'll talk about assembly later I'm sure
[01:23:33] we'll talk about assembly later I'm sure because that's its whole topic in
[01:23:34] because that's its whole topic in itself. But it's also celebrating people
[01:23:36] itself. But it's also celebrating people like Andreas Reinhardt who do
[01:23:38] like Andreas Reinhardt who do maintenance is I believe unpaid I
[01:23:41] maintenance is I believe unpaid I believe as a volunteer he's doing
[01:23:43] believe as a volunteer he's doing massive refactorings. Uh Andreas
[01:23:45] massive refactorings. Uh Andreas Reinhardt and Anton Kernov rewriting
[01:23:48] Reinhardt and Anton Kernov rewriting ffmpeg.c with threading celebrating
[01:23:50] ffmpeg.c with threading celebrating those guys celebrating the untold labor
[01:23:53] those guys celebrating the untold labor that's gone into this that actually
[01:23:55] that's gone into this that actually doesn't change anything from the user
[01:23:57] doesn't change anything from the user standpoint. The files are exactly the
[01:23:59] standpoint. The files are exactly the same, but wow, the the the airplane has
[01:24:01] same, but wow, the the the airplane has been rebuilt whilst it's in the air.
[01:24:03] been rebuilt whilst it's in the air. >> Christian Garcia said, "As a teenager
[01:24:05] >> Christian Garcia said, "As a teenager running this account," referring to the
[01:24:06] running this account," referring to the FFmpeg account, and you responded,
[01:24:08] FFmpeg account, and you responded, "Teeneageners have written more assembly
[01:24:11] "Teeneageners have written more assembly in FFmpeg than Google engineers, but
[01:24:14] in FFmpeg than Google engineers, but also just pointing out that there's a
[01:24:17] also just pointing out that there's a lot of incredible contributors who are
[01:24:18] lot of incredible contributors who are teenagers. Like JB said, we don't care
[01:24:21] teenagers. Like JB said, we don't care who you are, where you're from, what you
[01:24:23] who you are, where you're from, what you do. teenagers have written thousands of
[01:24:27] do. teenagers have written thousands of lines of assembly um over the years.
[01:24:30] lines of assembly um over the years. Give a shout out back in the days to
[01:24:32] Give a shout out back in the days to Daniel Kang. So, also highlighting the
[01:24:34] Daniel Kang. So, also highlighting the work of people like Ruka Peng. This is a
[01:24:36] work of people like Ruka Peng. This is a 16-year-old some of his first
[01:24:38] 16-year-old some of his first contributions to FFmpeg
[01:24:40] contributions to FFmpeg actually doing and putting some of these
[01:24:42] actually doing and putting some of these quote unquote security researchers to
[01:24:44] quote unquote security researchers to shame by act by actually finding issues
[01:24:46] shame by act by actually finding issues and fixing them and being 16.
[01:24:49] and fixing them and being 16. There's no barriers. There's no barriers
[01:24:51] There's no barriers. There's no barriers to you have to study on at college under
[01:24:53] to you have to study on at college under this person and understand these. It's
[01:24:55] this person and understand these. It's you can learn C and let's be honest that
[01:24:57] you can learn C and let's be honest that it's from it's from the KN&R book learn
[01:24:59] it's from it's from the KN&R book learn C. Uh you can learn assembly. We'll talk
[01:25:02] C. Uh you can learn assembly. We'll talk about that maybe a bit later. You can
[01:25:04] about that maybe a bit later. You can contribute to worldclass technologies in
[01:25:07] contribute to worldclass technologies in VC. Um one of the oldest contributors
[01:25:10] VC. Um one of the oldest contributors called Felix. He's the one doing
[01:25:12] called Felix. He's the one doing everything on Mac and iOS. He's starting
[01:25:14] everything on Mac and iOS. He's starting working on VC. He was 16. Um we had a
[01:25:17] working on VC. He was 16. Um we had a guy called Edward Wong who used to be a
[01:25:20] guy called Edward Wong who used to be a Google Summer of Code student who stayed
[01:25:23] Google Summer of Code student who stayed for 3 years around video land. He was
[01:25:25] for 3 years around video land. He was 14, right? Um and and part of Google
[01:25:28] 14, right? Um and and part of Google Summer of Code and Google coding which
[01:25:30] Summer of Code and Google coding which were programs where uh basically we have
[01:25:32] were programs where uh basically we have students or high school. Um we wrote a
[01:25:35] students or high school. Um we wrote a ton of assembly for x264 and for VC and
[01:25:38] ton of assembly for x264 and for VC and for ffmpeg, right? everyone can
[01:25:40] for ffmpeg, right? everyone can contribute.
[01:25:41] contribute. >> And he also did a good job because he
[01:25:42] >> And he also did a good job because he didn't play the alarmist um CVE high
[01:25:46] didn't play the alarmist um CVE high create a CVE which is like um public
[01:25:49] create a CVE which is like um public exposure of security and do these big
[01:25:52] exposure of security and do these big scary red 7.5 priority. He just fixed an
[01:25:56] scary red 7.5 priority. He just fixed an issue in git after three days and just
[01:25:58] issue in git after three days and just fixed it. He didn't need to go and play
[01:26:00] fixed it. He didn't need to go and play a big security drama about it. And I
[01:26:02] a big security drama about it. And I think I posted, you know, the kids are
[01:26:04] think I posted, you know, the kids are all right, whereas there's there's
[01:26:06] all right, whereas there's there's there's, you know, there is a I'm not
[01:26:08] there's, you know, there is a I'm not saying all security people do this, but
[01:26:09] saying all security people do this, but there is a portion of the security
[01:26:11] there is a portion of the security community, as Alex said, that likes to
[01:26:13] community, as Alex said, that likes to hype themselves up by creating drama.
[01:26:15] hype themselves up by creating drama. They would have happily raised this is a
[01:26:18] They would have happily raised this is a high priority CVE 8.0 or whatever on a
[01:26:21] high priority CVE 8.0 or whatever on a on an issue that actually was in Git. It
[01:26:23] on an issue that actually was in Git. It wasn't even in a release. It was in
[01:26:25] wasn't even in a release. It was in development and 3 days later was fixed.
[01:26:27] development and 3 days later was fixed. Well, I just want to put a little bit of
[01:26:29] Well, I just want to put a little bit of love out there even to the bigger
[01:26:32] love out there even to the bigger community. Uh much love and respect to
[01:26:36] community. Uh much love and respect to Google engineers. Like you said, they're
[01:26:38] Google engineers. Like you said, they're uh some of the the best software
[01:26:41] uh some of the the best software engineers in the world. They do
[01:26:42] engineers in the world. They do contribute a lot even on the security
[01:26:44] contribute a lot even on the security front. And also, you know, I'm a big fan
[01:26:46] front. And also, you know, I'm a big fan of Theo. Much love to Theo. He was part
[01:26:49] of Theo. Much love to Theo. He was part of this uh debacle and drama a little
[01:26:51] of this uh debacle and drama a little bit. I think when you just zoom out on
[01:26:54] bit. I think when you just zoom out on the grand arc of human history, the
[01:26:57] the grand arc of human history, the drama contributed positively to
[01:26:59] drama contributed positively to everybody involved. Donations went up.
[01:27:02] everybody involved. Donations went up. It brought more attention to the topic,
[01:27:04] It brought more attention to the topic, allowed uh everybody to bicker in a way
[01:27:07] allowed uh everybody to bicker in a way that ultimately uh got them to figure
[01:27:10] that ultimately uh got them to figure out what FFmpeg is all about.
[01:27:11] out what FFmpeg is all about. >> So the way the way we looked at this is
[01:27:13] >> So the way the way we looked at this is like it's a rap battle at the end of the
[01:27:15] like it's a rap battle at the end of the day, you know. No, but it it is we say
[01:27:17] day, you know. No, but it it is we say stuff. We say stuff.
[01:27:19] stuff. We say stuff. >> Yeah. But we can we can leave it on the
[01:27:21] >> Yeah. But we can we can leave it on the X is a perfect place for you know
[01:27:22] X is a perfect place for you know international rap battle. You say stuff
[01:27:24] international rap battle. You say stuff I say stuff about your mama but it
[01:27:26] I say stuff about your mama but it doesn't mean you know I'm have an actual
[01:27:28] doesn't mean you know I'm have an actual personal issue with her.
[01:27:29] personal issue with her. >> Yeah. And that's what it looks like. The
[01:27:31] >> Yeah. And that's what it looks like. The Theo situation you know JB can maybe
[01:27:32] Theo situation you know JB can maybe expand went a little bit too far and
[01:27:34] expand went a little bit too far and there was a little but
[01:27:36] there was a little but >> you know it's just a bit of fun. It's
[01:27:37] >> you know it's just a bit of fun. It's just a bit of rap battle. It's a bit
[01:27:39] just a bit of rap battle. It's a bit it's WWE. You know everyone's having a
[01:27:41] it's WWE. You know everyone's having a bit of fun on X. It it doesn't need to
[01:27:43] bit of fun on X. It it doesn't need to be taken seriously. you know that the
[01:27:45] be taken seriously. you know that the teenagers thing you know that so that
[01:27:47] teenagers thing you know that so that that guy was a Google employee saying
[01:27:48] that guy was a Google employee saying hey you know there are other ways to run
[01:27:50] hey you know there are other ways to run an open source business you know go just
[01:27:53] an open source business you know go just have a bit of fun you know that's what
[01:27:55] have a bit of fun you know that's what the point of this account is and and
[01:27:56] the point of this account is and and furthermore if you can teach people
[01:27:58] furthermore if you can teach people about the ways of open source projects
[01:28:01] about the ways of open source projects assembly etc by doing that I think
[01:28:03] assembly etc by doing that I think there's a lot to be offered here it's
[01:28:04] there's a lot to be offered here it's not dunking on people for dunking sake
[01:28:06] not dunking on people for dunking sake it's showing actually the story that I
[01:28:08] it's showing actually the story that I think X learned is these are not big
[01:28:10] think X learned is these are not big corporate open source projects this is
[01:28:12] corporate open source projects this is not Kubernetes where there's you
[01:28:14] not Kubernetes where there's you hundreds maybe thousands of people paid
[01:28:16] hundreds maybe thousands of people paid to develop this stuff. These are just
[01:28:17] to develop this stuff. These are just people in their basements in their spare
[01:28:19] people in their basements in their spare time.
[01:28:20] time. >> And if you can address that topic in a
[01:28:21] >> And if you can address that topic in a fun and entertaining way,
[01:28:23] fun and entertaining way, >> I think that that that's the good thing
[01:28:26] >> I think that that that's the good thing and that's that's the value of X and the
[01:28:28] and that's that's the value of X and the reach we have
[01:28:28] reach we have >> and and to be honest right like even on
[01:28:31] >> and and to be honest right like even on at Google, Google is
[01:28:34] at Google, Google is one entity but so many different people,
[01:28:36] one entity but so many different people, right? and and some and there is a ton
[01:28:38] right? and and some and there is a ton of Google engineer we work with um all
[01:28:42] of Google engineer we work with um all the time and even like Google from
[01:28:44] the time and even like Google from YouTube to Chrome to Chrome media to the
[01:28:47] YouTube to Chrome to Chrome media to the rest of Google those are very different
[01:28:48] rest of Google those are very different type of entities but what we do is
[01:28:52] type of entities but what we do is efficient and for example for for Theo
[01:28:54] efficient and for example for for Theo right it went a bit too far I had him
[01:28:56] right it went a bit too far I had him like I call everyone down I had him on
[01:28:59] like I call everyone down I had him on the phone we say okay like this goes too
[01:29:01] the phone we say okay like this goes too far and so on but in the end um yeah
[01:29:04] far and so on but in the end um yeah it's a hard battle but it's positive for
[01:29:06] it's a hard battle but it's positive for the project it's like the awareness we
[01:29:08] the project it's like the awareness we have on open source and and I mean true
[01:29:11] have on open source and and I mean true open source from communities right not
[01:29:13] open source from communities right not is increased dramatically in the last
[01:29:16] is increased dramatically in the last two years and this is useful
[01:29:19] two years and this is useful uh what do you think motivates all the
[01:29:21] uh what do you think motivates all the incredible contributors that we've been
[01:29:23] incredible contributors that we've been talking about like what's the what's the
[01:29:24] talking about like what's the what's the engine it's so interesting to see like
[01:29:26] engine it's so interesting to see like you said they're sitting in the basement
[01:29:28] you said they're sitting in the basement what's the driver what's the engine
[01:29:29] what's the driver what's the engine there
[01:29:29] there >> there are many uh drivers but weirdly
[01:29:32] >> there are many uh drivers but weirdly the main one is that what we do in
[01:29:35] the main one is that what we do in multimedia uh plays videos and video is
[01:29:37] multimedia uh plays videos and video is cool, right? And and for example, we
[01:29:40] cool, right? And and for example, we have so many people in the community who
[01:29:43] have so many people in the community who arrive because they loved watching
[01:29:45] arrive because they loved watching anime, right? And and this is like the
[01:29:48] anime, right? And and this is like the advice when people ask me what should I
[01:29:50] advice when people ask me what should I work on in open source? How do I start?
[01:29:52] work on in open source? How do I start? And my question, my answer is always the
[01:29:54] And my question, my answer is always the same. Work on something you love.
[01:29:56] same. Work on something you love. >> I am working on VC because I love
[01:29:59] >> I am working on VC because I love movies, right? And I love watching the
[01:30:01] movies, right? And I love watching the same movies over and over even if my
[01:30:03] same movies over and over even if my wife hates me when I do that right but
[01:30:06] wife hates me when I do that right but because it's interesting right because
[01:30:07] because it's interesting right because it's a topic that you like right the
[01:30:09] it's a topic that you like right the first that's the first thing where
[01:30:11] first that's the first thing where people come to usually to VC and FFMP
[01:30:14] people come to usually to VC and FFMP the second thing is that technically we
[01:30:17] the second thing is that technically we because we search for excellent this is
[01:30:19] because we search for excellent this is the best school ever right this is the
[01:30:23] the best school ever right this is the best school ever of programming if
[01:30:25] best school ever of programming if you're good in C in ffmpeg if you know
[01:30:28] you're good in C in ffmpeg if you know how to write assembly I sure you you're
[01:30:30] how to write assembly I sure you you're going to be one of the best programmers
[01:30:32] going to be one of the best programmers ever even if you're working on writing
[01:30:34] ever even if you're working on writing TypeScript because this is the most
[01:30:36] TypeScript because this is the most amazing thing to do and you will like
[01:30:40] amazing thing to do and you will like have to get reviews by some of the most
[01:30:43] have to get reviews by some of the most seasoned programmer ever who are going
[01:30:45] seasoned programmer ever who are going to look at every part of your code and
[01:30:47] to look at every part of your code and tell you why it's not great. It's like
[01:30:50] tell you why it's not great. It's like we are the best teachers that you've
[01:30:51] we are the best teachers that you've ever had in programming. Right.
[01:30:52] ever had in programming. Right. >> Andrew Kelly started Zig. He was an
[01:30:54] >> Andrew Kelly started Zig. He was an FFmpeg developer and started Zigg after
[01:30:56] FFmpeg developer and started Zigg after his FFmpeg school. I mean it it's the
[01:31:00] his FFmpeg school. I mean it it's the place to learn so many aspects of
[01:31:02] place to learn so many aspects of programming in the real world in a thing
[01:31:04] programming in the real world in a thing used by billions of people. You have
[01:31:07] used by billions of people. You have nowhere to hide. You have to be open and
[01:31:09] nowhere to hide. You have to be open and honest about your flaws and and how you
[01:31:11] honest about your flaws and and how you can learn and be better. And what is
[01:31:13] can learn and be better. And what is also interesting in multimedia is that
[01:31:16] also interesting in multimedia is that you have 16 milliseconds to display a
[01:31:18] you have 16 milliseconds to display a frame. It's not like a game engine where
[01:31:20] frame. It's not like a game engine where you can basically slow down and wait a
[01:31:22] you can basically slow down and wait a frame like so it's you need to be good,
[01:31:25] frame like so it's you need to be good, right? There is no choice else you don't
[01:31:27] right? There is no choice else you don't have your video. and and because of how
[01:31:29] have your video. and and because of how codecs if you miss a frame you're going
[01:31:31] codecs if you miss a frame you're going to destroy the look of the video, right?
[01:31:33] to destroy the look of the video, right? So, you need to be good. You need to be
[01:31:36] So, you need to be good. You need to be perfect to have the right thing. But
[01:31:38] perfect to have the right thing. But also is that it's not just pure
[01:31:40] also is that it's not just pure programming in the mathematical fence,
[01:31:43] programming in the mathematical fence, right? A lot of people don't understand,
[01:31:45] right? A lot of people don't understand, but um
[01:31:47] but um in order to program correctly on the
[01:31:50] in order to program correctly on the open source multimedia community, you
[01:31:52] open source multimedia community, you need to understand how computer works.
[01:31:54] need to understand how computer works. And and when you write assembly, you
[01:31:56] And and when you write assembly, you need to understand about uh CPU
[01:31:58] need to understand about uh CPU pipelining, right? You need to
[01:32:00] pipelining, right? You need to understand how SIMD works, how the AU
[01:32:02] understand how SIMD works, how the AU works, right? You need to understand how
[01:32:04] works, right? You need to understand how IO works, right? And this is what I
[01:32:07] IO works, right? And this is what I think that is missing to a lot of
[01:32:09] think that is missing to a lot of engineers and software engineers today
[01:32:11] engineers and software engineers today is understanding what we call computer
[01:32:12] is understanding what we call computer architecture. Um and like seriously like
[01:32:16] architecture. Um and like seriously like some of the debate is like should we use
[01:32:17] some of the debate is like should we use this assembly coil or this one and
[01:32:19] this assembly coil or this one and people say well no it's going to be like
[01:32:21] people say well no it's going to be like three cycle on this type of CPU and this
[01:32:24] three cycle on this type of CPU and this one and has massive impact on the output
[01:32:27] one and has massive impact on the output right we should expand ffmpeg is
[01:32:29] right we should expand ffmpeg is probably one of the biggest CPU users in
[01:32:31] probably one of the biggest CPU users in the world that's probably it's probably
[01:32:32] the world that's probably it's probably running
[01:32:33] running >> as we speak easily 100 order of
[01:32:36] >> as we speak easily 100 order of magnitude 100 million maybe even a
[01:32:38] magnitude 100 million maybe even a billion CPUs as we speak so every
[01:32:41] billion CPUs as we speak so every instruction matters there's not
[01:32:45] instruction matters there's not the the the impact at least in terms of
[01:32:47] the the the impact at least in terms of CPU is massive for everything that we
[01:32:50] CPU is massive for everything that we do. So first you come because it's an
[01:32:53] do. So first you come because it's an interesting subject then you stay
[01:32:54] interesting subject then you stay because it's excellent and in the end
[01:32:57] because it's excellent and in the end you're very proud of it because it's on
[01:32:59] you're very proud of it because it's on the end of everyone like so many people
[01:33:03] the end of everyone like so many people like oh I'm working for whatever
[01:33:06] like oh I'm working for whatever consulting company and I'm doing some
[01:33:10] consulting company and I'm doing some portal to download invoices for your PG&
[01:33:13] portal to download invoices for your PG& wow great like so many jobs are like
[01:33:16] wow great like so many jobs are like that you're not going to to tell that to
[01:33:18] that you're not going to to tell that to your grandma but if you go to see your
[01:33:20] your grandma but if you go to see your run mine say I do this so that you can
[01:33:22] run mine say I do this so that you can play video on your laptop they
[01:33:24] play video on your laptop they understand and this is very important
[01:33:26] understand and this is very important right because you're working on VC
[01:33:27] right because you're working on VC ffmpeg x264 it's in the end of hundreds
[01:33:31] ffmpeg x264 it's in the end of hundreds of million of people and you have an
[01:33:33] of million of people and you have an impact um and so you can be proud of
[01:33:35] impact um and so you can be proud of yourself and so I think that in addition
[01:33:38] yourself and so I think that in addition to great doing a great resume all those
[01:33:39] to great doing a great resume all those things are why people contribute
[01:33:42] things are why people contribute >> yeah those are side effects my favorite
[01:33:43] >> yeah those are side effects my favorite quote on this topic is John Collinsson
[01:33:45] quote on this topic is John Collinsson he said the world is a museum of passion
[01:33:47] he said the world is a museum of passion projects you know everything out there
[01:33:49] projects you know everything out there is is it's a passion project and open
[01:33:52] is is it's a passion project and open source multimedia and open source in
[01:33:53] source multimedia and open source in general you can just do that so much
[01:33:56] general you can just do that so much faster there's such a faster network
[01:33:58] faster there's such a faster network effect you know
[01:33:59] effect you know >> I can open a cafe and that can be my
[01:34:01] >> I can open a cafe and that can be my passion project but I have to get
[01:34:02] passion project but I have to get building codes I have to build a
[01:34:03] building codes I have to build a building I have to find a find a
[01:34:05] building I have to find a find a location I have to do all the you know
[01:34:07] location I have to do all the you know all sorts of things well in the software
[01:34:09] all sorts of things well in the software world that passion project can be can
[01:34:11] world that passion project can be can move quickly it can be amplified by the
[01:34:14] move quickly it can be amplified by the network effect and that amplification
[01:34:16] network effect and that amplification can be more than the the sum of the sum
[01:34:19] can be more than the the sum of the sum of the parts You know, you can be you
[01:34:21] of the parts You know, you can be you can find people interested in extremely
[01:34:25] can find people interested in extremely obscure things and have a network effect
[01:34:28] obscure things and have a network effect and make something that is truly
[01:34:30] and make something that is truly amazing.
[01:34:31] amazing. >> And uh on that topic of passion
[01:34:33] >> And uh on that topic of passion projects, Tim Sweeney actually said in a
[01:34:36] projects, Tim Sweeney actually said in a reply to a tweet that was uh
[01:34:38] reply to a tweet that was uh complimenting JB. He said, quote, "Many
[01:34:42] complimenting JB. He said, quote, "Many things in the world only happen because
[01:34:44] things in the world only happen because an awesome person decides to do it. This
[01:34:47] an awesome person decides to do it. This is the case with VLC. And that speaks to
[01:34:50] is the case with VLC. And that speaks to something interesting to me that it does
[01:34:51] something interesting to me that it does seem that a small number of people,
[01:34:53] seem that a small number of people, sometimes one person can create
[01:34:58] sometimes one person can create something incredible in the software
[01:35:00] something incredible in the software world. Like you said this over and over
[01:35:01] world. Like you said this over and over and over. I think JavaScript is an
[01:35:04] and over. I think JavaScript is an incredible thing created by by uh
[01:35:06] incredible thing created by by uh initially a single person. the some of
[01:35:07] initially a single person. the some of the programming languages like Python
[01:35:10] the programming languages like Python and C and Java like this one person has
[01:35:13] and C and Java like this one person has this vision has this design and brings
[01:35:15] this vision has this design and brings it sometimes over a weekend is the
[01:35:17] it sometimes over a weekend is the initial spark.
[01:35:18] initial spark. >> Yes. Linus built git in two weeks.
[01:35:21] >> Yes. Linus built git in two weeks. >> Yeah.
[01:35:22] >> Yeah. >> Wow.
[01:35:23] >> Wow. >> Yeah. Changed the world. Git I mean it
[01:35:24] >> Yeah. Changed the world. Git I mean it really changed the world.
[01:35:25] really changed the world. >> Linu passion project. Hey uploading this
[01:35:27] >> Linu passion project. Hey uploading this toble to an FTP like deal with it. But
[01:35:30] toble to an FTP like deal with it. But for me it's not just in software right
[01:35:32] for me it's not just in software right and I believe in in in individuals that
[01:35:35] and I believe in in in individuals that are going to change the world right and
[01:35:37] are going to change the world right and it's with a good as you said vision
[01:35:39] it's with a good as you said vision right I want to do that um it is useful
[01:35:42] right I want to do that um it is useful it will be useful and whether it's going
[01:35:44] it will be useful and whether it's going to like build trains or cars or rockets
[01:35:47] to like build trains or cars or rockets or something like I I believe people who
[01:35:50] or something like I I believe people who believe in themselves and have a vision
[01:35:52] believe in themselves and have a vision can have a huge impact for humanity.
[01:35:56] can have a huge impact for humanity. Let's actually uh zoom out before we
[01:35:58] Let's actually uh zoom out before we zoom back in. We'll just keep going up
[01:36:00] zoom back in. We'll just keep going up and down the the stack. So, uh you know,
[01:36:03] and down the the stack. So, uh you know, we've been talking back and forth VC and
[01:36:05] we've been talking back and forth VC and FFmpeg. Karen, you said that FFmpeg and
[01:36:08] FFmpeg. Karen, you said that FFmpeg and Video Land VC coexist and there's no
[01:36:13] Video Land VC coexist and there's no central point of importance. Is it kind
[01:36:15] central point of importance. Is it kind of what you call the binary star system?
[01:36:17] of what you call the binary star system? >> Yeah.
[01:36:18] >> Yeah. >> Uh they succeed because of each other.
[01:36:20] >> Uh they succeed because of each other. Can you explain the difference how they
[01:36:22] Can you explain the difference how they interact? What is the
[01:36:24] interact? What is the >> Are they competitors?
[01:36:26] >> Are they competitors? >> I don't I don't think they're
[01:36:27] >> I don't I don't think they're competitors. is I think I think the
[01:36:28] competitors. is I think I think the simple answer is the short answer before
[01:36:30] simple answer is the short answer before I go into detail is VC is to ffmpeg as
[01:36:34] I go into detail is VC is to ffmpeg as Android is to Linux. So they depend on
[01:36:37] Android is to Linux. So they depend on each other but they they coexist because
[01:36:39] each other but they they coexist because of each other. So they are a binary star
[01:36:41] of each other. So they are a binary star system is the analogy I used. By the way
[01:36:43] system is the analogy I used. By the way I feel horrible that I just recently
[01:36:46] I feel horrible that I just recently learned that Alpha Centauri the closest
[01:36:48] learned that Alpha Centauri the closest star system to us is a triple star
[01:36:50] star system to us is a triple star system.
[01:36:50] system. >> And and when you start doing the physics
[01:36:52] >> And and when you start doing the physics it's a nightmare right but
[01:36:55] it's a nightmare right but >> hence the threebody problem. But anyway,
[01:36:57] >> hence the threebody problem. But anyway, >> so a lot of ffmpeg pipelines involve the
[01:37:01] >> so a lot of ffmpeg pipelines involve the x264 project which is a videoand
[01:37:03] x264 project which is a videoand project. I I would put a finger in the
[01:37:05] project. I I would put a finger in the air that's say 80 plus% of those
[01:37:07] air that's say 80 plus% of those pipelines are dependent on a videoand
[01:37:09] pipelines are dependent on a videoand project.
[01:37:11] project. VC obviously we've discussed a videoand
[01:37:13] VC obviously we've discussed a videoand project uses ffmpeg gives it reach
[01:37:16] project uses ffmpeg gives it reach exposure to weird files. uh historically
[01:37:20] exposure to weird files. uh historically used some donation money to fund FFmp
[01:37:22] used some donation money to fund FFmp FFmpeg development and we'll talk a bit
[01:37:24] FFmpeg development and we'll talk a bit maybe about some of the reverse
[01:37:25] maybe about some of the reverse engineering later. So it's a binary star
[01:37:28] engineering later. So it's a binary star system. They they work and feed off each
[01:37:29] system. They they work and feed off each other. Many of the developers are
[01:37:31] other. Many of the developers are shared. There's no central location.
[01:37:33] shared. There's no central location. It's a virtuous cycle working together.
[01:37:36] It's a virtuous cycle working together. >> And we should mention that x264
[01:37:39] >> And we should mention that x264 is the encoder for H.264 video standard.
[01:37:42] is the encoder for H.264 video standard. So H.264 is the standard.
[01:37:44] So H.264 is the standard. >> Yes. X264
[01:37:46] >> Yes. X264 >> is the implementation open source
[01:37:47] >> is the implementation open source implementation of the standards
[01:37:49] implementation of the standards >> that's used by basically everybody for
[01:37:51] >> that's used by basically everybody for everything.
[01:37:52] everything. >> Yeah,
[01:37:52] >> Yeah, >> it's that is the main driver of this.
[01:37:54] >> it's that is the main driver of this. When you think of an MP4 file that has
[01:37:57] When you think of an MP4 file that has H.264 codec in it
[01:37:59] H.264 codec in it >> if it if it came from a software
[01:38:01] >> if it if it came from a software environment like a data center or
[01:38:03] environment like a data center or somewhere the chances are it was created
[01:38:05] somewhere the chances are it was created with x264
[01:38:06] with x264 >> and that's under the flag of uh video
[01:38:09] >> and that's under the flag of uh video land.
[01:38:09] land. >> That's a video land project. So in the
[01:38:10] >> That's a video land project. So in the video land graphic it sits in the
[01:38:12] video land graphic it sits in the videoand world
[01:38:13] videoand world >> and videoand has a say has a bunch of
[01:38:15] >> and videoand has a say has a bunch of stuff in it. Go to the video land
[01:38:18] stuff in it. Go to the video land website. There's a bunch of icons like
[01:38:21] website. There's a bunch of icons like if you look there is so many libraries
[01:38:23] if you look there is so many libraries right um lib DVD CSS lib DVD nav lib DVD
[01:38:30] right um lib DVD CSS lib DVD nav lib DVD PSI live VLC of course VC unity lib roof
[01:38:35] PSI live VLC of course VC unity lib roof Blu-ray
[01:38:38] Blu-ray >> uh yeah there
[01:38:38] >> uh yeah there >> and there is so many more right lately
[01:38:41] >> and there is so many more right lately the David project that we might talk
[01:38:43] the David project that we might talk about is the last project from video
[01:38:46] about is the last project from video it's everywhere right and we do we We
[01:38:48] it's everywhere right and we do we We have a lip special audio lately that we
[01:38:50] have a lip special audio lately that we announced. We have Czechm which is like
[01:38:54] announced. We have Czechm which is like an insane project but amazing. Um so and
[01:38:58] an insane project but amazing. Um so and x264 is one of those video land project
[01:39:00] x264 is one of those video land project and my opinion for example is that x264
[01:39:04] and my opinion for example is that x264 was it is the most amazing encoder ever
[01:39:07] was it is the most amazing encoder ever designed and this helped the adoption of
[01:39:11] designed and this helped the adoption of ffmpeg. A lot of people and large
[01:39:12] ffmpeg. A lot of people and large companies went through ffmpeg because
[01:39:15] companies went through ffmpeg because they wanted to use x264 and x264
[01:39:18] they wanted to use x264 and x264 increased the popularity on ffmpeg but
[01:39:20] increased the popularity on ffmpeg but also VC had its popularity because it's
[01:39:24] also VC had its popularity because it's played so many files that were done by
[01:39:26] played so many files that were done by ffmpeg right so it's it's many project
[01:39:30] ffmpeg right so it's it's many project that are interwined and work together.
[01:39:32] that are interwined and work together. Yeah, unfortunately there's a there's a
[01:39:33] Yeah, unfortunately there's a there's a thing on X where um VC is mentioned and
[01:39:36] thing on X where um VC is mentioned and there's people a quick reminder it's
[01:39:38] there's people a quick reminder it's FFmpeg inside doing the actual work and
[01:39:40] FFmpeg inside doing the actual work and and that's like I said it's not that
[01:39:43] and that's like I said it's not that that's not the case that we work
[01:39:45] that's not the case that we work together and to give you an idea right
[01:39:47] together and to give you an idea right when I compile VC for Windows I compiled
[01:39:50] when I compile VC for Windows I compiled around 16 million lines of code right 1
[01:39:54] around 16 million lines of code right 1 million of those are inside the VC
[01:39:56] million of those are inside the VC repository and ffmpeg in total is
[01:39:59] repository and ffmpeg in total is probably two around two right but that
[01:40:01] probably two around two right but that means that so any dependencies outside
[01:40:04] means that so any dependencies outside and if you also look at ffmpeg per se
[01:40:07] and if you also look at ffmpeg per se ffmpeg also is integrating third party
[01:40:09] ffmpeg also is integrating third party libraries like x264 but lib opus and so
[01:40:12] libraries like x264 but lib opus and so many others right so we all depend on
[01:40:14] many others right so we all depend on each other
[01:40:15] each other >> uh yeah that's why I was uh hoping to do
[01:40:17] >> uh yeah that's why I was uh hoping to do this episode as we are doing that just
[01:40:20] this episode as we are doing that just uh kind of joinsf and vc because it's
[01:40:23] uh kind of joinsf and vc because it's really it's it's really two two of the
[01:40:25] really it's it's really two two of the same like I said binary star system and
[01:40:28] same like I said binary star system and we're all just orbiting it uh can we
[01:40:30] we're all just orbiting it uh can we give a shout out to some of the people
[01:40:31] give a shout out to some of the people along the way. We didn't really quite
[01:40:34] along the way. We didn't really quite talk about the history of FFmpeg.
[01:40:37] talk about the history of FFmpeg. So, uh maybe can you tell me about Fris?
[01:40:41] So, uh maybe can you tell me about Fris? Can you tell me about Michael Nemar? Can
[01:40:44] Can you tell me about Michael Nemar? Can you tell me about some of the key
[01:40:45] you tell me about some of the key figures about the the eras of FMPE? Cuz
[01:40:48] figures about the the eras of FMPE? Cuz there there's key eras and key people
[01:40:51] there there's key eras and key people that made this possible. uh Fabric Bard
[01:40:55] that made this possible. uh Fabric Bard as you mentioned creating the concept
[01:40:57] as you mentioned creating the concept and then probably in the 2000's era I
[01:40:59] and then probably in the 2000's era I would call the era era tall of fmpg is
[01:41:01] would call the era era tall of fmpg is the 2000s era was Michael Neidmaya so
[01:41:05] the 2000s era was Michael Neidmaya so key things he got done was exhaustive
[01:41:08] key things he got done was exhaustive support for DivX and Xvid at the time
[01:41:11] support for DivX and Xvid at the time and all sorts of weird variants of
[01:41:12] and all sorts of weird variants of what's known as Meg 4 part 2. So this
[01:41:15] what's known as Meg 4 part 2. So this predates the Meg 4 part 10 that we used
[01:41:18] predates the Meg 4 part 10 that we used to. So this was 2000's era video codecs
[01:41:21] to. So this was 2000's era video codecs where there were oh flavor after flavor
[01:41:24] where there were oh flavor after flavor of weird weird decoders
[01:41:26] of weird weird decoders >> at the time in the 2000s you needed a
[01:41:29] >> at the time in the 2000s you needed a new player to play every different type
[01:41:30] new player to play every different type of file format. So there was Windows
[01:41:32] of file format. So there was Windows Media Player to play Windows media
[01:41:34] Media Player to play Windows media formats. There was real player to play
[01:41:36] formats. There was real player to play real media formats and those were the
[01:41:38] real media formats and those were the other the other key thing in FFmpeg at
[01:41:40] other the other key thing in FFmpeg at the time were native decoders for those.
[01:41:41] the time were native decoders for those. I actually do remember being a teenager.
[01:41:44] I actually do remember being a teenager. I must have been figuring out that there
[01:41:47] I must have been figuring out that there was this one player that could play
[01:41:49] was this one player that could play could decode these files without having
[01:41:51] could decode these files without having separate bloated players cuz at the time
[01:41:53] separate bloated players cuz at the time when you downloaded Real Player there
[01:41:55] when you downloaded Real Player there was a ton of other stuff in there, a ton
[01:41:57] was a ton of other stuff in there, a ton of ads, a ton of other things and just
[01:41:58] of ads, a ton of other things and just having a simple library that was fast
[01:42:01] having a simple library that was fast led to that. And then I think 2008 was
[01:42:04] led to that. And then I think 2008 was it 2008 onwards
[01:42:08] it 2008 onwards was a big change because that's when
[01:42:10] was a big change because that's when H.264 264 got its maturity and and I
[01:42:13] H.264 264 got its maturity and and I think something hopefully we'll talk
[01:42:15] think something hopefully we'll talk about a bit more. This was the beginning
[01:42:16] about a bit more. This was the beginning of high definition video.
[01:42:19] of high definition video. So H.264 was the key decoder of that. So
[01:42:22] So H.264 was the key decoder of that. So I'd call that the late 2000 and 2010s
[01:42:24] I'd call that the late 2000 and 2010s and that's when the big reverse
[01:42:26] and that's when the big reverse engineers came along and really did
[01:42:28] engineers came along and really did astonishing work. The beginning was a
[01:42:31] astonishing work. The beginning was a single player that could play Xvid,
[01:42:33] single player that could play Xvid, DivX,
[01:42:35] DivX, Windows Media, and Real Player was
[01:42:37] Windows Media, and Real Player was already in a massive achievement in
[01:42:38] already in a massive achievement in itself without codec packs, without
[01:42:40] itself without codec packs, without weird stuff you had to download that had
[01:42:41] weird stuff you had to download that had weird ads and weird spyware. VC 1.0 was
[01:42:45] weird ads and weird spyware. VC 1.0 was out on those times, 200, 2009, 2010. And
[01:42:49] out on those times, 200, 2009, 2010. And this is like where it exploded.
[01:42:52] this is like where it exploded. >> Yeah. without codec packs it just works
[01:42:55] >> Yeah. without codec packs it just works across all these different
[01:42:57] across all these different >> the fact so it's just like all the codec
[01:42:59] >> the fact so it's just like all the codec packs are ffmpeg inside VC plus we have
[01:43:02] packs are ffmpeg inside VC plus we have other modules for all the type of
[01:43:04] other modules for all the type of >> at the time that wasn't is there were
[01:43:05] >> at the time that wasn't is there were weird in the 2000s there were weird
[01:43:07] weird in the 2000s there were weird codec packs with DLS coming from this
[01:43:09] codec packs with DLS coming from this place coming with spyware with you know
[01:43:12] place coming with spyware with you know what it wasn't reliable you didn't know
[01:43:14] what it wasn't reliable you didn't know and having a single player that was open
[01:43:16] and having a single player that was open source or single playback module/player
[01:43:19] source or single playback module/player that could do this that was open source
[01:43:21] that could do this that was open source but I think the thing to emphasize is is
[01:43:24] but I think the thing to emphasize is is this task in the 2000s that Michael did
[01:43:26] this task in the 2000s that Michael did was was cisify and it was really the
[01:43:28] was was cisify and it was really the number of edge cases are beyond
[01:43:31] number of edge cases are beyond comprehension in terms of you could have
[01:43:33] comprehension in terms of you could have a Chinese CCTV system that did one weird
[01:43:36] a Chinese CCTV system that did one weird variant of meg 4 part two what's known
[01:43:39] variant of meg 4 part two what's known as meg 4 ASP and that was a weird
[01:43:41] as meg 4 ASP and that was a weird variant and you had to fix that without
[01:43:42] variant and you had to fix that without breaking everybody else
[01:43:44] breaking everybody else >> times a million.
[01:43:45] >> times a million. >> So that's uh so you said that that's
[01:43:47] >> So that's uh so you said that that's where a lot of the reverse engineering
[01:43:48] where a lot of the reverse engineering was happening. It start it started in
[01:43:50] was happening. It start it started in the 2000s with the Windows media stuff
[01:43:52] the 2000s with the Windows media stuff cuz that was proprietary. It started
[01:43:54] cuz that was proprietary. It started with the real media sort of with
[01:43:55] with the real media sort of with Benjamin Lson,
[01:43:56] Benjamin Lson, >> Koska Shishkovk,
[01:43:57] >> Koska Shishkovk, >> Koskashiskov that era those were the key
[01:43:59] >> Koskashiskov that era those were the key that was the key groundwork and then in
[01:44:01] that was the key groundwork and then in in the 2010s was kind of the Por Mahal
[01:44:04] in the 2010s was kind of the Por Mahal Costa era building doing some of the
[01:44:08] Costa era building doing some of the most difficult codecs. Um JB maybe can
[01:44:10] most difficult codecs. Um JB maybe can talk about go to meeting four and go to
[01:44:12] talk about go to meeting four and go to meeting five and
[01:44:13] meeting five and >> what
[01:44:14] >> what >> these what's the go
[01:44:15] >> these what's the go >> so so so so like let's talk about this
[01:44:18] >> so so so so like let's talk about this amazing Ukrainian guy called Costa who
[01:44:22] amazing Ukrainian guy called Costa who was uh at the time living in Germany and
[01:44:25] was uh at the time living in Germany and who was in love with Sweden right and
[01:44:29] who was in love with Sweden right and the guy was the most he he's like like a
[01:44:34] the guy was the most he he's like like a lot of the people in the community are
[01:44:36] lot of the people in the community are very clever he's one of those who are
[01:44:38] very clever he's one of those who are like borderline geniuses, right?
[01:44:40] like borderline geniuses, right? >> He was able to reverse engineering
[01:44:43] >> He was able to reverse engineering extremely complex codecs. Um, and he he
[01:44:46] extremely complex codecs. Um, and he he does that and we do a a bit of engineers
[01:44:48] does that and we do a a bit of engineers with Kieran, but clearly not at this
[01:44:50] with Kieran, but clearly not at this level.
[01:44:51] level. >> Um, he reverse engineered binary blobs
[01:44:54] >> Um, he reverse engineered binary blobs which are 20 megabytes.
[01:44:56] which are 20 megabytes. >> Yeah. So, just for reference, uh, a 1
[01:44:58] >> Yeah. So, just for reference, uh, a 1 megabyte binary blob to reverse engineer
[01:45:00] megabyte binary blob to reverse engineer is probably order of magnitude a month
[01:45:03] is probably order of magnitude a month of work. And this guy is doing 20 30
[01:45:05] of work. And this guy is doing 20 30 megabyte blobs. Maybe we'll talk about
[01:45:07] megabyte blobs. Maybe we'll talk about that in a minute about the subtleties of
[01:45:08] that in a minute about the subtleties of how you do that. But this guy is doing
[01:45:10] how you do that. But this guy is doing it for very difficult and very obscure
[01:45:12] it for very difficult and very obscure codecs
[01:45:13] codecs >> and and did that for fun, right? And and
[01:45:16] >> and and did that for fun, right? And and um so go to meeting uh was a big problem
[01:45:19] um so go to meeting uh was a big problem with VC because that was like the number
[01:45:22] with VC because that was like the number number one uh f request for a long time.
[01:45:24] number one uh f request for a long time. So I put a bounty and the guy at some
[01:45:27] So I put a bounty and the guy at some point say okay GB I'm going to do it.
[01:45:29] point say okay GB I'm going to do it. And in a matter of two months and then
[01:45:32] And in a matter of two months and then he explained how he did it. He was just
[01:45:34] he explained how he did it. He was just like, "Oh, I looked at the code like
[01:45:35] like, "Oh, I looked at the code like this looked like a a DCT that I used to
[01:45:38] this looked like a a DCT that I used to see on WMV and so on." He did that and
[01:45:41] see on WMV and so on." He did that and the funniest part is that the code is
[01:45:44] the funniest part is that the code is written is a ton of jokes and there is
[01:45:48] written is a ton of jokes and there is there is a ton of JB right and and Kemp
[01:45:52] there is a ton of JB right and and Kemp um and Kemp and Costia jokes inside the
[01:45:55] um and Kemp and Costia jokes inside the code. The code is beautiful, right? So,
[01:45:57] code. The code is beautiful, right? So, one of the things I I want to call is uh
[01:46:00] one of the things I I want to call is uh I've gotten a chance to speak to some of
[01:46:02] I've gotten a chance to speak to some of the developers, some of the assembly
[01:46:04] the developers, some of the assembly language level uh uh people and they all
[01:46:06] language level uh uh people and they all always make everything sound like it's
[01:46:08] always make everything sound like it's kind of easy. There's a a kind of
[01:46:11] kind of easy. There's a a kind of humility
[01:46:13] humility because I maybe just the level of what's
[01:46:16] because I maybe just the level of what's required to do this stuff is so high
[01:46:19] required to do this stuff is so high that everything else seems easy I guess
[01:46:21] that everything else seems easy I guess is the lesson to take away from that. So
[01:46:23] is the lesson to take away from that. So in the community like some of the most
[01:46:26] in the community like some of the most impressive people are the ones doing
[01:46:27] impressive people are the ones doing reverse engineering and the other ones
[01:46:28] reverse engineering and the other ones doing the assembly forks right um and
[01:46:32] doing the assembly forks right um and both of those type of people are
[01:46:35] both of those type of people are >> amazing. Um x264 for example became
[01:46:38] >> amazing. Um x264 for example became amazing because of a guy called Lauren
[01:46:41] amazing because of a guy called Lauren Merritt who is was from University of
[01:46:43] Merritt who is was from University of Washington I think
[01:46:44] Washington I think >> at the time. Yeah
[01:46:45] >> at the time. Yeah >> and who was like who made everything
[01:46:48] >> and who was like who made everything great and fast doing a ton of assembly.
[01:46:51] great and fast doing a ton of assembly. Um yeah so this is like the the the the
[01:46:55] Um yeah so this is like the the the the golden era I guess where so many things
[01:46:57] golden era I guess where so many things got
[01:46:57] got >> if you look at Costa for example he
[01:46:59] >> if you look at Costa for example he looked at the world as a binary
[01:47:00] looked at the world as a binary specification he he didn't need
[01:47:02] specification he he didn't need documentation or anything it's I have a
[01:47:04] documentation or anything it's I have a binary and I can figure all of this out
[01:47:06] binary and I can figure all of this out and and he he regularly used the phrase
[01:47:08] and and he he regularly used the phrase binary specification you know it's not a
[01:47:10] binary specification you know it's not a problem and he went and he would go away
[01:47:11] problem and he went and he would go away and he would come back and he he would
[01:47:14] and he would come back and he he would do interesting stuff
[01:47:15] do interesting stuff >> can you actually speak to the details or
[01:47:17] >> can you actually speak to the details or any any add color and texture to what it
[01:47:19] any any add color and texture to what it takes to reverse engineer here a blob.
[01:47:22] takes to reverse engineer here a blob. >> Yeah. So, so let's look at go to meeting
[01:47:23] >> Yeah. So, so let's look at go to meeting for example is a good one because um I
[01:47:26] for example is a good one because um I record a meeting uh on go to meeting for
[01:47:28] record a meeting uh on go to meeting for example. How do I play it back without
[01:47:31] example. How do I play it back without needing this go to meeting player? They
[01:47:34] needing this go to meeting player? They may not even be a player. I may I may
[01:47:36] may not even be a player. I may I may need to send a recording of a meeting to
[01:47:38] need to send a recording of a meeting to someone that doesn't have a player or
[01:47:39] someone that doesn't have a player or whatever. So f first of all there's a
[01:47:42] whatever. So f first of all there's a ton of other stuff there. There's a
[01:47:43] ton of other stuff there. There's a actual video conferencing client you
[01:47:45] actual video conferencing client you need to go and find. It may be easy. it
[01:47:47] need to go and find. It may be easy. it may not be easy to find the actual
[01:47:48] may not be easy to find the actual module doing the decompression.
[01:47:51] module doing the decompression. You need a way to actually dump the YUV
[01:47:54] You need a way to actually dump the YUV data from the module. So often it rever
[01:47:56] data from the module. So often it rever it involves opening in a disassembler
[01:47:58] it involves opening in a disassembler trying to guess where the hooks are to
[01:48:01] trying to guess where the hooks are to incorporate that module and run that
[01:48:04] incorporate that module and run that module natively to decode a sample file.
[01:48:06] module natively to decode a sample file. So figure out where this module is doing
[01:48:09] So figure out where this module is doing the decoding process and and find a way
[01:48:12] the decoding process and and find a way to hook in and output the raw YUV data
[01:48:14] to hook in and output the raw YUV data because you will need that. as a point
[01:48:17] because you will need that. as a point of comparison for when you actually do
[01:48:18] of comparison for when you actually do the reverse engineering because you'll
[01:48:20] the reverse engineering because you'll need to be bit exact or in some cases
[01:48:23] need to be bit exact or in some cases close to bit exact and then you you open
[01:48:26] close to bit exact and then you you open up your disassembler use a lot of
[01:48:28] up your disassembler use a lot of intuition to go and figure out you know
[01:48:30] intuition to go and figure out you know where the DCT is where's entropy coding
[01:48:32] where the DCT is where's entropy coding there there is a kind of not a rule book
[01:48:35] there there is a kind of not a rule book but there's always a pattern of some
[01:48:37] but there's always a pattern of some sort um for example go to meeting you
[01:48:39] sort um for example go to meeting you know it will be a a lot of screen codec
[01:48:41] know it will be a a lot of screen codec tools there's also different variants so
[01:48:44] tools there's also different variants so often I think what's go to meeting four
[01:48:46] often I think what's go to meeting four five
[01:48:46] five >> 2 3 4 I think
[01:48:47] >> 2 3 4 I think >> 234.
[01:48:48] >> 234. >> Yeah. So I should mention here going to
[01:48:50] >> Yeah. So I should mention here going to perplexity go to meeting uses its own
[01:48:52] perplexity go to meeting uses its own proprietary codec for older recorded
[01:48:55] proprietary codec for older recorded sessions historically stored in WMV
[01:48:58] sessions historically stored in WMV files that require a special decoder to
[01:49:00] files that require a special decoder to play properly on Windows. Without this
[01:49:03] play properly on Windows. Without this decoder installed, Windows Media Player
[01:49:06] decoder installed, Windows Media Player and some editors cannot decode the video
[01:49:08] and some editors cannot decode the video track. So you may only hear audio or see
[01:49:11] track. So you may only hear audio or see a black screen. Boy, do I remember that.
[01:49:13] a black screen. Boy, do I remember that. But this is reverse engineering that
[01:49:16] But this is reverse engineering that >> this is key, right? Because because the
[01:49:18] >> this is key, right? Because because the go to meeting is something that not many
[01:49:20] go to meeting is something that not many people know anymore, right? You know
[01:49:22] people know anymore, right? You know about Zoom and and Teams and so on. But
[01:49:23] about Zoom and and Teams and so on. But like now let's fast forward 10 years, 15
[01:49:26] like now let's fast forward 10 years, 15 years. And like this is Goto Meeting.exe
[01:49:29] years. And like this is Goto Meeting.exe for Windows 32 bits, right? Which is
[01:49:31] for Windows 32 bits, right? Which is like, oh yeah, but I'm on Android, I'm
[01:49:33] like, oh yeah, but I'm on Android, I'm on an iPad, I'm somewhere else, right?
[01:49:36] on an iPad, I'm somewhere else, right? How are you going to do that? I'm going
[01:49:37] How are you going to do that? I'm going to be on Ris 5 on ARM. Those are
[01:49:40] to be on Ris 5 on ARM. Those are blocked. But there are tons of files we
[01:49:42] blocked. But there are tons of files we need to support for the future. And this
[01:49:44] need to support for the future. And this is why those type of work are
[01:49:48] is why those type of work are in exceptionally useful for humanity. I
[01:49:51] in exceptionally useful for humanity. I just have to say though that reverse
[01:49:53] just have to say though that reverse engineering process is mindblowing. It's
[01:49:56] engineering process is mindblowing. It's crazy. It's like
[01:49:58] crazy. It's like >> it's a kind of like you know I've been
[01:50:00] >> it's a kind of like you know I've been reading a lot and I interview
[01:50:02] reading a lot and I interview archaeologists. I mean you you just have
[01:50:04] archaeologists. I mean you you just have so little signal. Yes. Yes. You know
[01:50:07] so little signal. Yes. Yes. You know over time you get so much experience you
[01:50:09] over time you get so much experience you understand the structure of the original
[01:50:11] understand the structure of the original coast you can kind of start inferring
[01:50:13] coast you can kind of start inferring basics
[01:50:14] basics >> but you're like
[01:50:17] >> but you're like like like archaeologist with a little
[01:50:19] like like archaeologist with a little brush trying to reconstruct the entire
[01:50:21] brush trying to reconstruct the entire human
[01:50:22] human >> K is too humble but Kieran has done some
[01:50:24] >> K is too humble but Kieran has done some reverse
[01:50:24] reverse >> engine of ciform yeah at the time um
[01:50:26] >> engine of ciform yeah at the time um >> capform nice
[01:50:27] >> capform nice >> yeah at the time before um actually led
[01:50:29] >> yeah at the time before um actually led to the open sourcing of that work and so
[01:50:32] to the open sourcing of that work and so in parallel to doing the binary side you
[01:50:34] in parallel to doing the binary side you obviously have samples in many cases you
[01:50:36] obviously have samples in many cases you don't have many samples. Um, so you have
[01:50:39] don't have many samples. Um, so you have to figure out what all the different
[01:50:40] to figure out what all the different flavors are and you may have a soap form
[01:50:42] flavors are and you may have a soap form for example is actually a collection of
[01:50:44] for example is actually a collection of different approaches and toolkits within
[01:50:46] different approaches and toolkits within that codec cuz often it it grows
[01:50:48] that codec cuz often it it grows naturally and and the hard part is
[01:50:50] naturally and and the hard part is finding the sample that gets you kind of
[01:50:53] finding the sample that gets you kind of somewhere to start without having to
[01:50:55] somewhere to start without having to implement 10 different other things. So
[01:50:58] implement 10 different other things. So start there. I think thankfully at the
[01:51:00] start there. I think thankfully at the time I found a sample by pure chance
[01:51:02] time I found a sample by pure chance that had a lot of flat blocks. It was
[01:51:03] that had a lot of flat blocks. It was animation. And so that really helped a
[01:51:05] animation. And so that really helped a lot because it wasn't using particularly
[01:51:08] lot because it wasn't using particularly complex coding tools etc. And you could
[01:51:10] complex coding tools etc. And you could kind of get somewhere and then and then
[01:51:11] kind of get somewhere and then and then build up and build up until you figure,
[01:51:13] build up and build up until you figure, hey, here's a few bits here. I missed
[01:51:14] hey, here's a few bits here. I missed this. I missed this this if branch that
[01:51:16] this. I missed this this if branch that it does and go, oh,
[01:51:17] it does and go, oh, >> so when we say samples, you mean a
[01:51:19] >> so when we say samples, you mean a sample of videos and then and then
[01:51:21] sample of videos and then and then you're tracking trying to infer like
[01:51:23] you're tracking trying to infer like what is this codec doing by observing
[01:51:26] what is this codec doing by observing the sample and then looking at what at
[01:51:29] the sample and then looking at what at the at the machine
[01:51:30] the at the machine >> the machine code saying I have this bite
[01:51:33] >> the machine code saying I have this bite is six take this branch
[01:51:35] is six take this branch >> and in a different sample. Oh, it's
[01:51:36] >> and in a different sample. Oh, it's >> nuts and and nuts. So you see this is
[01:51:40] >> nuts and and nuts. So you see this is not then you go to things like go to
[01:51:42] not then you go to things like go to meeting it's like
[01:51:43] meeting it's like >> mine was easy imagine
[01:51:45] >> mine was easy imagine >> right
[01:51:45] >> right >> two order of magnitude of more
[01:51:48] >> two order of magnitude of more complexity a guy alone
[01:51:51] complexity a guy alone >> somewhere in Germany doing that
[01:51:53] >> somewhere in Germany doing that >> and for a long time you work you're in a
[01:51:56] >> and for a long time you work you're in a black box
[01:51:57] black box >> because a decoder for a long time
[01:51:58] >> because a decoder for a long time because there is so many step from the
[01:52:00] because there is so many step from the entropy decoding the intra prediction
[01:52:02] entropy decoding the intra prediction the motion prediction the IDCT and so on
[01:52:05] the motion prediction the IDCT and so on for a long time you don't see anything
[01:52:07] for a long time you don't see anything right so you're debugging purely
[01:52:09] right so you're debugging purely in memory debugging guesswork and you
[01:52:11] in memory debugging guesswork and you may have the buffer that the
[01:52:12] may have the buffer that the coefficients are stored in completely
[01:52:14] coefficients are stored in completely wrong and so you may be going down a
[01:52:16] wrong and so you may be going down a complete rabbit hole thinking it's this
[01:52:17] complete rabbit hole thinking it's this and then oh damn that's not that's
[01:52:19] and then oh damn that's not that's that's something else and
[01:52:21] that's something else and >> and you're doing that on binaries that
[01:52:23] >> and you're doing that on binaries that are tens of megabytes
[01:52:25] are tens of megabytes millions of instructions right
[01:52:27] millions of instructions right >> so you're stepping through the debugger
[01:52:29] >> so you're stepping through the debugger like one by one you know instruction by
[01:52:31] like one by one you know instruction by instruction going hey this instruction
[01:52:32] instruction going hey this instruction changes this this does this
[01:52:34] changes this this does this >> posing the program on the CPU level
[01:52:36] >> posing the program on the CPU level >> it on the CPU level watching what's
[01:52:38] >> it on the CPU level watching what's going on trying to figure out
[01:52:39] going on trying to figure out >> sometimes you need to like be in a VM.
[01:52:41] >> sometimes you need to like be in a VM. So yes, you can pause the VM. Yeah,
[01:52:43] So yes, you can pause the VM. Yeah, pause the VM, dump the memory cuz it
[01:52:44] pause the VM, dump the memory cuz it could some of the codecs could have
[01:52:46] could some of the codecs could have encryption. There could be um like DRM
[01:52:48] encryption. There could be um like DRM on there. So you need to dump the memory
[01:52:50] on there. So you need to dump the memory from a virtual machine. Like when I
[01:52:53] from a virtual machine. Like when I joined the Colar in 2003, John Le
[01:52:56] joined the Colar in 2003, John Le Johansson basically broke the DVD
[01:52:58] Johansson basically broke the DVD specification and created DCSS. Showed
[01:53:01] specification and created DCSS. Showed us how he was breaking a DRM which was
[01:53:04] us how he was breaking a DRM which was MP4 fair play from Apple. what he did on
[01:53:07] MP4 fair play from Apple. what he did on his laptop and I was young. I was 21 was
[01:53:10] his laptop and I was young. I was 21 was just like mind-blowing because he was
[01:53:12] just like mind-blowing because he was basically debugging Windows inside a
[01:53:14] basically debugging Windows inside a type of VM with like wow it's
[01:53:18] type of VM with like wow it's incredible. It's mind-blowing and
[01:53:20] incredible. It's mind-blowing and inspiring. Does it get like from your
[01:53:22] inspiring. Does it get like from your experience and from what you've seen in
[01:53:24] experience and from what you've seen in the community does it get discouraging?
[01:53:25] the community does it get discouraging? Does it get people help you? People send
[01:53:27] Does it get people help you? People send you samples. People are keen. Sometimes
[01:53:30] you samples. People are keen. Sometimes you don't have access to an encoder. So
[01:53:32] you don't have access to an encoder. So this is even more difficult because you
[01:53:34] this is even more difficult because you just you just ask and you have to ask
[01:53:36] just you just ask and you have to ask for samples. I remember video land you
[01:53:38] for samples. I remember video land you used to tweet for samples at one stage.
[01:53:40] used to tweet for samples at one stage. Hey, I need this obscure sample and
[01:53:42] Hey, I need this obscure sample and >> for a long time I was oh I need this
[01:53:43] >> for a long time I was oh I need this codec and I need this codec
[01:53:45] codec and I need this codec >> and and and if you were really lucky you
[01:53:46] >> and and and if you were really lucky you would find like if you were if you were
[01:53:48] would find like if you were if you were unlucky you'd get like one or you'd get
[01:53:50] unlucky you'd get like one or you'd get nothing or you get one or two and then
[01:53:51] nothing or you get one or two and then they would sometimes you'd find a gold
[01:53:53] they would sometimes you'd find a gold mine like yeah my company has a 100,000
[01:53:55] mine like yeah my company has a 100,000 of these files cuz we depended on it for
[01:53:56] of these files cuz we depended on it for some reason. Um and so those are those
[01:53:59] some reason. Um and so those are those are kind of the best if if because then
[01:54:01] are kind of the best if if because then they can test bit exactness across the
[01:54:04] they can test bit exactness across the huge range of coding tools.
[01:54:06] huge range of coding tools. >> Can you explain bit exactness?
[01:54:07] >> Can you explain bit exactness? >> Bit exactness. So um most but not all
[01:54:11] >> Bit exactness. So um most but not all video codecs certainly from about the
[01:54:14] video codecs certainly from about the 2000s onwards have a bit exact
[01:54:17] 2000s onwards have a bit exact definition. So every implementation must
[01:54:19] definition. So every implementation must produce exactly the same bits bit for
[01:54:23] produce exactly the same bits bit for bits in exactly the same data that comes
[01:54:25] bits in exactly the same data that comes out of a decoder
[01:54:26] out of a decoder >> for like a large number of samples
[01:54:28] >> for like a large number of samples >> for a given sample. So Lex's
[01:54:30] >> for a given sample. So Lex's implementation, JB's implementation and
[01:54:32] implementation, JB's implementation and my implementation of H.264 must match
[01:54:34] my implementation of H.264 must match bit exactly. That wasn't the case in the
[01:54:37] bit exactly. That wasn't the case in the '90s of Meg 2. Probably fair to say one
[01:54:40] '90s of Meg 2. Probably fair to say one of the biggest mistakes the video
[01:54:42] of the biggest mistakes the video industry made. And I think people who
[01:54:43] industry made. And I think people who were in the room in '92, I most of both
[01:54:46] were in the room in '92, I most of both of us were in diapers I suspect, but um
[01:54:48] of us were in diapers I suspect, but um have acknowledged um I would give a
[01:54:50] have acknowledged um I would give a shout out to Yuri Resnik. He's
[01:54:53] shout out to Yuri Resnik. He's acknowledged that was one of the big
[01:54:54] acknowledged that was one of the big mistakes of the era. And you're saying
[01:54:56] mistakes of the era. And you're saying the encoders needed to be able to run
[01:54:58] the encoders needed to be able to run tests and then the the the bit
[01:55:00] tests and then the the the bit exactness.
[01:55:02] exactness. I mean that's a nice thing to guarantee
[01:55:04] I mean that's a nice thing to guarantee like there's a parallel sort of
[01:55:05] like there's a parallel sort of development here on uh the way the web
[01:55:08] development here on uh the way the web browser works which is a you know takes
[01:55:11] browser works which is a you know takes HTML and displays it and there's no bit
[01:55:13] HTML and displays it and there's no bit exactness there across the different
[01:55:15] exactness there across the different engine. I would I would point out
[01:55:16] engine. I would I would point out actually FFmpeg is unique in the sense
[01:55:18] actually FFmpeg is unique in the sense that it's it has been a winner takes all
[01:55:21] that it's it has been a winner takes all scenario. You have brow browsers is a
[01:55:23] scenario. You have brow browsers is a good analogy because it has to parse a
[01:55:25] good analogy because it has to parse a lot of different content and render it
[01:55:26] lot of different content and render it in a particular way like a decoder but
[01:55:29] in a particular way like a decoder but there still are multiple browser
[01:55:30] there still are multiple browser engines. There's Firefox's one, there's
[01:55:32] engines. There's Firefox's one, there's Chrome's one, there's a few Japanese
[01:55:33] Chrome's one, there's a few Japanese ones that are pretty decent. That's not
[01:55:36] ones that are pretty decent. That's not been the case in multimedia in general
[01:55:38] been the case in multimedia in general um across a wide range of codecs. FFmpeg
[01:55:40] um across a wide range of codecs. FFmpeg has kind of won it all, I suppose, in a
[01:55:43] has kind of won it all, I suppose, in a sense because of because of the fact
[01:55:45] sense because of because of the fact that you can get every new codec added
[01:55:47] that you can get every new codec added is actually worth more than the value of
[01:55:49] is actually worth more than the value of that codec itself because it makes the
[01:55:51] that codec itself because it makes the whole thing better.
[01:55:52] whole thing better. >> Man, this is really cool. Uh going to
[01:55:54] >> Man, this is really cool. Uh going to Perplexity, Yuri Resnik is a multimedia
[01:55:56] Perplexity, Yuri Resnik is a multimedia and signal processing researcher. got
[01:55:58] and signal processing researcher. got his PhD in computer science from KE
[01:56:00] his PhD in computer science from KE University with over 150 papers in more
[01:56:03] University with over 150 papers in more than 80 granted US patents, contributor
[01:56:06] than 80 granted US patents, contributor to major multimedia standards including
[01:56:08] to major multimedia standards including H64, MEG 4 AVC, H265,
[01:56:14] H64, MEG 4 AVC, H265, uh, MEG 4 ALS, G718.
[01:56:17] uh, MEG 4 ALS, G718. >> G71 is telco stuff.
[01:56:19] >> G71 is telco stuff. >> And so he was more connected to
[01:56:21] >> And so he was more connected to >> real audio, real video, right? That was
[01:56:23] >> real audio, real video, right? That was >> Oh yeah,
[01:56:24] >> Oh yeah, >> very important. coder Bride Cove context
[01:56:27] >> very important. coder Bride Cove context is man I need to hang out with Yuri he's
[01:56:29] is man I need to hang out with Yuri he's legit
[01:56:30] legit >> and he's like one of the most nice
[01:56:32] >> and he's like one of the most nice person ever right like like um for
[01:56:35] person ever right like like um for example for for my for my startup uh
[01:56:38] example for for my for my startup uh that I'm doing right now called Kyber
[01:56:40] that I'm doing right now called Kyber right I met Yuri because I met him every
[01:56:42] right I met Yuri because I met him every year at the mile high video uh
[01:56:44] year at the mile high video uh conference which is in Denver um and he
[01:56:47] conference which is in Denver um and he he gave me like so much good ideas and
[01:56:50] he gave me like so much good ideas and good things he's like really amazing
[01:56:52] good things he's like really amazing person
[01:56:52] person >> he tell he tells us how how you how
[01:56:55] >> he tell he tells us how how you how great it is to be, you know, even know
[01:56:56] great it is to be, you know, even know us and just, you know, you look at that
[01:56:58] us and just, you know, you look at that and it's I think it's I think it's the
[01:57:00] and it's I think it's I think it's the other way around. Yuri,
[01:57:01] other way around. Yuri, >> that reminds me of a thing that you uh
[01:57:03] >> that reminds me of a thing that you uh mentioned to me about fate testing and
[01:57:05] mentioned to me about fate testing and like the insanely rigorous process
[01:57:07] like the insanely rigorous process that's used to test everything that's
[01:57:10] that's used to test everything that's incorporated into um into FFmpeg. Can
[01:57:13] incorporated into um into FFmpeg. Can you take me through the testing process?
[01:57:14] you take me through the testing process? Yeah. So, FFmpeg has a system called
[01:57:16] Yeah. So, FFmpeg has a system called Fate FFmpeg automated testing
[01:57:18] Fate FFmpeg automated testing environment.
[01:57:19] environment. >> Because FFmpeg runs on so many different
[01:57:21] >> Because FFmpeg runs on so many different OSS and can be compiled with so many
[01:57:23] OSS and can be compiled with so many different compilers,
[01:57:25] different compilers, there's been a crazy number of
[01:57:27] there's been a crazy number of configurations. So, you can see the
[01:57:30] configurations. So, you can see the absurd combination of um compiler
[01:57:33] absurd combination of um compiler variants, operating system variants,
[01:57:36] variants, operating system variants, instruction sets. You can see at the top
[01:57:38] instruction sets. You can see at the top Mac OS has tons of different variants
[01:57:40] Mac OS has tons of different variants because it has iOS, it has tvOS. I'm
[01:57:42] because it has iOS, it has tvOS. I'm looking at a page fate. Ffmpeg.org
[01:57:47] looking at a page fate. Ffmpeg.org 81 minutes ago, 76 minutes ago. Looking
[01:57:50] 81 minutes ago, 76 minutes ago. Looking at the different architectures, the
[01:57:52] at the different architectures, the operating system, the different
[01:57:53] operating system, the different compilers, Apple clang version,
[01:57:57] compilers, Apple clang version, uh
[01:57:57] uh >> combinations are the combination.
[01:58:00] >> combinations are the combination. >> These are all run by volunteers. So
[01:58:01] >> These are all run by volunteers. So these are all volunteer systems. The
[01:58:03] these are all volunteer systems. The ones at the top, for example, the max I
[01:58:05] ones at the top, for example, the max I host in my office for example, uh host
[01:58:08] host in my office for example, uh host all sorts of different stuff. Other
[01:58:09] all sorts of different stuff. Other people host other things. So it's really
[01:58:12] people host other things. So it's really there to make sure and because FFmpeg
[01:58:14] there to make sure and because FFmpeg does quite complex C code for example
[01:58:17] does quite complex C code for example you do have miso compilations. So the
[01:58:19] you do have miso compilations. So the compiler will sometimes compile C code
[01:58:22] compiler will sometimes compile C code incorrectly for example. This happens
[01:58:24] incorrectly for example. This happens once in a while.
[01:58:25] once in a while. >> Oh there's like there's a log of all the
[01:58:27] >> Oh there's like there's a log of all the compilations.
[01:58:28] compilations. >> Yeah log of all the compilations all the
[01:58:30] >> Yeah log of all the compilations all the tests. I think one of the other ones
[01:58:31] tests. I think one of the other ones will show all the tests passing.
[01:58:33] will show all the tests passing. >> If you click you can see all the tests
[01:58:35] >> If you click you can see all the tests >> back all tests successful
[01:58:37] >> back all tests successful >> in logs test. Yeah. So you see all those
[01:58:40] >> in logs test. Yeah. So you see all those tests are passing
[01:58:42] tests are passing of all the different codecs, all the
[01:58:44] of all the different codecs, all the different filter transformations,
[01:58:46] different filter transformations, all the the level of scale is is quite
[01:58:49] all the the level of scale is is quite crazy on all the combinations. It's it's
[01:58:51] crazy on all the combinations. It's it's not just a matrix at this point. It's
[01:58:53] not just a matrix at this point. It's like a pivot table of different
[01:58:54] like a pivot table of different combinations.
[01:58:56] combinations. >> That's nuts.
[01:58:57] >> That's nuts. >> And it's a key part of what we do
[01:58:58] >> And it's a key part of what we do because you you may be able to test
[01:59:00] because you you may be able to test something locally. you make a change but
[01:59:03] something locally. you make a change but actually that breaks GCC version 11 on
[01:59:07] actually that breaks GCC version 11 on Mac or something like that and you're
[01:59:09] Mac or something like that and you're able to then fix that. We also have miso
[01:59:11] able to then fix that. We also have miso compilation. So the C code sometimes the
[01:59:13] compilation. So the C code sometimes the compiler can have a bug in it where it
[01:59:14] compiler can have a bug in it where it creates the wrong output and that can
[01:59:17] creates the wrong output and that can have quite a big effect sometimes on a
[01:59:18] have quite a big effect sometimes on a vid on a video because of the way frames
[01:59:21] vid on a video because of the way frames have dependencies. Even a small change
[01:59:23] have dependencies. Even a small change in the output can can cascade to
[01:59:26] in the output can can cascade to actually quite big glitches. You see
[01:59:28] actually quite big glitches. You see power PC, you see Risk, you see PC,
[01:59:31] power PC, you see Risk, you see PC, there was Risk, there was weird stuff in
[01:59:32] there was Risk, there was weird stuff in the past like DEC alpha,
[01:59:35] the past like DEC alpha, >> you see Visual Studio different,
[01:59:38] >> you see Visual Studio different, Intel compiler, Apple client, you name
[01:59:40] Intel compiler, Apple client, you name it.
[01:59:41] it. >> What are some of the pain points like
[01:59:43] >> What are some of the pain points like maybe do you have emotional triggers uh
[01:59:47] maybe do you have emotional triggers uh maybe nightmares about a particular
[01:59:49] maybe nightmares about a particular operating system, a particular container
[01:59:52] operating system, a particular container codec combination?
[01:59:53] codec combination? >> For me, it's really easy because so I
[01:59:55] >> For me, it's really easy because so I have a day job. Um my company builds the
[01:59:58] have a day job. Um my company builds the company I started builds um equipment
[02:00:01] company I started builds um equipment for broadcasting sports matches between
[02:00:02] for broadcasting sports matches between TV stadiums and studios for example. We
[02:00:05] TV stadiums and studios for example. We have to work with 10 bit video and 10
[02:00:08] have to work with 10 bit video and 10 bit video has um a set of challenges
[02:00:10] bit video has um a set of challenges that you can't process 10 bit data
[02:00:13] that you can't process 10 bit data natively on a CPU. So that means you
[02:00:15] natively on a CPU. So that means you have to stick it in um 16 bits. So that
[02:00:19] have to stick it in um 16 bits. So that means you have six wasted bits. So
[02:00:21] means you have six wasted bits. So there's different packing formats to
[02:00:23] there's different packing formats to actually pack the data more efficiently
[02:00:24] actually pack the data more efficiently because when you send that over a
[02:00:26] because when you send that over a network you lose because you need to you
[02:00:28] network you lose because you need to you need to save that 40%. For example on
[02:00:30] need to save that 40%. For example on PCI Express you may only have bus
[02:00:32] PCI Express you may only have bus bandwidth to do that and so I think
[02:00:34] bandwidth to do that and so I think internally we have about some some are
[02:00:36] internally we have about some some are some are industry ones and some are
[02:00:38] some are industry ones and some are internal to our own hardware that we
[02:00:40] internal to our own hardware that we build. We have a I think a 5x5 or 6x6
[02:00:44] build. We have a I think a 5x5 or 6x6 matrix of every single format to every
[02:00:46] matrix of every single format to every single other format conversion. In fact,
[02:00:49] single other format conversion. In fact, one one of them I sent you and they're
[02:00:50] one one of them I sent you and they're all written in handwritten assembly and
[02:00:52] all written in handwritten assembly and they're all written and they all support
[02:00:54] they're all written and they all support different CPU generations.
[02:00:57] different CPU generations. So, this is really traumatic handling
[02:00:59] So, this is really traumatic handling all these different combinations
[02:01:01] all these different combinations times a million.
[02:01:02] times a million. >> By the way, the company you're talking
[02:01:03] >> By the way, the company you're talking about is Open Broadcast.
[02:01:04] about is Open Broadcast. >> No, no relation to the free OBS
[02:01:06] >> No, no relation to the free OBS streaming service.
[02:01:07] streaming service. >> Yeah. But JB and I have started
[02:01:10] >> Yeah. But JB and I have started companies broadly speaking around the
[02:01:12] companies broadly speaking around the FFmpeg VC ethos. So, that's really
[02:01:15] FFmpeg VC ethos. So, that's really low-level work. So, so in most companies
[02:01:18] low-level work. So, so in most companies this this wouldn't be written in
[02:01:19] this this wouldn't be written in assembly. It would be accepted that C C
[02:01:22] assembly. It would be accepted that C C is fast. Um, as you can see from that C
[02:01:24] is fast. Um, as you can see from that C is not fast. Um,
[02:01:26] is not fast. Um, >> uh, so here it says 62 times faster than
[02:01:30] >> uh, so here it says 62 times faster than C.
[02:01:31] C. >> Yeah. So it's taking those the ethos of
[02:01:34] >> Yeah. So it's taking those the ethos of doing low-level programming, real-time
[02:01:37] doing low-level programming, real-time programming
[02:01:39] programming and take using that for commercial
[02:01:40] and take using that for commercial applications and JB and I have started
[02:01:42] applications and JB and I have started companies around that in many cases
[02:01:44] companies around that in many cases hiring developers from the open source
[02:01:46] hiring developers from the open source community
[02:01:48] community to use that ethos and and so that that's
[02:01:51] to use that ethos and and so that that's a great example of of some of the things
[02:01:54] a great example of of some of the things we're doing. In most companies, it would
[02:01:55] we're doing. In most companies, it would be say I'll write this in C and it's
[02:01:57] be say I'll write this in C and it's fast and we're done. But actually, you
[02:01:58] fast and we're done. But actually, you can get a lot better. For me like some
[02:02:01] can get a lot better. For me like some of the headaches we have is around some
[02:02:05] of the headaches we have is around some OS that are difficult to support, right?
[02:02:07] OS that are difficult to support, right? Because um uh
[02:02:10] Because um uh if you look at VC and thanks to Fate and
[02:02:13] if you look at VC and thanks to Fate and Fmpeg, we run on the last version of VC
[02:02:17] Fmpeg, we run on the last version of VC runs on Windows XP and still run there
[02:02:20] runs on Windows XP and still run there and runs on Windows 11. Um we work on
[02:02:24] and runs on Windows 11. Um we work on Mac OS 10.7 to the latest Mac OS
[02:02:28] Mac OS 10.7 to the latest Mac OS whatever it is, right? 26. Um we work on
[02:02:31] whatever it is, right? 26. Um we work on iOS since iOS 9. Well, we are actually
[02:02:34] iOS since iOS 9. Well, we are actually iOS 26, right? We support um we support
[02:02:38] iOS 26, right? We support um we support uh many types of Linuxes, BSD, Solaris.
[02:02:42] uh many types of Linuxes, BSD, Solaris. The last version still runs on OS2,
[02:02:44] The last version still runs on OS2, right? like there is maybe 10 users of
[02:02:47] right? like there is maybe 10 users of OS2 in the world and one of them is
[02:02:49] OS2 in the world and one of them is maintaining VC then you realize that
[02:02:51] maintaining VC then you realize that this very small team around VC and using
[02:02:54] this very small team around VC and using ffmpeg codecs and and all the other ones
[02:02:56] ffmpeg codecs and and all the other ones support more oss than Microsoft or
[02:03:00] support more oss than Microsoft or Google or Apple and they have infinite
[02:03:03] Google or Apple and they have infinite amount of uh of uh power and resources
[02:03:07] amount of uh of uh power and resources but for example the worst is iOS um for
[02:03:11] but for example the worst is iOS um for in order to build on iOS 9 we need to do
[02:03:14] in order to build on iOS 9 we need to do some very clever mixing of several
[02:03:17] some very clever mixing of several version of the Xcode ID and SDK from
[02:03:21] version of the Xcode ID and SDK from Apple from several versions and do a
[02:03:23] Apple from several versions and do a type of Frankenstein version of that so
[02:03:26] type of Frankenstein version of that so that we can still support iOS 9 which is
[02:03:28] that we can still support iOS 9 which is not supported at all by the the compiler
[02:03:31] not supported at all by the the compiler of uh of uh Apple in order to still run
[02:03:34] of uh of uh Apple in order to still run on ARM 32 um on iOS 9 and you've seen on
[02:03:38] on ARM 32 um on iOS 9 and you've seen on fate that it was still supporting iOS 9,
[02:03:41] fate that it was still supporting iOS 9, right? So, um, my headaches are mostly
[02:03:44] right? So, um, my headaches are mostly related to the the support of so many
[02:03:47] related to the the support of so many OSS and it's important because like we
[02:03:50] OSS and it's important because like we receive so many people saying, "Hey,
[02:03:52] receive so many people saying, "Hey, thank you. I still have my iPad 2 to
[02:03:54] thank you. I still have my iPad 2 to watch movies and it still works on iOS
[02:03:57] watch movies and it still works on iOS 9, right?" And it's also an impact of
[02:04:00] 9, right?" And it's also an impact of like not forcing people to buy new
[02:04:02] like not forcing people to buy new hardware when it works fine if you
[02:04:04] hardware when it works fine if you optimize it correctly. Which brings us
[02:04:06] optimize it correctly. Which brings us to what we were saying about assembly.
[02:04:08] to what we were saying about assembly. It's also fighting like the fact that
[02:04:11] It's also fighting like the fact that you need to buy something new non-stop
[02:04:13] you need to buy something new non-stop uh while you could optimize more which
[02:04:15] uh while you could optimize more which is a lost art. You got to tell me about
[02:04:20] is a lost art. You got to tell me about this lost art or this uh
[02:04:23] this lost art or this uh >> the the carriers of the flame of
[02:04:26] >> the the carriers of the flame of assembly. What what is what what is
[02:04:29] assembly. What what is what what is assembly? Why is it beautiful? Why is it
[02:04:32] assembly? Why is it beautiful? Why is it challenging? How does it work? So when
[02:04:35] challenging? How does it work? So when you write assembly code, you write this
[02:04:36] you write assembly code, you write this using the instructions the actual
[02:04:38] using the instructions the actual processor is using directly. So most of
[02:04:41] processor is using directly. So most of the time you would write in a language,
[02:04:43] the time you would write in a language, let's take C is a good example. The
[02:04:44] let's take C is a good example. The compiler would use that to create
[02:04:48] compiler would use that to create um assembly language and machine code
[02:04:49] um assembly language and machine code instructions for you based off your C
[02:04:52] instructions for you based off your C code. And there's a specific flavor of
[02:04:55] code. And there's a specific flavor of assembly that we use in FFmpeg that's
[02:04:57] assembly that we use in FFmpeg that's called SIMD, SIMD, single instruction
[02:04:59] called SIMD, SIMD, single instruction multiple data.
[02:05:01] multiple data. So this means for example say say I want
[02:05:04] So this means for example say say I want to add five to a number in scala
[02:05:08] to add five to a number in scala assembly. So this is what's known as you
[02:05:09] assembly. So this is what's known as you work on an individual element. So I want
[02:05:11] work on an individual element. So I want to I have a number of I have the number
[02:05:12] to I have a number of I have the number 10 and I want to add five. I use the add
[02:05:14] 10 and I want to add five. I use the add instruction and I add five to 10 and I
[02:05:18] instruction and I add five to 10 and I get 15. With SIMD with SIMD I can have a
[02:05:21] get 15. With SIMD with SIMD I can have a whole vector of 16 different numbers or
[02:05:23] whole vector of 16 different numbers or could all be different. If I want to add
[02:05:25] could all be different. If I want to add five to that, I can I can run one
[02:05:27] five to that, I can I can run one instruction
[02:05:29] instruction and that one instruction
[02:05:31] and that one instruction sums all 16 elements. And that, as you
[02:05:35] sums all 16 elements. And that, as you can imagine, lends itself very well to
[02:05:37] can imagine, lends itself very well to video. Video video is, you know, a pixel
[02:05:40] video. Video video is, you know, a pixel grid. So I can perform operations on
[02:05:43] grid. So I can perform operations on multiple pixels at the same time. The
[02:05:45] multiple pixels at the same time. The key thing that we do differently in
[02:05:46] key thing that we do differently in FFmpeg is we don't use any abstractions
[02:05:49] FFmpeg is we don't use any abstractions or any major abstractions on top of
[02:05:51] or any major abstractions on top of that. So there's a part of the world
[02:05:53] that. So there's a part of the world that uses what's known as intrinsics. So
[02:05:55] that uses what's known as intrinsics. So these are C functions that behave very
[02:05:57] these are C functions that behave very similarly but not quite the same to
[02:06:00] similarly but not quite the same to writing assembly by hand. So the
[02:06:02] writing assembly by hand. So the registers that data uh is stored in on
[02:06:06] registers that data uh is stored in on the CPU. The compiler allocates those
[02:06:09] the CPU. The compiler allocates those for you. And so the key thing to
[02:06:11] for you. And so the key thing to understand was when we write SIMD is we
[02:06:12] understand was when we write SIMD is we have a 10x and not percentage 10x to 50x
[02:06:18] have a 10x and not percentage 10x to 50x speed improvement that that function is
[02:06:20] speed improvement that that function is 62x
[02:06:21] 62x >> that's nuts
[02:06:22] >> that's nuts >> on the ffmpeg account as you know posts
[02:06:24] >> on the ffmpeg account as you know posts and tweets a lot about that to try and
[02:06:26] and tweets a lot about that to try and say hey we're doing this stuff
[02:06:29] say hey we're doing this stuff >> you are a person who sees the beauty in
[02:06:31] >> you are a person who sees the beauty in assembly but it's also extremely useful
[02:06:33] assembly but it's also extremely useful for these kinds of application to
[02:06:35] for these kinds of application to actually significantly outperform even C
[02:06:39] actually significantly outperform even C which is correct
[02:06:40] which is correct it is necessary, right? Because like one
[02:06:43] it is necessary, right? Because like one of the projects um that we need to talk
[02:06:45] of the projects um that we need to talk about is called David, right? So David
[02:06:48] about is called David, right? So David is a decoder for the format uh that was
[02:06:52] is a decoder for the format uh that was done by Aliens for Open Media, uh which
[02:06:55] done by Aliens for Open Media, uh which is an a video decoder called AV1.
[02:06:58] is an a video decoder called AV1. >> So for people who don't know, we've been
[02:07:00] >> So for people who don't know, we've been talking about H.264.
[02:07:02] talking about H.264. AV1 is another hugely popular standard
[02:07:06] AV1 is another hugely popular standard and codec that is increasingly taking
[02:07:10] and codec that is increasingly taking over uh the internet and when this
[02:07:13] over uh the internet and when this format was launched uh many people said
[02:07:16] format was launched uh many people said especially even from the aliens from
[02:07:17] especially even from the aliens from open media right which is Google,
[02:07:19] open media right which is Google, Netflix, Amazon, Modzilla they say well
[02:07:22] Netflix, Amazon, Modzilla they say well this format is so complex it must be
[02:07:24] this format is so complex it must be done in hardware to do decoding right
[02:07:27] done in hardware to do decoding right and well I arrived uh with a few other
[02:07:30] and well I arrived uh with a few other people uh mostly um Ronald uh Henrik and
[02:07:34] people uh mostly um Ronald uh Henrik and Martin and we said we need to have an
[02:07:36] Martin and we said we need to have an extremely good software decoder uh
[02:07:39] extremely good software decoder uh because it's going to take time to have
[02:07:41] because it's going to take time to have hardware and so we wrote this project
[02:07:44] hardware and so we wrote this project which is beyond insane um we are talking
[02:07:48] which is beyond insane um we are talking about 30,000 line of C but 240,000
[02:07:54] about 30,000 line of C but 240,000 lines of handwritten assembly right
[02:07:56] lines of handwritten assembly right handwritten assembly 240,000
[02:08:00] handwritten assembly 240,000 lines Nice.
[02:08:02] lines Nice. >> That's incredible. That mean I mean some
[02:08:04] >> That's incredible. That mean I mean some of the stuff we're talking about is
[02:08:05] of the stuff we're talking about is probably the biggest assembly
[02:08:08] probably the biggest assembly uh code bases to give you an idea and
[02:08:11] uh code bases to give you an idea and Ken can correct me but I think the
[02:08:12] Ken can correct me but I think the ffmpeg has 100,000 lines of assembly for
[02:08:16] ffmpeg has 100,000 lines of assembly for all the codecs and just this one has
[02:08:19] all the codecs and just this one has 240,000.
[02:08:20] 240,000. Um it's a vidoland project of course um
[02:08:23] Um it's a vidoland project of course um and it is optimized at the maximum
[02:08:28] and it is optimized at the maximum because the moto when we starting the
[02:08:30] because the moto when we starting the project is every cycle matters right
[02:08:34] project is every cycle matters right every cycle matters because David is
[02:08:36] every cycle matters because David is used in VC and in some software AV1
[02:08:39] used in VC and in some software AV1 playback stacks we are talking about
[02:08:41] playback stacks we are talking about probably 3 billion devices which are
[02:08:45] probably 3 billion devices which are going to decode video non-stop because
[02:08:47] going to decode video non-stop because for example 30% of the video from
[02:08:49] for example 30% of the video from Netflix are now in everyone on 50% of
[02:08:52] Netflix are now in everyone on 50% of YouTube, right? So, and you often don't
[02:08:54] YouTube, right? So, and you often don't have a hardware decoder because not many
[02:08:57] have a hardware decoder because not many devices have a hardware decoder. And
[02:08:59] devices have a hardware decoder. And with David, we realize that in one or
[02:09:01] with David, we realize that in one or two cores, you were able to decode 720p
[02:09:04] two cores, you were able to decode 720p correctly. So, it is like literally
[02:09:08] correctly. So, it is like literally >> incredible, right,
[02:09:09] >> incredible, right, >> David? Look at that.
[02:09:10] >> David? Look at that. >> Yeah. So, this is uh another spicy tweet
[02:09:14] >> Yeah. So, this is uh another spicy tweet from you. This is what peak video codec
[02:09:16] from you. This is what peak video codec should look like. 79.9% assembly, 19.6%
[02:09:21] should look like. 79.9% assembly, 19.6% C and uh 0.5%
[02:09:24] C and uh 0.5% other. And what's incredible is with
[02:09:27] other. And what's incredible is with those tweets, which is factual, people
[02:09:30] those tweets, which is factual, people get
[02:09:31] get crazy. They are unhappy, right? They
[02:09:33] crazy. They are unhappy, right? They say,
[02:09:33] say, >> well, yeah, for the last two years, they
[02:09:34] >> well, yeah, for the last two years, they go crazy. No, intrinsics is fine. The
[02:09:36] go crazy. No, intrinsics is fine. The compiler is Oh, they go
[02:09:38] compiler is Oh, they go >> you can optimize your compiler auto
[02:09:39] >> you can optimize your compiler auto vectorization. It's your fault. You
[02:09:41] vectorization. It's your fault. You don't understand. And we've tried that
[02:09:43] don't understand. And we've tried that forever, right?
[02:09:44] forever, right? >> For two years. and two years later
[02:09:46] >> For two years. and two years later showing hundreds of examples of of
[02:09:49] showing hundreds of examples of of handwritten assembly. No, no, no. You're
[02:09:51] handwritten assembly. No, no, no. You're doing it wrong. The compiler can do
[02:09:52] doing it wrong. The compiler can do this.
[02:09:52] this. >> So, we should actually just articulate a
[02:09:54] >> So, we should actually just articulate a little clearer. So, the intuition there
[02:09:56] little clearer. So, the intuition there from the software engineering folks when
[02:09:58] from the software engineering folks when you have code like okay, let's just take
[02:10:00] you have code like okay, let's just take an example C++. There's a compiler
[02:10:03] an example C++. There's a compiler that's doing a lot of the optimization.
[02:10:05] that's doing a lot of the optimization. >> Yes. And the presumption is if you have
[02:10:08] >> Yes. And the presumption is if you have a good enough compiler, if you continue
[02:10:10] a good enough compiler, if you continue to improve the compiler, you're going to
[02:10:12] to improve the compiler, you're going to generate code that can perform like
[02:10:15] generate code that can perform like optimal performance. You cannot possibly
[02:10:17] optimal performance. You cannot possibly beat it. And you're consistently
[02:10:19] beat it. And you're consistently challenging that thought that if you
[02:10:21] challenging that thought that if you >> by orders of magnitude
[02:10:22] >> by orders of magnitude >> by orders of magnitude, handcrafted
[02:10:24] >> by orders of magnitude, handcrafted assembly can outperform
[02:10:26] assembly can outperform >> C. The two things that they tell us is,
[02:10:29] >> C. The two things that they tell us is, yeah, but modern compilers have auto
[02:10:31] yeah, but modern compilers have auto vectorization, right? because SIMD that
[02:10:33] vectorization, right? because SIMD that we're doing is vectorization and like
[02:10:36] we're doing is vectorization and like it's not even close, right? It's not
[02:10:38] it's not even close, right? It's not even close, right? It's not like 5% 10%
[02:10:41] even close, right? It's not like 5% 10% slower. It's multiple times slower. So,
[02:10:43] slower. It's multiple times slower. So, can we I don't know if you can say
[02:10:44] can we I don't know if you can say something philosophically because
[02:10:46] something philosophically because there's a lot of there's a lot of great
[02:10:48] there's a lot of there's a lot of great software engineers, great engineers,
[02:10:50] software engineers, great engineers, great machine learning people like
[02:10:52] great machine learning people like Karpathy will listen to this and it's
[02:10:53] Karpathy will listen to this and it's like what's the intuition he's supposed
[02:10:55] like what's the intuition he's supposed to get from this? What are we supposed
[02:10:56] to get from this? What are we supposed to
[02:10:57] to >> He learned assembly because of the
[02:10:58] >> He learned assembly because of the tweets by the way. just he thought he
[02:11:00] tweets by the way. just he thought he went he's like oh I think this is a let
[02:11:02] went he's like oh I think this is a let me figure out and and you know the way
[02:11:03] me figure out and and you know the way he documents his work and so
[02:11:05] he documents his work and so >> philosophically what's important to
[02:11:07] >> philosophically what's important to realize is that we passed the time where
[02:11:11] realize is that we passed the time where hardware was going so much faster right
[02:11:13] hardware was going so much faster right we at the end of the m low we have
[02:11:15] we at the end of the m low we have limitation for uh for AI for memory you
[02:11:18] limitation for uh for AI for memory you need to go down in the stack and
[02:11:20] need to go down in the stack and optimize more to get more power from
[02:11:23] optimize more to get more power from what you have because uh request for
[02:11:26] what you have because uh request for power CPU power GPU power are exploding
[02:11:30] power CPU power GPU power are exploding while the hardware is not exploding in
[02:11:34] while the hardware is not exploding in speed, right? So you what people do is
[02:11:37] speed, right? So you what people do is that they add more cores, right? But
[02:11:38] that they add more cores, right? But that's basically like at some point you
[02:11:41] that's basically like at some point you can add 250 coursees, right? So what we
[02:11:43] can add 250 coursees, right? So what we do is to take every inch of the machine.
[02:11:46] do is to take every inch of the machine. >> Not just that, not just that we abuse
[02:11:48] >> Not just that, not just that we abuse the machine. We we go and use we use the
[02:11:51] the machine. We we go and use we use the machine in ways that the that the
[02:11:53] machine in ways that the that the creator didn't expect. Sometimes we use
[02:11:55] creator didn't expect. Sometimes we use an instruction that's completely
[02:11:56] an instruction that's completely unrelated to what we do. We use a
[02:11:58] unrelated to what we do. We use a cryptography instruction in video
[02:12:00] cryptography instruction in video processing to do nothing related.
[02:12:01] processing to do nothing related. >> And one of other things that we do for
[02:12:03] >> And one of other things that we do for example uh in David which is a bit crazy
[02:12:06] example uh in David which is a bit crazy is that we don't use the function
[02:12:10] is that we don't use the function calling convention from the operating
[02:12:12] calling convention from the operating system.
[02:12:13] system. >> We should explain that
[02:12:14] >> We should explain that >> that is extremely complex but basically
[02:12:16] >> that is extremely complex but basically usually when you do move from one
[02:12:18] usually when you do move from one function in in code to another there is
[02:12:21] function in in code to another there is a way to save the registry the state of
[02:12:23] a way to save the registry the state of the CPU to enter another function. Um,
[02:12:26] the CPU to enter another function. Um, and this is like standard.
[02:12:28] and this is like standard. >> It's a bit complex. So I would I would
[02:12:29] >> It's a bit complex. So I would I would simplify this a bit. So So David does
[02:12:32] simplify this a bit. So So David does things to abuse the calling convention.
[02:12:34] things to abuse the calling convention. You could define the calling convention
[02:12:35] You could define the calling convention as I've written a function and I want to
[02:12:38] as I've written a function and I want to call another function. How is the data
[02:12:40] call another function. How is the data shared between the functions? Because
[02:12:42] shared between the functions? Because there's a convention what's known as a
[02:12:44] there's a convention what's known as a calling convention. And what David does
[02:12:47] calling convention. And what David does for optimal reasons is create its own
[02:12:50] for optimal reasons is create its own calling convention sometimes. So if I
[02:12:52] calling convention sometimes. So if I want to call Lex Freedman's library, we
[02:12:54] want to call Lex Freedman's library, we got we've got to agree on a convention
[02:12:56] got we've got to agree on a convention so that I can share data with you in the
[02:12:58] so that I can share data with you in the assembly language space and one of the
[02:13:00] assembly language space and one of the challenges in assembly is uh every
[02:13:02] challenges in assembly is uh every operating well not every operating
[02:13:03] operating well not every operating system but there are at least four that
[02:13:05] system but there are at least four that I can think of on x86 Linux 32-bit,
[02:13:08] I can think of on x86 Linux 32-bit, Windows 32-bit, Windows 64, Linux 64.
[02:13:11] Windows 32-bit, Windows 64, Linux 64. They all have their own calling
[02:13:12] They all have their own calling conventions. And so one of the amazing
[02:13:15] conventions. And so one of the amazing things Lauren Merritt did who we talked
[02:13:16] things Lauren Merritt did who we talked about before was create a very
[02:13:18] about before was create a very lightweight abstraction layer. So you
[02:13:19] lightweight abstraction layer. So you could write your assembly code once and
[02:13:21] could write your assembly code once and it handled all the calling convention
[02:13:23] it handled all the calling convention stuff for you
[02:13:25] stuff for you which was always a problem because you
[02:13:27] which was always a problem because you had to manage four different variants.
[02:13:29] had to manage four different variants. But David takes this even further for
[02:13:31] But David takes this even further for speed reasons it does its own calling
[02:13:33] speed reasons it does its own calling convention um within itself to bypass
[02:13:37] convention um within itself to bypass the kind of rules the rules of the rules
[02:13:39] the kind of rules the rules of the rules of sort of of functions and say okay
[02:13:41] of sort of of functions and say okay actually I'm going to call a function
[02:13:42] actually I'm going to call a function this way because I know it's within my
[02:13:44] this way because I know it's within my library.
[02:13:45] library. >> Does it has to be special to every
[02:13:46] >> Does it has to be special to every single operating system? Well, if it's
[02:13:48] single operating system? Well, if it's custom, no. But the the the challenge is
[02:13:51] custom, no. But the the the challenge is in general, yes. And in terms of in
[02:13:54] in general, yes. And in terms of in terms of each instruction set. So the
[02:13:55] terms of each instruction set. So the thing to also emphasize is we do this on
[02:13:58] thing to also emphasize is we do this on every instruction set. So every
[02:13:59] every instruction set. So every instruction set has its own handwritten
[02:14:01] instruction set has its own handwritten assembly, which is even more crazy. And
[02:14:03] assembly, which is even more crazy. And that that that has um that matrix has
[02:14:06] that that that has um that matrix has got bigger in recent years because of
[02:14:07] got bigger in recent years because of risk 5, because of ARM 64, because of
[02:14:10] risk 5, because of ARM 64, because of the new SVE, there's x86 has AVX 512
[02:14:16] the new SVE, there's x86 has AVX 512 AVX. So we do runtime processor
[02:14:18] AVX. So we do runtime processor detection. We see what the machine
[02:14:20] detection. We see what the machine FFmpeg is running on or David's running
[02:14:22] FFmpeg is running on or David's running on is capable of because you could be on
[02:14:24] on is capable of because you could be on a laptop from 2008 where this isn't
[02:14:27] a laptop from 2008 where this isn't there. Runtime detection. We set
[02:14:29] there. Runtime detection. We set function pointers accordingly and then
[02:14:32] function pointers accordingly and then from then on off you go or you could be
[02:14:34] from then on off you go or you could be on a machine with risk 5. Yes. And in
[02:14:37] on a machine with risk 5. Yes. And in all that we don't even respect the
[02:14:39] all that we don't even respect the calling convention of the operating
[02:14:41] calling convention of the operating system in order to be faster because we
[02:14:43] system in order to be faster because we know that we are going to be called from
[02:14:45] know that we are going to be called from within our binary. So we can share data
[02:14:47] within our binary. So we can share data without saving all the registry in the
[02:14:49] without saving all the registry in the common way because that can lead to
[02:14:52] common way because that can lead to loading and saving registry on the L1
[02:14:55] loading and saving registry on the L1 and L2 CPU and gets us faster. So that's
[02:14:58] and L2 CPU and gets us faster. So that's why I said that understanding
[02:15:01] why I said that understanding CPU architecture computer architecture
[02:15:04] CPU architecture computer architecture is key. And this is also why it's
[02:15:06] is key. And this is also why it's handwritten. I don't know anyone I've
[02:15:07] handwritten. I don't know anyone I've never he than David doing that. This is
[02:15:11] never he than David doing that. This is why Kieran calls it calls it an art,
[02:15:13] why Kieran calls it calls it an art, right? It is an art. I think I think in
[02:15:15] right? It is an art. I think I think in a in a mass world there isn't something
[02:15:17] a in a mass world there isn't something on billions of devices. I I know there
[02:15:18] on billions of devices. I I know there are some specialist industries. I know
[02:15:20] are some specialist industries. I know in high frequency trading they take this
[02:15:22] in high frequency trading they take this really seriously where they're receiving
[02:15:24] really seriously where they're receiving feeds from a market and they need to
[02:15:25] feeds from a market and they need to react within x number of microsconds and
[02:15:28] react within x number of microsconds and so the instructions matter. But that's
[02:15:30] so the instructions matter. But that's not a mass, you know, a mass-produced
[02:15:32] not a mass, you know, a mass-produced thing that's on a billion devices.
[02:15:33] thing that's on a billion devices. That's hyper specialized running on
[02:15:34] That's hyper specialized running on hyperspecialized hardware. We're running
[02:15:37] hyperspecialized hardware. We're running on all hardware from
[02:15:39] on all hardware from >> Sorry to linger on it, but like that's a
[02:15:42] >> Sorry to linger on it, but like that's a really counterintuitive almost like
[02:15:43] really counterintuitive almost like revolutionary
[02:15:46] revolutionary idea here that there's a huge amount of
[02:15:49] idea here that there's a huge amount of value to assembly like what are we
[02:15:50] value to assembly like what are we supposed to take away from that? like
[02:15:52] supposed to take away from that? like what you know there's a bunch of people
[02:15:54] what you know there's a bunch of people listening to this they're basically like
[02:15:56] listening to this they're basically like sorry for myself included you know I I
[02:15:59] sorry for myself included you know I I programmed for many many years in C++
[02:16:02] programmed for many many years in C++ going up the standards of C++ fell in
[02:16:04] going up the standards of C++ fell in love with C++ even meta programming and
[02:16:06] love with C++ even meta programming and so on and then transition more and more
[02:16:08] so on and then transition more and more because of machine learning about 15
[02:16:10] because of machine learning about 15 years ago to Python and so like for me
[02:16:13] years ago to Python and so like for me in this Python world JavaScript world
[02:16:17] in this Python world JavaScript world now vibe coding where I'm just using
[02:16:19] now vibe coding where I'm just using natural language sitting in my jacuzzi
[02:16:21] natural language sitting in my jacuzzi drinking a drink and just talking to the
[02:16:23] drinking a drink and just talking to the computer like like record stops. Why is
[02:16:27] computer like like record stops. Why is the value to go back all the way down to
[02:16:30] the value to go back all the way down to the low level
[02:16:32] the low level >> because you can get more power per
[02:16:34] >> because you can get more power per dollar invested, right? And sometimes
[02:16:37] dollar invested, right? And sometimes it's going to be a problem that is
[02:16:39] it's going to be a problem that is limited by your hardware. A good analogy
[02:16:42] limited by your hardware. A good analogy is what you see in quant quantization in
[02:16:45] is what you see in quant quantization in in LLMs, right? And people are doing oh
[02:16:48] in LLMs, right? And people are doing oh I'm going to do that in FP8 or FP FP4 or
[02:16:51] I'm going to do that in FP8 or FP FP4 or some some crazy things like Microsoft
[02:16:53] some some crazy things like Microsoft feud who did in 1.5 because you're
[02:16:56] feud who did in 1.5 because you're constrained by memory because you're
[02:16:58] constrained by memory because you're constrained by the machine you can run
[02:17:00] constrained by the machine you can run because at some point uh we are doing
[02:17:02] because at some point uh we are doing real time and I believe this is going to
[02:17:05] real time and I believe this is going to happen on AI inference also is that at
[02:17:07] happen on AI inference also is that at some point you need to get faster and
[02:17:09] some point you need to get faster and you cannot always get harder uh more
[02:17:12] you cannot always get harder uh more powerful hardware right so you need to
[02:17:14] powerful hardware right so you need to analyze code and see where like where is
[02:17:18] analyze code and see where like where is the mission critical where is the things
[02:17:20] the mission critical where is the things that are called non-stops and for
[02:17:22] that are called non-stops and for example David is a good example it's
[02:17:24] example David is a good example it's going to be run billions of hours per
[02:17:28] going to be run billions of hours per day
[02:17:28] day >> that makes sense it doesn't make sense
[02:17:31] >> that makes sense it doesn't make sense to be on the glue of fmpe cli it makes
[02:17:34] to be on the glue of fmpe cli it makes sense over there
[02:17:35] sense over there >> yeah this has to do also we'll talk
[02:17:37] >> yeah this has to do also we'll talk about it more but uh your new effort
[02:17:39] about it more but uh your new effort your new company kyber is doing that
[02:17:42] your new company kyber is doing that kind of thing for ultra low latency so
[02:17:45] kind of thing for ultra low latency so the slogan being every millisecond
[02:17:47] the slogan being every millisecond count. So when you're actually extremely
[02:17:49] count. So when you're actually extremely highly constrained in some dimension, we
[02:17:53] highly constrained in some dimension, we also arriving at a point where we've
[02:17:55] also arriving at a point where we've done
[02:17:57] done so many great things but the hardware is
[02:17:59] so many great things but the hardware is getting back to us, right? because cost
[02:18:02] getting back to us, right? because cost is increasing because we need more power
[02:18:04] is increasing because we need more power and so you're limited by either uh your
[02:18:07] and so you're limited by either uh your CPU your RAM or your networking and you
[02:18:10] CPU your RAM or your networking and you need to optimize and this is where value
[02:18:13] need to optimize and this is where value is going to be especially because like
[02:18:15] is going to be especially because like doing AI is going to help do the
[02:18:18] doing AI is going to help do the programming of like business right and
[02:18:20] programming of like business right and so the core thing that you will not be
[02:18:24] so the core thing that you will not be able to uh v code are optimization for
[02:18:27] able to uh v code are optimization for the hardware to be as fast as it
[02:18:29] the hardware to be as fast as it possible So, I'd love to talk to you
[02:18:32] possible So, I'd love to talk to you about who and how should learn assembly,
[02:18:36] about who and how should learn assembly, but first, I think we need a bathroom
[02:18:38] but first, I think we need a bathroom break. Quick 10-second thank you to our
[02:18:40] break. Quick 10-second thank you to our sponsors. Check them out in the
[02:18:42] sponsors. Check them out in the description. It really is the best way
[02:18:43] description. It really is the best way to support this podcast. Go to
[02:18:45] to support this podcast. Go to lexfreedman.com/sponsors.
[02:18:48] lexfreedman.com/sponsors. And now back to the episode. All right,
[02:18:51] And now back to the episode. All right, and we're back. There's this uh nice
[02:18:54] and we're back. There's this uh nice repo with the assembly lessons. First of
[02:18:57] repo with the assembly lessons. First of all, do you think uh developers should
[02:18:59] all, do you think uh developers should learn how to program an assembly? And
[02:19:02] learn how to program an assembly? And how would you go about learning it? What
[02:19:04] how would you go about learning it? What is this ASM-
[02:19:07] is this ASM- lessons?
[02:19:08] lessons? >> So, I I personally wasn't happy with the
[02:19:11] >> So, I I personally wasn't happy with the way assembly is taught in books and
[02:19:13] way assembly is taught in books and online cuz it's very grammar focused and
[02:19:16] online cuz it's very grammar focused and you don't in general learn a language
[02:19:18] you don't in general learn a language from
[02:19:20] from learning the grammar and the structure.
[02:19:22] learning the grammar and the structure. You learn a language by asking someone
[02:19:24] You learn a language by asking someone what their name is and you start from
[02:19:26] what their name is and you start from there and you go and solve real problems
[02:19:28] there and you go and solve real problems uh that you have when you want to
[02:19:29] uh that you have when you want to communicate. You don't you don't learn
[02:19:31] communicate. You don't you don't learn sentence structure. This is the
[02:19:33] sentence structure. This is the interrogative and the adverb and all all
[02:19:34] interrogative and the adverb and all all the assembly books seem to be doing like
[02:19:36] the assembly books seem to be doing like like going through every instruction
[02:19:38] like going through every instruction even ones that aren't really relevant
[02:19:40] even ones that aren't really relevant explaining what they all do and how they
[02:19:42] explaining what they all do and how they actually doesn't really change much. So,
[02:19:44] actually doesn't really change much. So, and the other problem that we have in
[02:19:46] and the other problem that we have in our community is assembly is taught sort
[02:19:47] our community is assembly is taught sort of hand to hand like personto person
[02:19:50] of hand to hand like personto person like blacksmithing one by one. That's
[02:19:52] like blacksmithing one by one. That's the only logical sort of analogy and
[02:19:54] the only logical sort of analogy and that doesn't really scale online. It
[02:19:56] that doesn't really scale online. It doesn't um the other things. So this
[02:19:57] doesn't um the other things. So this this this
[02:19:59] this this I started a set of assembly lessons in
[02:20:01] I started a set of assembly lessons in the in the way it's done in FFmpeg which
[02:20:03] the in the way it's done in FFmpeg which is a little bit different to the way
[02:20:06] is a little bit different to the way assembly in general for I don't know I'm
[02:20:09] assembly in general for I don't know I'm trying to think the other good big use
[02:20:10] trying to think the other good big use case of assembly is in embedded devices
[02:20:12] case of assembly is in embedded devices in really really low power cheap devices
[02:20:14] in really really low power cheap devices and that's completely different to what
[02:20:16] and that's completely different to what we're doing here. I think it would be
[02:20:18] we're doing here. I think it would be good if you could highlight the
[02:20:19] good if you could highlight the requirements which are quite simple.
[02:20:20] requirements which are quite simple. It's high school mathematics and C and
[02:20:23] It's high school mathematics and C and actually not even C really really is
[02:20:24] actually not even C really really is pointers to emphasize. Yes, we've talked
[02:20:26] pointers to emphasize. Yes, we've talked about how brilliant this stuff is, but
[02:20:29] about how brilliant this stuff is, but high schoolers like Daniel Kang have
[02:20:31] high schoolers like Daniel Kang have written um Assembly and FFmpeg. I think
[02:20:34] written um Assembly and FFmpeg. I think there's been contributions because of
[02:20:36] there's been contributions because of these lessons. So, it's really about
[02:20:40] these lessons. So, it's really about trying to to get this dying art to
[02:20:42] trying to to get this dying art to continue because we've shown it's
[02:20:43] continue because we've shown it's possible with David to produce something
[02:20:45] possible with David to produce something amazing. Mhm.
[02:20:46] amazing. Mhm. >> There's still a lot of codecs in FFmpeg
[02:20:48] >> There's still a lot of codecs in FFmpeg that are only maybe partially assembly
[02:20:51] that are only maybe partially assembly um assembly optimized. And so it really
[02:20:54] um assembly optimized. And so it really it really starts with basics and
[02:20:55] it really starts with basics and continues explains a lot of the jargon,
[02:20:57] continues explains a lot of the jargon, a lot of the syntax. It doesn't really
[02:20:59] a lot of the syntax. It doesn't really try and explain to you, you know,
[02:21:02] try and explain to you, you know, interrupt handlers and interrupt
[02:21:04] interrupt handlers and interrupt instructions and all of these different
[02:21:05] instructions and all of these different jump targets. It actually makes this
[02:21:07] jump targets. It actually makes this really vector focused
[02:21:08] really vector focused >> and uh describes all kinds of registers
[02:21:11] >> and uh describes all kinds of registers uh general purpose registers, vector
[02:21:13] uh general purpose registers, vector registers.
[02:21:15] registers. Uh really nice examples. Oh, this is
[02:21:18] Uh really nice examples. Oh, this is cool.
[02:21:18] cool. >> It's a classic. Yes. It's a classic
[02:21:19] >> It's a classic. Yes. It's a classic example of But um some of this assembly
[02:21:22] example of But um some of this assembly language is is is really beautiful. And
[02:21:24] language is is is really beautiful. And and I think it's beautiful because it's
[02:21:26] and I think it's beautiful because it's kind of like flying a Spitfire. It's
[02:21:28] kind of like flying a Spitfire. It's really
[02:21:29] really aviation at its purest, but also pushing
[02:21:32] aviation at its purest, but also pushing the aircraft beyond what the designer
[02:21:34] the aircraft beyond what the designer thought was possible. So we're abusing,
[02:21:36] thought was possible. So we're abusing, for example, sometimes cryptography
[02:21:38] for example, sometimes cryptography instructions to do certain things. And
[02:21:40] instructions to do certain things. And there's a level of beauty in art where
[02:21:43] there's a level of beauty in art where it's really you and the processor.
[02:21:45] it's really you and the processor. >> Mhm.
[02:21:46] >> Mhm. >> There's there's there's nothing in
[02:21:47] >> There's there's there's nothing in between. It's you and the the the
[02:21:50] between. It's you and the the the joystick of the cockpit and and you you
[02:21:52] joystick of the cockpit and and you you move that joystick and it's physically
[02:21:53] move that joystick and it's physically connected to the aerons and you can push
[02:21:55] connected to the aerons and you can push that plane beyond what it can normally
[02:21:58] that plane beyond what it can normally do. And there's a level of yeah beauty
[02:22:01] do. And there's a level of yeah beauty and amazingness to go that but I don't
[02:22:04] and amazingness to go that but I don't think the sort of person by person
[02:22:06] think the sort of person by person assembly that is someone taught me and
[02:22:08] assembly that is someone taught me and I've taught multiple people is going to
[02:22:10] I've taught multiple people is going to work
[02:22:12] work long run
[02:22:13] long run because of the particular flavor and the
[02:22:15] because of the particular flavor and the way that we do it.
[02:22:16] way that we do it. >> It's literally uh No, I should I was
[02:22:19] >> It's literally uh No, I should I was going to say wizards handing it down. I
[02:22:22] going to say wizards handing it down. I realize I look like a wizard wearing
[02:22:23] realize I look like a wizard wearing this hat, but you're basically just like
[02:22:25] this hat, but you're basically just like the sages, the wise sages handing down
[02:22:28] the sages, the wise sages handing down the craft. Can can I ask about LLMs?
[02:22:30] the craft. Can can I ask about LLMs? Like, can they help?
[02:22:32] Like, can they help? >> They had more of an understanding than I
[02:22:34] >> They had more of an understanding than I expected, but they are still
[02:22:36] expected, but they are still >> I've asked it questions and it still
[02:22:38] >> I've asked it questions and it still goes and starts hallucinating. Not
[02:22:40] goes and starts hallucinating. Not hallucinating, but making modifications
[02:22:42] hallucinating, but making modifications and then I go, is it bit exact? No, fix
[02:22:45] and then I go, is it bit exact? No, fix it. And then it just goes and does the
[02:22:46] it. And then it just goes and does the same thing. and it's going it there
[02:22:49] same thing. and it's going it there isn't the corpus of information like
[02:22:51] isn't the corpus of information like stack overflow to work on
[02:22:52] stack overflow to work on >> there is not enough data to train on um
[02:22:56] >> there is not enough data to train on um and this is the biggest issue um I
[02:22:58] and this is the biggest issue um I started my career actually doing some uh
[02:23:01] started my career actually doing some uh assembly for itanium right so itanium is
[02:23:04] assembly for itanium right so itanium is a dead um processor type right which was
[02:23:07] a dead um processor type right which was done by Intel in HP a long time ago when
[02:23:10] done by Intel in HP a long time ago when they wanted to do 64 bits well they lost
[02:23:12] they wanted to do 64 bits well they lost and then we got AMD who did it AMD 64
[02:23:16] and then we got AMD who did it AMD 64 which became x6 executive or but itanium
[02:23:19] which became x6 executive or but itanium was extremely interesting in the sense
[02:23:21] was extremely interesting in the sense that um those were processors who had a
[02:23:24] that um those were processors who had a ton of computing power to do floats fas
[02:23:28] ton of computing power to do floats fas which is similar to what we need now for
[02:23:30] which is similar to what we need now for for LLMs right and you could pack three
[02:23:33] for LLMs right and you could pack three operations per line that could be loaded
[02:23:37] operations per line that could be loaded so basically you had an output of
[02:23:39] so basically you had an output of basically 6 billion of operation per
[02:23:41] basically 6 billion of operation per second but the bus um the memory bus
[02:23:45] second but the bus um the memory bus only allowed 1.5, right? So your your
[02:23:48] only allowed 1.5, right? So your your CPU was four times faster. So you had to
[02:23:51] CPU was four times faster. So you had to do crazy things to to pack things in
[02:23:53] do crazy things to to pack things in memory, reuse the registered and those
[02:23:56] memory, reuse the registered and those type of themantics. No language could do
[02:23:58] type of themantics. No language could do that, right? So um like I I I have the
[02:24:02] that, right? So um like I I I have the Italian programming uh book because
[02:24:04] Italian programming uh book because Intel did amazing books, but that's
[02:24:07] Intel did amazing books, but that's exactly what Kieran says. If you don't
[02:24:09] exactly what Kieran says. If you don't know what you're you you're going to do,
[02:24:11] know what you're you you're going to do, it's impossible to read, right? It's a
[02:24:13] it's impossible to read, right? It's a ton of jargon and so on. While those
[02:24:16] ton of jargon and so on. While those lessons are amazing because they are
[02:24:19] lessons are amazing because they are targeted to a real problem and you can
[02:24:20] targeted to a real problem and you can do it yourself
[02:24:21] do it yourself >> and people have people have their
[02:24:23] >> and people have people have their patches and they said I studied your
[02:24:24] patches and they said I studied your lessons and here's my first changes.
[02:24:26] lessons and here's my first changes. >> That's amazing. And part of that um in
[02:24:29] >> That's amazing. And part of that um in the lessons is is a framework called x86
[02:24:32] the lessons is is a framework called x86 inc written by Lauren when we was
[02:24:34] inc written by Lauren when we was working on x264 and it allows you to do
[02:24:37] working on x264 and it allows you to do more things about that to create a type
[02:24:40] more things about that to create a type of like not caring too much about
[02:24:42] of like not caring too much about different cooling convention and we had
[02:24:45] different cooling convention and we had a lot of students who gave code to x264
[02:24:49] a lot of students who gave code to x264 using uh that a long time ago right so
[02:24:52] using uh that a long time ago right so it's really doable and I believe it's um
[02:24:56] it's really doable and I believe it's um necess necessarily to understand uh
[02:24:58] necess necessarily to understand uh assembly language even if you don't do
[02:25:00] assembly language even if you don't do it much to understand what's going on
[02:25:03] it much to understand what's going on inside your computer and that will make
[02:25:05] inside your computer and that will make you a better programmer and I assure you
[02:25:08] you a better programmer and I assure you that because doing that you will
[02:25:09] that because doing that you will understand some of the architecture of
[02:25:11] understand some of the architecture of the memory inside your your computer
[02:25:13] the memory inside your your computer right understanding register L1 L2 L3
[02:25:16] right understanding register L1 L2 L3 RAMs SSDs disk and so on um which are
[02:25:21] RAMs SSDs disk and so on um which are very important because then you have a
[02:25:23] very important because then you have a good programming culture that will make
[02:25:25] good programming culture that will make you a as a programmer.
[02:25:27] you a as a programmer. >> Uh what do you think about the Rust
[02:25:28] >> Uh what do you think about the Rust programming language cuz that's a bit of
[02:25:30] programming language cuz that's a bit of a meme.
[02:25:31] a meme. >> We have very different opinions with
[02:25:32] >> We have very different opinions with Khan.
[02:25:33] Khan. >> I think it's valuable what they're doing
[02:25:35] >> I think it's valuable what they're doing in terms of memory safety as a concept.
[02:25:37] in terms of memory safety as a concept. >> Can it achieve this some of the speed up
[02:25:40] >> Can it achieve this some of the speed up that assembly achieves?
[02:25:42] that assembly achieves? >> Not not assembly by hand. No, I think
[02:25:44] >> Not not assembly by hand. No, I think that that's a given. C potentially, but
[02:25:46] that that's a given. C potentially, but I see it very it has a very big
[02:25:48] I see it very it has a very big Espiranto vibe about it. It's like we're
[02:25:50] Espiranto vibe about it. It's like we're going to solve this and we're doing this
[02:25:53] going to solve this and we're doing this in a particular way. meaning is a bit
[02:25:55] in a particular way. meaning is a bit too utopian.
[02:25:56] too utopian. >> There's a lot of focus on the
[02:25:57] >> There's a lot of focus on the self-importance rather than solving real
[02:25:59] self-importance rather than solving real world problems. It reminds me of the
[02:26:01] world problems. It reminds me of the Sinclair C5. Slive Sinclair of Sinclair
[02:26:04] Sinclair C5. Slive Sinclair of Sinclair Computers built a car and he said, "Oh,
[02:26:07] Computers built a car and he said, "Oh, everyone will be traveling around in you
[02:26:08] everyone will be traveling around in you know, one of these electric cars and it
[02:26:10] know, one of these electric cars and it was um Frost reminds me of that where I
[02:26:15] was um Frost reminds me of that where I think the community doesn't the
[02:26:16] think the community doesn't the community doesn't quite understand that
[02:26:19] community doesn't quite understand that in order to get people to move, you have
[02:26:20] in order to get people to move, you have to build something that's as good as if
[02:26:22] to build something that's as good as if not better than what you have now." Um,
[02:26:25] not better than what you have now." Um, yes, people are doing Rust rewrites, but
[02:26:27] yes, people are doing Rust rewrites, but if they're if they only if they only do
[02:26:30] if they're if they only if they only do 85 90% of the feature set of what what
[02:26:34] 85 90% of the feature set of what what we need, like things like core utils,
[02:26:36] we need, like things like core utils, that last 1% takes 99% of the time um to
[02:26:40] that last 1% takes 99% of the time um to to use Elon's famous quote, prototypes
[02:26:42] to use Elon's famous quote, prototypes are easy. Like this kind of stuff is
[02:26:43] are easy. Like this kind of stuff is easy, but this to get a real electric
[02:26:45] easy, but this to get a real electric car, you have to make a car as good as
[02:26:47] car, you have to make a car as good as if not better than what we have now. And
[02:26:48] if not better than what we have now. And Rust isn't in that stage yet. I think we
[02:26:51] Rust isn't in that stage yet. I think we I don't think anyone would object to
[02:26:53] I don't think anyone would object to seeing
[02:26:54] seeing ROST code in FFmpeg, but it needs to
[02:26:57] ROST code in FFmpeg, but it needs to work as well and support the same unit
[02:26:59] work as well and support the same unit testing as everything else. It needs to
[02:27:01] testing as everything else. It needs to be flawless. It can't just randomly
[02:27:03] be flawless. It can't just randomly break. They can't just randomly break AI
[02:27:05] break. They can't just randomly break AI when they want to. It needs to it needs
[02:27:08] when they want to. It needs to it needs to have I think more I think it still
[02:27:09] to have I think more I think it still has only one compiler implementation.
[02:27:11] has only one compiler implementation. >> Yes.
[02:27:12] >> Yes. >> So,
[02:27:13] >> So, it's got to be as good as if not better.
[02:27:16] it's got to be as good as if not better. And saying, "Hey, here's my utopia of
[02:27:18] And saying, "Hey, here's my utopia of memory safety isn't enough." even though
[02:27:20] memory safety isn't enough." even though we we probably all agree that that's the
[02:27:23] we we probably all agree that that's the goal.
[02:27:24] goal. >> So, um I've done a ton of Rust and the
[02:27:27] >> So, um I've done a ton of Rust and the two major topics I had was adding Rust
[02:27:30] two major topics I had was adding Rust modules inside VC. One of the reasons VC
[02:27:33] modules inside VC. One of the reasons VC got popular um and which was one of the
[02:27:35] got popular um and which was one of the main architecture decision is that VC is
[02:27:38] main architecture decision is that VC is a very small core and a ton of modules
[02:27:41] a very small core and a ton of modules right and so you can write modules in C
[02:27:43] right and so you can write modules in C in
[02:27:45] in C++ in Objective C and anything that is
[02:27:48] C++ in Objective C and anything that is basically interoperable with C and and
[02:27:50] basically interoperable with C and and and so we we we did some Ras modules and
[02:27:54] and so we we we did some Ras modules and so I have experience on that and I wrote
[02:27:56] so I have experience on that and I wrote some of it and also like my my new
[02:27:58] some of it and also like my my new startup called Kyber is an open source
[02:28:00] startup called Kyber is an open source project mainly done in Rust. Um what
[02:28:04] project mainly done in Rust. Um what Rust is extremely good in in the sense
[02:28:07] Rust is extremely good in in the sense that um it's a better C++ that cares
[02:28:11] that um it's a better C++ that cares about memory and allows you to do things
[02:28:13] about memory and allows you to do things about memory ownership that no one else
[02:28:16] about memory ownership that no one else can do so far. However, it's great when
[02:28:21] can do so far. However, it's great when you start a new project from scratch and
[02:28:23] you start a new project from scratch and you do everything in Rust, but it's very
[02:28:26] you do everything in Rust, but it's very not good when you interrupt with
[02:28:28] not good when you interrupt with existing part. And some part of the Rust
[02:28:31] existing part. And some part of the Rust community believe that they need to
[02:28:32] community believe that they need to rewrite everything and everything will
[02:28:34] rewrite everything and everything will be better with Rust. And and the answer
[02:28:36] be better with Rust. And and the answer is like no. Like I'm almost always and
[02:28:39] is like no. Like I'm almost always and all my years of being engineer, manager,
[02:28:44] all my years of being engineer, manager, CTO, startup and so on. don't rewrite,
[02:28:47] CTO, startup and so on. don't rewrite, >> right? is that that's the that's the
[02:28:49] >> right? is that that's the that's the initial instinct for a lot of people
[02:28:51] initial instinct for a lot of people when they show up to a codebase probably
[02:28:53] when they show up to a codebase probably before LMS is like probably because they
[02:28:57] before LMS is like probably because they don't understand the the the wisdom of
[02:29:00] don't understand the the the wisdom of the way things have been done in the
[02:29:01] the way things have been done in the past they say well we need to rewrite it
[02:29:03] past they say well we need to rewrite it hence why there's a thousand JavaScript
[02:29:06] hence why there's a thousand JavaScript frameworks but the reason is the
[02:29:07] frameworks but the reason is the following and this is very important to
[02:29:10] following and this is very important to understand it is an order of magnitude
[02:29:13] understand it is an order of magnitude easier to write code than read code
[02:29:16] easier to write code than read code >> and you see that also with LLM they can
[02:29:18] >> and you see that also with LLM they can vibe code that analyzing is a lot more
[02:29:22] vibe code that analyzing is a lot more difficult and so when you arrive and
[02:29:26] difficult and so when you arrive and when you arrive to a very complex piece
[02:29:28] when you arrive to a very complex piece of code right you you don't understand
[02:29:30] of code right you you don't understand it right um because it's so much more
[02:29:33] it right um because it's so much more effort to understand the code from
[02:29:35] effort to understand the code from someone else because you don't have the
[02:29:36] someone else because you don't have the thought process um and often I joke
[02:29:39] thought process um and often I joke about um some some languages mostly uh
[02:29:43] about um some some languages mostly uh pearl for example um which has very
[02:29:45] pearl for example um which has very complex syntax And imagine I am at my
[02:29:48] complex syntax And imagine I am at my maximum intellectual efficiency in
[02:29:51] maximum intellectual efficiency in programming right and I write the best
[02:29:54] programming right and I write the best code ever. I will not be able to
[02:29:56] code ever. I will not be able to understanding myself 6 months later
[02:29:58] understanding myself 6 months later right because reading code is more
[02:30:00] right because reading code is more difficult. So very often you arrive you
[02:30:03] difficult. So very often you arrive you don't understand all the wisdom all the
[02:30:05] don't understand all the wisdom all the business logic the reasons that were
[02:30:07] business logic the reasons that were done that is maybe not documented and
[02:30:09] done that is maybe not documented and you say well I'm going to rewrite it and
[02:30:12] you say well I'm going to rewrite it and the thing is no you don't right because
[02:30:14] the thing is no you don't right because that's as Kieran said right I'm going to
[02:30:16] that's as Kieran said right I'm going to rewrite core utils in rust and then of
[02:30:18] rewrite core utils in rust and then of course you arrive very quickly at 80% um
[02:30:22] course you arrive very quickly at 80% um then 90% takes a bit more time and then
[02:30:24] then 90% takes a bit more time and then you got the last ones right on the other
[02:30:28] you got the last ones right on the other side right so for new projects it's
[02:30:30] side right so for new projects it's great everything related to parsing
[02:30:32] great everything related to parsing files uh networking uh because of the
[02:30:35] files uh networking uh because of the memory uh checker bary checker it's
[02:30:37] memory uh checker bary checker it's amazing and there is nothing else to
[02:30:40] amazing and there is nothing else to answer a bit differently for us
[02:30:44] answer a bit differently for us imagine I take a piece of software like
[02:30:47] imagine I take a piece of software like David or x264 right which has a ton of
[02:30:50] David or x264 right which has a ton of runtime in assembly right um I rewrite
[02:30:54] runtime in assembly right um I rewrite the C part in Rust right so it's more
[02:30:55] the C part in Rust right so it's more secure yes but then you arrive into the
[02:30:58] secure yes but then you arrive into the assembly and you can jump anywhere where
[02:31:00] assembly and you can jump anywhere where in the memory because we are doing
[02:31:02] in the memory because we are doing handwritten assembly. So even if I
[02:31:05] handwritten assembly. So even if I rewrite the C part in rust for security
[02:31:08] rewrite the C part in rust for security reason you you break all the security
[02:31:12] reason you you break all the security when you re you write handwritten
[02:31:13] when you re you write handwritten assembly because we can jump anywhere.
[02:31:16] assembly because we can jump anywhere. So in my opinion we need to do something
[02:31:18] So in my opinion we need to do something that is um secure assembly right so
[02:31:22] that is um secure assembly right so which is compile time check the assembly
[02:31:25] which is compile time check the assembly which is similar to the check assembly
[02:31:27] which is similar to the check assembly project that we're doing on on David and
[02:31:29] project that we're doing on on David and x264 with video is to start
[02:31:34] x264 with video is to start instrumenting your assembly at compile
[02:31:36] instrumenting your assembly at compile time to check that it's not jumping
[02:31:38] time to check that it's not jumping anywhere in the memory because else you
[02:31:40] anywhere in the memory because else you might rewrite a part of C in Rust but if
[02:31:43] might rewrite a part of C in Rust but if you want to have the same performances
[02:31:44] you want to have the same performances you're going to have inline assembly And
[02:31:46] you're going to have inline assembly And so you destroy your whole security
[02:31:47] so you destroy your whole security model. Um so that's a bit what I think
[02:31:50] model. Um so that's a bit what I think about rest.
[02:31:51] about rest. >> I just want I would say on a personal
[02:31:53] >> I just want I would say on a personal level I'm so in awe about the assembly I
[02:31:56] level I'm so in awe about the assembly I actually
[02:31:58] actually once and it never gets old the speed
[02:32:00] once and it never gets old the speed improvements to show 62x.
[02:32:02] improvements to show 62x. Uh there are months on a personal level
[02:32:04] Uh there are months on a personal level I I run our internal test suite at work
[02:32:08] I I run our internal test suite at work and just see I'm still in awe at the
[02:32:10] and just see I'm still in awe at the gains we have. there there's a source of
[02:32:11] gains we have. there there's a source of joy and happiness for programming for
[02:32:13] joy and happiness for programming for different reasons,
[02:32:15] different reasons, >> but I think one of the greatest
[02:32:18] >> but I think one of the greatest happinesses is in the optimization of
[02:32:20] happinesses is in the optimization of code and it sounds like you're like at
[02:32:23] code and it sounds like you're like at the cutting edge of that.
[02:32:25] the cutting edge of that. >> That was cool.
[02:32:25] >> That was cool. >> In the community, I want to speak about
[02:32:27] >> In the community, I want to speak about two people who are
[02:32:30] two people who are >> wizards of assembly, right? The two of
[02:32:33] >> wizards of assembly, right? The two of them are actually working uh living in
[02:32:35] them are actually working uh living in north of Europe uh Sweden and uh and
[02:32:39] north of Europe uh Sweden and uh and Finland. And um
[02:32:42] Finland. And um Henrik Grner knows so much about
[02:32:46] Henrik Grner knows so much about Intel x86 assembly that when we ask
[02:32:49] Intel x86 assembly that when we ask questions at Intel about things they
[02:32:52] questions at Intel about things they tell like why are you asking us Intel?
[02:32:54] tell like why are you asking us Intel? You have Henrik. Henrik knows better. He
[02:32:57] You have Henrik. Henrik knows better. He knows all the cycles of almost all the
[02:32:59] knows all the cycles of almost all the SIMD instruction by all the CPU
[02:33:03] SIMD instruction by all the CPU generation. Oh yes, this is a P4, this
[02:33:05] generation. Oh yes, this is a P4, this is a NHL, this is a core 2, etc. That
[02:33:08] is a NHL, this is a core 2, etc. That person is like the best person on
[02:33:11] person is like the best person on assembly in the world and he's the
[02:33:13] assembly in the world and he's the nicest person that you've seen like very
[02:33:18] nicest person that you've seen like very he he arrived you don't see he's
[02:33:20] he he arrived you don't see he's amazing. And the other one is is called
[02:33:23] amazing. And the other one is is called Martin Martin Stio and he's they're
[02:33:26] Martin Martin Stio and he's they're doing mostly the same on ARM, right? So
[02:33:29] doing mostly the same on ARM, right? So Neon, right? And iPhones and Androids
[02:33:32] Neon, right? And iPhones and Androids and so on. And he codes in assembly on
[02:33:37] and so on. And he codes in assembly on his phone, editing it with the crappy uh
[02:33:42] his phone, editing it with the crappy uh uh keyboard, like virtual keyboard you
[02:33:45] uh keyboard, like virtual keyboard you have while watching his kids play in the
[02:33:48] have while watching his kids play in the playground, right? like like this is
[02:33:50] playground, right? like like this is just like wizard level. So those two
[02:33:53] just like wizard level. So those two people are like
[02:33:55] people are like >> Yes.
[02:33:56] >> Yes. >> So a part when you're programming
[02:33:58] >> So a part when you're programming assembly at that high level a part of
[02:34:00] assembly at that high level a part of that is knowing the architecture that
[02:34:02] that is knowing the architecture that you're program x86
[02:34:04] you're program x86 >> in particular. Yes.
[02:34:04] >> in particular. Yes. >> ARM in particular but x I mean these are
[02:34:08] >> ARM in particular but x I mean these are complicated architectures right?
[02:34:09] complicated architectures right? >> Yeah but ARM um in some ways is more x86
[02:34:13] >> Yeah but ARM um in some ways is more x86 with uh out of order execution it's not
[02:34:15] with uh out of order execution it's not so bad. ARM you really need to
[02:34:16] so bad. ARM you really need to understand all the different generations
[02:34:18] understand all the different generations of ARM processor cuz they're all
[02:34:19] of ARM processor cuz they're all different. There's A72 uh etc. etc. And
[02:34:23] different. There's A72 uh etc. etc. And there's the Apple variant, there's this
[02:34:24] there's the Apple variant, there's this variant, there's that. And you need to
[02:34:25] variant, there's that. And you need to write code that works efficiently on all
[02:34:28] write code that works efficiently on all of them. x86, well, broadly speaking,
[02:34:30] of them. x86, well, broadly speaking, you have Intel, AMD, you have sub
[02:34:31] you have Intel, AMD, you have sub variants, but generally speaking,
[02:34:34] variants, but generally speaking, there's
[02:34:36] there's something fast is going to remain fast
[02:34:37] something fast is going to remain fast on all of the variants, whereas an ARM
[02:34:39] on all of the variants, whereas an ARM is a completely much more complicated
[02:34:42] is a completely much more complicated ball game. We're uh taking a nonlinear
[02:34:45] ball game. We're uh taking a nonlinear journey through history here, but we
[02:34:46] journey through history here, but we were talking about uh Michael Nemmer. Uh
[02:34:50] were talking about uh Michael Nemmer. Uh I wanted to ask about this for a time
[02:34:56] I wanted to ask about this for a time there was a split in ffmpeg and a lib
[02:34:59] there was a split in ffmpeg and a lib av. Yes. So um in open source projects
[02:35:04] av. Yes. So um in open source projects um sometimes you disagree, right? Um
[02:35:10] um sometimes you disagree, right? Um >> that's such a nice way of putting it.
[02:35:12] >> that's such a nice way of putting it. Yeah. And the good thing is because of
[02:35:13] Yeah. And the good thing is because of the license you're allowed to basically
[02:35:15] the license you're allowed to basically do your own, right?
[02:35:17] do your own, right? >> Um, and this is normal and this has
[02:35:19] >> Um, and this is normal and this has happened all the time, right? At the
[02:35:21] happened all the time, right? At the point there was uh GCC at the time of
[02:35:24] point there was uh GCC at the time of GCC 2 and EGCS uh which became then
[02:35:28] GCC 2 and EGCS uh which became then GCC3, right? There is what we told KHTML
[02:35:31] GCC3, right? There is what we told KHTML with WebKit with Blink. um it is a sane
[02:35:35] with WebKit with Blink. um it is a sane process and also like when I want to do
[02:35:37] process and also like when I want to do a new feature today in VC I fork I do my
[02:35:40] a new feature today in VC I fork I do my thing on my own and then I merge back to
[02:35:42] thing on my own and then I merge back to the community. So there was a split in
[02:35:44] the community. So there was a split in the open source community on ffmpeg
[02:35:46] the open source community on ffmpeg which become libe and ffmpeg and after a
[02:35:48] which become libe and ffmpeg and after a few years well the community merged back
[02:35:51] few years well the community merged back and people moved on. It's a bit um drama
[02:35:55] and people moved on. It's a bit um drama that is normal in open source community,
[02:35:57] that is normal in open source community, but forks are even they're important
[02:36:00] but forks are even they're important because they they they change the statue
[02:36:03] because they they they change the statue core of a community. Um um not talking
[02:36:06] core of a community. Um um not talking about f and and eb here, but the GCC
[02:36:10] about f and and eb here, but the GCC fork made GCC a ton better because the
[02:36:13] fork made GCC a ton better because the some people wanted to change the
[02:36:15] some people wanted to change the architecture fundamentally to make it
[02:36:17] architecture fundamentally to make it faster. And of course it's always
[02:36:20] faster. And of course it's always question of people and so on but in the
[02:36:22] question of people and so on but in the end you realize that ffmpeg today is
[02:36:25] end you realize that ffmpeg today is better than it was uh before the fork.
[02:36:28] better than it was uh before the fork. Um and now well we're back all together
[02:36:31] Um and now well we're back all together right and I spend a lot of time and K
[02:36:34] right and I spend a lot of time and K can say in the in in the community um
[02:36:37] can say in the in in the community um it's not often to be honest uh very well
[02:36:41] it's not often to be honest uh very well explained uh because a ton of the
[02:36:43] explained uh because a ton of the reasons are not very public uh but I
[02:36:46] reasons are not very public uh but I think that's that's normal and that's
[02:36:48] think that's that's normal and that's good.
[02:36:49] good. >> Yeah. I mean you're making it sound
[02:36:50] >> Yeah. I mean you're making it sound really nice but there there is battle
[02:36:52] really nice but there there is battle there's pretty heated battles inside
[02:36:54] there's pretty heated battles inside open source projects. I mean it is a
[02:36:55] open source projects. I mean it is a very passionate community and you are
[02:36:58] very passionate community and you are kind of in a distributed way have to
[02:37:00] kind of in a distributed way have to define the direction of things. So here
[02:37:02] define the direction of things. So here looking at uh perplexity ffmpeg and live
[02:37:05] looking at uh perplexity ffmpeg and live aav split in 2011 mainly over project
[02:37:08] aav split in 2011 mainly over project governance leadership style and
[02:37:09] governance leadership style and development processes not because of a
[02:37:11] development processes not because of a fundamental technical disagreement. Uh
[02:37:13] fundamental technical disagreement. Uh ffmpeg effectively absorbed lib's work
[02:37:17] ffmpeg effectively absorbed lib's work while lib withered and most
[02:37:19] while lib withered and most distributions and developers moved back
[02:37:21] distributions and developers moved back to ffmpeg. Yeah, that was a that was a
[02:37:23] to ffmpeg. Yeah, that was a that was a weird that was a from a user's
[02:37:24] weird that was a from a user's perspective that was a weird experience
[02:37:27] perspective that was a weird experience because you know I'm a Linux user so you
[02:37:29] because you know I'm a Linux user so you know whether it's a bondu and so on all
[02:37:31] know whether it's a bondu and so on all of a sudden I think for for a for a
[02:37:34] of a sudden I think for for a for a little bit bond to I feel like am I
[02:37:36] little bit bond to I feel like am I remembering correct they switched to lib
[02:37:38] remembering correct they switched to lib a
[02:37:39] a >> 12 14 something like that something like
[02:37:41] >> 12 14 something like that something like that
[02:37:41] that >> and then they switched back I was like
[02:37:43] >> and then they switched back I was like what is happening so on the sort of you
[02:37:47] what is happening so on the sort of you you get to feel the ripple effects of
[02:37:50] you get to feel the ripple effects of the different internal debates that are
[02:37:52] the different internal debates that are happening.
[02:37:53] happening. >> To be fair, on Apple when you type GCC
[02:37:55] >> To be fair, on Apple when you type GCC you get Cang like they they did
[02:37:57] you get Cang like they they did something like that as well. So
[02:37:58] something like that as well. So >> yeah. So to me it's like the fork was
[02:38:02] >> yeah. So to me it's like the fork was like heated drama but
[02:38:05] like heated drama but >> most of the development from Libby was
[02:38:07] >> most of the development from Libby was merged back into FFmpeg, right? So the
[02:38:09] merged back into FFmpeg, right? So the factomp got a a supererset around lib
[02:38:13] factomp got a a supererset around lib and so that gave the user because in the
[02:38:15] and so that gave the user because in the end we work the user for the users a
[02:38:17] end we work the user for the users a larger set of features and a ton of
[02:38:19] larger set of features and a ton of things that were um discussed. For
[02:38:22] things that were um discussed. For example the debate on reviews and on how
[02:38:25] example the debate on reviews and on how we push are something that now is
[02:38:28] we push are something that now is completely settled in FFmpeg and is
[02:38:30] completely settled in FFmpeg and is following what mostly what everyone in
[02:38:33] following what mostly what everyone in the community agrees. Right? So the
[02:38:35] the community agrees. Right? So the facto everyone who was active on the
[02:38:38] facto everyone who was active on the baby came back in work on FFmpeg because
[02:38:42] baby came back in work on FFmpeg because the disagreement were fixed and in the
[02:38:44] the disagreement were fixed and in the end ffmpeg is stronger uh than it was
[02:38:48] end ffmpeg is stronger uh than it was before right and
[02:38:49] before right and >> I know people love drama but um well my
[02:38:52] >> I know people love drama but um well my my main concern I I understand I think
[02:38:56] my main concern I I understand I think looking at the the long history it's all
[02:39:00] looking at the the long history it's all for the good but I do I am concerned
[02:39:03] for the good but I do I am concerned because there's so few humans
[02:39:06] because there's so few humans that are critical to the success of open
[02:39:08] that are critical to the success of open source projects that I have seen it uh
[02:39:12] source projects that I have seen it uh be a psychological toll on folks and you
[02:39:17] be a psychological toll on folks and you know sometimes leads to burnout. So you
[02:39:18] know sometimes leads to burnout. So you have these incredible people that are at
[02:39:20] have these incredible people that are at the core of open source projects. There
[02:39:22] the core of open source projects. There is a moment that happens cuz like what
[02:39:24] is a moment that happens cuz like what is the motivation of doing it?
[02:39:26] is the motivation of doing it? Ultimately, it's because you're
[02:39:27] Ultimately, it's because you're passionate about it and it makes you
[02:39:28] passionate about it and it makes you happy and a certain point you wake up
[02:39:30] happy and a certain point you wake up and it's like this been a bit too much
[02:39:33] and it's like this been a bit too much heat from the drama. So, like at the at
[02:39:36] heat from the drama. So, like at the at the project level, the project continues
[02:39:38] the project level, the project continues and often flourishes. But sometimes
[02:39:40] and often flourishes. But sometimes there's these individual humans, they're
[02:39:43] there's these individual humans, they're just like I've had enough.
[02:39:45] just like I've had enough. >> Yeah. But it's not just about folks,
[02:39:46] >> Yeah. But it's not just about folks, right? So it's a very what what you what
[02:39:49] right? So it's a very what what you what you are um referring to is one of the
[02:39:52] you are um referring to is one of the most challenging and most interesting
[02:39:54] most challenging and most interesting part of open source today is maintainers
[02:39:58] part of open source today is maintainers burnout right and AI is a problem
[02:40:02] burnout right and AI is a problem because of that and Daniel Steinberg
[02:40:04] because of that and Daniel Steinberg which is um the maintainer of curl uh
[02:40:07] which is um the maintainer of curl uh who's probably one of the best promoter
[02:40:10] who's probably one of the best promoter of open source in the world he's by the
[02:40:13] of open source in the world he's by the way a member of the European open source
[02:40:14] way a member of the European open source academy with me so I'm very like humbled
[02:40:17] academy with me so I'm very like humbled to be on the same communities as him,
[02:40:18] to be on the same communities as him, right? He's against what he call AI
[02:40:21] right? He's against what he call AI slop, right? because it gives a ton of
[02:40:24] slop, right? because it gives a ton of um uh fake reports or bad reports, bad
[02:40:28] um uh fake reports or bad reports, bad patches and then a lot of maintainers
[02:40:31] patches and then a lot of maintainers have a lot of burden to maintain the
[02:40:33] have a lot of burden to maintain the software
[02:40:34] software >> and this is uh straining the um the the
[02:40:38] >> and this is uh straining the um the the the mind of open-source developers much
[02:40:41] the mind of open-source developers much more than forks and for example the XZ
[02:40:44] more than forks and for example the XZ fiasco was because there was one guy
[02:40:47] fiasco was because there was one guy maintaining it and he got basically
[02:40:49] maintaining it and he got basically hammered by two attackants who were
[02:40:51] hammered by two attackants who were asking asking him questions non-stop at
[02:40:53] asking asking him questions non-stop at weird times at night to block him and at
[02:40:56] weird times at night to block him and at some point he got fed up and says okay I
[02:40:58] some point he got fed up and says okay I can't do that and gave the commit access
[02:41:00] can't do that and gave the commit access to the attackant um so burnout in open
[02:41:03] to the attackant um so burnout in open source community is something that
[02:41:06] source community is something that exists uh but mostly it's about
[02:41:09] exists uh but mostly it's about maintaining things right
[02:41:11] maintaining things right >> no for sure I wonder how do we we help
[02:41:13] >> no for sure I wonder how do we we help that because those people are so
[02:41:15] that because those people are so important the human beings are so
[02:41:17] important the human beings are so important at the core of these projects
[02:41:19] important at the core of these projects >> so so for example now I am maintaining a
[02:41:21] >> so so for example now I am maintaining a ton of multimedia and non multimedia
[02:41:23] ton of multimedia and non multimedia library as maintainer uh because the
[02:41:26] library as maintainer uh because the maintainers got fed up right um some on
[02:41:29] maintainers got fed up right um some on video land some outside of video land uh
[02:41:32] video land some outside of video land uh because it's sometimes you need a a
[02:41:35] because it's sometimes you need a a tough skins right because you get like
[02:41:37] tough skins right because you get like it's not really attacks but oh this is
[02:41:40] it's not really attacks but oh this is not working this is not working and you
[02:41:41] not working this is not working and you feel it personally um and this is also
[02:41:44] feel it personally um and this is also why resources or the the Google fiasco
[02:41:47] why resources or the the Google fiasco um is was a problem right they don't
[02:41:49] um is was a problem right they don't realize that in the And you have um you
[02:41:52] realize that in the And you have um you know it's like the same graph where you
[02:41:53] know it's like the same graph where you see like everything and it's just like
[02:41:56] see like everything and it's just like one random open source project that is
[02:41:58] one random open source project that is maintaining the whole internet. You see
[02:42:00] maintaining the whole internet. You see the one right there?
[02:42:01] the one right there? >> Yeah. This is the meme. I mean it
[02:42:03] >> Yeah. This is the meme. I mean it applies to to a lot of projects but this
[02:42:06] applies to to a lot of projects but this is
[02:42:07] is >> it's the all modern digital multimedia
[02:42:09] >> it's the all modern digital multimedia infrastructure and then that thing at
[02:42:11] infrastructure and then that thing at the very bottom that everything relies
[02:42:13] the very bottom that everything relies on is FFmpeg. It's true. And then
[02:42:16] on is FFmpeg. It's true. And then there's usually you know a handful of
[02:42:18] there's usually you know a handful of folks that are maintaining that
[02:42:19] folks that are maintaining that >> and FFmpeg or VC right you have a
[02:42:22] >> and FFmpeg or VC right you have a community of 10 15 core developers are
[02:42:25] community of 10 15 core developers are not the worst open source project XZ
[02:42:27] not the worst open source project XZ which is even in more installations is
[02:42:30] which is even in more installations is one person right there is one guy lib
[02:42:33] one person right there is one guy lib XML is
[02:42:34] XML is >> yeah lib XML right there was a big stop
[02:42:36] >> yeah lib XML right there was a big stop no one is maintaining XML anymore which
[02:42:38] no one is maintaining XML anymore which is like pass the only library that is
[02:42:40] is like pass the only library that is able to pass XML everywhere
[02:42:41] able to pass XML everywhere >> all the crazy edge cases of XML under
[02:42:44] >> all the crazy edge cases of XML under ridiculous circumstances and they get
[02:42:46] ridiculous circumstances and they get attacked by security researchers because
[02:42:48] attacked by security researchers because there's one other crazy edge case that
[02:42:50] there's one other crazy edge case that they haven't thought of and yeah but the
[02:42:52] they haven't thought of and yeah but the body of knowledge to actually resolve
[02:42:54] body of knowledge to actually resolve that is massive. There is one guy
[02:42:57] that is massive. There is one guy maintaining all the time zones for
[02:42:59] maintaining all the time zones for everyone who is in the middle of I think
[02:43:01] everyone who is in the middle of I think was it Nebraska or South Dakota like
[02:43:04] was it Nebraska or South Dakota like >> the mental health of the open source
[02:43:07] >> the mental health of the open source maintainers is something that large
[02:43:09] maintainers is something that large corporations don't care or don't see
[02:43:12] corporations don't care or don't see right it's just like oh yeah I'm just
[02:43:13] right it's just like oh yeah I'm just doing an open source uh report and so on
[02:43:16] doing an open source uh report and so on some of it is financial but some of it
[02:43:19] some of it is financial but some of it and people should definitely support
[02:43:21] and people should definitely support open source financially all across the
[02:43:23] open source financially all across the board but some of it is also like
[02:43:25] board but some of it is also like spiritual on a basic human level,
[02:43:27] spiritual on a basic human level, there's something that happens like with
[02:43:30] there's something that happens like with this image of FFmpeg and so much of the
[02:43:32] this image of FFmpeg and so much of the internet depending on it where people
[02:43:34] internet depending on it where people almost like talk down to the folks who
[02:43:38] almost like talk down to the folks who are carrying these projects forward and
[02:43:40] are carrying these projects forward and maintaining
[02:43:40] maintaining >> in the security community. They
[02:43:42] >> in the security community. They certainly did. That was one of that was
[02:43:43] certainly did. That was one of that was one of the things I think that argument
[02:43:45] one of the things I think that argument came out is
[02:43:47] came out is there was there was a portion of the
[02:43:48] there was there was a portion of the security community like no these guys
[02:43:50] security community like no these guys write crap code. They need to fix their
[02:43:52] write crap code. They need to fix their crap code. I'm like, "No, no, no, no.
[02:43:53] crap code. I'm like, "No, no, no, no. This is a guy's hobby project. You've
[02:43:56] This is a guy's hobby project. You've you've have a security bot that's gone
[02:43:57] you've have a security bot that's gone and found some AI generator stuff. That
[02:44:00] and found some AI generator stuff. That guy didn't write crap code. It's just an
[02:44:02] guy didn't write crap code. It's just an edge case to the 99th.99999
[02:44:05] edge case to the 99th.99999 percentile. He didn't think about
[02:44:07] percentile. He didn't think about because it's his hobby project decoding
[02:44:10] because it's his hobby project decoding Star Wars games.
[02:44:10] Star Wars games. >> I get the hobby project aspect of it.
[02:44:12] >> I get the hobby project aspect of it. It's it's just hard work and it's it's
[02:44:15] It's it's just hard work and it's it's beautiful and like the right approach
[02:44:16] beautiful and like the right approach there is to celebrate people for doing
[02:44:19] there is to celebrate people for doing incredible
[02:44:21] incredible >> incredible work. It's It's just
[02:44:22] >> incredible work. It's It's just incredible that humans step up.
[02:44:24] incredible that humans step up. >> Sure.
[02:44:25] >> Sure. >> Not getting really paid at first or
[02:44:27] >> Not getting really paid at first or maybe ever and then they doing it out of
[02:44:30] maybe ever and then they doing it out of the love of it. And we need to like
[02:44:32] the love of it. And we need to like human civilization runs on people like
[02:44:34] human civilization runs on people like that. We need to celebrate that.
[02:44:36] that. We need to celebrate that. >> To give you an idea, I received death
[02:44:37] >> To give you an idea, I received death threats on video land, right? And um
[02:44:40] threats on video land, right? And um >> you mentioned that to me like what what
[02:44:41] >> you mentioned that to me like what what is what is behind that?
[02:44:43] is what is behind that? >> So that must be what 2009 2010, right?
[02:44:47] >> So that must be what 2009 2010, right? Um uh Apple is moving from Power PC to
[02:44:50] Um uh Apple is moving from Power PC to Core Duo. Um that's probably in 2006 and
[02:44:54] Core Duo. Um that's probably in 2006 and by 2009 or 2010 I decide that we are not
[02:44:57] by 2009 or 2010 I decide that we are not going to do new versions of VC for poor
[02:45:00] going to do new versions of VC for poor PC.
[02:45:01] PC. >> Mhm.
[02:45:02] >> Mhm. >> At that time like VC we were close to
[02:45:04] >> At that time like VC we were close to the number 1.0 release we were four of
[02:45:07] the number 1.0 release we were four of us right like just like no this is not
[02:45:10] us right like just like no this is not possible. So I receive a death threat
[02:45:12] possible. So I receive a death threat with some powder in it, right? It
[02:45:14] with some powder in it, right? It remember there was some anthrax threats
[02:45:17] remember there was some anthrax threats at that time, right? Um and it was
[02:45:20] at that time, right? Um and it was because I had taken the decision to not
[02:45:23] because I had taken the decision to not maintain the power PC port anymore. And
[02:45:26] maintain the power PC port anymore. And of course it wasn't anthrax of course it
[02:45:28] of course it wasn't anthrax of course it was some type of flaw and so on. I
[02:45:30] was some type of flaw and so on. I received that as a with a later of like
[02:45:33] received that as a with a later of like uh you you're a piece of you
[02:45:35] uh you you're a piece of you should die. Uh Boa PC forever and so on.
[02:45:39] should die. Uh Boa PC forever and so on. And was 2009 or 2010, right? I was I was
[02:45:43] And was 2009 or 2010, right? I was I was uh I was young. I was just like why?
[02:45:46] uh I was young. I was just like why? What did I do? Right.
[02:45:47] What did I do? Right. >> Yeah, they can break your spirit. It's
[02:45:49] >> Yeah, they can break your spirit. It's like why?
[02:45:49] like why? >> My mother freaked out, right? We had to
[02:45:52] >> My mother freaked out, right? We had to go to see the police and so on. And now
[02:45:54] go to see the police and so on. And now like
[02:45:55] like I'm going to say that I'm quite happy
[02:45:57] I'm going to say that I'm quite happy that this happened at that time. It it
[02:45:59] that this happened at that time. It it folds me a lot, right? I am I I can see
[02:46:02] folds me a lot, right? I am I I can see I can take a lot of uh of hate on me.
[02:46:05] I can take a lot of uh of hate on me. I'm I'm okay with it, right? It sucks
[02:46:08] I'm I'm okay with it, right? It sucks that that's part of reality cuz all the
[02:46:10] that that's part of reality cuz all the people that love VC, all the people that
[02:46:12] people that love VC, all the people that love FFmpeg,
[02:46:14] love FFmpeg, like me, you know, I I legitimately
[02:46:19] like me, you know, I I legitimately hund probably thousands of times in my
[02:46:21] hund probably thousands of times in my life had a smile on my face because
[02:46:24] life had a smile on my face because FFmpeg made me happy. Period. And how
[02:46:27] FFmpeg made me happy. Period. And how many times did I get a chance to say
[02:46:28] many times did I get a chance to say that? Zero. Until I realize there's a
[02:46:31] that? Zero. Until I realize there's a Twitter account and every once in a
[02:46:33] Twitter account and every once in a while I'm like messaging it. One of the
[02:46:36] while I'm like messaging it. One of the things I like on the Reddit me about me,
[02:46:38] things I like on the Reddit me about me, which I don't like this me for a lot of
[02:46:40] which I don't like this me for a lot of reasons, but and someone says, "Oh, JB
[02:46:42] reasons, but and someone says, "Oh, JB is on is on Reddit," which I am, right?
[02:46:44] is on is on Reddit," which I am, right? And I say and say hello, right? And then
[02:46:46] And I say and say hello, right? And then I got so many people who say, "Oh, thank
[02:46:48] I got so many people who say, "Oh, thank you for VC." And like I take pictures
[02:46:50] you for VC." And like I take pictures and then I share that to to to the
[02:46:53] and then I share that to to to the signal to IRC. Uh yes, we use IRC on on
[02:46:57] signal to IRC. Uh yes, we use IRC on on different
[02:46:58] different >> I I saw as a quick tangent you mentioned
[02:47:00] >> I I saw as a quick tangent you mentioned IRC is like Slack for old people. So you
[02:47:02] IRC is like Slack for old people. So you still use IRC?
[02:47:03] still use IRC? >> Of course.
[02:47:04] >> Of course. >> Yep. I have it on my phone as well.
[02:47:05] >> Yep. I have it on my phone as well. >> Of course.
[02:47:05] >> Of course. >> Every day.
[02:47:06] >> Every day. >> Works fine.
[02:47:07] >> Works fine. >> Wow. It works fine.
[02:47:08] >> Wow. It works fine. >> You have to power with the crank.
[02:47:10] >> You have to power with the crank. >> No, but there's no
[02:47:13] >> No, but there's no there's no tracking. There's nothing
[02:47:15] there's no tracking. There's nothing like it's uh
[02:47:15] like it's uh >> the the biggest issue to be honest,
[02:47:17] >> the the biggest issue to be honest, right, compared to Slack is that it
[02:47:19] right, compared to Slack is that it doesn't have threads. That's annoying.
[02:47:21] doesn't have threads. That's annoying. It doesn't have emojis for reaction.
[02:47:23] It doesn't have emojis for reaction. Sometimes it would be nice.
[02:47:24] Sometimes it would be nice. >> V3 does.
[02:47:25] >> V3 does. >> Yes, V3. But no one does it and you
[02:47:27] >> Yes, V3. But no one does it and you cannot edit your messages. Yeah.
[02:47:29] cannot edit your messages. Yeah. >> Right. And the rest it works perfectly
[02:47:31] >> Right. And the rest it works perfectly fine for
[02:47:31] fine for >> How do you communicate without emojis?
[02:47:33] >> How do you communicate without emojis? >> Well, that's that's why I said it's for
[02:47:35] >> Well, that's that's why I said it's for old people.
[02:47:36] old people. >> Old people and we don't we know emojis
[02:47:38] >> Old people and we don't we know emojis with like you know columns and
[02:47:43] >> so anyway you communicate on IRC. What
[02:47:45] >> so anyway you communicate on IRC. What were you even talking about? Yeah, we
[02:47:47] were you even talking about? Yeah, we are talking about death threats and but
[02:47:49] are talking about death threats and but having people thanking you and sometime
[02:47:51] having people thanking you and sometime they got people who send me a message
[02:47:53] they got people who send me a message and mail, oh, thank you for VC and I
[02:47:56] and mail, oh, thank you for VC and I always answer because I want to to to
[02:47:59] always answer because I want to to to validate the fact that you need to thank
[02:48:01] validate the fact that you need to thank the the the open source community. Yeah,
[02:48:03] the the the open source community. Yeah, please everybody listening to this
[02:48:06] please everybody listening to this celebrate
[02:48:07] celebrate celebrate FFMPEX, celebrate VC,
[02:48:10] celebrate FFMPEX, celebrate VC, celebrate all the incredible uh open
[02:48:13] celebrate all the incredible uh open source projects, Linux, everything.
[02:48:15] source projects, Linux, everything. There's so many there's so many and and
[02:48:17] There's so many there's so many and and you know what I mean even outside of
[02:48:19] you know what I mean even outside of open source just celebrate companies
[02:48:22] open source just celebrate companies that create software that you use a lot
[02:48:25] that create software that you use a lot and love.
[02:48:26] and love. >> Celebrate human endeavor. Celebrate the
[02:48:28] >> Celebrate human endeavor. Celebrate the human effort to not just build something
[02:48:30] human effort to not just build something that's okay.
[02:48:31] that's okay. >> Yeah.
[02:48:32] >> Yeah. >> Build something that is damn good.
[02:48:34] >> Build something that is damn good. >> Yeah. This is important, right? Because
[02:48:36] >> Yeah. This is important, right? Because as we said right, we work for technic we
[02:48:39] as we said right, we work for technic we do something very complex for
[02:48:42] do something very complex for the normal people like we want our
[02:48:45] the normal people like we want our excellence in tech to be useful for
[02:48:48] excellence in tech to be useful for everyone and this is why like this is
[02:48:50] everyone and this is why like this is why we work right this is why I wake up
[02:48:52] why we work right this is why I wake up in the morning is because I want people
[02:48:54] in the morning is because I want people to use our stuff uh because it's making
[02:48:56] to use our stuff uh because it's making everyone's life easier
[02:48:58] everyone's life easier >> want to solve hard problems work on
[02:49:00] >> want to solve hard problems work on something interesting work on some
[02:49:01] something interesting work on some interesting technical challenges
[02:49:03] interesting technical challenges >> we engineers we love to build things
[02:49:04] >> we engineers we love to build things right When I was young, like very early,
[02:49:06] right When I was young, like very early, I knew I wanted to build to be engineer.
[02:49:08] I knew I wanted to build to be engineer. I wanted to do to do cars, right? Maybe
[02:49:10] I wanted to do to do cars, right? Maybe at some point I will go back to cars,
[02:49:12] at some point I will go back to cars, right? But um this is like we want to
[02:49:15] right? But um this is like we want to build things that are cool and useful
[02:49:17] build things that are cool and useful and they need to be challenging, right?
[02:49:19] and they need to be challenging, right? Because you want your brain to turn on.
[02:49:21] Because you want your brain to turn on. >> When did the two of you first fall in
[02:49:23] >> When did the two of you first fall in love with programming, with with
[02:49:25] love with programming, with with building, with engineering?
[02:49:26] building, with engineering? >> When is the first time you programmed K?
[02:49:29] >> When is the first time you programmed K? >> Microsoft QBasic as on Windows 3.1 and
[02:49:32] >> Microsoft QBasic as on Windows 3.1 and Windows 95. Microsoft QBasic.
[02:49:34] Windows 95. Microsoft QBasic. >> Wow. What did you build?
[02:49:36] >> Wow. What did you build? >> Uh like a multiplication just counting
[02:49:38] >> Uh like a multiplication just counting loops like 10 20 30 40.
[02:49:40] loops like 10 20 30 40. >> Nice.
[02:49:41] >> Nice. >> Uh then I thought I could do everything
[02:49:42] >> Uh then I thought I could do everything after that. I wanted I jumped from doing
[02:49:44] after that. I wanted I jumped from doing that to I want to create a soccer no a
[02:49:47] that to I want to create a soccer no a football soccer video game and I drew
[02:49:49] football soccer video game and I drew all the I drew everything out. I was
[02:49:50] all the I drew everything out. I was like I want to do it and I didn't quite
[02:49:51] like I want to do it and I didn't quite grasp that actually
[02:49:54] grasp that actually >> uh I didn't grasp actually it's a
[02:49:55] >> uh I didn't grasp actually it's a massive piece of work to jump from basic
[02:49:57] massive piece of work to jump from basic and drawings and pictures to a video
[02:49:58] and drawings and pictures to a video game. But there we go. I I think I did
[02:50:01] game. But there we go. I I think I did um also basics and then the turbo pascal
[02:50:05] um also basics and then the turbo pascal when I was yeah um end of elementary
[02:50:10] when I was yeah um end of elementary school. Uh but mostly the first time I
[02:50:13] school. Uh but mostly the first time I actually did some serious programming
[02:50:14] actually did some serious programming was the the first uh year of um would
[02:50:19] was the the first uh year of um would you call that middle school when you're
[02:50:20] you call that middle school when you're 11?
[02:50:21] 11? >> Mhm.
[02:50:21] >> Mhm. >> Um I was uh I lived in Italy for a year
[02:50:25] >> Um I was uh I lived in Italy for a year in Florence and it was amazing year. Um,
[02:50:29] in Florence and it was amazing year. Um, and like the math teacher told us to to
[02:50:32] and like the math teacher told us to to work in a programming language called
[02:50:33] work in a programming language called logo where you had a turtle that was
[02:50:36] logo where you had a turtle that was designing things uh on the screen and
[02:50:38] designing things uh on the screen and you turn left and right and in the end
[02:50:40] you turn left and right and in the end we use that to do uh very complex
[02:50:43] we use that to do uh very complex programming because of course you could
[02:50:45] programming because of course you could do things and and this changed like I
[02:50:48] do things and and this changed like I knew I was I wanted to do things with
[02:50:50] knew I was I wanted to do things with computers and program.
[02:50:51] computers and program. >> I don't think we quite talked about x264
[02:50:54] >> I don't think we quite talked about x264 properly. We talked about David. Can we
[02:50:56] properly. We talked about David. Can we return backtrack a little bit to x264?
[02:51:00] return backtrack a little bit to x264? This thing that powers basically all of
[02:51:02] This thing that powers basically all of the video on the internet. So, uh, can
[02:51:05] the video on the internet. So, uh, can you tell me the story of x264? And
[02:51:09] you tell me the story of x264? And Kieran, you're actually a contributor to
[02:51:11] Kieran, you're actually a contributor to x64.
[02:51:12] x64. >> So, so x264 is is a video encoder for
[02:51:15] >> So, so x264 is is a video encoder for the H.264 video standard. It dominates
[02:51:18] the H.264 video standard. It dominates internet video, but also other areas
[02:51:20] internet video, but also other areas such as Blu-ray discs. And Blu-ray discs
[02:51:21] such as Blu-ray discs. And Blu-ray discs are interesting because the people that
[02:51:23] are interesting because the people that make them really want the highest
[02:51:24] make them really want the highest quality. And there's some really cool
[02:51:26] quality. And there's some really cool high-end films that have been encoded uh
[02:51:28] high-end films that have been encoded uh broadcasting and all sorts of other
[02:51:30] broadcasting and all sorts of other areas. X264 was a big step change um
[02:51:34] areas. X264 was a big step change um because it kind of happened at the right
[02:51:36] because it kind of happened at the right time as well. A lot of the development
[02:51:37] time as well. A lot of the development took place when HD video was coming out.
[02:51:40] took place when HD video was coming out. Intel CU and NHLM CPUs were getting
[02:51:43] Intel CU and NHLM CPUs were getting fast. You could do real-time video. But
[02:51:45] fast. You could do real-time video. But the mo the most important thing was um a
[02:51:49] the mo the most important thing was um a key sort of focus on visual metrics. So
[02:51:53] key sort of focus on visual metrics. So industry and academia for 20 years
[02:51:55] industry and academia for 20 years before was obsessed with um mathematical
[02:51:59] before was obsessed with um mathematical metrics. So what's known as peak signal
[02:52:00] metrics. So what's known as peak signal to noise ratio. So mean squed error
[02:52:03] to noise ratio. So mean squed error logarithm of mean squed error. And that
[02:52:04] logarithm of mean squed error. And that led to tons of issues because mean
[02:52:06] led to tons of issues because mean squared error um leads to blurring
[02:52:09] squared error um leads to blurring because you actually want to you want to
[02:52:10] because you actually want to you want to minimize you want to add a little bit of
[02:52:11] minimize you want to add a little bit of error to everything to to reduce the
[02:52:13] error to everything to to reduce the mean squared error as opposed to having
[02:52:14] mean squared error as opposed to having a big error and that led to loads and
[02:52:16] a big error and that led to loads and loads of blurring. So but hobbyists buck
[02:52:18] loads of blurring. So but hobbyists buck that trend. It was for their own
[02:52:19] that trend. It was for their own personal videos, mostly anime.
[02:52:22] personal videos, mostly anime. So there were two there were two things
[02:52:23] So there were two there were two things they did differently and there was a big
[02:52:25] they did differently and there was a big iterative feedback loop with the
[02:52:26] iterative feedback loop with the community. They did some stuff
[02:52:28] community. They did some stuff differently. Um two two big things.
[02:52:31] differently. Um two two big things. Psycho visual rate distortion. So using
[02:52:34] Psycho visual rate distortion. So using block energy trying to compensate for
[02:52:36] block energy trying to compensate for human perception when making decisions.
[02:52:38] human perception when making decisions. >> So the psycho visual distortion that's
[02:52:41] >> So the psycho visual distortion that's the critical thing. That's the thing. I
[02:52:44] the critical thing. That's the thing. I mean it's kind of revolutionary like
[02:52:47] mean it's kind of revolutionary like that we can like rethink
[02:52:50] that we can like rethink don't don't make it like this kind of
[02:52:52] don't don't make it like this kind of theoretic thing of compression
[02:52:54] theoretic thing of compression >> make it all about
[02:52:56] >> make it all about >> being pleasing visually to the eye.
[02:52:58] >> being pleasing visually to the eye. >> Yeah. Yeah. So compressing in a way that
[02:53:00] >> Yeah. Yeah. So compressing in a way that loses the least amount of information
[02:53:02] loses the least amount of information for the stuff that matters for us
[02:53:04] for the stuff that matters for us humans.
[02:53:04] humans. >> Yes. Exactly. as opposed to what
[02:53:06] >> Yes. Exactly. as opposed to what industry some parts of industry are
[02:53:08] industry some parts of industry are still obsessed by this which is
[02:53:09] still obsessed by this which is mathematical numbers that don't look um
[02:53:12] mathematical numbers that don't look um good in reality. And then adaptive
[02:53:14] good in reality. And then adaptive quantization was the other big one where
[02:53:16] quantization was the other big one where it was biasing bits against um complex
[02:53:20] it was biasing bits against um complex areas and redistributing them to less
[02:53:22] areas and redistributing them to less complex areas like grass. Grass has some
[02:53:25] complex areas like grass. Grass has some high frequencies but it's kind of it's
[02:53:28] high frequencies but it's kind of it's less complex overall compared to more
[02:53:29] less complex overall compared to more complicated things. And this came around
[02:53:31] complicated things. And this came around by um Pock Joy. to Park Joy was really
[02:53:34] by um Pock Joy. to Park Joy was really the canonical sample that was the
[02:53:37] the canonical sample that was the running around in the park. Yeah. So
[02:53:38] running around in the park. Yeah. So this guy was really the um
[02:53:42] this guy was really the um >> so this this was created by Swedish
[02:53:44] >> so this this was created by Swedish television in um the beginning of HD and
[02:53:47] television in um the beginning of HD and it was done on film and it was no
[02:53:49] it was done on film and it was no expense spared in terms of production
[02:53:51] expense spared in terms of production quality and it was given away for free.
[02:53:53] quality and it was given away for free. This was really and this is the sample
[02:53:56] This was really and this is the sample really that sorts the men from the boys
[02:53:57] really that sorts the men from the boys in terms of it has so many challenges
[02:53:59] in terms of it has so many challenges with the trees, with the water, with the
[02:54:02] with the trees, with the water, with the grass, with the motion, with the I don't
[02:54:05] grass, with the motion, with the I don't think there's there's still been any any
[02:54:08] think there's there's still been any any public test sequence as good as that
[02:54:10] public test sequence as good as that these days. So for people who are just
[02:54:12] these days. So for people who are just listening, we're looking at a bunch of
[02:54:15] listening, we're looking at a bunch of humans running along a river as you have
[02:54:18] humans running along a river as you have the reflection, a lot of really high
[02:54:21] the reflection, a lot of really high information textures everywhere, the
[02:54:23] information textures everywhere, the leaves and the lighting playing with the
[02:54:25] leaves and the lighting playing with the leaves and all of this.
[02:54:27] leaves and all of this. >> You could show clearly that encoders
[02:54:29] >> You could show clearly that encoders with high PSNR
[02:54:30] with high PSNR >> will blur everything
[02:54:31] >> will blur everything >> will blur everything. And you could see
[02:54:33] >> will blur everything. And you could see actually I could turn on psycho visual
[02:54:34] actually I could turn on psycho visual stuff. I could turn off reactive
[02:54:35] stuff. I could turn off reactive quantization and it would just look so
[02:54:38] quantization and it would just look so much better. But your your metrics and
[02:54:40] much better. But your your metrics and these metrics are at the time what at
[02:54:42] these metrics are at the time what at the time were considered so holy.
[02:54:44] the time were considered so holy. >> These are the holy metrics that are
[02:54:45] >> These are the holy metrics that are untouchable. PSNR is the most important
[02:54:47] untouchable. PSNR is the most important thing. Uh can you speak to how do you
[02:54:50] thing. Uh can you speak to how do you measure psycho visual stuff like how do
[02:54:53] measure psycho visual stuff like how do you turn how pleasing a compression is
[02:54:56] you turn how pleasing a compression is for a human eye into a number? Is that
[02:54:59] for a human eye into a number? Is that even possible?
[02:55:00] even possible? >> That's what that's what Netflix have
[02:55:01] >> That's what that's what Netflix have been trying to do with Vmath. They said
[02:55:02] been trying to do with Vmath. They said they've used a machine learning model.
[02:55:04] they've used a machine learning model. >> That's a more recent thing. But back in
[02:55:06] >> That's a more recent thing. But back in uh when X6 was being developed, that's
[02:55:09] uh when X6 was being developed, that's by eye. You're basically
[02:55:10] by eye. You're basically >> it was by eye. It was developers on
[02:55:12] >> it was by eye. It was developers on their laptops. So it's not like even
[02:55:13] their laptops. So it's not like even with big companies with prof
[02:55:15] with big companies with prof professional screens or anything. It's
[02:55:16] professional screens or anything. It's and that was actually one of the goals
[02:55:18] and that was actually one of the goals which was I don't the the developers at
[02:55:20] which was I don't the the developers at the time, Lauren Merritt in particular,
[02:55:22] the time, Lauren Merritt in particular, it's I don't want to test this on a
[02:55:23] it's I don't want to test this on a $30,000 screen. It's I want this to look
[02:55:25] $30,000 screen. It's I want this to look good on someone's laptop at home.
[02:55:27] good on someone's laptop at home. >> Yeah. Brilliant. And there is another
[02:55:30] >> Yeah. Brilliant. And there is another sample which is um a sample that is
[02:55:33] sample which is um a sample that is planet earth's kila sample that I
[02:55:36] planet earth's kila sample that I absolutely love and you we're going to
[02:55:38] absolutely love and you we're going to see why this uh
[02:55:41] see why this uh >> um it's a ton of birds right flying and
[02:55:45] >> um it's a ton of birds right flying and the more it goes the more there are
[02:55:47] the more it goes the more there are birds and at at the end right it's
[02:55:50] birds and at at the end right it's almost like you have millions of birds
[02:55:53] almost like you have millions of birds it's the most complex thing ever to
[02:55:56] it's the most complex thing ever to encode right and well you're watching it
[02:55:58] encode right and well you're watching it on YouTube and you see how bad the
[02:56:00] on YouTube and you see how bad the YouTube encoding is actually, right? Um,
[02:56:03] YouTube encoding is actually, right? Um, and this is like phenomenal to to
[02:56:06] and this is like phenomenal to to optimize and get um perfect quality in a
[02:56:10] optimize and get um perfect quality in a constant bit rate. Um, there was a lot
[02:56:12] constant bit rate. Um, there was a lot of optimization mostly by by Lauren also
[02:56:15] of optimization mostly by by Lauren also um on anime, right? For a long time,
[02:56:18] um on anime, right? For a long time, anime was very badly encoded because
[02:56:20] anime was very badly encoded because there was a ton of bending, right? And
[02:56:22] there was a ton of bending, right? And you see those um those issue and there
[02:56:25] you see those um those issue and there was a ton of things. Um, so x264 is like
[02:56:28] was a ton of things. Um, so x264 is like and today it's still the reference to
[02:56:31] and today it's still the reference to any encoder, new encoder, AV1, AV2, VVC,
[02:56:35] any encoder, new encoder, AV1, AV2, VVC, HVC, everyone compares to x264. One of
[02:56:38] HVC, everyone compares to x264. One of my favorite films, um, Cinema Paradiso,
[02:56:41] my favorite films, um, Cinema Paradiso, I know the engineer who created the
[02:56:43] I know the engineer who created the Blu-ray and he showed me the comparisons
[02:56:44] Blu-ray and he showed me the comparisons of X.264 versus others. And, uh, it's
[02:56:48] of X.264 versus others. And, uh, it's completely different. And I think a
[02:56:50] completely different. And I think a bunch a bunch of guys in the Blu-ray
[02:56:52] bunch a bunch of guys in the Blu-ray world started using x.264. Um, I think
[02:56:54] world started using x.264. Um, I think the big one was Chris Henderson from
[02:56:55] the big one was Chris Henderson from Warner Brothers. He did the whole French
[02:56:57] Warner Brothers. He did the whole French box set with that's a quite like a thing
[02:56:59] box set with that's a quite like a thing that a person on the street actually
[02:57:01] that a person on the street actually watches and wants to look good. And so
[02:57:02] watches and wants to look good. And so they kind of took a risk in their jobs
[02:57:04] they kind of took a risk in their jobs doing that cuz they're in a big company.
[02:57:06] doing that cuz they're in a big company. That big company can buy whatever they
[02:57:07] That big company can buy whatever they want. And they said, "No, no, no. I want
[02:57:09] want. And they said, "No, no, no. I want to use this free and open source thing
[02:57:11] to use this free and open source thing so that things look good for my my
[02:57:13] so that things look good for my my customers and build the best." And to
[02:57:15] customers and build the best." And to this day, I personally still try and
[02:57:17] this day, I personally still try and avoid watching the most cinematic films
[02:57:20] avoid watching the most cinematic films on streaming services and buy the
[02:57:22] on streaming services and buy the physical discs because they look they
[02:57:25] physical discs because they look they look good without even having to buy an
[02:57:26] look good without even having to buy an expensive TV. I think that's the key
[02:57:28] expensive TV. I think that's the key thing. And x264 is yet another example
[02:57:30] thing. And x264 is yet another example of open source project. It was started
[02:57:32] of open source project. It was started by Lauron Emar when he was at the San
[02:57:35] by Lauron Emar when he was at the San Alpare where VC was born. And then you
[02:57:37] Alpare where VC was born. And then you got a generation of people like Lauren,
[02:57:40] got a generation of people like Lauren, like Jason, like Monz, like uh so many
[02:57:44] like Jason, like Monz, like uh so many Henrik and this is Anton. And this is
[02:57:47] Henrik and this is Anton. And this is where the assembly thing that we use now
[02:57:50] where the assembly thing that we use now on FFmpeg David and so on was born.
[02:57:52] on FFmpeg David and so on was born. Right. So X264 is like amazing project
[02:57:54] Right. So X264 is like amazing project with people who were really all over the
[02:57:57] with people who were really all over the world and I think most of them never met
[02:57:59] world and I think most of them never met each other but all of them according to
[02:58:01] each other but all of them according to Kieran or large percentage love anime.
[02:58:05] Kieran or large percentage love anime. There's several things I've never got
[02:58:07] There's several things I've never got into and one of them is anime and I need
[02:58:08] into and one of them is anime and I need to
[02:58:09] to >> I watch anime so much. Um especially at
[02:58:12] >> I watch anime so much. Um especially at the time um like at the time it was like
[02:58:17] the time um like at the time it was like a lot of anime content doesn't exist
[02:58:20] a lot of anime content doesn't exist commercially right we before crunchy
[02:58:22] commercially right we before crunchy roll right so what happens is usually
[02:58:24] roll right so what happens is usually people who love anime who take some
[02:58:26] people who love anime who take some things some DVDs in Japan and rip them
[02:58:31] things some DVDs in Japan and rip them because there is no commercial offering
[02:58:33] because there is no commercial offering and some of the people who are what we
[02:58:35] and some of the people who are what we call fan subers are basically
[02:58:37] call fan subers are basically translating themselves to to make
[02:58:39] translating themselves to to make subtitles right and at that time you
[02:58:41] subtitles right and at that time you download completely illegally. It was
[02:58:43] download completely illegally. It was the only way to do that, right? And so
[02:58:46] the only way to do that, right? And so all of that was handcrafted and it fits
[02:58:49] all of that was handcrafted and it fits the open source community, right?
[02:58:50] the open source community, right? Because they needed tools to encode to
[02:58:52] Because they needed tools to encode to do fun subbing, right? One of the most
[02:58:54] do fun subbing, right? One of the most amazing open source projects for
[02:58:56] amazing open source projects for subtitles. It's called a AEG sub and
[02:58:59] subtitles. It's called a AEG sub and it's it's a subtitle. It's known for
[02:59:01] it's it's a subtitle. It's known for anime for for for um South Asian in
[02:59:05] anime for for for um South Asian in Japanese languages. There are weird
[02:59:07] Japanese languages. There are weird textures in anime that I don't think you
[02:59:09] textures in anime that I don't think you get in real life content. I think that
[02:59:12] get in real life content. I think that was a key one which was optimizing these
[02:59:14] was a key one which was optimizing these weird textures that you get because
[02:59:15] weird textures that you get because anime is not done in a normal fashion.
[02:59:17] anime is not done in a normal fashion. >> Yeah. The way you produce is not you
[02:59:20] >> Yeah. The way you produce is not you mostly produce it like on screens right
[02:59:22] mostly produce it like on screens right since a bit of time and you have all
[02:59:24] since a bit of time and you have all those gradients right in colors because
[02:59:27] those gradients right in colors because they are very easy to produce digitally
[02:59:29] they are very easy to produce digitally very complex to to produce in real life.
[02:59:31] very complex to to produce in real life. Um and um and the subtitles also are
[02:59:34] Um and um and the subtitles also are very complex because you need to have
[02:59:36] very complex because you need to have often the Japanese and then um you need
[02:59:39] often the Japanese and then um you need to have the diiocritics, right? The what
[02:59:41] to have the diiocritics, right? The what we call the ribeye, right? Which is the
[02:59:43] we call the ribeye, right? Which is the hiragana and the katakana for the ki and
[02:59:45] hiragana and the katakana for the ki and then because of course you so that you
[02:59:47] then because of course you so that you have the official subtitling, but you
[02:59:50] have the official subtitling, but you also need the English subtitles or the
[02:59:52] also need the English subtitles or the French subtitles because you want to to
[02:59:53] French subtitles because you want to to learn that, right? Um and there is so
[02:59:55] learn that, right? Um and there is so many things crazy on subtitles. uh and
[02:59:58] many things crazy on subtitles. uh and we had like crazy samples on on
[02:59:59] we had like crazy samples on on subtitles that we've seen all around. So
[03:00:02] subtitles that we've seen all around. So um this is an important part of the
[03:00:06] um this is an important part of the culture but also because there was no
[03:00:08] culture but also because there was no official offering there was no way of
[03:00:10] official offering there was no way of doing that. Uh can you speak to the
[03:00:12] doing that. Uh can you speak to the difference between H.264 and AV1 and
[03:00:15] difference between H.264 and AV1 and then X264 and David this is this big
[03:00:20] then X264 and David this is this big step. Can you help people understand are
[03:00:22] step. Can you help people understand are some of the streaming sites moving more
[03:00:24] some of the streaming sites moving more towards that direction of AV1? Let's be
[03:00:27] towards that direction of AV1? Let's be honest, all of those codecs since Meeek
[03:00:31] honest, all of those codecs since Meeek 2 video are the same concepts.
[03:00:36] 2 video are the same concepts. The same concept about inverse
[03:00:38] The same concept about inverse transform, about intra prediction,
[03:00:40] transform, about intra prediction, motion compensate, entropy coding, all
[03:00:43] motion compensate, entropy coding, all of them. However, each generation gives
[03:00:46] of them. However, each generation gives you a bump between 25 and 50%
[03:00:51] you a bump between 25 and 50% more compression for the same quality.
[03:00:54] more compression for the same quality. Um and so you had the MEC 2, you had the
[03:00:57] Um and so you had the MEC 2, you had the Divix area, you have H.264 which was
[03:01:01] Divix area, you have H.264 which was like changing, right? H.264 improved so
[03:01:04] like changing, right? H.264 improved so much. And then you had more, right? You
[03:01:06] much. And then you had more, right? You had HVC. Um you had VP9 at the same time
[03:01:10] had HVC. Um you had VP9 at the same time of HVC. VP9 is a bit similar to HVC in
[03:01:14] of HVC. VP9 is a bit similar to HVC in terms of quality compression, but it's
[03:01:16] terms of quality compression, but it's royalty-free because in multimedia there
[03:01:19] royalty-free because in multimedia there is ton of patents and the licensing
[03:01:22] is ton of patents and the licensing after H.264 264 became out of Android
[03:01:24] after H.264 264 became out of Android and could cost hundreds of millions of
[03:01:27] and could cost hundreds of millions of dollars per year. So it made no sense.
[03:01:29] dollars per year. So it made no sense. So Google did this VP9 and the Alons for
[03:01:32] So Google did this VP9 and the Alons for Open Media did this new codec called
[03:01:34] Open Media did this new codec called AV1.
[03:01:35] AV1. Um so you can imagine that AV1 saves
[03:01:39] Um so you can imagine that AV1 saves between 40 and 60% less bandwidth than
[03:01:43] between 40 and 60% less bandwidth than H.264 for for the same quality visual
[03:01:47] H.264 for for the same quality visual quality
[03:01:48] quality >> at a given bit rate.
[03:01:49] >> at a given bit rate. >> At a given bit rate, right? So that's
[03:01:51] >> At a given bit rate, right? So that's really like you increase the quality.
[03:01:53] really like you increase the quality. Either you you set the bit rate and you
[03:01:55] Either you you set the bit rate and you increase the quality or you set the
[03:01:56] increase the quality or you set the quality and you decrease your bit rate.
[03:01:58] quality and you decrease your bit rate. But because now you move from from SD to
[03:02:01] But because now you move from from SD to HD and HD to 4K and 4K to 4K HDR like
[03:02:06] HD and HD to 4K and 4K to 4K HDR like you increasing the size by like two
[03:02:09] you increasing the size by like two factor two three four right so you need
[03:02:11] factor two three four right so you need to have better compression to keep it in
[03:02:14] to have better compression to keep it in terms of something that is manageable.
[03:02:16] terms of something that is manageable. It's more coding tools, more bigger
[03:02:19] It's more coding tools, more bigger blocks, lots more subartitions in each
[03:02:21] blocks, lots more subartitions in each block. It's just exponentially more
[03:02:22] block. It's just exponentially more complex.
[03:02:23] complex. >> It's more complex because the the
[03:02:25] >> It's more complex because the the encoder needs to search more
[03:02:28] encoder needs to search more possibilities, right? So you for example
[03:02:30] possibilities, right? So you for example um one of the things that is easy to
[03:02:32] um one of the things that is easy to understand is to predict a block, a
[03:02:35] understand is to predict a block, a color block to another, you have
[03:02:37] color block to another, you have directions, right? So you can go left,
[03:02:39] directions, right? So you can go left, right, bottom, up and then in terms of
[03:02:42] right, bottom, up and then in terms of uh uh like the other quadrants, right?
[03:02:45] uh uh like the other quadrants, right? uh I called north northeast northwest
[03:02:47] uh I called north northeast northwest and and so on right but that's eight
[03:02:49] and and so on right but that's eight directions then you can do more
[03:02:50] directions then you can do more direction you can do 16 or 69 or 128
[03:02:54] direction you can do 16 or 69 or 128 right you can and every time your
[03:02:57] right you can and every time your encoder is going to spend more time to
[03:02:58] encoder is going to spend more time to see oh well these blocks is exactly this
[03:03:00] see oh well these blocks is exactly this one um and those type of tools that you
[03:03:04] one um and those type of tools that you can bring and the encoder needs to check
[03:03:06] can bring and the encoder needs to check which of the tools are going to compress
[03:03:08] which of the tools are going to compress you better and so
[03:03:11] you better and so I guess that AV1 encoding is two order
[03:03:15] I guess that AV1 encoding is two order of magnitudes more than H.264 in terms
[03:03:17] of magnitudes more than H.264 in terms of CPU cycle, right? Order of magnitude,
[03:03:21] of CPU cycle, right? Order of magnitude, right?
[03:03:21] right? >> Yeah.
[03:03:21] >> Yeah. >> And as we discussed, CPUs are not
[03:03:23] >> And as we discussed, CPUs are not getting faster. You're just throwing
[03:03:24] getting faster. You're just throwing more cores at the problem.
[03:03:25] more cores at the problem. >> But also, it's a fact that you encode
[03:03:27] >> But also, it's a fact that you encode once and you have hundreds of millions
[03:03:29] once and you have hundreds of millions of users, right? So, for example,
[03:03:31] of users, right? So, for example, YouTube, a very good example, YouTube
[03:03:33] YouTube, a very good example, YouTube encodes almost everything in H.264, 264
[03:03:36] encodes almost everything in H.264, 264 but the popular video gets re-encoded in
[03:03:39] but the popular video gets re-encoded in AV1 because
[03:03:40] AV1 because >> it costs more of course to encode but
[03:03:43] >> it costs more of course to encode but you encode once and you send that to
[03:03:45] you encode once and you send that to millions right so it's a trade-off
[03:03:47] millions right so it's a trade-off between encoding time and complexity and
[03:03:50] between encoding time and complexity and and CPU usage on the server side and on
[03:03:53] and CPU usage on the server side and on the client side because at the end if
[03:03:56] the client side because at the end if you're distributing a video to hundreds
[03:03:58] you're distributing a video to hundreds of thousands of people and the size is
[03:04:00] of thousands of people and the size is half of the other then it's better it's
[03:04:03] half of the other then it's better it's better for your battery it's better for
[03:04:04] better for your battery it's better for your them etc etc. So we can lay out
[03:04:08] your them etc etc. So we can lay out let's say the top five codec container
[03:04:13] let's say the top five codec container combos would be H264 inside MP4
[03:04:17] combos would be H264 inside MP4 containers
[03:04:19] containers AV1 inside MP4 webm containers uh ProRes
[03:04:25] AV1 inside MP4 webm containers uh ProRes for nonlinear editing
[03:04:27] for nonlinear editing uh inside containers. So for people who
[03:04:30] uh inside containers. So for people who don't know, I guess ProRes is it's
[03:04:33] don't know, I guess ProRes is it's Apple's codec for editing originally for
[03:04:35] Apple's codec for editing originally for Final Cut Pro and it's designed to be
[03:04:37] Final Cut Pro and it's designed to be fast to decode, fast to seek because an
[03:04:39] fast to decode, fast to seek because an editor will need to move very quickly.
[03:04:41] editor will need to move very quickly. So it's a different use case to the
[03:04:43] So it's a different use case to the distribution element.
[03:04:45] distribution element. >> There's no or very minimal uh temporal
[03:04:47] >> There's no or very minimal uh temporal compression.
[03:04:48] compression. >> There's none yet. There's none in so you
[03:04:50] >> There's none yet. There's none in so you can cut. So you can do cuts.
[03:04:52] can cut. So you can do cuts. >> This is what we call intra only codecs.
[03:04:54] >> This is what we call intra only codecs. Right? So I'm going to explain quickly
[03:04:57] Right? So I'm going to explain quickly what is IPB frames.
[03:05:00] what is IPB frames. >> Yes please.
[03:05:00] >> Yes please. >> Um so I frames often key frames but is
[03:05:06] >> Um so I frames often key frames but is complete frames. It's like an image.
[03:05:08] complete frames. It's like an image. It's a JPEG right you have you can start
[03:05:10] It's a JPEG right you have you can start you see everything right um and then you
[03:05:14] you see everything right um and then you the next image can be a P frame which is
[03:05:17] the next image can be a P frame which is a predicted frame. So you take some part
[03:05:20] a predicted frame. So you take some part of the previous image saying well I need
[03:05:22] of the previous image saying well I need the block five and seven and 42 and you
[03:05:26] the block five and seven and 42 and you replace it and then you just give the
[03:05:28] replace it and then you just give the extra information right but that means
[03:05:30] extra information right but that means that in order to decode this P frame you
[03:05:32] that in order to decode this P frame you need to have access the previous I frame
[03:05:35] need to have access the previous I frame right
[03:05:35] right >> and then of course you have more complex
[03:05:37] >> and then of course you have more complex one which are B frames which are be
[03:05:40] one which are B frames which are be predicted um frames which can be depend
[03:05:43] predicted um frames which can be depend on different type of frames some in the
[03:05:46] on different type of frames some in the past some in the future
[03:05:49] past some in the future Um and so prozz is an intra only codec
[03:05:52] Um and so prozz is an intra only codec for the people who can see this is a
[03:05:54] for the people who can see this is a very good one right so I frames are
[03:05:56] very good one right so I frames are complete frames um uh p frame basically
[03:06:00] complete frames um uh p frame basically uh depend only on I frame and b frames
[03:06:02] uh depend only on I frame and b frames can depend on in front
[03:06:04] can depend on in front >> in this uh gop group of pictures I think
[03:06:07] >> in this uh gop group of pictures I think the default for actually ffmpeg
[03:06:11] the default for actually ffmpeg for h64
[03:06:15] is uh like 250 50 a frame something like
[03:06:18] is uh like 250 50 a frame something like this.
[03:06:19] this. >> Yes.
[03:06:19] >> Yes. >> And to me it's just it's like magic like
[03:06:22] >> And to me it's just it's like magic like that you could predict you could have a
[03:06:24] that you could predict you could have a complete frame every
[03:06:26] complete frame every >> several seconds that means
[03:06:28] >> several seconds that means >> several seconds and then you could still
[03:06:30] >> several seconds and then you could still you can have this chain of predictions
[03:06:32] you can have this chain of predictions you make and the fact that you can the
[03:06:34] you make and the fact that you can the fact that somebody like me can can use
[03:06:37] fact that somebody like me can can use fmp to compress something and not notice
[03:06:39] fmp to compress something and not notice that the result still plays back
[03:06:41] that the result still plays back smoothly is is like magic. You can even
[03:06:44] smoothly is is like magic. You can even have and we use that in tons on Kyber is
[03:06:48] have and we use that in tons on Kyber is what we call intra refresh where
[03:06:51] what we call intra refresh where basically it's there is no frame.
[03:06:54] basically it's there is no frame. >> You you have no you have one at the
[03:06:56] >> You you have no you have one at the beginning and you never send an iframe.
[03:06:57] beginning and you never send an iframe. >> How does that work? What is
[03:06:58] >> How does that work? What is >> you you build up an iframe gradually
[03:07:01] >> you you build up an iframe gradually across as the stream continues. So, so
[03:07:03] across as the stream continues. So, so you refresh certain parts of the image,
[03:07:05] you refresh certain parts of the image, >> but so you never have an iframe like
[03:07:07] >> but so you never have an iframe like this is in refresh that we use, right?
[03:07:10] this is in refresh that we use, right? But for me, for me, the the the biggest
[03:07:12] But for me, for me, the the the biggest mind-b blown when I started was the B
[03:07:14] mind-b blown when I started was the B frames.
[03:07:14] frames. >> Yeah.
[03:07:15] >> Yeah. >> B frames means B predicted frames can
[03:07:18] >> B frames means B predicted frames can depend on frames that are coming in the
[03:07:21] depend on frames that are coming in the future.
[03:07:22] future. >> Mhm.
[03:07:23] >> Mhm. That means that in order to decode this
[03:07:26] That means that in order to decode this B frame, you need to wait for the next
[03:07:28] B frame, you need to wait for the next frame that is dependent buffers that
[03:07:32] frame that is dependent buffers that decode that one so that you can decode
[03:07:34] decode that one so that you can decode the B frame. Right? So the the the way
[03:07:36] the B frame. Right? So the the the way you decode the frame, the decoding order
[03:07:40] you decode the frame, the decoding order is not the same as the display order,
[03:07:42] is not the same as the display order, right? That means the encoder needs to
[03:07:44] right? That means the encoder needs to be very clever and decide that well, you
[03:07:47] be very clever and decide that well, you know, I'm going to depend on things like
[03:07:48] know, I'm going to depend on things like in the future. So
[03:07:50] in the future. So >> this is like mindblowing.
[03:07:53] >> this is like mindblowing. >> Yeah.
[03:07:53] >> Yeah. >> The fact it works so smoothly every day
[03:07:55] >> The fact it works so smoothly every day is is kind of miraculous in some ways.
[03:07:57] is is kind of miraculous in some ways. It works so you can have a stream that
[03:07:59] It works so you can have a stream that works
[03:08:01] works >> across the world on their decoder versus
[03:08:03] >> across the world on their decoder versus one in the US versus one here of
[03:08:05] one in the US versus one here of different manufacturers and they produce
[03:08:07] different manufacturers and they produce bit for bit exactly the same material.
[03:08:10] bit for bit exactly the same material. >> That's quite remarkable. And do quite
[03:08:11] >> That's quite remarkable. And do quite complex things and getting more and more
[03:08:13] complex things and getting more and more complex and still be bit exact. There's
[03:08:16] complex and still be bit exact. There's a lot of work that goes into that.
[03:08:18] a lot of work that goes into that. There's a lot of knobs you can control
[03:08:19] There's a lot of knobs you can control in this whole process. There's a lot of
[03:08:21] in this whole process. There's a lot of really fascinating parameters that I've
[03:08:23] really fascinating parameters that I've gotten to know more and more over the
[03:08:25] gotten to know more and more over the years that FFmpeg gives you complete
[03:08:27] years that FFmpeg gives you complete access to. Maybe you can speak to some
[03:08:29] access to. Maybe you can speak to some of them. So, first of all, like
[03:08:30] of them. So, first of all, like obviously we can lower the resolution.
[03:08:32] obviously we can lower the resolution. We can lower the frame rate. We can use
[03:08:34] We can lower the frame rate. We can use different kinds of codecs as we
[03:08:36] different kinds of codecs as we mentioned from H.64 to AV1. There's ways
[03:08:40] mentioned from H.64 to AV1. There's ways to tune the trade-off between bit rate
[03:08:42] to tune the trade-off between bit rate and quality as we've kind of spoken to.
[03:08:45] and quality as we've kind of spoken to. You know, you could do constant bit
[03:08:46] You know, you could do constant bit rate. You can do constant quality serq
[03:08:49] rate. You can do constant quality serq QP. You can uh do the longer or shorter
[03:08:54] QP. You can uh do the longer or shorter group of pictures go that we mentioned.
[03:08:57] group of pictures go that we mentioned. I mean all that kind of stuff. It's
[03:08:58] I mean all that kind of stuff. It's crazy number of B frames.
[03:09:00] crazy number of B frames. >> Yeah. What is crazy is that
[03:09:03] >> Yeah. What is crazy is that >> a ton of people's job is to optimize
[03:09:08] >> a ton of people's job is to optimize those parameters, right? A ton of people
[03:09:10] those parameters, right? A ton of people that you see at YouTube, at Netflix, at
[03:09:12] that you see at YouTube, at Netflix, at Meta, and so on, they're not writing
[03:09:14] Meta, and so on, they're not writing codecs. They're just like finding the
[03:09:16] codecs. They're just like finding the right parameters for the file they have
[03:09:20] right parameters for the file they have for the format they have, right? Because
[03:09:22] for the format they have, right? Because like something that is for a movie or
[03:09:24] like something that is for a movie or something that is user generated content
[03:09:26] something that is user generated content from your phone or a screen recording or
[03:09:29] from your phone or a screen recording or something that you're going to video
[03:09:30] something that you're going to video edit, you don't want the same things.
[03:09:33] edit, you don't want the same things. And there are thousands of people whose
[03:09:36] And there are thousands of people whose job is just to optimize all that.
[03:09:38] job is just to optimize all that. >> Yeah, they're wizards. Hats off to them.
[03:09:42] >> Yeah, they're wizards. Hats off to them. uh YouTube like to deliver all the
[03:09:44] uh YouTube like to deliver all the streaming sites actually to deliver at
[03:09:46] streaming sites actually to deliver at scale and like YouTube is is really
[03:09:48] scale and like YouTube is is really magical because it's not just doing like
[03:09:50] magical because it's not just doing like what Netflix it does which is one way
[03:09:54] what Netflix it does which is one way like broadcasting type thing it's also
[03:09:58] like broadcasting type thing it's also has to upload videos from all the
[03:10:01] has to upload videos from all the places. So they're also doing encoding
[03:10:03] places. So they're also doing encoding at scale
[03:10:04] at scale >> for videos that are going to be watched
[03:10:05] >> for videos that are going to be watched by like five people and it still has to
[03:10:08] by like five people and it still has to deliver them re like on a moment's
[03:10:11] deliver them re like on a moment's notice. No, no delay, nothing. No late I
[03:10:13] notice. No, no delay, nothing. No late I mean very minimal latency and and also
[03:10:16] mean very minimal latency and and also serve it in all different resolutions
[03:10:20] serve it in all different resolutions like YouTube is basically the web
[03:10:23] like YouTube is basically the web version of VC.
[03:10:26] version of VC. Well, actually it's funny because like
[03:10:27] Well, actually it's funny because like Google video, which was uh something
[03:10:30] Google video, which was uh something they did before uh they acquired
[03:10:32] they did before uh they acquired YouTube, uh was actually using the VC
[03:10:35] YouTube, uh was actually using the VC plug-in so that you could run VLC inside
[03:10:37] plug-in so that you could run VLC inside the web browser uh using the Activix
[03:10:40] the web browser uh using the Activix plug-in and so it worked in Internet
[03:10:43] plug-in and so it worked in Internet Explorer and you were actually running
[03:10:46] Explorer and you were actually running um VC inside your browser
[03:10:49] um VC inside your browser >> which is funny because today we have the
[03:10:51] >> which is funny because today we have the opposite where we have VC web assembly
[03:10:53] opposite where we have VC web assembly where we compile all VC NF ffmpeg to
[03:10:56] where we compile all VC NF ffmpeg to decode to run VLC in type of inside the
[03:10:59] decode to run VLC in type of inside the JavaScript virtual machine um with web
[03:11:02] JavaScript virtual machine um with web assembly. Okay, there's this legendary
[03:11:06] assembly. Okay, there's this legendary story that you pointed me to that uh it
[03:11:10] story that you pointed me to that uh it was discovered uh via Wikileaks release
[03:11:13] was discovered uh via Wikileaks release of all seven documents the CIA was using
[03:11:16] of all seven documents the CIA was using a modified version of VC
[03:11:18] a modified version of VC uh to basically try and trick people
[03:11:21] uh to basically try and trick people what to steal their data.
[03:11:23] what to steal their data. >> Yes, exactly. So can you explain what
[03:11:25] >> Yes, exactly. So can you explain what the heck happened? What?
[03:11:27] the heck happened? What? >> So So this was a surprise, right? Um
[03:11:29] >> So So this was a surprise, right? Um because at some point mentioned some
[03:11:32] because at some point mentioned some documents. There were a few ones with
[03:11:34] documents. There were a few ones with something related to Blu-rays and VC,
[03:11:36] something related to Blu-rays and VC, but the the most interesting one was the
[03:11:39] but the the most interesting one was the CIA vault 7 which if I understand
[03:11:42] CIA vault 7 which if I understand correctly um was the CIA had like a
[03:11:45] correctly um was the CIA had like a custom version of VC uh where they had a
[03:11:48] custom version of VC uh where they had a specific plug-in. Yeah, exactly. This is
[03:11:51] specific plug-in. Yeah, exactly. This is like we had to to write a press release
[03:11:53] like we had to to write a press release on that. Video land wrote a press
[03:11:55] on that. Video land wrote a press release saying the only safe source for
[03:11:57] release saying the only safe source for getting VLC media player is the official
[03:11:59] getting VLC media player is the official video land website. I mean I suppose
[03:12:01] video land website. I mean I suppose that's a security vulnerability for
[03:12:04] that's a security vulnerability for basically any piece of open source
[03:12:07] basically any piece of open source software. Somebody can trick you to
[03:12:10] software. Somebody can trick you to download in a fake website or targeted
[03:12:12] download in a fake website or targeted advertisement. Right? That was a
[03:12:14] advertisement. Right? That was a targeted advertisement. To watch a
[03:12:16] targeted advertisement. To watch a specific file, you need to watch with
[03:12:19] specific file, you need to watch with this custom version of VC. And it was
[03:12:21] this custom version of VC. And it was the normal binaries of VC except they
[03:12:23] the normal binaries of VC except they added one DL. I think it was psappi.dll,
[03:12:27] added one DL. I think it was psappi.dll, which was basically
[03:12:29] which was basically reading your your document folder,
[03:12:32] reading your your document folder, encrypting that and sending that. And
[03:12:35] encrypting that and sending that. And the thing is, this is very clever to be
[03:12:37] the thing is, this is very clever to be honest because once you're watching a
[03:12:40] honest because once you're watching a movie, right, you're going to do that
[03:12:41] movie, right, you're going to do that for two hours and you're not going to
[03:12:42] for two hours and you're not going to touch your computer and sometimes it's
[03:12:44] touch your computer and sometimes it's normal because it's HD that your your
[03:12:46] normal because it's HD that your your fans are going up and say and there is
[03:12:49] fans are going up and say and there is ton of TPU usage because you're using
[03:12:50] ton of TPU usage because you're using VC, right? That's normal. But the thing
[03:12:52] VC, right? That's normal. But the thing is what you don't see is that actually
[03:12:56] is what you don't see is that actually a a powered version of VC that is used
[03:12:59] a a powered version of VC that is used by CIA. Um, we had exactly the same
[03:13:01] by CIA. Um, we had exactly the same problem with Chinese hackers that were
[03:13:05] problem with Chinese hackers that were targeting Indian people and that got VC
[03:13:09] targeting Indian people and that got VC banned from India until I had to to
[03:13:12] banned from India until I had to to fight in courts in India, the Indian
[03:13:14] fight in courts in India, the Indian government to unban VLC. They didn't use
[03:13:17] government to unban VLC. They didn't use VC. They took just one DLL because we
[03:13:20] VC. They took just one DLL because we signed the DLL correctly. Um, and they
[03:13:23] signed the DLL correctly. Um, and they used that DLL to do another program. Uh,
[03:13:27] used that DLL to do another program. Uh, so you had a VC.exe XZ and was calling
[03:13:29] so you had a VC.exe XZ and was calling lib VLC but it was calling it into a
[03:13:32] lib VLC but it was calling it into a fake one and they use that to to target.
[03:13:34] fake one and they use that to to target. Um there is not much we can do actually
[03:13:37] Um there is not much we can do actually to to to block those type of hacks.
[03:13:39] to to to block those type of hacks. >> Yeah. And I think people should for all
[03:13:41] >> Yeah. And I think people should for all open source software for all software in
[03:13:44] open source software for all software in general people should pay attention
[03:13:45] general people should pay attention where they download the thing.
[03:13:46] where they download the thing. >> Yes. Because that means that they were
[03:13:48] >> Yes. Because that means that they were not downloading it from our website.
[03:13:50] not downloading it from our website. >> Do the search engines help you?
[03:13:52] >> Do the search engines help you? >> No they don't. Just to clarify because
[03:13:54] >> No they don't. Just to clarify because you can, you know, to prevent threats
[03:13:56] you can, you know, to prevent threats from people manipulating SEO to get up
[03:13:59] from people manipulating SEO to get up there and links.
[03:14:00] there and links. >> Absolutely not. Right. We have a big
[03:14:02] >> Absolutely not. Right. We have a big issue for like more than 10 years is
[03:14:04] issue for like more than 10 years is that there is a fake version of VC in
[03:14:07] that there is a fake version of VC in Germany that was reported for now for 12
[03:14:11] Germany that was reported for now for 12 years and Google basically decides to
[03:14:14] years and Google basically decides to not they know what's in it, but the
[03:14:16] not they know what's in it, but the binary is too big for their virus
[03:14:18] binary is too big for their virus analyzer to analyze it. And so, well, if
[03:14:22] analyzer to analyze it. And so, well, if you're in Germany, you can go to a
[03:14:23] you're in Germany, you can go to a website that is a fake version of VC
[03:14:25] website that is a fake version of VC with a custom installer and it's very
[03:14:28] with a custom installer and it's very popular in Germany because their website
[03:14:30] popular in Germany because their website is in German and and Google mentioned
[03:14:32] is in German and and Google mentioned that before Vidolan and the weirdest
[03:14:35] that before Vidolan and the weirdest thing is that it doesn't do anything on
[03:14:36] thing is that it doesn't do anything on your machine for 3 weeks.
[03:14:38] your machine for 3 weeks. >> Mhm. because that's how they they do the
[03:14:40] >> Mhm. because that's how they they do the detection and after three weeks there is
[03:14:42] detection and after three weeks there is a small program that is a service that
[03:14:44] a small program that is a service that they installed at the same time that
[03:14:45] they installed at the same time that wakes up after 3 weeks and it start
[03:14:47] wakes up after 3 weeks and it start downloading spyware and adear and Google
[03:14:50] downloading spyware and adear and Google knows about it they've decided not to do
[03:14:52] knows about it they've decided not to do anything the guys use dark SEO in
[03:14:54] anything the guys use dark SEO in Germany to to to to do that at some
[03:14:57] Germany to to to to do that at some point um and this is very damaging right
[03:15:01] point um and this is very damaging right because one of the things that they're
[03:15:03] because one of the things that they're downloading is actually something that
[03:15:04] downloading is actually something that is replacing your ads inside your
[03:15:07] is replacing your ads inside your machine It's actually quite surprisingly
[03:15:10] machine It's actually quite surprisingly effective.
[03:15:12] effective. Whoever is doing it with Twitter and X
[03:15:15] Whoever is doing it with Twitter and X with X, I'll get emails about your ex
[03:15:18] with X, I'll get emails about your ex account has been hacked. And however
[03:15:20] account has been hacked. And however they phrase it, it gets me to like at
[03:15:23] they phrase it, it gets me to like at least click on the email not to follow
[03:15:25] least click on the email not to follow the thing. And then you're like, man,
[03:15:28] the thing. And then you're like, man, whatever they're doing with the
[03:15:29] whatever they're doing with the psychology to try to trick you, they're
[03:15:31] psychology to try to trick you, they're quite good. There is a security version
[03:15:33] quite good. There is a security version of VC, right? You received an email
[03:15:35] of VC, right? You received an email saying, "Hey, there is a security
[03:15:37] saying, "Hey, there is a security version update on VC. Think about
[03:15:39] version update on VC. Think about updating right now because it can hack
[03:15:41] updating right now because it can hack your computer." You come, it's a website
[03:15:43] your computer." You come, it's a website that looks decent and and you download
[03:15:46] that looks decent and and you download it's a new version of VC. Great. You
[03:15:47] it's a new version of VC. Great. You don't know a month later you're hacked.
[03:15:49] don't know a month later you're hacked. You have no idea you're part of a
[03:15:50] You have no idea you're part of a botnet.
[03:15:51] botnet. >> Yeah.
[03:15:52] >> Yeah. >> So, make make sure wherever you're
[03:15:54] >> So, make make sure wherever you're downloading stuff, it's legitimate part
[03:15:57] downloading stuff, it's legitimate part of the botnet. Uh speaking of which, so
[03:15:59] of the botnet. Uh speaking of which, so uh you've mentioned that uh VC sandbox
[03:16:03] uh you've mentioned that uh VC sandbox is something is something you're working
[03:16:04] is something is something you're working on and it's actually something quite
[03:16:06] on and it's actually something quite challenging. Why is it important? Why is
[03:16:08] challenging. Why is it important? Why is it hard? So VLC is
[03:16:11] it hard? So VLC is a core with around 500 plugins, right?
[03:16:14] a core with around 500 plugins, right? One of them is FFmpeg, but we have we
[03:16:17] One of them is FFmpeg, but we have we support so many other formats. We
[03:16:18] support so many other formats. We support uh new protocols. We support new
[03:16:22] support uh new protocols. We support new filters. We support weird architectures.
[03:16:24] filters. We support weird architectures. And in this release of VC, you have
[03:16:28] And in this release of VC, you have modules that are going to call your
[03:16:30] modules that are going to call your drivers, right? Uh mostly the hardware
[03:16:32] drivers, right? Uh mostly the hardware decoders which are going to call uh your
[03:16:36] decoders which are going to call uh your Intel, your Nvidia, your IMD driver. Um
[03:16:40] Intel, your Nvidia, your IMD driver. Um and or calling FFmpeg, right? And there
[03:16:43] and or calling FFmpeg, right? And there might be a security issue. There might
[03:16:45] might be a security issue. There might be a security issue in the shader. There
[03:16:47] be a security issue in the shader. There might be a security issue in VC in
[03:16:49] might be a security issue in VC in FFmpeg that is going to basically crash.
[03:16:52] FFmpeg that is going to basically crash. The issue is that you're running VC like
[03:16:56] The issue is that you're running VC like every every other program like Adobe,
[03:16:58] every every other program like Adobe, right? You're running it on your machine
[03:17:00] right? You're running it on your machine and it has access to all your documents,
[03:17:02] and it has access to all your documents, right? though the idea is to be sure
[03:17:05] right? though the idea is to be sure that you do a sandbox so that we can
[03:17:08] that you do a sandbox so that we can protect from ourselves because inside
[03:17:11] protect from ourselves because inside the VC process is running some code that
[03:17:13] the VC process is running some code that is not even ours either it's open source
[03:17:16] is not even ours either it's open source the other projects that we integrate in
[03:17:17] the other projects that we integrate in VC or it's your GPU driver or something
[03:17:21] VC or it's your GPU driver or something that is provided by someone else inside
[03:17:23] that is provided by someone else inside and so when we crash we want to not
[03:17:26] and so when we crash we want to not allow people to do bad things right
[03:17:29] allow people to do bad things right because one of the common way of hacking
[03:17:30] because one of the common way of hacking people is to crash a program very often
[03:17:34] people is to crash a program very often done with a web browser very often done
[03:17:35] done with a web browser very often done with PDF files less often with
[03:17:38] with PDF files less often with multimedia but that could happen and
[03:17:40] multimedia but that could happen and when you crash you launch something on
[03:17:42] when you crash you launch something on the on the machine of the person could
[03:17:44] the on the machine of the person could be a ransomware could be a botnet right
[03:17:46] be a ransomware could be a botnet right so security of desktop application is
[03:17:49] so security of desktop application is important on mobile it's a bit different
[03:17:51] important on mobile it's a bit different because most of the mobile application
[03:17:53] because most of the mobile application are running on inside their own sandbox
[03:17:56] are running on inside their own sandbox but for VC we could run it inside one
[03:17:59] but for VC we could run it inside one sandbox but the problem is that we need
[03:18:01] sandbox but the problem is that we need access to so many things that it's
[03:18:04] access to so many things that it's basically we we would do we would have
[03:18:07] basically we we would do we would have all the permissions right and so if you
[03:18:09] all the permissions right and so if you have a sun sandbox and you put some
[03:18:11] have a sun sandbox and you put some holes everywhere it defeats the purpose
[03:18:14] holes everywhere it defeats the purpose right so what we are trying to do and
[03:18:15] right so what we are trying to do and we're actually doing is splitting VC
[03:18:18] we're actually doing is splitting VC into several processes one is decoding
[03:18:21] into several processes one is decoding one is demoxing one is filters and all
[03:18:25] one is demoxing one is filters and all of them run into their own sandbox so
[03:18:28] of them run into their own sandbox so that the whole VC a part of VC see crash
[03:18:32] that the whole VC a part of VC see crash like Chrome crashes on some on some tab,
[03:18:35] like Chrome crashes on some on some tab, right? It crashes a crash, but it did
[03:18:37] right? It crashes a crash, but it did not crash the whole program. And this is
[03:18:39] not crash the whole program. And this is what we're trying to do. And it's
[03:18:41] what we're trying to do. And it's difficult because it's a sandbox that
[03:18:42] difficult because it's a sandbox that needs to sustain gigabits per second of
[03:18:46] needs to sustain gigabits per second of of mem copies. Now, it's not a website
[03:18:48] of mem copies. Now, it's not a website which is 5 megabytes or 10 megabytes.
[03:18:50] which is 5 megabytes or 10 megabytes. We're talking about hundreds of megabits
[03:18:52] We're talking about hundreds of megabits per second. So this is why it is quite
[03:18:54] per second. So this is why it is quite challenging and this is a a research
[03:18:56] challenging and this is a a research topic that we we are working on in order
[03:18:59] topic that we we are working on in order to have multimedia player that is
[03:19:02] to have multimedia player that is secure. This is all the kind of stuff
[03:19:04] secure. This is all the kind of stuff you have to think about when millions of
[03:19:05] you have to think about when millions of people are using you. You've mentioned
[03:19:07] people are using you. You've mentioned something somewhere where like uh all
[03:19:10] something somewhere where like uh all the different features of VC
[03:19:12] the different features of VC when you have that many people using it
[03:19:15] when you have that many people using it somebody will use every single feature
[03:19:18] somebody will use every single feature and they will tell you about it. best
[03:19:20] and they will tell you about it. best feature in VC is called the puzzle
[03:19:22] feature in VC is called the puzzle filter, right? So, you click the puzzle
[03:19:26] filter, right? So, you click the puzzle filter and it transforms your video into
[03:19:29] filter and it transforms your video into a jigsaw puzzle, right? And you can
[03:19:31] a jigsaw puzzle, right? And you can click
[03:19:31] click >> and move the pieces, right?
[03:19:34] >> and move the pieces, right? >> Um, it's very very useful when you're
[03:19:36] >> Um, it's very very useful when you're watching a French movie, right? You're
[03:19:37] watching a French movie, right? You're bored and because it's like like very
[03:19:40] bored and because it's like like very long things or or a love triangle,
[03:19:42] long things or or a love triangle, right? We've seen that so many times,
[03:19:44] right? We've seen that so many times, right? But but you need to watch it
[03:19:45] right? But but you need to watch it because someone your wife or told you to
[03:19:48] because someone your wife or told you to do that or or your your boyfriend told
[03:19:50] do that or or your your boyfriend told you to do that, right? So you're doing
[03:19:52] you to do that, right? So you're doing that, right? And you can click and move
[03:19:54] that, right? And you can click and move the pieces around.
[03:19:55] the pieces around. >> Yeah.
[03:19:55] >> Yeah. >> It's absolutely useless, right? Like who
[03:19:58] >> It's absolutely useless, right? Like who cares about that? First, it was done by
[03:20:01] cares about that? First, it was done by a mass um teacher in high school in
[03:20:04] a mass um teacher in high school in south of France to teach his students
[03:20:06] south of France to teach his students about basier curves, which is something
[03:20:08] about basier curves, which is something that everyone should know about, right?
[03:20:10] that everyone should know about, right? It's very useful.
[03:20:11] It's very useful. >> But the was clean. So it got in VC. It
[03:20:14] >> But the was clean. So it got in VC. It was merged in 2010. 5 years later, I
[03:20:16] was merged in 2010. 5 years later, I received an email and saying, "Hello,
[03:20:18] received an email and saying, "Hello, JB. I have a problem with VC. The puzzle
[03:20:21] JB. I have a problem with VC. The puzzle is too simple." And I was just like,
[03:20:23] is too simple." And I was just like, "What?" And yes, the puzzle was in the
[03:20:27] "What?" And yes, the puzzle was in the UI maximus by 16 by 16, right? Only 256
[03:20:31] UI maximus by 16 by 16, right? Only 256 pieces. And he says, "I'm sorry, but in
[03:20:33] pieces. And he says, "I'm sorry, but in a movie, I love puzzle. This is too too
[03:20:36] a movie, I love puzzle. This is too too simple." Right? So there is a com of me
[03:20:38] simple." Right? So there is a com of me you can check it online which is JB
[03:20:40] you can check it online which is JB changing that the dimensions are 256 by
[03:20:43] changing that the dimensions are 256 by 256
[03:20:45] 256 but my point is
[03:20:47] but my point is so many use features are used by a few
[03:20:51] so many use features are used by a few people right there is a way to watch VC
[03:20:54] people right there is a way to watch VC movies in common line without any UI
[03:20:57] movies in common line without any UI right it's
[03:20:57] right it's >> I saw that you can do as
[03:21:00] >> I saw that you can do as >> art is it useful very useful imagine
[03:21:03] >> art is it useful very useful imagine you're debugging imagine you're
[03:21:05] you're debugging imagine you're debugging a multiccast network, right?
[03:21:07] debugging a multiccast network, right? You have thousands very complex,
[03:21:09] You have thousands very complex, >> very complex networking stack, right?
[03:21:12] >> very complex networking stack, right? You can SSH to all of the routters and
[03:21:14] You can SSH to all of the routters and put VC on it with no UI and you're going
[03:21:17] put VC on it with no UI and you're going to see whether it's black or it's not
[03:21:18] to see whether it's black or it's not black, right? So you see if or it's all
[03:21:20] black, right? So you see if or it's all green or not all green, right? So you
[03:21:22] green or not all green, right? So you can see
[03:21:23] can see >> right
[03:21:24] >> right >> people don't realize there is so many
[03:21:26] >> people don't realize there is so many things in VC uh that are useful um and
[03:21:31] things in VC uh that are useful um and they are they have users because once
[03:21:33] they are they have users because once you have hundreds of millions of users
[03:21:36] you have hundreds of millions of users you have people who use every feature.
[03:21:40] you have people who use every feature. >> I would love to sort of uh zoom in and
[03:21:43] >> I would love to sort of uh zoom in and and and talk a little bit more about the
[03:21:45] and and talk a little bit more about the distinction between kind of downloading
[03:21:48] distinction between kind of downloading a file and watching it offline versus
[03:21:50] a file and watching it offline versus streaming. So the the complexities the
[03:21:53] streaming. So the the complexities the challenges of streaming. Is there
[03:21:55] challenges of streaming. Is there something we could say about what it
[03:21:56] something we could say about what it takes to
[03:21:58] takes to uh stream files because we've been
[03:22:00] uh stream files because we've been talking about codecs and I think a lot
[03:22:03] talking about codecs and I think a lot of that implies
[03:22:05] of that implies encoding and decoding
[03:22:08] encoding and decoding uh without the having to communicate
[03:22:10] uh without the having to communicate over the network.
[03:22:11] over the network. >> Sure.
[03:22:11] >> Sure. >> Sure.
[03:22:12] >> Sure. >> Uh so can you can you elaborate like
[03:22:14] >> Uh so can you can you elaborate like what's required to do over the network
[03:22:16] what's required to do over the network stuff? Yeah, but it is less complex than
[03:22:18] stuff? Yeah, but it is less complex than it seems compared to everything that
[03:22:20] it seems compared to everything that we've talked about. Um especially
[03:22:22] we've talked about. Um especially because the most complex thing is not
[03:22:25] because the most complex thing is not about streaming in terms of um uh
[03:22:28] about streaming in terms of um uh streaming services, but it was was what
[03:22:31] streaming services, but it was was what was done to actually broadcast through
[03:22:33] was done to actually broadcast through satellites. Um because in in most of the
[03:22:37] satellites. Um because in in most of the modern uh broadcasting services, you can
[03:22:40] modern uh broadcasting services, you can pose and you can go on. When you're
[03:22:43] pose and you can go on. When you're sending live streaming whether it's
[03:22:44] sending live streaming whether it's broadcast or live for streaming services
[03:22:47] broadcast or live for streaming services which are live this is much more
[03:22:49] which are live this is much more difficult because you need to encode in
[03:22:51] difficult because you need to encode in real time you when you go on a satellite
[03:22:53] real time you when you go on a satellite you have a specific size of the link
[03:22:56] you have a specific size of the link right you cannot have a burst of
[03:22:58] right you cannot have a burst of bandwidths even for a second right
[03:23:00] bandwidths even for a second right because you don't have the space for
[03:23:02] because you don't have the space for that in your your total file however
[03:23:05] that in your your total file however there is different types of challenges
[03:23:07] there is different types of challenges which are interesting challenges but I
[03:23:10] which are interesting challenges but I think they are less complex than the one
[03:23:11] think they are less complex than the one we've seen with um late '9s and early
[03:23:14] we've seen with um late '9s and early 2000s about broadcasting and streaming
[03:23:17] 2000s about broadcasting and streaming through satellite
[03:23:17] through satellite >> they're different they're control
[03:23:18] >> they're different they're control systems challenges whereas whereas some
[03:23:20] systems challenges whereas whereas some are more mathematical I think there's a
[03:23:22] are more mathematical I think there's a difference
[03:23:23] difference >> in the streaming world what you have is
[03:23:24] >> in the streaming world what you have is called what we call adaptive streaming
[03:23:26] called what we call adaptive streaming because the difficulty and it's not
[03:23:28] because the difficulty and it's not really a video problem it's mostly a CDN
[03:23:31] really a video problem it's mostly a CDN problem is that you might have too many
[03:23:33] problem is that you might have too many people watching the same thing at the
[03:23:34] people watching the same thing at the same time and it's a congestion of the
[03:23:36] same time and it's a congestion of the network right so your player um has
[03:23:40] network right so your player um has difficulty downloading things fast
[03:23:41] difficulty downloading things fast enough to play them. So what happens is
[03:23:44] enough to play them. So what happens is that locally the player is going to read
[03:23:48] that locally the player is going to read a lower resolution of it. Um but
[03:23:53] a lower resolution of it. Um but >> there are some very clever algorithms to
[03:23:56] >> there are some very clever algorithms to do that but most of it is quite basics
[03:23:59] do that but most of it is quite basics to be honest.
[03:23:59] to be honest. >> Even on the buffering side is pretty
[03:24:01] >> Even on the buffering side is pretty basic.
[03:24:02] basic. >> Yeah. Yeah, you you you start to
[03:24:04] >> Yeah. Yeah, you you you start to download a a segment, what we call a
[03:24:06] download a a segment, what we call a segment, and then you you time, right?
[03:24:08] segment, and then you you time, right? And if you if it takes more than 50% of
[03:24:10] And if you if it takes more than 50% of the time to download a segment, you go
[03:24:12] the time to download a segment, you go down to, right? And the difficulty is
[03:24:15] down to, right? And the difficulty is more about when do you go up in
[03:24:17] more about when do you go up in bandwidth in quality, but this is not
[03:24:20] bandwidth in quality, but this is not very complex to do. When you encode,
[03:24:22] very complex to do. When you encode, you're going to encode seven
[03:24:23] you're going to encode seven resolutions, right? And and you're going
[03:24:25] resolutions, right? And and you're going to give the bit rate. Um, the difficulty
[03:24:28] to give the bit rate. Um, the difficulty is to have your encoder gives the same
[03:24:29] is to have your encoder gives the same bit rate, but it's not as strict as it
[03:24:33] bit rate, but it's not as strict as it used to be. So, uh, probably YouTube has
[03:24:35] used to be. So, uh, probably YouTube has to figure out how the human psychology
[03:24:38] to figure out how the human psychology side of that, like how pissed off do you
[03:24:41] side of that, like how pissed off do you get when it's like very low bit rate and
[03:24:48] get when it's like very low bit rate and uh, how long should it wait before it
[03:24:50] uh, how long should it wait before it increases the bit rate even though the
[03:24:52] increases the bit rate even though the connection is better? because maybe the
[03:24:55] connection is better? because maybe the changes in the bit rate is what like
[03:24:57] changes in the bit rate is what like affects you psychologically.
[03:24:59] affects you psychologically. >> I I think actually the interesting one
[03:25:00] >> I I think actually the interesting one is the audio that you can kind of notice
[03:25:02] is the audio that you can kind of notice when they move from um full fat AAC to
[03:25:06] when they move from um full fat AAC to the um there are compressed versions of
[03:25:09] the um there are compressed versions of AAC that use spectral band replication.
[03:25:10] AAC that use spectral band replication. You can kind of see it goes a bit tiny
[03:25:12] You can kind of see it goes a bit tiny and that up and down is very jarring.
[03:25:14] and that up and down is very jarring. The video side is a lot smoother and
[03:25:17] The video side is a lot smoother and there's less notice. It's really the
[03:25:18] there's less notice. It's really the audio. You can you can definitely you
[03:25:20] audio. You can you can definitely you can definitely feel it from when it's
[03:25:21] can definitely feel it from when it's moved you from a different audio profile
[03:25:23] moved you from a different audio profile to one or the other.
[03:25:24] to one or the other. >> I don't know. We're surprisingly
[03:25:26] >> I don't know. We're surprisingly tolerant at skipping audio glitches. I
[03:25:28] tolerant at skipping audio glitches. I I'm surprised people I know who are not
[03:25:31] I'm surprised people I know who are not video engineers how tolerant they are
[03:25:33] video engineers how tolerant they are how tolerant they are to watching sports
[03:25:36] how tolerant they are to watching sports at 30 fps for example whereas it should
[03:25:38] at 30 fps for example whereas it should really be 60. The world is a lot more
[03:25:40] really be 60. The world is a lot more tolerant to that. But audio people are
[03:25:42] tolerant to that. But audio people are very there it's an immediate feedback
[03:25:44] very there it's an immediate feedback mechanism of oh
[03:25:45] mechanism of oh >> if you hear a glitch you realize it
[03:25:47] >> if you hear a glitch you realize it directly. Yeah, I get to fully realize
[03:25:49] directly. Yeah, I get to fully realize that. I suppose one of the things I'm
[03:25:50] that. I suppose one of the things I'm afraid of when I listen to audio more
[03:25:52] afraid of when I listen to audio more and more that I get to notice every
[03:25:54] and more that I get to notice every single tiny detail and that you can
[03:25:56] single tiny detail and that you can oversess when uh people people in
[03:25:59] oversess when uh people people in general are able to kind of
[03:26:02] general are able to kind of kind of blur their consumption. They
[03:26:05] kind of blur their consumption. They they can they can look past certain
[03:26:07] they can they can look past certain imperfections. But then when you combine
[03:26:10] imperfections. But then when you combine like um an event that is for example a
[03:26:14] like um an event that is for example a sport event that is probably going
[03:26:15] sport event that is probably going through satellite or somewhere else and
[03:26:18] through satellite or somewhere else and goes to a central place for encoding and
[03:26:21] goes to a central place for encoding and then you need to encode this all the
[03:26:22] then you need to encode this all the resolution you in real time you don't
[03:26:25] resolution you in real time you don't have time for QA you need to push that
[03:26:27] have time for QA you need to push that to CDNs you need to add probably DRM uh
[03:26:31] to CDNs you need to add probably DRM uh protection you need to have that over a
[03:26:33] protection you need to have that over a ton of different um devices then yes it
[03:26:37] ton of different um devices then yes it is complex X um but and also like you're
[03:26:42] is complex X um but and also like you're in the web browser or in very much
[03:26:44] in the web browser or in very much different devices that you use for
[03:26:46] different devices that you use for television where you had like a a
[03:26:48] television where you had like a a defined setup box or cable box that that
[03:26:51] defined setup box or cable box that that you know where you control end to end.
[03:26:53] you know where you control end to end. Um so it's a challenge but it's less I
[03:26:56] Um so it's a challenge but it's less I think the networking part uh while you
[03:27:00] think the networking part uh while you agree to have 10 20 seconds of latency I
[03:27:02] agree to have 10 20 seconds of latency I don't think this is very difficult.
[03:27:05] don't think this is very difficult. Speaking of networking and latency. So
[03:27:09] Speaking of networking and latency. So your new effort as we mentioned is Kyber
[03:27:11] your new effort as we mentioned is Kyber which is uh aimed at ultra low latency
[03:27:17] which is uh aimed at ultra low latency as you say every millisecond counts and
[03:27:20] as you say every millisecond counts and uh you're applying that to remote
[03:27:21] uh you're applying that to remote control machines like robots, drones,
[03:27:23] control machines like robots, drones, computers. Can you tell me about it?
[03:27:25] computers. Can you tell me about it? >> Sure. Um if you start from where we used
[03:27:28] >> Sure. Um if you start from where we used to be, right? You used to use ffmpeg to
[03:27:30] to be, right? You used to use ffmpeg to encode files, right? And then we used
[03:27:33] encode files, right? And then we used fmpeg and VC to encode in streaming
[03:27:37] fmpeg and VC to encode in streaming services, right? And then you need to go
[03:27:40] services, right? And then you need to go lower and lower. And the question was
[03:27:42] lower and lower. And the question was where up to where we can can we go?
[03:27:45] where up to where we can can we go? >> Um, and this question is very important
[03:27:47] >> Um, and this question is very important because there are many use cases where
[03:27:50] because there are many use cases where you need to be fast and it's when you
[03:27:52] you need to be fast and it's when you have feedback interaction, right? We are
[03:27:55] have feedback interaction, right? We are not just listening to something, you're
[03:27:57] not just listening to something, you're actually controlling it, right? because
[03:27:59] actually controlling it, right? because and that's the biggest difference that
[03:28:01] and that's the biggest difference that compared to what we've done so far is
[03:28:03] compared to what we've done so far is that I I need video to have a feedback
[03:28:06] that I I need video to have a feedback on something that is happening live
[03:28:08] on something that is happening live whether it's a drone flying whether it's
[03:28:11] whether it's a drone flying whether it's um controlling a humanoid robots from
[03:28:13] um controlling a humanoid robots from distance whether it's controlling a
[03:28:15] distance whether it's controlling a rover whether it's playing a video game
[03:28:18] rover whether it's playing a video game in the cloud gaming because this is um
[03:28:21] in the cloud gaming because this is um what I did on a previous job right I was
[03:28:24] what I did on a previous job right I was CTO of a cloud gaming startup um and
[03:28:27] CTO of a cloud gaming startup um and this is an very interesting topic
[03:28:29] this is an very interesting topic because you push to the limits the
[03:28:31] because you push to the limits the network. You need to be to care not
[03:28:35] network. You need to be to care not about the quality like we've done on
[03:28:37] about the quality like we've done on video and we've talked about with x264.
[03:28:40] video and we've talked about with x264. You care about latency because a
[03:28:43] You care about latency because a milliseconds
[03:28:44] milliseconds is meaningful when you're controlling a
[03:28:48] is meaningful when you're controlling a car, right? For well you we've you've
[03:28:49] car, right? For well you we've you've seen you've used Whimos, right? when
[03:28:51] seen you've used Whimos, right? when whimos don't work and that happens even
[03:28:54] whimos don't work and that happens even if 1% of the time there is someone that
[03:28:56] if 1% of the time there is someone that is basically remote controlling that um
[03:29:00] is basically remote controlling that um and this is exactly the stuff that we're
[03:29:02] and this is exactly the stuff that we're building it's a really an SDK um
[03:29:05] building it's a really an SDK um platform uh to do end to end control of
[03:29:09] platform uh to do end to end control of machines so this comes up quite a lot in
[03:29:12] machines so this comes up quite a lot in a lot of different context in robotics
[03:29:14] a lot of different context in robotics so obviously teley operation teley op is
[03:29:16] so obviously teley operation teley op is becoming more and more important and
[03:29:20] becoming more and more important and including for training
[03:29:23] including for training uh robots uh via machine learning.
[03:29:25] uh robots uh via machine learning. >> Yes. And what we do is a bit different
[03:29:27] >> Yes. And what we do is a bit different from any everyone else is that we take
[03:29:30] from any everyone else is that we take only one socket, one connection which is
[03:29:32] only one socket, one connection which is a quick uh protocol based on UDP. Um
[03:29:36] a quick uh protocol based on UDP. Um which is interesting because it's done
[03:29:38] which is interesting because it's done for low latency. It doesn't have two of
[03:29:40] for low latency. It doesn't have two of the what we call the TCP end ofline
[03:29:41] the what we call the TCP end ofline problem and HTTP end offline problem.
[03:29:44] problem and HTTP end offline problem. It's safer by default. But on the same
[03:29:46] It's safer by default. But on the same wire we send multiple streams like
[03:29:48] wire we send multiple streams like multiple track. We send audio, we send
[03:29:50] multiple track. We send audio, we send in video, but we also send the comments,
[03:29:52] in video, but we also send the comments, right? Uh mouse, keyboard, gamepad and
[03:29:55] right? Uh mouse, keyboard, gamepad and so on. And we do that while maintaining
[03:29:59] so on. And we do that while maintaining coherence, right? Synchronization.
[03:30:01] coherence, right? Synchronization. Because what people don't realize is
[03:30:03] Because what people don't realize is that all the clocks actually drift. And
[03:30:07] that all the clocks actually drift. And when you're controlling a robot, a robot
[03:30:09] when you're controlling a robot, a robot is going to have like two cameras, five
[03:30:11] is going to have like two cameras, five cameras, 10 cameras, a ton of captors,
[03:30:13] cameras, 10 cameras, a ton of captors, GPS, and so on. Um, and if you want to
[03:30:15] GPS, and so on. Um, and if you want to train correctly your robotic AI model,
[03:30:17] train correctly your robotic AI model, you need to have all those that are in
[03:30:20] you need to have all those that are in sync and currents. And what we've done,
[03:30:22] sync and currents. And what we've done, and it's all the stuff that we learn on
[03:30:24] and it's all the stuff that we learn on VC in broadcast in real time and impact
[03:30:27] VC in broadcast in real time and impact TS that Karan know well, is that we
[03:30:30] TS that Karan know well, is that we account for clock drifting and so when I
[03:30:33] account for clock drifting and so when I record a kyber stream a robot, I am sure
[03:30:37] record a kyber stream a robot, I am sure that it's going to be predictive in the
[03:30:39] that it's going to be predictive in the way you play it back. And so when you're
[03:30:42] way you play it back. And so when you're going to do recording and training of
[03:30:44] going to do recording and training of your AI model, you need to be sure that
[03:30:46] your AI model, you need to be sure that every time you retrain based on the
[03:30:48] every time you retrain based on the data, the data is going to stay coherent
[03:30:51] data, the data is going to stay coherent and clocks actually drift. Like the
[03:30:54] and clocks actually drift. Like the existing solution works with one camera.
[03:30:57] existing solution works with one camera. Once you're going to a five or seven,
[03:30:59] Once you're going to a five or seven, it's more complex.
[03:31:00] it's more complex. >> So you want to make sure that the visual
[03:31:04] >> So you want to make sure that the visual snapshot perfectly matches the time it
[03:31:07] snapshot perfectly matches the time it actually happened.
[03:31:08] actually happened. >> Exactly. And also if you're going to
[03:31:10] >> Exactly. And also if you're going to control right I do something on robot I
[03:31:12] control right I do something on robot I need to be sure that it is actually
[03:31:14] need to be sure that it is actually happening at that precise time right and
[03:31:16] happening at that precise time right and so we have on the the server which would
[03:31:19] so we have on the the server which would be a robot a time of like
[03:31:21] be a robot a time of like rettimestamping mechanism accounting for
[03:31:23] rettimestamping mechanism accounting for clock drift for that right so that's one
[03:31:25] clock drift for that right so that's one of the use case um of kyber to to
[03:31:27] of the use case um of kyber to to control robots um I also see like remote
[03:31:31] control robots um I also see like remote drones remote uh whether it's defense or
[03:31:34] drones remote uh whether it's defense or non-defense remote cars remote
[03:31:36] non-defense remote cars remote submarines there is many places in
[03:31:39] submarines there is many places in industry or remote surgery where the
[03:31:43] industry or remote surgery where the expert cannot go everywhere the machine
[03:31:46] expert cannot go everywhere the machine is because either dangerous or it's too
[03:31:48] is because either dangerous or it's too costly. Right? So you you allow people
[03:31:50] costly. Right? So you you allow people to have machines
[03:31:53] to have machines next to you, right? The goal of Kyber is
[03:31:55] next to you, right? The goal of Kyber is to make distance disappear um because
[03:31:57] to make distance disappear um because it's either projection of skills or
[03:32:00] it's either projection of skills or projection of power, right? So imagine
[03:32:02] projection of power, right? So imagine we we all like you've seen the meta
[03:32:05] we we all like you've seen the meta reban and everyone else, right? you need
[03:32:07] reban and everyone else, right? you need to stream there, right? Because you're
[03:32:08] to stream there, right? Because you're not going to run anything over there,
[03:32:10] not going to run anything over there, right? So, you need GPU power, whether
[03:32:12] right? So, you need GPU power, whether it's on a cloud, on a phone to stream
[03:32:13] it's on a cloud, on a phone to stream that. And so, all of these use cases
[03:32:16] that. And so, all of these use cases needs to be not about extremely low
[03:32:18] needs to be not about extremely low latency, but real time latency for
[03:32:20] latency, but real time latency for video. And so, that means you need we're
[03:32:24] video. And so, that means you need we're toing with the encoders so that the
[03:32:26] toing with the encoders so that the encoders encoder frame in for
[03:32:28] encoders encoder frame in for milliseconds. and and and Kiran with his
[03:32:31] milliseconds. and and and Kiran with his company also goes under those type of
[03:32:33] company also goes under those type of license of of latency because you need
[03:32:37] license of of latency because you need to optimize at max the
[03:32:41] to optimize at max the local latency right because it's the
[03:32:43] local latency right because it's the decoder the encoder um and so on um
[03:32:47] decoder the encoder um and so on um because this time is going to be added
[03:32:49] because this time is going to be added to your networking time um so and it's
[03:32:53] to your networking time um so and it's not just about low latency it's also
[03:32:54] not just about low latency it's also about like reliability we do clever
[03:32:57] about like reliability we do clever things like uh forward error correction,
[03:33:00] things like uh forward error correction, right? So forward error correction is
[03:33:03] right? So forward error correction is you overransmit a bit of data, right? A
[03:33:05] you overransmit a bit of data, right? A few percent um and while over transmit
[03:33:08] few percent um and while over transmit you're allowed to lose some packets
[03:33:11] you're allowed to lose some packets because all of that is very difficult
[03:33:13] because all of that is very difficult over a internet network uh where you're
[03:33:16] over a internet network uh where you're going to do things very far away. Um and
[03:33:20] going to do things very far away. Um and if you check that all packets are
[03:33:22] if you check that all packets are delivered, you add a ton of latency. If
[03:33:24] delivered, you add a ton of latency. If you don't want latency, what we do is
[03:33:26] you don't want latency, what we do is that we over transmit some data that you
[03:33:29] that we over transmit some data that you can retrans reconstruct on the client
[03:33:31] can retrans reconstruct on the client side when there is um things that are
[03:33:34] side when there is um things that are broken, right? So um and we um a few a
[03:33:37] broken, right? So um and we um a few a few days weeks ago we were doing the
[03:33:39] few days weeks ago we were doing the demo around Las Vegas for the CES about
[03:33:42] demo around Las Vegas for the CES about we had a a rover that is fully 3D
[03:33:45] we had a a rover that is fully 3D printed. It's very simple. It's a car,
[03:33:47] printed. It's very simple. It's a car, right? It's a small car with a a
[03:33:49] right? It's a small car with a a telescopic arm and it was actually
[03:33:52] telescopic arm and it was actually controlled from France, right? And the
[03:33:53] controlled from France, right? And the the video uh was uh with a webcam in a
[03:33:57] the video uh was uh with a webcam in a very small server, right? A small a
[03:33:59] very small server, right? A small a small PCB was basically running and send
[03:34:02] small PCB was basically running and send that to someone that is on the other
[03:34:04] that to someone that is on the other side of the planet. And so there is so
[03:34:07] side of the planet. And so there is so many use cases. You can also think about
[03:34:09] many use cases. You can also think about having AI who are going to control many
[03:34:12] having AI who are going to control many drones and so on. And the technically we
[03:34:14] drones and so on. And the technically we need to be amazing in video. We need to
[03:34:17] need to be amazing in video. We need to be amazing at networking. We need to
[03:34:19] be amazing at networking. We need to care about any milliseconds in
[03:34:21] care about any milliseconds in networking in encoding time in decoding
[03:34:23] networking in encoding time in decoding time and also you need to integrate very
[03:34:26] time and also you need to integrate very low level.
[03:34:26] low level. >> So sync everything together well but how
[03:34:29] >> So sync everything together well but how like what kind of latency can you get to
[03:34:31] like what kind of latency can you get to like what when you say milliseconds what
[03:34:33] like what when you say milliseconds what what's the goal?
[03:34:34] what's the goal? >> So my goal is 4 milliseconds glassto
[03:34:37] >> So my goal is 4 milliseconds glassto glass latency. Um
[03:34:38] glass latency. Um >> what's glass to glass mean?
[03:34:40] >> what's glass to glass mean? >> So it's easy right? You have a computer
[03:34:42] >> So it's easy right? You have a computer which is running a program right
[03:34:43] which is running a program right probably a video game and this one is
[03:34:45] probably a video game and this one is actually running right? It could be it's
[03:34:48] actually running right? It could be it's an example of a robot, right?
[03:34:50] an example of a robot, right? >> And you have the replicate that is done
[03:34:53] >> And you have the replicate that is done to the network
[03:34:54] to the network >> and and you want
[03:34:56] >> and and you want >> if you take a a 1,00 Hz camera, you can
[03:34:59] >> if you take a a 1,00 Hz camera, you can take a picture and you want that to be
[03:35:01] take a picture and you want that to be at 4 milliseconds. 4 millconds means 240
[03:35:04] at 4 milliseconds. 4 millconds means 240 Hz, right?
[03:35:06] Hz, right? >> Yes. Not
[03:35:07] >> Yes. Not >> um so far we we achieve um 7
[03:35:10] >> um so far we we achieve um 7 milliseconds from a Windows to Windows
[03:35:13] milliseconds from a Windows to Windows or window to Mac. Um and if you look in
[03:35:16] or window to Mac. Um and if you look in the timing most there is around 3.5
[03:35:19] the timing most there is around 3.5 milliseconds inside the Nvidia uh
[03:35:23] milliseconds inside the Nvidia uh hardware encoder and around 2
[03:35:25] hardware encoder and around 2 milliseconds on the Intel decoder right
[03:35:28] milliseconds on the Intel decoder right so like the encoder plus the decoder is
[03:35:30] so like the encoder plus the decoder is already 6 milliseconds right so in order
[03:35:32] already 6 milliseconds right so in order to go down we need either to have some
[03:35:35] to go down we need either to have some other type of codecs
[03:35:37] other type of codecs um or some better encoder that are
[03:35:40] um or some better encoder that are faster uh but 4 milliseconds is would be
[03:35:43] faster uh but 4 milliseconds is would be the growl That's pretty nuts. I love it
[03:35:46] the growl That's pretty nuts. I love it though. I I don't think anyone's ever
[03:35:48] though. I I don't think anyone's ever achieved that, right?
[03:35:49] achieved that, right? >> That's fast.
[03:35:50] >> That's fast. >> You can achieve that with custom
[03:35:52] >> You can achieve that with custom hardware,
[03:35:53] hardware, >> with SDI, with professional hardware,
[03:35:55] >> with SDI, with professional hardware, but I want that to work over the
[03:35:58] but I want that to work over the internet. I want that to work with any
[03:36:01] internet. I want that to work with any robots where you're going to have a
[03:36:02] robots where you're going to have a small Jetson Nano in it or or N150,
[03:36:06] small Jetson Nano in it or or N150, right? I want that because there is
[03:36:08] right? I want that because there is going to be millions of robots or drones
[03:36:11] going to be millions of robots or drones are just rolling robots or flying robots
[03:36:14] are just rolling robots or flying robots or or swimming robots, right? It's just
[03:36:15] or or swimming robots, right? It's just you a machine that you control and in
[03:36:18] you a machine that you control and in order
[03:36:19] order >> either you need to teleoperate them
[03:36:23] >> either you need to teleoperate them or when everything will be fully
[03:36:25] or when everything will be fully autonomous, you need to te observe them,
[03:36:27] autonomous, you need to te observe them, right? You need to check what's
[03:36:29] right? You need to check what's happening.
[03:36:29] happening. >> Yeah. And in my view in the future like
[03:36:32] >> Yeah. And in my view in the future like all those remote cars will be teleobserv
[03:36:35] all those remote cars will be teleobserv observed by an AI model which is just
[03:36:38] observed by an AI model which is just going to say well everything is gone
[03:36:40] going to say well everything is gone good and when it's not good say hey
[03:36:42] good and when it's not good say hey there is a problem and then you have an
[03:36:43] there is a problem and then you have an operator right and this is going to be
[03:36:45] operator right and this is going to be about safety right when you have your
[03:36:47] about safety right when you have your humanoid taking care of your grandma or
[03:36:48] humanoid taking care of your grandma or my grandma I want to be sure that
[03:36:51] my grandma I want to be sure that everything goes well and I'm not in
[03:36:52] everything goes well and I'm not in those type of horrible scenarios where
[03:36:54] those type of horrible scenarios where the robot is dangerous or when I'm
[03:36:57] the robot is dangerous or when I'm driving I I want like the car to to stop
[03:37:00] driving I I want like the car to to stop when it should stop and if needed
[03:37:02] when it should stop and if needed someone takes care of that right and so
[03:37:04] someone takes care of that right and so there is so many cases scenarios about
[03:37:07] there is so many cases scenarios about real time and so the goal of Kyber is to
[03:37:10] real time and so the goal of Kyber is to make real time control of machine
[03:37:14] make real time control of machine distance disappear
[03:37:15] distance disappear >> it's it's incredible and some of the
[03:37:17] >> it's it's incredible and some of the same technology some of the same ideas
[03:37:18] same technology some of the same ideas we've been talking about is is connected
[03:37:21] we've been talking about is is connected to what you're doing
[03:37:22] to what you're doing >> and for for me it's amazingly
[03:37:23] >> and for for me it's amazingly challenging right because I would say
[03:37:25] challenging right because I would say that on video I'm doing okay but
[03:37:28] that on video I'm doing okay but networking I'm I so much more to learn,
[03:37:30] networking I'm I so much more to learn, right? It's uh um about like congestion
[03:37:33] right? It's uh um about like congestion protocols, bit rate adaptation in real
[03:37:35] protocols, bit rate adaptation in real time. Um but it's it's quite funny and
[03:37:38] time. Um but it's it's quite funny and and so I created this project and and we
[03:37:41] and so I created this project and and we we have fundraised in the US of course
[03:37:44] we have fundraised in the US of course but it's open source, right? This is
[03:37:45] but it's open source, right? This is important, right? Like we've not said
[03:37:46] important, right? Like we've not said that, right? But everything on Kyber is
[03:37:49] that, right? But everything on Kyber is open source.
[03:37:50] open source. >> So how do you make money?
[03:37:51] >> So how do you make money? >> It's a dual license commercial and AGPL,
[03:37:54] >> It's a dual license commercial and AGPL, right? You remember what you said about
[03:37:56] right? You remember what you said about about uh licenses? Uh basically if you
[03:37:59] about uh licenses? Uh basically if you want to use Sky Kyber in your product
[03:38:02] want to use Sky Kyber in your product you must have your full product open
[03:38:03] you must have your full product open source. If you want to use this amazing
[03:38:06] source. If you want to use this amazing technology but not open source you pay
[03:38:08] technology but not open source you pay the commercial license right. So the
[03:38:10] the commercial license right. So the small people or the the the hobbyist and
[03:38:13] small people or the the the hobbyist and the the very small guys who want to do
[03:38:15] the the very small guys who want to do that they can use the technology they
[03:38:17] that they can use the technology they build something that is open source and
[03:38:18] build something that is open source and cool. Awesome. And if you're a large
[03:38:20] cool. Awesome. And if you're a large company you're going to have the support
[03:38:22] company you're going to have the support all the IP the right modification and so
[03:38:24] all the IP the right modification and so on. So, um, yeah, it's really cool and
[03:38:28] on. So, um, yeah, it's really cool and and and also I'm building robots and I
[03:38:30] and and also I'm building robots and I love that, right? Like like the the
[03:38:32] love that, right? Like like the the rover we have is 3D printed. Um, we are
[03:38:36] rover we have is 3D printed. Um, we are finishing a demo where it's an actual
[03:38:37] finishing a demo where it's an actual wing, right? Like a type of drone wings
[03:38:39] wing, right? Like a type of drone wings that is also fully 3D printed. Um, we
[03:38:42] that is also fully 3D printed. Um, we are trying to do a a sailboat that is 3D
[03:38:46] are trying to do a a sailboat that is 3D printed. Uh, and and and we'll work on
[03:38:49] printed. Uh, and and and we'll work on some humanoids. Of course, they are not
[03:38:51] some humanoids. Of course, they are not going to be very good robots, right?
[03:38:52] going to be very good robots, right? It's not our job, but we're here for
[03:38:55] It's not our job, but we're here for everyone to make robots. Cool. Ah,
[03:38:57] everyone to make robots. Cool. Ah, you're talking to the right guy. I love
[03:38:59] you're talking to the right guy. I love robots. There's a bunch of them
[03:39:00] robots. There's a bunch of them upstairs. Uh, and tell is going to be
[03:39:03] upstairs. Uh, and tell is going to be really, really important, especially as
[03:39:04] really, really important, especially as the number of robots goes across the
[03:39:06] the number of robots goes across the world. So, 100%. Let's talk about the
[03:39:08] world. So, 100%. Let's talk about the future of um multimedia
[03:39:12] future of um multimedia ffmpeg VC, but some of the codecs. We
[03:39:15] ffmpeg VC, but some of the codecs. We didn't really mention AV2. So, can we
[03:39:18] didn't really mention AV2. So, can we just lay out what is AV2? What is the
[03:39:22] just lay out what is AV2? What is the hope for it? What is H.265, H.266.
[03:39:26] hope for it? What is H.265, H.266. So AV1 is this codec that is done by the
[03:39:29] So AV1 is this codec that is done by the aliens for open media, right? Where
[03:39:30] aliens for open media, right? Where there is uh Google, Netflix, Amazon,
[03:39:34] there is uh Google, Netflix, Amazon, Apple, uh Vidolan where we try to make a
[03:39:36] Apple, uh Vidolan where we try to make a royaltyfree very good codec, right? And
[03:39:39] royaltyfree very good codec, right? And now it's being deployed. But actually
[03:39:41] now it's being deployed. But actually the codec was finished in 2018,
[03:39:44] the codec was finished in 2018, but a codec takes years to be used in
[03:39:49] but a codec takes years to be used in wide scenarios, right? So uh AV2 is the
[03:39:53] wide scenarios, right? So uh AV2 is the next generation of this codec. It's 30%
[03:39:55] next generation of this codec. It's 30% better, right? So if you keep the same
[03:39:59] better, right? So if you keep the same uh quality, you got 30% bandwidth
[03:40:01] uh quality, you got 30% bandwidth reduction compared to AV1.
[03:40:03] reduction compared to AV1. >> What's the connection with the David and
[03:40:05] >> What's the connection with the David and AV2?
[03:40:06] AV2? >> We are going to do a David 2, right?
[03:40:08] >> We are going to do a David 2, right? That I called David because D is two in
[03:40:12] That I called David because D is two in French.
[03:40:13] French. >> Well done.
[03:40:13] >> Well done. >> And and you have to know that David is
[03:40:15] >> And and you have to know that David is an actual uh what we call recursive
[03:40:17] an actual uh what we call recursive acronym, right? because it means D David
[03:40:20] acronym, right? because it means D David is an AV1 decoder, right? So
[03:40:23] is an AV1 decoder, right? So >> Oh, nice. Nice. I didn't even think
[03:40:25] >> Oh, nice. Nice. I didn't even think about that. And uh people should know
[03:40:27] about that. And uh people should know that David is spelled with a one.
[03:40:29] that David is spelled with a one. >> Yes, it's um and so David 2.
[03:40:32] >> Yes, it's um and so David 2. >> It's going to be spelled with a two.
[03:40:33] >> It's going to be spelled with a two. >> It's going to be day a V2D. Sorry, I
[03:40:36] >> It's going to be day a V2D. Sorry, I don't know how you pronounce that. Uh
[03:40:38] don't know how you pronounce that. Uh and again, we did a demo at the CES of
[03:40:42] and again, we did a demo at the CES of VC running the first demo of AV2. So can
[03:40:44] VC running the first demo of AV2. So can you clarify to me the specification
[03:40:47] you clarify to me the specification of AV2
[03:40:50] of AV2 and then the encoding and the decoding?
[03:40:52] and then the encoding and the decoding? >> Sure. So the specification is like the
[03:40:56] >> Sure. So the specification is like the document who explains how the codec is
[03:40:58] document who explains how the codec is supposed to work. Right.
[03:41:00] supposed to work. Right. >> And that's really AV2.
[03:41:02] >> And that's really AV2. >> That is AV2 like H.264.
[03:41:04] >> That is AV2 like H.264. >> Yeah. Right.
[03:41:05] >> Yeah. Right. >> Then you have an encoder. The current
[03:41:07] >> Then you have an encoder. The current encoder is called AVM. And there will
[03:41:10] encoder is called AVM. And there will probably be other encoders probably one
[03:41:12] probably be other encoders probably one called SVT AV2 and those are the
[03:41:14] called SVT AV2 and those are the encoder. The same way x264 is an encoder
[03:41:17] encoder. The same way x264 is an encoder to H.264. The same way that X265 is an
[03:41:22] to H.264. The same way that X265 is an encoder for the H.265 codec. And the
[03:41:26] encoder for the H.265 codec. And the decoders for AV1 is David. The decoder
[03:41:30] decoders for AV1 is David. The decoder for AV2 is David 2. The decoder for
[03:41:35] for AV2 is David 2. The decoder for H.264
[03:41:37] H.264 is FFH.264 264 inside FFmpeg.
[03:41:41] is FFH.264 264 inside FFmpeg. The decoder for HVC is FFHVC inside
[03:41:45] The decoder for HVC is FFHVC inside FFmpeg. Um, and there is a next
[03:41:48] FFmpeg. Um, and there is a next generation codec from the Meg world
[03:41:51] generation codec from the Meg world after H.264, H.265. There is one that is
[03:41:54] after H.264, H.265. There is one that is called H.266
[03:41:56] called H.266 or also known as VVC. So, HEVC is H.265.
[03:42:02] or also known as VVC. So, HEVC is H.265. VVC is H.266.
[03:42:06] VVC is H.266. Why is H.266 266 super sexy so much
[03:42:09] Why is H.266 266 super sexy so much better.
[03:42:10] better. >> So the question often we have is why are
[03:42:12] >> So the question often we have is why are they two names? Because most of the time
[03:42:15] they two names? Because most of the time it is a conjunct work from the ISO world
[03:42:19] it is a conjunct work from the ISO world and the ITU which is the international
[03:42:22] and the ITU which is the international telecommunication union.
[03:42:24] telecommunication union. >> These are these two regulatory bodies.
[03:42:26] >> These are these two regulatory bodies. >> One is a private entity and one is the
[03:42:27] >> One is a private entity and one is the United Nations.
[03:42:28] United Nations. >> Which one is the private?
[03:42:29] >> Which one is the private? >> ISO is private.
[03:42:31] >> ISO is private. >> In theory H.264 264 is MPEG 4 part 10
[03:42:37] >> In theory H.264 264 is MPEG 4 part 10 H.264/AVC.
[03:42:41] >> Mhm.
[03:42:42] >> Mhm. >> And this is the full name.
[03:42:43] >> And this is the full name. >> Nice.
[03:42:44] >> Nice. >> So it's the concatenation of the ISO
[03:42:46] >> So it's the concatenation of the ISO name and the ITU name.
[03:42:48] name and the ITU name. >> Yeah.
[03:42:48] >> Yeah. >> Even though they work together, this is
[03:42:50] >> Even though they work together, this is this is politics historical, you know.
[03:42:53] this is politics historical, you know. >> And for HVC, it's MEGH H.265 HVC.
[03:42:58] >> And for HVC, it's MEGH H.265 HVC. >> Got it.
[03:42:59] >> Got it. >> And there is H.266, 266 which is also
[03:43:02] >> And there is H.266, 266 which is also named VVC.
[03:43:02] named VVC. >> Is there a high level thing to say about
[03:43:04] >> Is there a high level thing to say about the improvement?
[03:43:05] the improvement? >> 30% each generation is a best summary.
[03:43:09] >> 30% each generation is a best summary. >> This is true both for the AV codeex and
[03:43:12] >> This is true both for the AV codeex and the H26456.
[03:43:15] the H26456. >> So the professional who are listening to
[03:43:17] >> So the professional who are listening to us are going to kill us because they say
[03:43:18] us are going to kill us because they say no it's 35% 25% blah blah blah. But
[03:43:22] no it's 35% 25% blah blah blah. But globally you need to know that HVC is
[03:43:24] globally you need to know that HVC is 30% better than than H.264. H26 is 30%
[03:43:28] 30% better than than H.264. H26 is 30% better than H.26. 265 because there is
[03:43:30] better than H.26. 265 because there is so many case and so many scenarios. For
[03:43:33] so many case and so many scenarios. For example, there are cases especially for
[03:43:35] example, there are cases especially for screen recording where the gains are
[03:43:39] screen recording where the gains are humongous because you arrive you have
[03:43:41] humongous because you arrive you have the right tool that is done for that and
[03:43:43] the right tool that is done for that and so for a specific video a new generation
[03:43:45] so for a specific video a new generation is going to give you 70% gain or 80%
[03:43:48] is going to give you 70% gain or 80% gain, right?
[03:43:50] gain, right? >> Um but there used to be a ton more
[03:43:52] >> Um but there used to be a ton more codecs but now the two main codecs for
[03:43:55] codecs but now the two main codecs for transmission are the H.264 H264, H265,
[03:43:57] transmission are the H.264 H264, H265, H266 and the other is AV1, AV2.
[03:44:00] H266 and the other is AV1, AV2. >> And I guess the the major difference
[03:44:02] >> And I guess the the major difference would be the cost of encoding. Yes.
[03:44:05] would be the cost of encoding. Yes. >> Yes. and the royalty of the patents and
[03:44:08] >> Yes. and the royalty of the patents and this is the reasons why you see the AV
[03:44:10] this is the reasons why you see the AV version of Codex is because um they try
[03:44:14] version of Codex is because um they try to be as royalty-free which means no
[03:44:16] to be as royalty-free which means no cost for the patents as much as possible
[03:44:19] cost for the patents as much as possible because what you need to know and we've
[03:44:21] because what you need to know and we've not talked about that so far is that
[03:44:23] not talked about that so far is that multimedia is what we call a patent
[03:44:26] multimedia is what we call a patent minefield there is two places where it's
[03:44:29] minefield there is two places where it's you have the most patents it's
[03:44:30] you have the most patents it's everything related to 3G 4G 5G RF and
[03:44:35] everything related to 3G 4G 5G RF and multimedia um because it's very
[03:44:36] multimedia um because it's very mathematical and you can get great gains
[03:44:39] mathematical and you can get great gains and so on. Um so Google and Meta and
[03:44:43] and so on. Um so Google and Meta and Netflix wanted something where it was
[03:44:45] Netflix wanted something where it was royalty-free. There are people who says
[03:44:47] royalty-free. There are people who says that they have patents outside but they
[03:44:49] that they have patents outside but they are fringe patents right so it's mostly
[03:44:52] are fringe patents right so it's mostly true that is patent free
[03:44:53] true that is patent free >> we should extend patent patent checking
[03:44:56] >> we should extend patent patent checking was done as part of the standardization
[03:44:58] was done as part of the standardization process in AV1 AV2 whereas patents are
[03:45:01] process in AV1 AV2 whereas patents are not even discussed in the meg world
[03:45:06] not even discussed in the meg world patents are off topic completely
[03:45:08] patents are off topic completely >> can you educate me at the patents side
[03:45:10] >> can you educate me at the patents side so usually so meg does a format right
[03:45:13] so usually so meg does a format right and then there is um everyone comes
[03:45:15] and then there is um everyone comes around and say Well, I have all those
[03:45:17] around and say Well, I have all those patents for the format and they do
[03:45:18] patents for the format and they do usually a union called um was called Meg
[03:45:22] usually a union called um was called Meg LA, Meg Licensing Association. Um and
[03:45:26] LA, Meg Licensing Association. Um and you put all the all your patents in and
[03:45:28] you put all the all your patents in and then you ask everyone who's using this
[03:45:30] then you ask everyone who's using this format to pay for it.
[03:45:31] format to pay for it. >> Wait, can you elaborate? What does it
[03:45:33] >> Wait, can you elaborate? What does it mean to have a patent of a codec? Why is
[03:45:34] mean to have a patent of a codec? Why is there many patents?
[03:45:36] there many patents? >> Uh imagine I'm doing something where I'm
[03:45:39] >> Uh imagine I'm doing something where I'm going to def instead of doing blocks
[03:45:41] going to def instead of doing blocks which are square, I'm going to do
[03:45:43] which are square, I'm going to do rectangles, right?
[03:45:44] rectangles, right? >> Oh, so every idea Yes. Somebody patents
[03:45:46] >> Oh, so every idea Yes. Somebody patents it. Yes.
[03:45:49] it. Yes. >> Oh man.
[03:45:51] >> Oh man. >> Yes. Oh man. People and how many lawyers
[03:45:56] >> Yes. Oh man. People and how many lawyers or I mean it pays for a lot of lawyers,
[03:45:57] or I mean it pays for a lot of lawyers, right?
[03:45:58] right? >> The biggest issue is not the following,
[03:45:59] >> The biggest issue is not the following, right? Because at time of H.264 the
[03:46:01] right? Because at time of H.264 the patents were let's call it like sane but
[03:46:05] patents were let's call it like sane but there was there was so much money in
[03:46:08] there was there was so much money in that
[03:46:09] that >> that for HVC
[03:46:11] >> that for HVC >> a lot there were a ton of things that
[03:46:13] >> a lot there were a ton of things that were pushed inside the specification
[03:46:15] were pushed inside the specification which are not useful in 99.9%
[03:46:18] which are not useful in 99.9% of the time but so just one could add a
[03:46:21] of the time but so just one could add a patent on it and so it became that for
[03:46:24] patent on it and so it became that for HVC licensing there was MELA plus
[03:46:28] HVC licensing there was MELA plus another patent pool called HV VC advance
[03:46:31] another patent pool called HV VC advance plus um I think was Nokia was outside of
[03:46:34] plus um I think was Nokia was outside of the patent pool. a few of them are
[03:46:35] the patent pool. a few of them are outside and some other one
[03:46:36] outside and some other one >> and so it was impossible to license
[03:46:38] >> and so it was impossible to license right and I think that several months
[03:46:40] right and I think that several months ago HP decided that they were going to
[03:46:43] ago HP decided that they were going to remove support from HVC in their Windows
[03:46:46] remove support from HVC in their Windows laptops because the the cost was
[03:46:49] laptops because the the cost was increasing of those patents and it
[03:46:52] increasing of those patents and it arrived where a point where and there
[03:46:54] arrived where a point where and there was uncapped pad and so for YouTube or
[03:46:57] was uncapped pad and so for YouTube or Netflix we could talk about hundreds of
[03:47:00] Netflix we could talk about hundreds of millions of dollars of licensing for
[03:47:03] millions of dollars of licensing for patents per year and they said you know
[03:47:05] patents per year and they said you know what as 100 millions per year you know I
[03:47:09] what as 100 millions per year you know I could create my own codec and this is
[03:47:10] could create my own codec and this is what they did and so that's why we have
[03:47:12] what they did and so that's why we have the open media alliance alliance for
[03:47:15] the open media alliance alliance for open media where we are part of that is
[03:47:17] open media where we are part of that is that created AV1 and creates AV2 uh we
[03:47:20] that created AV1 and creates AV2 uh we create also audio codecs but yes so the
[03:47:24] create also audio codecs but yes so the main difference would be that and
[03:47:26] main difference would be that and because you need to work around the
[03:47:27] because you need to work around the patents or go do some things that are
[03:47:30] patents or go do some things that are not patented
[03:47:32] not patented a lot of things are different Right. The
[03:47:34] a lot of things are different Right. The basing things that were done in MEG 2 30
[03:47:37] basing things that were done in MEG 2 30 years ago are caught out of patents. But
[03:47:40] years ago are caught out of patents. But so for example, there is things like a
[03:47:42] so for example, there is things like a golden frame or S frame or or different
[03:47:46] golden frame or S frame or or different type of
[03:47:46] type of >> These are all patented ideas.
[03:47:48] >> These are all patented ideas. >> Yeah. No, it's I can't believe it's not
[03:47:49] >> Yeah. No, it's I can't believe it's not butter. I can't believe it's not a B
[03:47:51] butter. I can't believe it's not a B frame.
[03:47:52] frame. I mean I mean it's kind of what it is in
[03:47:54] I mean I mean it's kind of what it is in some ways. It's like a
[03:47:55] some ways. It's like a >> Oh, so it's a different variant of a B.
[03:47:57] >> Oh, so it's a different variant of a B. >> Yeah. That's to try and sidestep things
[03:48:00] >> Yeah. That's to try and sidestep things like that.
[03:48:00] like that. >> And so you need to have double
[03:48:02] >> And so you need to have double creativity, right? creativity in terms
[03:48:04] creativity, right? creativity in terms of being more efficient but creativity
[03:48:06] of being more efficient but creativity of being sure that you don't infringe
[03:48:09] of being sure that you don't infringe existing patents and so for example VVC
[03:48:12] existing patents and so for example VVC is has all the patents of HVC plus new
[03:48:15] is has all the patents of HVC plus new ones right while AV2 tries to be as
[03:48:19] ones right while AV2 tries to be as royalty-free as possible
[03:48:20] royalty-free as possible >> to what degree does FFmpeg and VC have
[03:48:22] >> to what degree does FFmpeg and VC have to think about this kind of stuff
[03:48:24] to think about this kind of stuff >> we don't and one of the reasons why VC
[03:48:27] >> we don't and one of the reasons why VC was in France is that France rejects
[03:48:29] was in France is that France rejects software patents so most of those
[03:48:32] software patents so most of those patents
[03:48:33] patents are illegal in France because I once
[03:48:37] are illegal in France because I once made the comput the calculus that if I
[03:48:41] made the comput the calculus that if I had to pay all the licency fee for VC I
[03:48:44] had to pay all the licency fee for VC I needed to pay more than €200 per user
[03:48:47] needed to pay more than €200 per user right the same in dollars um but most of
[03:48:49] right the same in dollars um but most of those patents are invalid in Europe
[03:48:52] those patents are invalid in Europe because those are called it's basically
[03:48:54] because those are called it's basically mathematical patents or idea patents and
[03:48:57] mathematical patents or idea patents and they are not valid in Europe uh let me
[03:49:00] they are not valid in Europe uh let me just at a high level just out of
[03:49:02] just at a high level just out of curiosity. So the meme online and the
[03:49:05] curiosity. So the meme online and the interwebs on X and Twitter and so on and
[03:49:08] interwebs on X and Twitter and so on and my own I have friends in Europe uh this
[03:49:13] my own I have friends in Europe uh this the sense is that Europe is not friendly
[03:49:15] the sense is that Europe is not friendly to entrepreneurship. They overregulate
[03:49:18] to entrepreneurship. They overregulate there's too much bureaucracy and so on.
[03:49:22] there's too much bureaucracy and so on. Is is there any anything positive to
[03:49:24] Is is there any anything positive to say? Is there hope for entrepreneurship
[03:49:26] say? Is there hope for entrepreneurship in the future of Europe? Is Europe over
[03:49:30] in the future of Europe? Is Europe over from a tech perspective?
[03:49:32] from a tech perspective? >> Just just look at the two of us, right?
[03:49:33] >> Just just look at the two of us, right? Is it's notable that there's two people
[03:49:35] Is it's notable that there's two people from the European continent on this
[03:49:37] from the European continent on this podcast talking about video. It's fair
[03:49:40] podcast talking about video. It's fair to say the community's weighted heavily.
[03:49:42] to say the community's weighted heavily. >> What you probably don't see yet is that
[03:49:45] >> What you probably don't see yet is that there is a new generation of
[03:49:47] there is a new generation of entrepreneurs in Europe and mostly in
[03:49:51] entrepreneurs in Europe and mostly in France. uh UK has done it since a long
[03:49:53] France. uh UK has done it since a long time because well it's more it's more
[03:49:56] time because well it's more it's more Anglo-Saxon type of uh business look at
[03:49:59] Anglo-Saxon type of uh business look at business but especially like what
[03:50:01] business but especially like what happens in France and of course
[03:50:03] happens in France and of course sometimes a bit overdone with everything
[03:50:05] sometimes a bit overdone with everything called French tech but today most of the
[03:50:08] called French tech but today most of the people who come on the market wants to
[03:50:10] people who come on the market wants to create a startups 15 years ago it wasn't
[03:50:13] create a startups 15 years ago it wasn't the case everyone wanted to work on big
[03:50:16] the case everyone wanted to work on big companies because when you failed in in
[03:50:18] companies because when you failed in in in France for example 20 years ago 15
[03:50:21] in France for example 20 years ago 15 years ago
[03:50:22] years ago and you destroy your company which is
[03:50:24] and you destroy your company which is normal for startup right you you were
[03:50:26] normal for startup right you you were not allowed to create a new company
[03:50:28] not allowed to create a new company right there was a lot of stigma this
[03:50:30] right there was a lot of stigma this stigma is gone um there is so many
[03:50:32] stigma is gone um there is so many things happening on AI in France and so
[03:50:35] things happening on AI in France and so on right so there is sure
[03:50:38] on right so there is sure overregulations I I know that right I'm
[03:50:41] overregulations I I know that right I'm an entrepreneur um but it has some good
[03:50:45] an entrepreneur um but it has some good things also I mean is there some
[03:50:46] things also I mean is there some paralyzing aspects you know if I look at
[03:50:49] paralyzing aspects you know if I look at at the case of somebody u become close
[03:50:52] at the case of somebody u become close with Paul Durov.
[03:50:54] with Paul Durov. You know, he was uh uh blamed directly
[03:50:58] You know, he was uh uh blamed directly by the French government for the kind of
[03:51:01] by the French government for the kind of things his quote platform was hosting. I
[03:51:05] things his quote platform was hosting. I could see the same kind of stuff
[03:51:07] could see the same kind of stuff basically just as an example VC being
[03:51:10] basically just as an example VC being blamed for the kind of videos that
[03:51:12] blamed for the kind of videos that people are watching.
[03:51:13] people are watching. >> We tried, right? like we we like we we
[03:51:16] >> We tried, right? like we we like we we had issues like we like
[03:51:18] had issues like we like >> I mean is that that's the pressure that
[03:51:20] >> I mean is that that's the pressure that people worry about because if you have
[03:51:22] people worry about because if you have to think about that kind of stuff when
[03:51:24] to think about that kind of stuff when you're kind of just obsessed about
[03:51:25] you're kind of just obsessed about >> no you don't think about it and that's
[03:51:28] >> no you don't think about it and that's that's okay right like
[03:51:29] that's okay right like >> but what if they come in when what if
[03:51:31] >> but what if they come in when what if they show up and
[03:51:31] they show up and >> there is no office doesn't have office
[03:51:33] >> there is no office doesn't have office >> I mean this what happened with Pablo
[03:51:35] >> I mean this what happened with Pablo they arrested him right so arrested him
[03:51:37] they arrested him right so arrested him for particular videos or or or
[03:51:39] for particular videos or or or particular content that's being shared
[03:51:41] particular content that's being shared on the platform
[03:51:42] on the platform >> sure I don't have any platform
[03:51:43] >> sure I don't have any platform everything is on the claim side.
[03:51:45] everything is on the claim side. >> Yeah, but they're they can still arrest
[03:51:46] >> Yeah, but they're they can still arrest you on what ground? I'm not sharing
[03:51:49] you on what ground? I'm not sharing anything. I'm not the content doesn't go
[03:51:51] anything. I'm not the content doesn't go through my my stuff
[03:51:52] through my my stuff >> for sure. But it's still lawyer fees.
[03:51:54] >> for sure. But it's still lawyer fees. That's the problem.
[03:51:55] That's the problem. >> Yes, that's correct. Paperwork. So like
[03:51:57] >> Yes, that's correct. Paperwork. So like actually if you had infinite trillions
[03:52:00] actually if you had infinite trillions of dollars
[03:52:02] of dollars uh you would win
[03:52:04] uh you would win uh easily because you're on the right
[03:52:06] uh easily because you're on the right side. But the thing is there is a a
[03:52:09] side. But the thing is there is a a degree to which they suffocate you with
[03:52:11] degree to which they suffocate you with paperwork. That's that's the downside of
[03:52:12] paperwork. That's that's the downside of bureaucracy through paperwork, through
[03:52:14] bureaucracy through paperwork, through process.
[03:52:15] process. >> Yes.
[03:52:15] >> Yes. >> You know, it's the Kafka thing.
[03:52:17] >> You know, it's the Kafka thing. >> You you have to to to realize that one
[03:52:19] >> You you have to to to realize that one of the good things, for example, um in
[03:52:21] of the good things, for example, um in in France or most of Europe is that the
[03:52:24] in France or most of Europe is that the the
[03:52:27] the answering to a court order does not make
[03:52:29] answering to a court order does not make you bankrupt, right? It's not like in
[03:52:31] you bankrupt, right? It's not like in the US where it can actually bankrupt
[03:52:33] the US where it can actually bankrupt you, right? There is the way the law
[03:52:36] you, right? There is the way the law system work is that like I I receive
[03:52:40] system work is that like I I receive lawyers letters every week. Right. And I
[03:52:44] lawyers letters every week. Right. And I can tell you that the cost of lawyer
[03:52:47] can tell you that the cost of lawyer fees for vidoland is less than $10,000
[03:52:50] fees for vidoland is less than $10,000 per year. Right.
[03:52:52] per year. Right. >> Right. So that's not really scary.
[03:52:53] >> Right. So that's not really scary. >> I mean similar with Pavo. Did
[03:52:55] >> I mean similar with Pavo. Did intelligence agencies try to like say
[03:52:57] intelligence agencies try to like say can you put a back door in VLC?
[03:52:59] can you put a back door in VLC? >> Yes. Two of them.
[03:53:01] >> Yes. Two of them. >> What what do you say?
[03:53:02] >> What what do you say? >> No. I was a lot less polite.
[03:53:05] >> No. I was a lot less polite. >> I I see you're uh Yeah. Yeah. You're
[03:53:08] >> I I see you're uh Yeah. Yeah. You're basically saying hell no.
[03:53:09] basically saying hell no. >> Like if we had to compromise our
[03:53:11] >> Like if we had to compromise our software, we would shut it down. This is
[03:53:13] software, we would shut it down. This is clear.
[03:53:14] clear. >> And what's the definition of compromise?
[03:53:16] >> And what's the definition of compromise? Like allowing a government to do a back
[03:53:19] Like allowing a government to do a back door. There is no code that gets into VC
[03:53:22] door. There is no code that gets into VC that we don't control. And the way we
[03:53:24] that we don't control. And the way we compile VC, you would call me completely
[03:53:26] compile VC, you would call me completely paranoid. like we compile on boxes that
[03:53:31] paranoid. like we compile on boxes that are offline where we start by compiling
[03:53:33] are offline where we start by compiling the compiler. We do everything offline
[03:53:37] the compiler. We do everything offline on on on places that have never been
[03:53:39] on on on places that have never been connected to the internet. Uh we the way
[03:53:41] connected to the internet. Uh we the way we do signing there is double signature.
[03:53:44] we do signing there is double signature. Um especially because for example we've
[03:53:47] Um especially because for example we've seen and we believe it's a it's a
[03:53:49] seen and we believe it's a it's a governmental agency that is not from uh
[03:53:52] governmental agency that is not from uh the western world who tried to push a
[03:53:54] the western world who tried to push a fake binary into our own servers. Uh,
[03:53:57] fake binary into our own servers. Uh, and that scared us a lot. And Vidolan is
[03:54:01] and that scared us a lot. And Vidolan is open source. How can you kill it? Like I
[03:54:04] open source. How can you kill it? Like I moved to where, right? I moved to Malta.
[03:54:06] moved to where, right? I moved to Malta. I moved to uh, I don't know, Cayman
[03:54:08] I moved to uh, I don't know, Cayman Islands. And I I change the domain name
[03:54:10] Islands. And I I change the domain name and I I start again, right? Like
[03:54:14] and I I start again, right? Like VC is a tool. Um, it's a tool that is
[03:54:17] VC is a tool. Um, it's a tool that is going to help people doing things. Uh,
[03:54:20] going to help people doing things. Uh, we're not a platform. Um, and for
[03:54:23] we're not a platform. Um, and for patents, well, I'm sorry, but most of
[03:54:26] patents, well, I'm sorry, but most of the patents, like you shouldn't be able
[03:54:28] the patents, like you shouldn't be able to patent math and matrixes, like this
[03:54:30] to patent math and matrixes, like this is wrong. Does VLC ever like censor the
[03:54:35] is wrong. Does VLC ever like censor the kind of videos it can play and not based
[03:54:37] kind of videos it can play and not based on the content of the video?
[03:54:39] on the content of the video? >> No, never. We never do that because like
[03:54:43] >> No, never. We never do that because like a VC is completely offline, doesn't talk
[03:54:46] a VC is completely offline, doesn't talk to any server. So, we don't know
[03:54:47] to any server. So, we don't know anything that you're using the software
[03:54:49] anything that you're using the software for. So again there's no government that
[03:54:51] for. So again there's no government that can say you know like the French
[03:54:54] can say you know like the French government come in and say we don't want
[03:54:57] government come in and say we don't want uh I think anime is destructive to
[03:54:59] uh I think anime is destructive to society we don't want any anime not
[03:55:01] society we don't want any anime not allowed to be
[03:55:02] allowed to be >> no they can they cannot do that and also
[03:55:04] >> no they can they cannot do that and also what they tried is to say hey I want to
[03:55:07] what they tried is to say hey I want to know if that person watch that type of
[03:55:08] know if that person watch that type of video and the answer is like no idea
[03:55:11] video and the answer is like no idea >> so no on that too so for surveillance no
[03:55:14] >> so no on that too so for surveillance no >> no no because the only infrastructure we
[03:55:16] >> no no because the only infrastructure we have is a downloading infrastructure
[03:55:18] have is a downloading infrastructure there is no telemetry in VC, right?
[03:55:20] there is no telemetry in VC, right? >> It would be difficult because of the
[03:55:21] >> It would be difficult because of the international nature. It would be
[03:55:24] international nature. It would be difficult for you to incorporate that
[03:55:25] difficult for you to incorporate that code because there would be someone in
[03:55:27] code because there would be someone in the UK and someone in Germany and
[03:55:28] the UK and someone in Germany and someone in in the US as part of video
[03:55:31] someone in in the US as part of video and who' be able to see that it would be
[03:55:32] and who' be able to see that it would be extremely difficult. The only thing that
[03:55:34] extremely difficult. The only thing that we can do which happen is like we had
[03:55:36] we can do which happen is like we had the issue we had the case with some
[03:55:38] the issue we had the case with some police in the US who said we have a
[03:55:40] police in the US who said we have a murder case right and the file is
[03:55:43] murder case right and the file is destructed or doesn't play in that
[03:55:44] destructed or doesn't play in that version of VC could you help us right we
[03:55:46] version of VC could you help us right we never have access to the video it's like
[03:55:48] never have access to the video it's like a normal support right oh it's really
[03:55:50] a normal support right oh it's really about playing the file
[03:55:52] about playing the file >> yes and like I remember in the middle of
[03:55:54] >> yes and like I remember in the middle of the Afghan war right I received an email
[03:55:57] the Afghan war right I received an email from someone in the army right I don't
[03:55:59] from someone in the army right I don't remember the grades right it was just
[03:56:01] remember the grades right it was just like we have a big issue with the latest
[03:56:03] like we have a big issue with the latest version of VC because it doesn't play
[03:56:07] version of VC because it doesn't play correctly the file on an RTSP server
[03:56:09] correctly the file on an RTSP server that we have where there is all the the
[03:56:12] that we have where there is all the the movies and he says VC is very important
[03:56:14] movies and he says VC is very important for the moral on the troop on the ground
[03:56:16] for the moral on the troop on the ground right because at night I think it might
[03:56:17] right because at night I think it might be boring right so they have a
[03:56:19] be boring right so they have a collection of videos to watch or movies
[03:56:21] collection of videos to watch or movies over there right and and of course I did
[03:56:24] over there right and and of course I did an update and I broke some support of
[03:56:26] an update and I broke some support of RTSP right so I gave them another
[03:56:28] RTSP right so I gave them another version just for them right because it
[03:56:30] version just for them right because it was important and because VC is
[03:56:33] was important and because VC is completely open source. I think it is
[03:56:35] completely open source. I think it is allowed on the US army laptops, right?
[03:56:38] allowed on the US army laptops, right? Because the I guess someone in in the in
[03:56:40] Because the I guess someone in in the in the US military actually looked at it
[03:56:43] the US military actually looked at it and say, well, okay, this is okay,
[03:56:44] and say, well, okay, this is okay, right? And the way we document how we
[03:56:46] right? And the way we document how we process that was okay, right? So the
[03:56:48] process that was okay, right? So the only way we we we work with authorities
[03:56:50] only way we we we work with authorities is to help them uh doing support.
[03:56:53] is to help them uh doing support. >> That's amazing. That's really that's
[03:56:55] >> That's amazing. That's really that's amazing story. Yeah,
[03:56:56] amazing story. Yeah, >> we don't see anything happening on how
[03:56:58] >> we don't see anything happening on how people use VC and this is strong. Do you
[03:57:01] people use VC and this is strong. Do you feel the stress of this? So, first of
[03:57:03] feel the stress of this? So, first of all, millions of people using it. Second
[03:57:05] all, millions of people using it. Second of all, the military using it, maybe
[03:57:09] of all, the military using it, maybe sometimes pressure from governments.
[03:57:11] sometimes pressure from governments. Does that does that that's a that's a
[03:57:14] Does that does that that's a that's a small team, right?
[03:57:16] small team, right? >> Yeah.
[03:57:16] >> Yeah. >> How big is VC like the core contri? How
[03:57:20] >> How big is VC like the core contri? How many?
[03:57:21] many? >> Six. Eight.
[03:57:23] >> Six. Eight. >> But and and and everything legally is
[03:57:25] >> But and and and everything legally is only me. Everything that is legal is
[03:57:27] only me. Everything that is legal is only me.
[03:57:28] only me. >> You're not stressed about this. I used
[03:57:30] >> You're not stressed about this. I used to stress about that a lot.
[03:57:32] to stress about that a lot. >> Yeah.
[03:57:32] >> Yeah. >> But the thing is we're doing what we can
[03:57:35] >> But the thing is we're doing what we can for everyone for the greater good. We we
[03:57:38] for everyone for the greater good. We we work that we make some extremely complex
[03:57:41] work that we make some extremely complex technology easy for everyone. We a tool
[03:57:44] technology easy for everyone. We a tool and every tool is going to be used for
[03:57:46] and every tool is going to be used for great things and for bad things, right?
[03:57:49] great things and for bad things, right? You you cannot blame a tool. Uh I think
[03:57:51] You you cannot blame a tool. Uh I think um and and this is like very important
[03:57:54] um and and this is like very important for us. Um I used to be a lot in a lot
[03:57:58] for us. Um I used to be a lot in a lot of stress. I'm not anymore. Right.
[03:58:00] of stress. I'm not anymore. Right. >> What's the secret to your zen? I mean, o
[03:58:02] >> What's the secret to your zen? I mean, o over and over in the chats I've had with
[03:58:06] over and over in the chats I've had with you uh in the in the conversation today
[03:58:09] you uh in the in the conversation today about every even tense topic, you're
[03:58:11] about every even tense topic, you're very zen. What's the uh what's the
[03:58:12] very zen. What's the uh what's the source of zen?
[03:58:14] source of zen? >> I have um a way of um thinking about
[03:58:18] >> I have um a way of um thinking about what is the worst case scenario always,
[03:58:21] what is the worst case scenario always, right? And the answer is at the end if I
[03:58:25] right? And the answer is at the end if I take like like a chess player, right? In
[03:58:27] take like like a chess player, right? In the end, am I dead?
[03:58:28] the end, am I dead? >> Yes or no?
[03:58:30] >> Yes or no? >> Right? And and I do that non-stop,
[03:58:32] >> Right? And and I do that non-stop, right? And that's also how I do my my
[03:58:35] right? And that's also how I do my my startups, right? Is that I'm here to get
[03:58:38] startups, right? Is that I'm here to get something great? What is the worst case?
[03:58:40] something great? What is the worst case? It got bankrupts. That's life. A company
[03:58:42] It got bankrupts. That's life. A company lives, a company dies. That's okay,
[03:58:44] lives, a company dies. That's okay, right? Like, and so my my moral way is
[03:58:47] right? Like, and so my my moral way is always like, am I dying in the end? Am I
[03:58:49] always like, am I dying in the end? Am I hurting someone? If I'm the answer is
[03:58:52] hurting someone? If I'm the answer is no, then too bad, right? Like, oh, some
[03:58:55] no, then too bad, right? Like, oh, some lawyers are not going to be unhappy.
[03:58:57] lawyers are not going to be unhappy. What are they going to do? Take all the
[03:58:58] What are they going to do? Take all the money on your video land. Wow. They're
[03:59:00] money on your video land. Wow. They're going to have 50 grand. Amazing, right?
[03:59:03] going to have 50 grand. Amazing, right? What are they going to do with that? The
[03:59:04] What are they going to do with that? The cuts, the source code is out there. It's
[03:59:07] cuts, the source code is out there. It's not stoppable. Also, because what we do
[03:59:09] not stoppable. Also, because what we do is good and it's done for everyone.
[03:59:14] is good and it's done for everyone. >> That's beautiful. Uh Karen, you you said
[03:59:16] >> That's beautiful. Uh Karen, you you said that there's an active archiving uh
[03:59:19] that there's an active archiving uh preservation community. I think that's
[03:59:21] preservation community. I think that's super fascinating. You wrote that
[03:59:23] super fascinating. You wrote that they're stretching budget, but they see
[03:59:24] they're stretching budget, but they see the extreme importance of FFmpeg as a
[03:59:27] the extreme importance of FFmpeg as a Rosetta Stone so that multimedia can be
[03:59:30] Rosetta Stone so that multimedia can be played a thousand years from now. I
[03:59:32] played a thousand years from now. I mean, that's a beautiful way to see
[03:59:35] mean, that's a beautiful way to see FFmpeg VC as a tool for preserving
[03:59:39] FFmpeg VC as a tool for preserving visual knowledge. Yes. So, one of the
[03:59:42] visual knowledge. Yes. So, one of the coolest communities um in open source
[03:59:45] coolest communities um in open source multimedia uh mainly led by someone
[03:59:47] multimedia uh mainly led by someone called Dave Rice. I'll give him a shout
[03:59:48] called Dave Rice. I'll give him a shout out I think from um City University of
[03:59:50] out I think from um City University of New York is the archiving community.
[03:59:52] New York is the archiving community. They've done so much stuff that they
[03:59:54] They've done so much stuff that they they value that they value open source
[03:59:56] they value that they value open source one because yes they lack budgets but
[03:59:58] one because yes they lack budgets but two they see the fact that archiving
[04:00:01] two they see the fact that archiving video is important for the world and but
[04:00:05] video is important for the world and but being able to play that is a big
[04:00:07] being able to play that is a big problem. Um, famously in the UK there
[04:00:09] problem. Um, famously in the UK there was a something called the the new
[04:00:10] was a something called the the new doomsday book and they they archived
[04:00:12] doomsday book and they they archived lots of stuff on BBC microcomputers and
[04:00:15] lots of stuff on BBC microcomputers and within 10 to 15 years no one had the
[04:00:17] within 10 to 15 years no one had the right software to play that. I think it
[04:00:19] right software to play that. I think it was 20 years or something like that and
[04:00:21] was 20 years or something like that and someone had to go and reverse engineer
[04:00:22] someone had to go and reverse engineer this and that was 20 years. Imagine that
[04:00:24] this and that was 20 years. Imagine that in a thousand years. I think one of the
[04:00:27] in a thousand years. I think one of the the great things about ffmpeg is it's
[04:00:29] the great things about ffmpeg is it's written in C. C is the closest to
[04:00:31] written in C. C is the closest to mathematics you're probably going to
[04:00:33] mathematics you're probably going to get. The closest to logic is
[04:00:34] get. The closest to logic is >> Do you think in 10,000 years we'll still
[04:00:36] >> Do you think in 10,000 years we'll still have C compilers? Yes, we have languages
[04:00:38] have C compilers? Yes, we have languages that exist that haven't changed too
[04:00:40] that exist that haven't changed too much. We have mathematic mathematical
[04:00:42] much. We have mathematic mathematical notation that exists. Um, it will be
[04:00:44] notation that exists. Um, it will be like Latin. C will be like Latin. It
[04:00:45] like Latin. C will be like Latin. It will be a thing that you learn from the
[04:00:48] will be a thing that you learn from the past, but it will still be usable in
[04:00:49] past, but it will still be usable in certain contexts. Um, so the archiving
[04:00:52] certain contexts. Um, so the archiving community are really great practically.
[04:00:53] community are really great practically. They they again limited funds. They
[04:00:56] They they again limited funds. They funded the development of the FFV1
[04:00:58] funded the development of the FFV1 codec. So that's a lossless codec. So
[04:00:59] codec. So that's a lossless codec. So for so that the archiving community is
[04:01:01] for so that the archiving community is really scared about the act of
[04:01:03] really scared about the act of compression losing things. And this
[04:01:05] compression losing things. And this could this is they have a fair point in
[04:01:06] could this is they have a fair point in this, you know.
[04:01:07] this, you know. >> Yeah.
[04:01:08] >> Yeah. >> If they compress too hard, it could
[04:01:10] >> If they compress too hard, it could change the view of the material. There
[04:01:13] change the view of the material. There could be something slightly different
[04:01:14] could be something slightly different here and there. So they're really
[04:01:16] here and there. So they're really concerned about things need to be not
[04:01:18] concerned about things need to be not just compressed well, but lossless and
[04:01:20] just compressed well, but lossless and be fast and and so they worked with
[04:01:22] be fast and and so they worked with FFmpeg to develop a whole new codec
[04:01:24] FFmpeg to develop a whole new codec designed for fast softwarebased
[04:01:26] designed for fast softwarebased encoding. They're really concerned about
[04:01:28] encoding. They're really concerned about resilience. So they if they're storing
[04:01:31] resilience. So they if they're storing on tapes or other hard discs, I lose
[04:01:34] on tapes or other hard discs, I lose some bits. I need to recover quickly. I
[04:01:36] some bits. I need to recover quickly. I can't lose a whole GOP because I've lost
[04:01:38] can't lose a whole GOP because I've lost a bit. Something like that. So they're a
[04:01:41] a bit. Something like that. So they're a really great bunch of people. They
[04:01:42] really great bunch of people. They funded GPU encoding in FFmpeg um to make
[04:01:45] funded GPU encoding in FFmpeg um to make FFV1 encode faster. And and it's really
[04:01:47] FFV1 encode faster. And and it's really about preserving the world's multimedia
[04:01:51] about preserving the world's multimedia heritage in a way that's usable. And
[04:01:54] heritage in a way that's usable. And there's there's a lot of great teams and
[04:01:55] there's there's a lot of great teams and a lot of archival groups across the
[04:01:58] a lot of archival groups across the world who who've chosen FFmpeg and FFV1
[04:02:01] world who who've chosen FFmpeg and FFV1 as their archiving solution. And they
[04:02:04] as their archiving solution. And they can really provide us also super
[04:02:06] can really provide us also super specialist advice. They can explain ah
[04:02:09] specialist advice. They can explain ah in the 1950s colorimemetry was done like
[04:02:12] in the 1950s colorimemetry was done like this on this certain type of tape and so
[04:02:16] this on this certain type of tape and so there is this special case that you need
[04:02:18] there is this special case that you need to handle and you'll never get this
[04:02:20] to handle and you'll never get this anywhere else. is they they know things
[04:02:22] anywhere else. is they they know things on video like we don't
[04:02:24] on video like we don't >> like every time I talk to with a or the
[04:02:27] >> like every time I talk to with a or the people from the British uh British film
[04:02:29] people from the British uh British film >> film it's just like every time I just
[04:02:32] >> film it's just like every time I just learn something new and I've been doing
[04:02:33] learn something new and I've been doing video for 20 years they they have
[04:02:36] video for 20 years they they have especially on col colorimemetry and and
[04:02:39] especially on col colorimemetry and and colors
[04:02:39] colors >> storage these other things
[04:02:40] >> storage these other things >> I mean they have a deep deep
[04:02:42] >> I mean they have a deep deep appreciation of the content itself of
[04:02:44] appreciation of the content itself of the video itself and like especially
[04:02:47] the video itself and like especially when you're thinking lossless
[04:02:49] when you're thinking lossless >> they're terrified of losing something
[04:02:51] >> they're terrified of losing something essential about the thing and in so
[04:02:54] essential about the thing and in so doing they're deeply understanding the
[04:02:56] doing they're deeply understanding the thing that is to be preserved which you
[04:02:58] thing that is to be preserved which you sometimes might not be thinking about
[04:02:59] sometimes might not be thinking about when you're obsessing about the actual
[04:03:01] when you're obsessing about the actual technology of the encoding and so on
[04:03:03] technology of the encoding and so on >> and when you enter the rabbit hole of
[04:03:06] >> and when you enter the rabbit hole of film scanners right so you take those
[04:03:09] film scanners right so you take those >> those things to make to digital it's
[04:03:12] >> those things to make to digital it's like a huge topic that like would take
[04:03:15] like a huge topic that like would take another 5 hours of podcast just on that
[04:03:18] another 5 hours of podcast just on that topic
[04:03:18] topic >> film and there's a lot of film film that
[04:03:20] >> film and there's a lot of film film that needs to be archived film is degrading.
[04:03:21] needs to be archived film is degrading. It's maybe not stored in the right
[04:03:23] It's maybe not stored in the right environment. The other thing is they can
[04:03:25] environment. The other thing is they can what they also do is because it's open
[04:03:26] what they also do is because it's open source, they give this away their
[04:03:28] source, they give this away their workflows to countries who can't afford
[04:03:30] workflows to countries who can't afford to have archiving institutions where
[04:03:32] to have archiving institutions where archiving is done by volunteers. It's
[04:03:33] archiving is done by volunteers. It's done by other things. They go and teach,
[04:03:36] done by other things. They go and teach, you know, in India they teach children
[04:03:38] you know, in India they teach children to do to do FFmpeg commands. They're
[04:03:40] to do to do FFmpeg commands. They're really great. They're really
[04:03:43] really great. They're really uh they're really the model community,
[04:03:44] uh they're really the model community, the model ethos of what we're trying to
[04:03:46] the model ethos of what we're trying to achieve. that they are
[04:03:48] achieve. that they are such a great bunch of people so
[04:03:50] such a great bunch of people so interested in participating and being
[04:03:52] interested in participating and being part of something much bigger because
[04:03:54] part of something much bigger because they realize the work they're doing in a
[04:03:56] they realize the work they're doing in a thousand years is going to tell a lot
[04:03:59] thousand years is going to tell a lot you know in a thousand years we may be
[04:04:00] you know in a thousand years we may be drowning in AI slop this stuff needs to
[04:04:03] drowning in AI slop this stuff needs to be important and you know archived well
[04:04:05] be important and you know archived well what was what was life like yeah it
[04:04:07] what was what was life like yeah it feels like capturing the 20th century
[04:04:09] feels like capturing the 20th century and the 21st century is essential
[04:04:11] and the 21st century is essential because it feels like a transition point
[04:04:13] because it feels like a transition point where we went from scarcity
[04:04:17] where we went from scarcity of data to slop. Yeah.
[04:04:20] of data to slop. Yeah. >> Oceans of slop.
[04:04:22] >> Oceans of slop. >> And that transition point is good to
[04:04:24] >> And that transition point is good to archive.
[04:04:25] archive. >> People don't realize we are losing today
[04:04:27] >> People don't realize we are losing today a ton of films. There is a ton of things
[04:04:31] a ton of films. There is a ton of things from the 30s, from the 40s and the 50s
[04:04:34] from the 30s, from the 40s and the 50s that were there is no value.
[04:04:37] that were there is no value. >> And tape 70s and ' 80s there's tape and
[04:04:39] >> And tape 70s and ' 80s there's tape and there's not enough tape heads in the
[04:04:41] there's not enough tape heads in the world
[04:04:41] world >> to read all the tapes
[04:04:42] >> to read all the tapes >> to read those. So they have to decide
[04:04:43] >> to read those. So they have to decide what they want to archive and throw away
[04:04:45] what they want to archive and throw away the rest of the tapes.
[04:04:46] the rest of the tapes. >> There's huge moral hazard I guess uh for
[04:04:49] >> There's huge moral hazard I guess uh for one of a better phrase around this topic
[04:04:51] one of a better phrase around this topic because
[04:04:52] because this is a digital record of human
[04:04:54] this is a digital record of human history and they have to make decisions
[04:04:57] history and they have to make decisions but and there's digital stewardship I
[04:04:59] but and there's digital stewardship I suppose for one I made that phrase up.
[04:05:00] suppose for one I made that phrase up. That's not a real phrase. um to make
[04:05:03] That's not a real phrase. um to make sure the world can have this information
[04:05:05] sure the world can have this information in in something that's playable by
[04:05:07] in in something that's playable by everybody, not
[04:05:08] everybody, not >> playable on some device that well
[04:05:11] >> playable on some device that well doesn't exist anymore.
[04:05:13] doesn't exist anymore. >> And then there's like realistically
[04:05:15] >> And then there's like realistically speaking, there's a needle in the
[04:05:16] speaking, there's a needle in the haststack where there's a lot of value
[04:05:18] haststack where there's a lot of value in in archiving all that footage and
[04:05:21] in in archiving all that footage and then over time finding the gems that we
[04:05:24] then over time finding the gems that we don't know are there. Hey, there was
[04:05:26] don't know are there. Hey, there was something in that corner that we just
[04:05:27] something in that corner that we just did, you know, and that would have been
[04:05:29] did, you know, and that would have been compressed away or some little thing.
[04:05:31] compressed away or some little thing. Oh, wow. There's something there. And
[04:05:33] Oh, wow. There's something there. And that's they've made sure that it's
[04:05:34] that's they've made sure that it's lossless. They can prove mathematically
[04:05:37] lossless. They can prove mathematically that it's lossless. They can run
[04:05:40] that it's lossless. They can run different tradeoffs for if there's bit
[04:05:42] different tradeoffs for if there's bit they lose a bit, a single bit flips. I
[04:05:44] they lose a bit, a single bit flips. I can make sure that I only lose a portion
[04:05:46] can make sure that I only lose a portion of a given frame. We can do error rec
[04:05:48] of a given frame. We can do error rec they can do error recovery on previous
[04:05:50] they can do error recovery on previous frames. They can do all sorts of
[04:05:51] frames. They can do all sorts of different things. Do you think uh VC and
[04:05:54] different things. Do you think uh VC and FFmpeg will be here 100 years from now?
[04:05:57] FFmpeg will be here 100 years from now? >> FFmpeg. Yes.
[04:05:58] >> FFmpeg. Yes. >> Yeah. Yes.
[04:05:59] >> Yeah. Yes. >> VC maybe.
[04:06:01] >> VC maybe. >> What's the future of where's where where
[04:06:03] >> What's the future of where's where where is FFmpeg going? Where is VC going? Like
[04:06:06] is FFmpeg going? Where is VC going? Like in the next if you think about like 5
[04:06:08] in the next if you think about like 5 years, 10 years, 20 years.
[04:06:10] years, 10 years, 20 years. >> 5 years, 10 years is easy. The question
[04:06:12] >> 5 years, 10 years is easy. The question is after that, right?
[04:06:15] is after that, right? >> Um the question is do we arrive at
[04:06:17] >> Um the question is do we arrive at something called holograms? Right.
[04:06:19] something called holograms? Right. >> Yeah. So will VC and FFmpeg expand to
[04:06:24] >> Yeah. So will VC and FFmpeg expand to whatever
[04:06:25] whatever >> yes
[04:06:26] >> yes >> multimedia. So multimedia might become
[04:06:29] >> multimedia. So multimedia might become sorry for the pod head expansion of
[04:06:31] sorry for the pod head expansion of topic but you know um if you look at
[04:06:35] topic but you know um if you look at something like neurolink with brain
[04:06:36] something like neurolink with brain computer interfaces it's very possible
[04:06:39] computer interfaces it's very possible that we start to consume what multimedia
[04:06:42] that we start to consume what multimedia means is whatever
[04:06:45] means is whatever codec whatever data that our brain wants
[04:06:47] codec whatever data that our brain wants to consume through the brain computer
[04:06:49] to consume through the brain computer interfaces that's one then virtual
[04:06:51] interfaces that's one then virtual reality of course
[04:06:52] reality of course >> you will have VC for neural link
[04:06:55] >> you will have VC for neural link >> and you you'll have ffmpeg i input
[04:06:57] >> and you you'll have ffmpeg i input format human brain.
[04:06:58] format human brain. >> Yeah,
[04:06:58] >> Yeah, >> there's going to be codecs for the
[04:07:00] >> there's going to be codecs for the brain.
[04:07:01] brain. >> Sure, 100%.
[04:07:03] >> Sure, 100%. >> Of course, I mean compressed neural
[04:07:05] >> Of course, I mean compressed neural information.
[04:07:05] information. >> Today there is like there are new codecs
[04:07:07] >> Today there is like there are new codecs for for example what we call point
[04:07:09] for for example what we call point cloud, right? Or voluometric videos,
[04:07:11] cloud, right? Or voluometric videos, right? There is a ton of research on
[04:07:13] right? There is a ton of research on what we call RGBD, right? The codeex for
[04:07:15] what we call RGBD, right? The codeex for depths that is useful for robotics and
[04:07:17] depths that is useful for robotics and for 3D things. There is a ton of codeex
[04:07:20] for 3D things. There is a ton of codeex for compression of 3D elements,
[04:07:22] for compression of 3D elements, >> compression for astronomy. For example,
[04:07:24] >> compression for astronomy. For example, on VC, we also have already a VR and XR
[04:07:27] on VC, we also have already a VR and XR version of VC. And also on Kyber, right,
[04:07:30] version of VC. And also on Kyber, right, we talk about Kyber. On Kyber, we also
[04:07:31] we talk about Kyber. On Kyber, we also like do streaming of XR content on for
[04:07:34] like do streaming of XR content on for the glasses who cannot have enough power
[04:07:37] the glasses who cannot have enough power or inside the Apple Vision or the Quest.
[04:07:39] or inside the Apple Vision or the Quest. Um, so we already work on uh streaming
[04:07:43] Um, so we already work on uh streaming 3D, XR, interactive, low latency. There
[04:07:47] 3D, XR, interactive, low latency. There is something called Volutric video,
[04:07:48] is something called Volutric video, point cloud videos. So, it's not
[04:07:51] point cloud videos. So, it's not stopping. And yes, at some point it will
[04:07:54] stopping. And yes, at some point it will manage 3D data inside VT and FFMPE,
[04:07:56] manage 3D data inside VT and FFMPE, right? It's obvious.
[04:07:58] right? It's obvious. >> So that's where it is moving like the
[04:07:59] >> So that's where it is moving like the community is open.
[04:08:01] community is open. >> Not everyone in the community sees that.
[04:08:03] >> Not everyone in the community sees that. But like as Kiran and I, we we
[04:08:06] But like as Kiran and I, we we entrepreneurs, we know where it's going.
[04:08:08] entrepreneurs, we know where it's going. We we see that, right?
[04:08:10] We we see that, right? >> So I I suppose that there is a tension
[04:08:12] >> So I I suppose that there is a tension probably inside FFmpeg is like, hey,
[04:08:14] probably inside FFmpeg is like, hey, listen
[04:08:16] listen folks, we're really good at doing video
[04:08:18] folks, we're really good at doing video and audio.
[04:08:20] and audio. So like why expand? Like let's do the
[04:08:23] So like why expand? Like let's do the thing we're really good at doing.
[04:08:25] thing we're really good at doing. >> In order to answer that question, we
[04:08:26] >> In order to answer that question, we need to answer the definition of what is
[04:08:29] need to answer the definition of what is multimedia.
[04:08:30] multimedia. >> Yeah.
[04:08:31] >> Yeah. >> And multimedia
[04:08:33] >> And multimedia is a digital representation of several
[04:08:38] is a digital representation of several streams for the human senses
[04:08:42] streams for the human senses and we will do that. Right? So imagine
[04:08:45] and we will do that. Right? So imagine there is now a way to not have a mic but
[04:08:48] there is now a way to not have a mic but have a odor sensor and a diffuser of
[04:08:51] have a odor sensor and a diffuser of orers.
[04:08:52] orers. >> Mhm.
[04:08:52] >> Mhm. >> It will get into ffmpeg.
[04:08:54] >> It will get into ffmpeg. >> So your demox
[04:08:56] >> So your demox yes of course your demoxer has a new
[04:08:59] yes of course your demoxer has a new track type that is basically right and
[04:09:01] track type that is basically right and you already have
[04:09:02] you already have >> smell touch.
[04:09:03] >> smell touch. >> It's like audio. You have a left and
[04:09:04] >> It's like audio. You have a left and right nose track. You have a left and
[04:09:05] right nose track. You have a left and right audio pair. It's easy.
[04:09:07] right audio pair. It's easy. >> Yes. Of course the stereo smell.
[04:09:10] >> Yes. Of course the stereo smell. >> Stereo smell. Yeah.
[04:09:10] >> Stereo smell. Yeah. >> So in VC for example we already have a
[04:09:12] >> So in VC for example we already have a plugin for optic. It's mostly for what
[04:09:15] plugin for optic. It's mostly for what we call 4D cinemas right you know those
[04:09:17] we call 4D cinemas right you know those ones on hydraulic I don't know how you
[04:09:19] ones on hydraulic I don't know how you say that hydraulic arms hydraulic arms
[04:09:22] say that hydraulic arms hydraulic arms and where everything is moving like you
[04:09:23] and where everything is moving like you have a in theme parks right and there is
[04:09:26] have a in theme parks right and there is a data feed synchronized where which is
[04:09:30] a data feed synchronized where which is basically transporting this information
[04:09:32] basically transporting this information >> is there yet a standard for
[04:09:33] >> is there yet a standard for >> there are many standards right um
[04:09:35] >> there are many standards right um >> this is you're making me so happy and so
[04:09:39] >> this is you're making me so happy and so of course like we have a plugin which is
[04:09:41] of course like we have a plugin which is not in the normal version of that is
[04:09:44] not in the normal version of that is basically transporting those type of
[04:09:46] basically transporting those type of movements which is physical movements
[04:09:47] movements which is physical movements which is haptic movements right it is a
[04:09:50] which is haptic movements right it is a a human sense so it will get in that's
[04:09:54] a human sense so it will get in that's such an exciting future was it I mean
[04:09:57] such an exciting future was it I mean it's a small community of developers how
[04:10:00] it's a small community of developers how do you pull that off like if you're a
[04:10:02] do you pull that off like if you're a contributor at ffmpeg or VC it feels
[04:10:05] contributor at ffmpeg or VC it feels stressful like it just looking on
[04:10:08] stressful like it just looking on Twitter it's like it's it's a huge
[04:10:11] Twitter it's like it's it's a huge amount of work to make it work on all
[04:10:14] amount of work to make it work on all these different operating systems an
[04:10:16] these different operating systems an incredible effort. No, see it in the
[04:10:18] incredible effort. No, see it in the other direction. We are not the
[04:10:21] other direction. We are not the contributors, we are the maintainers,
[04:10:24] contributors, we are the maintainers, right? So we maintain for everyone.
[04:10:27] right? So we maintain for everyone. Meaning that for example, every year
[04:10:30] Meaning that for example, every year there is around 150 people who
[04:10:33] there is around 150 people who contribute to VC and maybe 300 on
[04:10:35] contribute to VC and maybe 300 on FFmpeg. Right? Our goal as a small team
[04:10:38] FFmpeg. Right? Our goal as a small team is to get all the contribution in. So if
[04:10:42] is to get all the contribution in. So if there is more usage there will be more
[04:10:44] there is more usage there will be more contributions and those people will do
[04:10:46] contributions and those people will do the right module the new format and so
[04:10:49] the right module the new format and so on. We care about the architecture of
[04:10:51] on. We care about the architecture of VC, the architecture of FFmpeg. Right
[04:10:54] VC, the architecture of FFmpeg. Right now, we're doing things in FF in VC
[04:10:57] now, we're doing things in FF in VC which is a special audio. Right? We did
[04:10:58] which is a special audio. Right? We did the demo not not long ago. There was
[04:11:02] the demo not not long ago. There was changes needed on the architecture and
[04:11:04] changes needed on the architecture and we did the first special audio module
[04:11:07] we did the first special audio module when it's going to add the second one.
[04:11:08] when it's going to add the second one. It's going to be easy or the third one
[04:11:09] It's going to be easy or the third one is going to be easy. Right? Our goal and
[04:11:12] is going to be easy. Right? Our goal and going to be the same for orders or
[04:11:14] going to be the same for orders or optic. Right? we need to work with the
[04:11:16] optic. Right? we need to work with the architecture. So that's modules can be
[04:11:18] architecture. So that's modules can be added to add future capabilities. So
[04:11:21] added to add future capabilities. So yes, we are going we have multimedia
[04:11:24] yes, we are going we have multimedia framework. So that's not just audio and
[04:11:25] framework. So that's not just audio and video. It's everything that is timed and
[04:11:30] video. It's everything that is timed and represent
[04:11:32] represent something that you can sense. And if
[04:11:34] something that you can sense. And if it's brain waves, it's going to be brain
[04:11:36] it's brain waves, it's going to be brain waves.
[04:11:36] waves. >> I think that's inevitable. Sorry. I I
[04:11:38] >> I think that's inevitable. Sorry. I I love this on so many fronts because so
[04:11:41] love this on so many fronts because so FFmpeg and VC are pushing
[04:11:45] FFmpeg and VC are pushing companies and pushing the world to
[04:11:47] companies and pushing the world to standardize. So for example to
[04:11:48] standardize. So for example to standardize brain waves so standardize
[04:11:52] standardize brain waves so standardize >> it's it would push like I hope Neurolink
[04:11:54] >> it's it would push like I hope Neurolink comes up with a standard for for
[04:11:57] comes up with a standard for for multimedia
[04:12:00] multimedia via brain computer interfaces or for
[04:12:03] via brain computer interfaces or for robots with haptic. By experience, what
[04:12:06] robots with haptic. By experience, what happens is always the same, right? You
[04:12:08] happens is always the same, right? You start, it's a new topic, there is like
[04:12:10] start, it's a new topic, there is like five different standards because
[04:12:12] five different standards because everyone starts to do this, the hype
[04:12:14] everyone starts to do this, the hype goes down because every time the hype
[04:12:16] goes down because every time the hype goes down, then people start say, "Well,
[04:12:18] goes down, then people start say, "Well, you know what? We need to do a standard.
[04:12:20] you know what? We need to do a standard. People because two or three companies,
[04:12:22] People because two or three companies, usually not the leader, but the two or
[04:12:24] usually not the leader, but the two or three followers do a standard
[04:12:27] three followers do a standard and then we implement the standard and
[04:12:29] and then we implement the standard and and then it's the end of the curve. it
[04:12:31] and then it's the end of the curve. it starts doing more pepper
[04:12:32] starts doing more pepper >> and then the leaders kind of pressured
[04:12:34] >> and then the leaders kind of pressured into it because it's better to do a
[04:12:36] into it because it's better to do a standard
[04:12:37] standard >> example 3D audio right yeah six or seven
[04:12:39] >> example 3D audio right yeah six or seven years ago it was everything about 3D you
[04:12:42] years ago it was everything about 3D you had the cardboard on Android you had two
[04:12:44] had the cardboard on Android you had two two audio formats they're all dead right
[04:12:47] two audio formats they're all dead right and now it's coming back with actual use
[04:12:49] and now it's coming back with actual use cases and we learn from the mistakes of
[04:12:52] cases and we learn from the mistakes of the past standard so it will be the same
[04:12:55] the past standard so it will be the same everywhere and not try to avoid closed I
[04:12:59] everywhere and not try to avoid closed I saw somewhere you you didn't have too
[04:13:01] saw somewhere you you didn't have too many nice things to say about Dolby.
[04:13:03] many nice things to say about Dolby. >> No, I don't.
[04:13:04] >> No, I don't. >> What is uh Can you educate uh me on why
[04:13:08] >> What is uh Can you educate uh me on why where they went? What what what did they
[04:13:10] where they went? What what what did they do bad that made you mad?
[04:13:13] do bad that made you mad? >> It used to be an amazing company doing
[04:13:17] >> It used to be an amazing company doing tons of great things with amazing
[04:13:19] tons of great things with amazing engineers. They defined what sound was
[04:13:22] engineers. They defined what sound was >> and now it's mostly lawyers and
[04:13:26] >> and now it's mostly lawyers and licensing things.
[04:13:27] licensing things. >> Oh, so they're Yeah. They're they're
[04:13:28] >> Oh, so they're Yeah. They're they're closing stuff off. They're trying.
[04:13:30] closing stuff off. They're trying. >> It's just like they don't innovate as
[04:13:32] >> It's just like they don't innovate as much as they did and so on. It's a bit
[04:13:34] much as they did and so on. It's a bit like I'm sorry to say right like HP,
[04:13:36] like I'm sorry to say right like HP, right?
[04:13:38] right? >> True.
[04:13:40] >> True. >> Oh, since we talked about Twitter a
[04:13:42] >> Oh, since we talked about Twitter a bunch in a bunch of different contexts.
[04:13:43] bunch in a bunch of different contexts. Do you have a do you have a favorite you
[04:13:46] Do you have a do you have a favorite you have and least favorite most
[04:13:48] have and least favorite most embarrassing tweet on either video landf
[04:13:52] embarrassing tweet on either video landf Twitters?
[04:13:53] Twitters? >> The two my two favorites are talk is
[04:13:55] >> The two my two favorites are talk is cheap send patches. I think that that
[04:13:57] cheap send patches. I think that that embodies a lot of the
[04:13:59] embodies a lot of the >> stuff doesn't get as as we've talked
[04:14:00] >> stuff doesn't get as as we've talked about stuff doesn't get built unless
[04:14:02] about stuff doesn't get built unless someone does it. It doesn't just appear
[04:14:04] someone does it. It doesn't just appear from the ether.
[04:14:05] from the ether. >> Uh the other one that I like is um
[04:14:07] >> Uh the other one that I like is um ffmpeg nothing is beyond our reach. I
[04:14:10] ffmpeg nothing is beyond our reach. I think that comes from a US military
[04:14:12] think that comes from a US military satellite patch where I think they they
[04:14:14] satellite patch where I think they they invented some kind of um
[04:14:16] invented some kind of um monitoring system that could see the
[04:14:17] monitoring system that could see the whole world and this was released.
[04:14:19] whole world and this was released. >> Wasn't there something where FFmpeg was
[04:14:20] >> Wasn't there something where FFmpeg was running on a rover on Mars also?
[04:14:22] running on a rover on Mars also? >> Yes. So ffmpeg is used by the Mars rover
[04:14:24] >> Yes. So ffmpeg is used by the Mars rover uh the Mars 2020 rover to compress uh
[04:14:28] uh the Mars 2020 rover to compress uh pictures. Um they really wanted um they
[04:14:30] pictures. Um they really wanted um they wrote a paper about it and they really
[04:14:32] wrote a paper about it and they really wanted to use as much commercial
[04:14:33] wanted to use as much commercial offtheshelf technology as possible.
[04:14:34] offtheshelf technology as possible. Feneg runs on Mars. So we are
[04:14:37] Feneg runs on Mars. So we are >> we are a multilanetary open source
[04:14:39] >> we are a multilanetary open source library.
[04:14:39] library. >> Nice.
[04:14:40] >> Nice. >> Um very often we've seen um uh tweets
[04:14:45] >> Um very often we've seen um uh tweets for people using um VC in weird places.
[04:14:49] for people using um VC in weird places. Uh a lot of the people doing Formula 1
[04:14:51] Uh a lot of the people doing Formula 1 ones are in all the paddocks. They use
[04:14:53] ones are in all the paddocks. They use VC to play the live feed. Uh we've seen
[04:14:56] VC to play the live feed. Uh we've seen uh the European Space Agency. We've seen
[04:14:58] uh the European Space Agency. We've seen SpaceX like monitoring the the launches
[04:15:01] SpaceX like monitoring the the launches with with VC and like it's like fills
[04:15:04] with with VC and like it's like fills you with joy, right? So
[04:15:05] you with joy, right? So >> I've seen uh particle accelerator.
[04:15:07] >> I've seen uh particle accelerator. >> Oh yeah. Yeah. We had one of the most
[04:15:10] >> Oh yeah. Yeah. We had one of the most amazing thing that I went for was to go
[04:15:13] amazing thing that I went for was to go to the sand at the LHC because they were
[04:15:16] to the sand at the LHC because they were using VC to monitor all the captors on
[04:15:20] using VC to monitor all the captors on the ring because the ring is 27
[04:15:22] the ring because the ring is 27 kilometers and so they had some analog
[04:15:26] kilometers and so they had some analog cameras
[04:15:27] cameras >> and they were using some of the capture
[04:15:29] >> and they were using some of the capture cards to go to analog to VC so VC could
[04:15:32] cards to go to analog to VC so VC could stream on their multiccast network for
[04:15:34] stream on their multiccast network for the whole CERN to access to that. Um and
[04:15:36] the whole CERN to access to that. Um and like I visited that in 2010 with Lauron
[04:15:41] like I visited that in 2010 with Lauron and and and
[04:15:43] and and and like we fixed their issue in an hour or
[04:15:45] like we fixed their issue in an hour or something like that, right? Because was
[04:15:47] something like that, right? Because was some parameters uh maybe not well
[04:15:50] some parameters uh maybe not well documented at that time. And he said,
[04:15:52] documented at that time. And he said, "Okay, for the whole day, what do you
[04:15:53] "Okay, for the whole day, what do you want to do?" And we visited everything
[04:15:55] want to do?" And we visited everything like things were with antimatter and and
[04:15:58] like things were with antimatter and and and and colliders and so on. And that
[04:16:00] and and colliders and so on. And that was like one of the most
[04:16:02] was like one of the most >> amazing day of my my physic.
[04:16:06] >> amazing day of my my physic. Yeah, it's used like everywhere. Uh any
[04:16:09] Yeah, it's used like everywhere. Uh any any tweets? Uh Karen, you regret
[04:16:13] any tweets? Uh Karen, you regret >> tweets of regret
[04:16:14] >> tweets of regret >> or is it like that? Well, how's the
[04:16:16] >> or is it like that? Well, how's the French song go? Regret nothing.
[04:16:19] French song go? Regret nothing. >> Yes, that's very important for me,
[04:16:21] >> Yes, that's very important for me, right? Don't regret anything. No, it's
[04:16:24] right? Don't regret anything. No, it's because regrets are attacks on your
[04:16:27] because regrets are attacks on your mind, right? So, learn from your
[04:16:29] mind, right? So, learn from your mistakes, but don't regret
[04:16:31] mistakes, but don't regret >> because you've done it. So except if you
[04:16:34] >> because you've done it. So except if you have a time machine to go back in time,
[04:16:37] have a time machine to go back in time, don't regret, right? It's going to just
[04:16:39] don't regret, right? It's going to just tax your brain. Learn from your mistake.
[04:16:41] tax your brain. Learn from your mistake. Sure, don't regret.
[04:16:43] Sure, don't regret. It's like it reminds me, it's beautiful.
[04:16:45] It's like it reminds me, it's beautiful. It's a tax in your brain. Reminds me of
[04:16:47] It's a tax in your brain. Reminds me of the Johnny Depp quote I saw where he was
[04:16:50] the Johnny Depp quote I saw where he was saying hate, you know, I don't hate.
[04:16:52] saying hate, you know, I don't hate. That's hate is a very expensive emotion.
[04:16:56] That's hate is a very expensive emotion. >> Are you comparing me to Johnny Depp?
[04:16:58] >> Are you comparing me to Johnny Depp? Because that would be your first one.
[04:17:00] Because that would be your first one. Well, gentlemen, like I said, I'm
[04:17:03] Well, gentlemen, like I said, I'm internally grateful for the the software
[04:17:06] internally grateful for the the software that the two of you and the bigger
[04:17:08] that the two of you and the bigger community have been part of building
[04:17:09] community have been part of building with FFmpeg and VC and everything else.
[04:17:12] with FFmpeg and VC and everything else. I'm eternally grateful for the spicy
[04:17:15] I'm eternally grateful for the spicy tweets. Never stop. And I'm grateful
[04:17:18] tweets. Never stop. And I'm grateful that you would uh talk with me today and
[04:17:21] that you would uh talk with me today and give me this sexy hat. I feel like a
[04:17:25] give me this sexy hat. I feel like a wizard. I feel special. Um, and I feel
[04:17:28] wizard. I feel special. Um, and I feel special to get a chance to talk and
[04:17:30] special to get a chance to talk and celebrate the piece of software that
[04:17:32] celebrate the piece of software that brought me so much joy over the years.
[04:17:33] brought me so much joy over the years. So, thank you for everything and thank
[04:17:35] So, thank you for everything and thank you for talking today.
[04:17:36] you for talking today. >> Thank you for having me.
[04:17:36] >> Thank you for having me. >> Thank you so much.
[04:17:38] >> Thank you so much. >> Thanks for listening to this
[04:17:39] >> Thanks for listening to this conversation with John Baptist Kemp and
[04:17:41] conversation with John Baptist Kemp and Kieran Kuna. To support this podcast,
[04:17:44] Kieran Kuna. To support this podcast, please check out our sponsors in the
[04:17:45] please check out our sponsors in the description where you can also find
[04:17:47] description where you can also find links to contact me, ask questions, give
[04:17:50] links to contact me, ask questions, give feedback and so on. And now, let me
[04:17:52] feedback and so on. And now, let me leave you with some words from the
[04:17:54] leave you with some words from the legendary Lionus Tvalds. Most good
[04:17:57] legendary Lionus Tvalds. Most good programmers do programming not because
[04:18:00] programmers do programming not because they expect to get paid or get
[04:18:02] they expect to get paid or get edgulation by the public, but because it
[04:18:06] edgulation by the public, but because it is fun to program. Thank you for
[04:18:08] is fun to program. Thank you for listening and hope to see you next time.