# But what is a neural network? | Deep learning chapter 1

https://www.youtube.com/watch?v=aircAruvnKk
Translation: zh-CN

[00:04] This is a 3.
  这是一个3。

[00:06] It's sloppily written and rendered at an extremely low resolution of 28x28 pixels, but your brain has no trouble recognizing it as a 3.
  它写得很潦草，并且以28x28像素的极低分辨率呈现，但你的大脑可以毫不费力地将其识别为3。

[00:14] And I want you to take a moment to appreciate how crazy it is that brains can do this so effortlessly.
  我想让你花点时间去体会一下，大脑能如此轻松地做到这一点是多么不可思议。

[00:19] I mean, this, this and this are also recognizable as 3s, even though the specific values of each pixel is very different from one image to the next.
  我的意思是，这个、这个还有这个也都可以被识别为3，尽管每个像素的具体数值在不同图像之间有很大差异。

[00:28] The particular light-sensitive cells in your eye that are firing when you see this 3 are very different from the ones firing when you see this 3.
  当你看到这个3时，你眼睛里活跃的特定感光细胞，与你看到那个3时活跃的细胞非常不同。

[00:37] But something in that crazy-smart visual cortex of yours resolves these as representing the same idea, while at the same time recognizing other images as their own distinct ideas.
  但你那极其聪明的视觉皮层中的某种机制，将这些图像解析为代表同一个概念，同时又能将其他图像识别为它们各自独特的概念。

[00:49] But if I told you, hey, sit down and write for me a program that takes in a grid of 28x28 pixels like this and outputs a single number between 0 and 10, telling you what it thinks the digit is, well the task goes from comically trivial to
  但如果我告诉你，嘿，坐下来给我写一个程序，输入像这样28x28像素的网格，并输出一个0到10之间的数字，告诉你它认为这是什么数字，那么这项任务就从极其简单变得……

[01:04] Dauntingly difficult.
  极其困难。

[01:07] Unless you've been living under a rock, I think I hardly need to motivate the relevance and importance of machine learning and neural networks to the present and to the future.
  除非你与世隔绝，否则我认为我无需多言，也能让你明白机器学习和神经网络对现在及未来的相关性和重要性。

[01:15] But what I want to do here is show you what a neural network actually is, assuming no background, and to help visualize what it's doing, not as a buzzword but as a piece of math.
  但我在这里想做的是，在假设你没有任何背景知识的情况下，向你展示神经网络到底是什么，并帮助你直观地理解它在做什么，不是将其视为一个流行词，而是将其视为一种数学工具。

[01:25] My hope is that you come away feeling like the structure itself is motivated, and to feel like you know what it means when you read, or you hear about a neural network quote-unquote learning.
  我希望你看完后会觉得这种结构本身是有理可循的，并且当你读到或听到关于神经网络“学习”的内容时，能明白其中的含义。

[01:35] This video is just going to be devoted to the structure component of that, and the following one is going to tackle learning.
  本视频将专门探讨其结构部分，而下一期视频将讨论学习部分。

[01:41] What we're going to do is put together a neural network that can learn to recognize handwritten digits.
  我们将要构建一个能够学习识别手写数字的神经网络。

[01:49] This is a somewhat classic example for introducing the topic, and I'm happy to stick with the status quo here, because at the end of the two videos I want to point you to a couple good resources where you can learn more, and where you can download the code that does this and play with it on your own computer.
  这是一个介绍该主题的经典案例，我很乐意沿用这个惯例，因为在两期视频结束时，我想为你推荐一些好的资源，让你深入学习，并下载实现此功能的代码，在自己的电脑上进行尝试。

[02:05] There are many many variants of neural networks.
  神经网络有很多很多变体。

[02:07] And in recent years there's been sort of a boom in research towards these variants.
  近年来，针对这些变体的研究出现了一种热潮。

[02:12] But in these two introductory videos you and I are just going to look at the simplest plain vanilla form with no added frills.
  但在接下来的两段入门视频中，你和我将只关注最简单、最基础、没有任何额外修饰的形式。

[02:19] This is kind of a necessary prerequisite for understanding any of the more powerful modern variants, and trust me it still has plenty of complexity for us to wrap our minds around.
  这是理解任何更强大的现代变体的必要前提，相信我，它仍然有足够的复杂性供我们去理解。

[02:29] But even in this simplest form it can learn to recognize handwritten digits, which is a pretty cool thing for a computer to be able to do.
  但即使在这种最简单的形式下，它也能学会识别手写数字，这对计算机来说是一件非常酷的事情。

[02:37] And at the same time you'll see how it does fall short of a couple hopes that we might have for it.
  同时，你也会看到它在某些方面是如何未能达到我们对它的期望的。

[02:43] As the name suggests neural networks are inspired by the brain, but let's break that down.
  顾名思义，神经网络的灵感来源于大脑，但让我们来详细剖析一下。

[02:48] What are the neurons, and in what sense are they linked together?
  什么是神经元？它们又是以何种方式连接在一起的？

[02:52] Right now when I say neuron all I want you to think about is a thing that holds a number, specifically a number between 0 and 1.
  现在，当我提到神经元时，我只想让你把它想象成一个存储数字的东西，具体来说是一个 0 到 1 之间的数字。

[03:00] It's really not more than that.
  它确实仅此而已。

[03:03] For example the network starts with a bunch of neurons corresponding to
  例如，网络从一堆神经元开始，这些神经元对应于

[03:08] Each of the 28x28 pixels of the input image, which is 784 neurons in total.
  输入图像的28x28个像素中的每一个，总共有784个神经元。

[03:14] Each one of these holds a number that represents the grayscale value of the corresponding pixel, ranging from 0 for black pixels up to 1 for white pixels.
  其中每一个都包含一个数字，代表对应像素的灰度值，范围从黑色像素的0到白色像素的1。

[03:25] This number inside the neuron is called its activation.
  神经元内部的这个数字被称为它的激活值。

[03:28] And the image you might have in mind here is that each neuron is lit up when its activation is a high number.
  你脑海中可能会浮现出这样的画面：当每个神经元的激活值较高时，它就会被点亮。

[03:36] So all of these 784 neurons make up the first layer of our network.
  因此，所有这784个神经元构成了我们网络的第一层。

[03:46] Now jumping over to the last layer, this has 10 neurons, each representing one of the digits.
  现在跳到最后一层，它有10个神经元，每一个代表一个数字。

[03:52] The activation in these neurons, again some number that's between 0 and 1, represents how much the system thinks that a given image corresponds with a given digit.
  这些神经元中的激活值（同样是0到1之间的一个数字）代表了系统认为给定图像与特定数字的匹配程度。

[04:03] There's also a couple layers in between called the hidden layers, which for the time being should just be a giant question mark for.
  中间还有几层被称为隐藏层，目前它们对你来说应该只是一个巨大的问号。

[04:09] How on earth this process of recognizing digits is going to be handled?
  这个识别数字的过程到底将如何处理？

[04:14] In this network I chose two hidden layers, each one with 16 neurons.
  在这个网络中，我选择了两个隐藏层，每一层有16个神经元。

[04:17] And admittedly that's kind of an arbitrary choice.
  诚然，这是一种随意的选择。

[04:21] To be honest I chose two layers based on how I want to motivate the structure in just a moment, and 16, well that was just a nice number to fit on the screen.
  老实说，我选择两层是基于我稍后想要如何激发这种结构，而16，嗯，那只是一个适合显示在屏幕上的好数字。

[04:28] In practice there is a lot of room for experiment with a specific structure here.
  在实践中，这里有很大的空间来试验特定的结构。

[04:33] The way the network operates, activations in one layer determine the activations of the next layer.
  网络运作的方式是，一层的激活决定了下一层的激活。

[04:39] And of course the heart of the network as an information processing mechanism comes down to exactly how those activations from one layer bring about activations in the next layer.
  当然，作为信息处理机制，网络的核心归根结底在于一层中的那些激活如何准确地引发下一层中的激活。

[04:49] It's meant to be loosely analogous to how in biological networks of neurons, some groups of neurons firing cause certain others to fire.
  它旨在松散地类比生物神经元网络中，一些神经元群的放电如何导致其他神经元的放电。

[04:58] Now the network I'm showing here has already been trained to recognize digits, and let me show you what I mean by that.
  现在我展示的这个网络已经经过训练可以识别数字，让我向你展示我的意思。

[05:03] It means if you feed in an image, lighting up all 784 neurons of the input layer according to the brightness of each pixel in the image,
  这意味着如果你输入一张图像，根据图像中每个像素的亮度点亮输入层的所有784个神经元，

[05:11] That pattern of activations causes some very specific pattern in the next layer.
  那种激活模式会在下一层引起某种非常特定的模式。

[05:16] Which causes some pattern in the one after it.
  这又会在再下一层引起某种模式。

[05:18] Which finally gives some pattern in the output layer.
  最终在输出层产生某种模式。

[05:22] And the brightest neuron of that output layer is the network's choice, so to speak, for what digit this image represents.
  而输出层中亮度最高的神经元，可以说是网络对于这张图像代表哪个数字的选择。

[05:32] And before jumping into the math for how one layer influences the next, or how training works, let's just talk about why it's even reasonable to expect a layered structure like this to behave intelligently.
  在深入探讨一层如何影响下一层，或者训练是如何运作的数学原理之前，我们先来谈谈为什么期望这样一种分层结构能够表现出智能是合理的。

[05:44] What are we expecting here?
  我们在这里期待什么呢？

[05:45] What is the best hope for what those middle layers might be doing?
  对于那些中间层可能在做的事情，最好的期望是什么？

[05:48] Well, when you or I recognize digits, we piece together various components.
  嗯，当你或我识别数字时，我们会将各种组件拼凑在一起。

[05:54] A 9 has a loop up top and a line on the right.
  数字 9 上面有一个圈，右边有一条线。

[05:57] An 8 also has a loop up top, but it's paired with another loop down low.
  数字 8 上面也有一个圈，但它下面还配有另一个圈。

[06:02] A 4 basically breaks down into three specific lines, and things like that.
  数字 4 基本上可以分解为三条特定的线，诸如此类。

[06:07] Now in a perfect world, we might hope that each neuron in the second.
  在理想的情况下，我们或许希望第二层中的每个神经元都能（发挥作用）。

[06:11] To last layer corresponds with one of these subcomponents.
  到最后一层对应于这些子组件之一。

[06:15] That anytime you feed in an image with, say, a loop up top, like a 9 or an 8, there's some specific neuron whose activation is going to be close to 1.
  这意味着每当你输入一张图像，比如顶部有一个环的图像，像 9 或 8，就会有一个特定的神经元，其激活值会接近 1。

[06:24] And I don't mean this specific loop of pixels.
  我指的并不是这组特定的像素环。

[06:26] The hope would be that any generally loopy pattern towards the top sets off this neuron.
  希望在于，顶部任何大致的环状图案都能触发这个神经元。

[06:32] That way, going from the third layer to the last one just requires learning which combination of subcomponents corresponds to which digits.
  这样一来，从第三层到最后一层，只需要学习哪些子组件的组合对应于哪些数字。

[06:41] Of course, that just kicks the problem down the road, because how would you recognize these subcomponents, or even learn what the right subcomponents should be?
  当然，这只是把问题推迟了，因为你该如何识别这些子组件，甚至学习什么是正确的子组件呢？

[06:48] And I still haven't even talked about how one layer influences the next, but run with me on this one for a moment.
  而且我甚至还没谈到一层是如何影响下一层的，但请先跟着我的思路走一会儿。

[06:53] Recognizing a loop can also break down into subproblems.
  识别一个环也可以分解为子问题。

[06:57] One reasonable way to do this would be to first recognize the various little edges that make it up.
  实现这一点的一种合理方法是首先识别构成它的各种小边缘。

[07:03] Similarly, a long line, like the kind you might see in the digits 1 or 4 or 7, is really just a long edge, or maybe you think of it as a certain pattern of several.
  同样地，一条长线，比如你在数字 1、4 或 7 中可能看到的那种，实际上只是一条长边缘，或者你可能认为它是某种由几个部分组成的特定图案。

[07:13] Smaller edges.
  更小的边缘。

[07:15] So maybe our hope is that each neuron in the second layer of the network corresponds with the various relevant little edges.
  所以也许我们的希望是，网络第二层中的每个神经元都对应于各种相关的小边缘。

[07:23] Maybe when an image like this one comes in, it lights up all of the neurons associated with around 8 to 10 specific little edges, which in turn lights up the neurons associated with the upper loop and a long vertical line, and those light up the neuron associated with a 9.
  也许当像这样的一张图像输入时，它会点亮所有与大约 8 到 10 个特定小边缘相关的神经元，这些神经元进而点亮与上环和长垂直线相关的神经元，而这些神经元又会点亮与数字 9 相关的神经元。

[07:40] Whether or not this is what our final network actually does is another question, one that I'll come back to once we see how to train the network, but this is a hope that we might have, a sort of goal with the layered structure like this.
  这是否是我们最终网络实际执行的操作是另一个问题，一旦我们了解了如何训练网络，我就会回到这个问题，但这可能是我们所怀有的一种希望，即这种分层结构的一种目标。

[07:53] Moreover, you can imagine how being able to detect edges and patterns like this would be really useful for other image recognition tasks.
  此外，你可以想象能够检测这样的边缘和模式对于其他图像识别任务将是多么有用。

[08:00] And even beyond image recognition, there are all sorts of intelligent things you might want to do that break down into layers of abstraction.
  甚至在图像识别之外，还有各种各样你可能想要做的智能事情，它们可以分解为抽象层。

[08:08] Parsing speech, for example, involves taking raw audio and picking out distinct sounds, which combine to make certain syllables, which combine to form words,
  例如，解析语音涉及获取原始音频并挑选出不同的声音，这些声音组合起来构成特定的音节，进而组合形成单词，

[08:16] Which combine to make up phrases and more abstract thoughts, etc.
  这些组合在一起构成了短语和更抽象的思想等等。

[08:21] But getting back to how any of this actually works, picture yourself right now designing how exactly the activations in one layer might determine the activations in the next.
  但回到这一切究竟是如何运作的问题上，现在想象一下，你正在设计一层中的激活值如何精确地决定下一层的激活值。

[08:30] The goal is to have some mechanism that could conceivably combine pixels into edges, or edges into patterns, or patterns into digits.
  目标是拥有一种机制，可以设想将像素组合成边缘，或将边缘组合成图案，或将图案组合成数字。

[08:39] And to zoom in on one very specific example, let's say the hope is for one particular neuron in the second layer to pick up on whether or not the image has an edge in this region here.
  为了深入探讨一个非常具体的例子，假设我们希望第二层中的某个特定神经元能够识别出图像的这个区域是否有边缘。

[08:51] The question at hand is what parameters should the network have?
  当前的问题是，网络应该具备哪些参数？

[08:55] What dials and knobs should you be able to tweak so that it's expressive enough to potentially capture this pattern, or any other pixel pattern, or the pattern that several edges can make a loop, and other such things?
  你应该能够调节哪些旋钮，使其具有足够的表达能力，从而能够捕捉到这种模式、任何其他像素模式，或者由几条边缘构成一个环的模式，以及其他类似的事物？

[09:08] Well, what we'll do is assign a weight to each one of the connections between our neuron and the neurons from the first layer.
  那么，我们将要做的是为我们的神经元与第一层神经元之间的每一个连接分配一个权重。

[09:16] These weights are just numbers.
  这些权重仅仅是数字。

[09:18] Then take all of those activations from the first layer and compute their weighted sum according to these weights.
  然后获取第一层的所有激活值，并根据这些权重计算它们的加权和。

[09:27] I find it helpful to think of these weights as being organized into a little grid of their own, and I'm going to use green pixels to indicate positive weights, and red pixels to indicate negative weights, where the brightness of that pixel is some loose depiction of the weight's value.
  我觉得把这些权重想象成一个它们自己的小网格会很有帮助，我将用绿色像素表示正权重，用红色像素表示负权重，像素的亮度大致代表了权重的值。

[09:42] Now if we made the weights associated with almost all of the pixels zero except for some positive weights in this region that we care about, then taking the weighted sum of all the pixel values really just amounts to adding up the values of the pixel just in the region that we care about.
  现在，如果我们除了我们关心的这个区域内的一些正权重外，将与几乎所有其他像素相关的权重都设为零，那么对所有像素值进行加权求和，实际上就等同于只将我们关心的区域内的像素值相加。

[09:59] And if you really wanted to pick up on whether there's an edge here, what you might do is have some negative weights associated with the surrounding pixels.
  如果你真的想检测这里是否存在边缘，你可以做的是为周围的像素分配一些负权重。

[10:07] Then the sum is largest when those middle pixels are bright but the surrounding pixels are darker.
  这样，当中间的像素明亮而周围的像素较暗时，总和就会达到最大。

[10:14] When you compute a weighted sum like this, you might come out with any number,
  当你像这样计算加权和时，你可能会得到任何数值，

[10:18] But for this network, what we want is for activations to be some value between 0 and 1.
  但对于这个网络，我们希望激活值是 0 到 1 之间的一个数值。

[10:24] So a common thing to do is to pump this weighted sum into some function that squishes the real number line into the range between 0 and 1.
  因此，一种常见的做法是将这个加权和输入到某个函数中，将实数轴压缩到 0 到 1 的范围内。

[10:32] And a common function that does this is called the sigmoid function, also known as a logistic curve.
  实现这一点的常用函数被称为 Sigmoid 函数，也称为逻辑曲线。

[10:38] Basically, very negative inputs end up close to 0, positive inputs end up close to 1, and it just steadily increases around the input 0.
  基本上，非常负的输入最终会接近 0，正的输入最终会接近 1，并且它在输入 0 附近稳步增加。

[10:49] So the activation of the neuron here is basically a measure of how positive the relevant weighted sum is.
  因此，这里神经元的激活基本上是对相关加权和有多正的度量。

[10:57] But maybe it's not that you want the neuron to light up when the weighted sum is bigger than 0.
  但也许你并不希望神经元在加权和大于 0 时被激活。

[11:02] Maybe you only want it to be active when the sum is bigger than, say, 10.
  也许你只希望它在总和大于（比如说）10 时才处于激活状态。

[11:06] That is, you want some bias for it to be inactive.
  也就是说，你需要某种偏置来让它保持不活跃。

[11:11] What we'll do then is just add in some other number like negative 10 to this weighted sum before plugging it through the sigmoid squishification function.
  那么我们要做的就是在这个加权和中加入另一个数字，比如负 10，然后再将其输入到 Sigmoid 压缩函数中。

[11:20] That additional number is called the bias.
  那个额外的数字被称为偏置。

[11:23] So the weights tell you what pixel pattern this neuron in the second layer is picking up on, and the bias tells you how high the weighted sum needs to be before the neuron starts getting meaningfully active.
  所以权重告诉你第二层的这个神经元正在捕捉什么样的像素模式，而偏置则告诉你加权和需要达到多高，神经元才会开始变得有意义地活跃。

[11:36] And that is just one neuron.
  而那仅仅是一个神经元。

[11:38] Every other neuron in this layer is going to be connected to all 784 pixel neurons from the first layer, and each one of those 784 connections has its own weight associated with it.
  这一层中的其他每个神经元都将连接到第一层的所有 784 个像素神经元，并且这 784 个连接中的每一个都有其关联的权重。

[11:51] Also, each one has some bias, some other number that you add on to the weighted sum before squishing it with the sigmoid.
  此外，每一个都有一些偏置，即你在用 Sigmoid 函数压缩加权和之前加到上面的另一个数字。

[11:58] And that's a lot to think about!
  这需要考虑的内容可真多！

[12:00] With this hidden layer of 16 neurons, that's a total of 784 times 16 weights, along with 16 biases.
  有了这个包含 16 个神经元的隐藏层，总共就有 784 乘以 16 个权重，以及 16 个偏置。

[12:08] And all of that is just the connections from the first layer to the second.
  而所有这些仅仅是从第一层到第二层的连接。

[12:12] The connections between the other layers also have a bunch of weights and biases associated with them.
  其他层之间的连接也关联着许多权重和偏置。

[12:18] All said and done, this network has almost exactly 13,000 total weights and biases.
  总而言之，这个网络总共拥有几乎正好 13,000 个权重和偏置。

[12:23] 13,000 knobs and dials that can be tweaked and turned to make this network behave in different ways.
  13,000个旋钮和拨盘可以进行调整和转动，使该网络以不同的方式运行。

[12:31] So when we talk about learning, what that's referring to is getting the computer to find a valid setting for all of these many many numbers so that it'll actually solve the problem at hand.
  所以当我们谈论学习时，指的是让计算机为所有这些成千上万的数字找到一个有效的设置，以便它能真正解决手头的问题。

[12:42] One thought experiment that is at once fun and kind of horrifying is to imagine sitting down and setting all of these weights and biases by hand, purposefully tweaking the numbers so that the second layer picks up on edges, the third layer picks up on patterns, etc.
  一个既有趣又有点可怕的思想实验是：想象一下坐下来手动设置所有这些权重和偏差，有目的地调整这些数字，使得第二层能够识别边缘，第三层能够识别模式，等等。

[12:57] I personally find this satisfying rather than just treating the network as a total black box, because when the network doesn't perform the way you anticipate, if you've built up a little bit of a relationship with what those weights and biases actually mean, you have a starting place for experimenting with how to change the structure to improve.
  我个人觉得这样做比仅仅把网络当作一个完全的黑盒要令人满意，因为当网络的表现不如你预期时，如果你已经对这些权重和偏差的实际含义有了一定的了解，你就有了尝试改变结构以进行改进的切入点。

[13:15] Or when the network does work but not for the reasons you might expect, digging into what the weights and biases are doing is a good way to challenge your assumptions and really expose the full space of possible solutions.
  或者当网络确实有效，但原因并非你所预料的那样时，深入研究权重和偏差的作用是挑战你假设并真正揭示所有可能解决方案空间的好方法。

[13:26] By the way, the actual function here is a little cumbersome to write down, don't you think?
  顺便说一下，这里的实际函数写起来有点麻烦，你不觉得吗？

[13:32] So let me show you a more notationally compact way that these connections are represented.
  所以我来向你展示一种在符号表示上更简洁的连接表示方式。

[13:37] This is how you'd see it if you choose to read up more about neural networks.
  如果你选择深入阅读有关神经网络的内容，你就会看到这种表示方法。

[13:40] Organize all of the activations from one layer into a column as a vector.
  将来自一层的所有激活值组织成一个列向量。

[13:48] Then organize all of the weights as a matrix, where each row of that matrix corresponds to the connections between one layer and a particular neuron in the next layer.
  然后将所有的权重组织成一个矩阵，其中矩阵的每一行对应于一层与下一层中特定神经元之间的连接。

[13:58] What that means is that taking the weighted sum of the activations in the first layer according to these weights corresponds to one of the terms in the matrix vector product of everything we have on the left here.
  这意味着根据这些权重对第一层的激活值进行加权求和，对应于我们左侧所有内容的矩阵向量乘积中的一项。

[14:14] By the way, so much of machine learning just comes down to having a good grasp of linear algebra, so for any of you who want a nice visual understanding for matrices and what matrix vector multiplication means, take a look at the series I did on linear algebra, especially chapter 3.
  顺便提一下，机器学习的很大一部分归根结底在于对线性代数的良好掌握，所以对于任何想要对矩阵以及矩阵向量乘法的含义有良好直观理解的人，请看看我做的线性代数系列视频，特别是第三章。

[14:29] Back to our expression, instead of talking about adding the bias to each one of these values independently, we represent it by organizing all those biases into a vector, and adding the entire vector to the previous matrix vector product.
  回到我们的表达式，与其讨论将偏置独立地加到每一个值上，我们通过将所有这些偏置组织成一个向量，并将整个向量加到之前的矩阵向量乘积上来表示它。

[14:43] Then as a final step, I'll wrap a sigmoid around the outside here, and what that's supposed to represent is that you're going to apply the sigmoid function to each specific component of the resulting vector inside.
  然后作为最后一步，我会在外面套上一个 sigmoid 函数，这所要表达的意思是，你将把 sigmoid 函数应用到内部结果向量的每一个特定分量上。

[14:55] So once you write down this weight matrix and these vectors as their own symbols, you can communicate the full transition of activations from one layer to the next in an extremely tight and neat little expression, and this makes the relevant code both a lot simpler and a lot faster, since many libraries optimize the heck out of matrix multiplication.
  所以一旦你把这个权重矩阵和这些向量写成它们自己的符号，你就可以用一个极其紧凑简洁的表达式来传达从一层到下一层的完整激活转换，这使得相关的代码既简单得多又快得多，因为许多库对矩阵乘法进行了极大的优化。

[15:17] Remember how earlier I said these neurons are simply things that hold numbers?
  还记得我之前说过这些神经元只是保存数字的东西吗？

[15:22] Well of course the specific numbers that they hold depends on the image you feed in, so it's actually more accurate to think of each neuron as a function,
  当然，它们所保存的具体数字取决于你输入的图像，所以实际上把每个神经元看作一个函数会更准确。

[15:31] One that takes in the outputs of all the neurons in the previous layer and spits out a number between 0 and 1.
  它接收上一层所有神经元的输出，并吐出一个 0 到 1 之间的数字。

[15:39] Really the entire network is just a function, one that takes in 784 numbers as an input and spits out 10 numbers as an output.
  实际上，整个网络只是一个函数，它接收 784 个数字作为输入，并吐出 10 个数字作为输出。

[15:47] It's an absurdly complicated function, one that involves 13,000 parameters in the forms of these weights and biases that pick up on certain patterns, and which involves iterating many matrix vector products and the sigmoid squishification function, but it's just a function nonetheless.
  这是一个极其复杂的函数，它涉及 13,000 个以权重和偏置形式存在的参数，这些参数用于捕捉特定的模式，并且涉及多次矩阵向量乘法和 sigmoid 压缩函数，但它终究只是一个函数。

[16:03] And in a way it's kind of reassuring that it looks complicated.
  从某种程度上说，它看起来很复杂反而让人感到安心。

[16:07] I mean if it were any simpler, what hope would we have that it could take on the challenge of recognizing digits?
  我的意思是，如果它再简单一点，我们还有什么希望指望它能承担起识别数字的挑战呢？

[16:13] And how does it take on that challenge?
  它是如何应对这一挑战的呢？

[16:15] How does this network learn the appropriate weights and biases just by looking at data?
  这个网络是如何仅通过观察数据来学习合适的权重和偏置的呢？

[16:20] Well that's what I'll show in the next video, and I'll also dig a little more into what this particular network we're seeing is really doing.
  嗯，这就是我将在下一个视频中展示的内容，我还会深入探讨我们所看到的这个特定网络到底在做什么。

[16:27] Now is the point I suppose I should say subscribe to stay notified about when that video or any new videos come out.
  我想现在是时候说订阅以获取该视频或任何新视频发布通知的时候了。

[16:33] But realistically, most of you don't actually receive notifications from YouTube, do you?
  但现实地说，你们大多数人实际上并没有收到 YouTube 的通知，对吧？

[16:38] Maybe more honestly I should say subscribe so that the neural networks that underlie YouTube's recommendation algorithm are primed to believe that you want to see content from this channel get recommended to you.
  也许更诚实地说，我应该说订阅，这样 YouTube 推荐算法背后的神经网络就会被设定为相信你希望看到这个频道的视频被推荐给你。

[16:48] Anyway, stay posted for more.
  总之，请保持关注以获取更多内容。

[16:50] Thank you very much to everyone supporting these videos on Patreon.
  非常感谢每一位在 Patreon 上支持这些视频的人。

[16:54] I've been a little slow to progress in the probability series this summer, but I'm jumping back into it after this project, so patrons you can look out for updates there.
  今年夏天我在概率系列视频上的进展有点慢，但在完成这个项目后我会重新投入其中，所以赞助者们可以留意那里的更新。

[17:03] To close things off here I have with me Lisha Li who did her PhD work on the theoretical side of deep learning and who currently works at a venture capital firm called Amplify Partners who kindly provided some of the funding for this video.
  最后，我请来了 Lisha Li，她曾从事深度学习理论方面的博士研究，目前在一家名为 Amplify Partners 的风险投资公司工作，该公司慷慨地为本视频提供了一些资金支持。

[17:15] So Lisha, one thing I think we should quickly bring up is this sigmoid function.
  那么 Lisha，我认为我们应该快速提一下这个 Sigmoid 函数。

[17:19] As I understand it, early networks use this to squish the relevant weighted sum into that interval between zero and one, you know kind of motivated by this biological analogy of neurons either being inactive or active.
  据我了解，早期的网络使用它将相关的加权和压缩到零到一之间的区间，你知道，这在某种程度上是受神经元处于非活动或活动状态的生物学类比所驱动的。

[17:30] Exactly.
  没错。

[17:30] But relatively few modern networks actually use sigmoid anymore.
  但现在实际上很少有现代网络使用 Sigmoid 了。

[17:34] Yeah.
  是的。

[17:34] It's kind of old school right?
  这有点老派，对吧？

[17:35] Yeah or rather ReLU seems to be much easier to train.
  是的，或者说 ReLU 似乎更容易训练。

[17:39] And ReLU, ReLU stands for rectified linear unit?
  那么 ReLU，ReLU 代表线性整流单元吗？

[17:42] Yes it's this kind of function where you're just taking a max of zero and a where a is given by what you were explaining in the video and what this was sort of motivated from I think was a partially by a biological analogy with how neurons would either be activated or not.
  是的，这是一种函数，你只需要取零和 a 的最大值，其中 a 是你在视频中解释的内容，我认为这在某种程度上是受到神经元如何被激活或不被激活的生物学类比的启发。

[18:01] And so if it passes a certain threshold it would be the identity function but if it did not then it would just not be activated so it'd be zero so it's kind of a simplification.
  因此，如果它超过了某个阈值，它就是恒等函数；但如果没有，它就不会被激活，所以结果为零，这算是一种简化。

[18:11] Using sigmoids didn't help training or it was very difficult to train at some point and people just tried ReLU and it happened to work very well for these incredibly deep neural networks.
  使用 sigmoid 函数对训练没有帮助，或者在某些时候很难训练，于是人们尝试了 ReLU，结果它在这些极其深层的神经网络中效果非常好。

[18:25] All right thank you Lisha.
  好的，谢谢你，Lisha。