chapter2/2.2-deep-learning-basic-mathematics.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 2.2 深度学习基础及数学原理\n",
    "深度学习并没有想象的那么难，甚至比有些传统的机器学习更简单。所用到的数学知识也不需要特别的高深，本章将会一边讲解深度学习中的基本理论，一边通过动手使用PyTorch实现一些简单的理论，本章内容很多，所以只做一个简短的介绍\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.1 监督学习和无监督学习\n",
    "监督学习、无监督学习、半监督学习、强化学习是我们日常接触到的常见的四个机器学习方法：\n",
    "\n",
    "- 监督学习：通过已有的训练样本（即已知数据以及其对应的输出）去训练得到一个最优模型（这个模型属于某个函数的集合，最优则表示在某个评价准则下是最佳的），再利用这个模型将所有的输入映射为相应的输出。\n",
    "- 无监督学习：它与监督学习的不同之处，在于我们事先没有任何训练样本，而需要直接对数据进行建模。 \n",
    "- 半监督学习 ：在训练阶段结合了大量未标记的数据和少量标签数据。与使用所有标签数据的模型相比，使用训练集的训练模型在训练时可以更为准确。\n",
    "- 强化学习：我们设定一个回报函数（reward function），通过这个函数来确认否越来越接近目标，类似我们训练宠物，如果做对了就给他奖励，做错了就给予惩罚，最后来达到我们的训练目的。\n",
    "\n",
    "这里我们只着重介绍监督学习，因为我们后面的绝大部们课程都是使用的监督学习的方法，在训练和验证时输入的数据既包含输入x，又包含x对应的输出y，即学习数据已经事先给出了正确答案。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.2 线性回归 （Linear Regreesion）\n",
    "线性回归是利用数理统计中回归分析，来确定两种或两种以上变量间相互依赖的定量关系的一种统计分析方法，运用十分广泛。其表达形式为y = w'x+e，e为误差服从均值为0的正态分布。 \n",
    "\n",
    "回归分析中，只包括一个自变量和一个因变量，且二者的关系可用一条直线近似表示，这种回归分析称为一元线性回归分析。如果回归分析中包括两个或两个以上的自变量，且因变量和自变量之间是线性关系，则称为多元线性回归分析。\n",
    "摘自[百度百科](https://baike.baidu.com/item/线性回归/8190345)\n",
    "\n",
    "简单的说：\n",
    "线性回归对于输入x与输出y有一个映射f，y=f(x),而f的形式为aX+b。其中a和b是两个可调的参数，我们训练的时候就是训练a，b这两个参数。\n",
    "\n",
    "下面我们来用PyTorch的代码来做一个详细的解释"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.0.1.post2'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 引用\n",
    "# 注意，这里我们使用了一个新库叫 seaborn 如果报错找不到包的话请使用pip install seaborn 来进行安装\n",
    "import torch\n",
    "from torch.nn import Linear, Module, MSELoss\n",
    "from torch.optim import SGD\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "torch.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面定义一个线性函数，这里使用 $y = 5x + 7$，这里的5和7就是上面说到的参数a和b，我们先使用matplot可视化一下这个函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x7fd40bbe57f0>]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD8CAYAAAB5Pm/hAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl4VPXd/vH3F0iAQFhCWAIhhH1LQDHsLigugAgial2KuJVq61NrWyGKC+6gtta2Lg+u2Fo3EhYFVETcFQUK2QgQIEAgJEAgCWTPfH9/ZHgufjRBYGZyZib367pyZTJzJufmzMydkzMnH4y1FhERCV6NnA4gIiK+paIXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcip6EZEgp6IXEQlyKnoRkSDXxOkAAJGRkTY2NtbpGCIiAWXdunUHrLXtf245vyj62NhY1q5d63QMEZGAYozZeSrL6dCNiEiQU9GLiAQ5Fb2ISJBT0YuIBDkVvYhIkFPRi4gEORW9iEiQU9GLiDigtKKap1ZsIudQic/X5Rd/MCUi0pB8t+0AiUmp7CooIbptGNNGdPPp+lT0IiL1pKiskqeWb+KdH3cT2y6Md2eMYESPdj5fr4peRKQerMzI44HFqewvLufXF/Tgnov70Cykcb2sW0UvIuJDB46UM2dpOh+l5NKvUziv3JTAoOg29ZpBRS8i4gPWWpZs2MsjH6ZztLyaP17Sh19f0JPQJvV/DoyKXkTEy/YeLuWBxWl8npnP2TFteHrqIHp3DHcsj4peRMRLXC7Lv3/cxdwVmVS7LA9NHMD0UbE0bmQczaWiFxHxgh0HjjIrKYUfdxRwbq9Inroqnq4RYU7HAlT0IiIeqap28eo3O3hu5RZCmzTi6amDuCYhGmOc3Ys/nopeROQMZewtYlZSCql7Crl0QEceuzKOjq2aOR3rv/xs0RtjXgcmAvnW2jj3dRHAe0AskA1ca609ZGp+hD0PTABKgJuttet9E11ExBnlVdX84/MsXvpiG23CQnjhhiFMiO/kV3vxxzuV83zeBMadcF0isMpa2xtY5f4aYDzQ2/0xA3jJOzFFRPzDup2HuPxv3/D3z7OYdFZnVt5zAZcPivLbkodT2KO31n5ljIk94erJwBj35QXAF8As9/VvWWst8IMxpo0xJspam+utwCIiTiipqOKZTzbz5nfZdG7dnDdvGcqYvh2cjnVKzvQYfcdj5W2tzTXGHPvXdgF2H7dcjvs6Fb2IBKxvth4gMTmFnEOl3DSyGzPH9aNl08B5i9PbSWv73cXWuqAxM6g5vENMTIyXY4iIeK6wpJInlmfw/tocekS24P1fj2RY9winY522My36vGOHZIwxUUC++/ocoOtxy0UDe2v7Btba+cB8gISEhFp/GIiIOOXjtH08uCSNgqMV3DmmJ3eP7V1vQ8i87UyLfikwHZjr/rzkuOvvMsa8CwwHCnV8XkQCyf7imiFky1JzGRDVijduHkpcl9ZOx/LIqZxe+Q41b7xGGmNygIepKfj3jTG3AbuAa9yLL6fm1Mosak6vvMUHmUVEvM5aS/L6PTz6UQalFdXce1lfZpzfg5DGgf8f8Z3KWTfX13HT2FqWtcBvPQ0lIlKf9hwu5f7kVL7csp9zurVl3tRB9OrQ0ulYXhM4bxuLiHiZy2X515qdzFuRiQUemTSQaSO60cjhIWTepqIXkQZp2/4jJCal8FP2Ic7rHcmTU/xnCJm3qehFpEGprHbxytfb+etnW2ke0phnrxnM1CFd/PovWz2loheRBiNtTyGzklJI31vE+LhOPDJ5IB3C/W8Imbep6EUk6JVVVvP3z7fy8pfbaRsWyks3DmF8fJTTseqNil5Egtra7AJmJqWwff9RrjknmtmX96dNWKjTseqVil5EgtKR8iqe+TiTt37YSefWzXnr1mGc36e907EcoaIXkaDz5Zb93J+cyt7CUqaPjOXey/rSIoCGkHlbw/2Xi0jQOVxSwWMfbSJpfQ4927fgg1+PJCE28IaQeZuKXkSCworUXB5cks6hkgruurAXd13UK2CHkHmbil5EAlp+URkPLUnn4/R9xHVpxYJbhzKwc2APIfM2Fb2IBCRrLQvX5fDYRxmUVbmYNa4fvzqvO02CYAiZt6noRSTg7C4o4f5FqXy99QDDYiOYOzWeHu2DZwiZt6noRSRgVLssb32fzTOfbMYAj00eyI3Dg28Imbep6EUkIGTlFzMrKZV1Ow9xQZ/2PHlVPF3aNHc6VkBQ0YuIX6usdvG/X27jb6uyCGvamL9cO5gpZwf3EDJvU9GLiN9KzSlkZlIKm3KLuHxQFHOuGEj78KZOxwo4KnoR8TtlldX89bOtvPL1dtq1COV/p53DZQM7OR0rYKnoRcSvrNl+kMTkVHYcOMovErpy/+X9ad08xOlYAU1FLyJ+obiskqc/3sw/f9hJ14jmvH37cEb3inQ6VlBQ0YuI41Zvzmd2ciq5RWXcOro7f7qsD2Ghqidv0ZYUEcccOlrBYx9lkPyfPfTu0JKkO0cxJKat07GCjopeROqdtZZlqbk8vCSdwtJKfje2N7+9sCdNm2gImS+o6EWkXuUVlfHA4jRWZuQxKLo1/7p9OP2jWjkdK6ip6EWkXlhreX/tbh5ftomKKhf3T+jHraM1hKw+qOhFxOd2HSwhMTmF77YdZHj3COZNHURsZAunYzUYKnoR8Zlql+XN77J59pPNNG5keGJKHNcPjdEQsnqmohcRn9iSV8zMhSls2H2Yi/p14IkpcUS11hAyJ6joRcSrKqpcvPTFNv6xeistmzbh+evOYtLgzhpC5iAVvYh4zcbdh5mVlELmvmImDe7Mw1cMoF1LDSFzmopeRDxWWlHNc59t4dWvt9MhvBmv3pTAxQM6Oh1L3DwqemPMPcDtgAVSgVuAKOBdIAJYD0yz1lZ4mFNE/NT32w5yX3IK2QdLuH5YDPdN6EerZhpC5k/O+ARWY0wX4HdAgrU2DmgMXAfMA56z1vYGDgG3eSOoiPiXorJK7l+UyvWv/IAF/v2r4Tx1VbxK3g95euimCdDcGFMJhAG5wEXADe7bFwBzgJc8XI+I+JFVm/KYvSiN/OIyfnVed/5wSV+ah2p8gb8646K31u4xxjwL7AJKgU+BdcBha22Ve7EcoIvHKUXELxw8Us4jH2awdONe+nYM5+Vp53BW1zZOx5KfccZFb4xpC0wGugOHgQ+A8bUsauu4/wxgBkBMTMyZxhCRemCtZenGvTzyYQbFZZXcc3Ef7hzTk9AmGl8QCDw5dHMxsMNaux/AGJMMjALaGGOauPfqo4G9td3ZWjsfmA+QkJBQ6w8DEXFebmEpDyxKY1VmPoO7tuHpqYPo2ync6VhyGjwp+l3ACGNMGDWHbsYCa4HVwNXUnHkzHVjiaUgRqX8ul+Xdn3bz1PJNVLpcPHB5f24Z3Z3GGl8QcDw5Rr/GGLOQmlMoq4D/ULOHvgx41xjzuPu617wRVETqT/aBoyQmp/DD9gJG9mjH3KnxdGunIWSByqOzbqy1DwMPn3D1dmCYJ99XRJxRVe3ijW+z+fPKzYQ0asTcq+L5xdCuGl8Q4PSXsSICQOa+ImYtTGFjTiEX9+/I41fG0al1M6djiReo6EUauPKqal5YvY0XV2fRunkIf7/+bCYOitJefBBR0Ys0YP/ZdYhZSSlsyTvClLO78ODEAUS0CHU6lniZil6kASqpqOLPn27h9W930KlVM16/OYGL+mkIWbBS0Ys0MN9lHSAxOZVdBSX8ckQMs8b1I1zzaYKail6kgSgsreSp5Zt496fddI9swbszRjCiRzunY0k9UNGLNACfpu/jgcVpHDhSzq8v6ME9F/ehWYiGkDUUKnqRIHbgSDlzlqbzUUou/TqF8+r0BAZFawhZQ6OiFwlC1loWb9jDIx9mUFJezR8v6cMdY3oS0lhDyBoiFb1IkNl7uJTZi1JZvXk/Z8fUDCHr3VFDyBoyFb1IkHC5LG//uIt5KzKpdlkemjiA6aNiNYRMVPQiwWD7/iMkJqXyY3YB5/aK5Kmr4ukaEeZ0LPETKnqRAFZV7eLVb3bw3MotNG3SiKevHsQ150RrfIH8f1T0IgEqY28RM5M2kraniMsGduSxyXF0aKUhZPLfVPQiAaa8qpp/fJ7FS19so01YCC/eOITxcZ20Fy91UtGLBJB1OwuYlZRKVv4RrhrShQcvH0BbDSGTn6GiFwkAR8ureOaTzSz4PpvOrZvz5i1DGdO3g9OxJECo6EX83Ndb93Nfcio5h0qZPrIb947rR8umeunKqdOzRcRPFZZU8viyDD5Yl0OP9i344I6RDI2NcDqWBCAVvYgf+jhtHw8uSaPgaAW/GdOT343trSFkcsZU9CJ+JL+4jDlL01meuo8BUa144+ahxHVp7XQsCXAqehE/YK0laf0eHvsog9LKau69rC8zzu+hIWTiFSp6EYflHCrh/kVpfLVlPwnd2jJ36iB6dWjpdCwJIip6EYe4XJZ//rCTeR9nAvDIpIFMG9GNRhpCJl6mohdxwLb9R5i1MIW1Ow9xfp/2PDkljui2GkImvqGiF6lHldUu5n+1nedXbaV5SGOevWYwU4d00fgC8SkVvUg9SdtTyMyFKWTkFjEhvhNzJg2kQ7iGkInvqehFfKyssprnV21l/lfbaRsWysu/HMK4uCinY0kDoqIX8aGfsguYtTCF7QeOcs050Txw+QBah4U4HUsaGBW9iA8cKa/i6Y8zeev7nUS3bc4/bxvGeb3bOx1LGiiPit4Y0wZ4FYgDLHArsBl4D4gFsoFrrbWHPEopEkC+3LKf+5NT2VtYys2jYrn3sr600BAycZCnf3b3PPCxtbYfMBjYBCQCq6y1vYFV7q9Fgt7hkgr+8P4Gpr/+I81CGrHwjpHMmTRQJS+OO+NnoDGmFXA+cDOAtbYCqDDGTAbGuBdbAHwBzPIkpIg/s9ayIm0fDy1J43BJJXdd2Iu7LuqlIWTiNzzZ1egB7AfeMMYMBtYBdwMdrbW5ANbaXGOM/ncECVr5RWU8uCSNT9LziOvSigW3DmNgZw0hE//iSdE3AYYA/2OtXWOMeZ7TOExjjJkBzACIiYnxIIZI/bPW8sG6HB7/KIPyKheJ4/tx+7ndaaIhZOKHPCn6HCDHWrvG/fVCaoo+zxgT5d6bjwLya7uztXY+MB8gISHBepBDpF7tLijhvuRUvsk6wLDYCOZOjadHew0hE/91xkVvrd1njNltjOlrrd0MjAUy3B/Tgbnuz0u8klTEYdUuy1vfZ/P0x5tpZOCxK+O4cViMhpCJ3/P0dID/Ad42xoQC24FbqDmT531jzG3ALuAaD9ch4ris/GJmLkxh/a7DjOnbniemxNOlTXOnY4mcEo+K3lq7AUio5aaxnnxfEX9RWe3i5S+28ffPswhr2pjnfjGYK8/SEDIJLDrBV6QOqTmF3LtwI5n7ipk4KIo5kwYS2bKp07FETpuKXuQEZZXVPPfZFl75ajuRLZsyf9o5XDqwk9OxRM6Yil7kOGu2HyQxOZUdB45y3dCu3DehP62bawiZBDYVvQhQXFbJvI8z+dcPu+ga0Zy3bx/O6F6RTscS8QoVvTR4qzPzuX9RKvuKyrjt3O788dI+hIXqpSHBQ89mabAKjlbw6IfpLN6wl94dWpJ05yiGxLR1OpaI16nopcGx1vJRSi5zlqZTWFrJ78b25rcX9qRpEw0hk+CkopcGJa+ojNmL0vhsUx6Dolvz9q+G069TK6djifiUil4aBGst7/20myeWb6KiysXsCf25ZXSshpBJg6Cil6C362AJickpfLftIMO7RzBv6iBiI1s4HUuk3qjoJWhVuyxvfLuDZz/dTJNGjXhySjzXDe2qIWTS4KjoJSht3lfMzKQUNu4+zEX9OvDElDiiWmsImTRMKnoJKhVVLl78IosXVmcR3iyE5687i0mDO2sImTRoKnoJGht3H2bmwhQ25xUz+azOPDRxAO00hExERS+Br7Simr+s3Mxr3+ygQ3gzXr0pgYsHdHQ6lojfUNFLQPtu2wHuS05l58ESbhgeQ+L4frRqpiFkIsdT0UtAKiqr5Knlmbzz4y66tQvj378azqieGkImUhsVvQSczzLymL04lf3F5cw4vwf3XNyH5qEaXyBSFxW9BIyDR8p55MMMlm7cS79O4cyflsDgrm2cjiXi91T04vestSzduJc5S9M5Ul7FPRf34c4xPQltovEFIqdCRS9+LbewlAcWpbEqM5+zurbh6asH0adjuNOxRAKKil78kstleeenXTy1PJMql4sHLu/PLaO701jjC0ROm4pe/M6OA0dJTEphzY4CRvVsx9yrBhHTLszpWCIBS0UvfqOq2sXr3+7gz59uIbRJI+ZNjefahK4aXyDiIRW9+IVNuUXMSkohJaeQSwZ05PEr4+jYqpnTsUSCgopeHFVeVc0Lq7fx4uosWjcP4R83nM3l8VHaixfxIhW9OGb9rkPMWpjC1vwjTDm7Cw9NHEDbFqFOxxIJOip6qXclFVU8+8kW3vhuB51aNeONm4dyYb8OTscSCVoqeqlX32YdIDE5hd0FpfxyRAyzxvUjXEPIRHxKRS/1orC0kieXbeK9tbvpHtmC92aMYHiPdk7HEmkQVPTic5+m7+OBxWkcPFrBHRf05PcX96ZZiIaQidQXj4veGNMYWAvssdZONMZ0B94FIoD1wDRrbYWn65HAs7+4nDkfprMsJZf+Ua14bfpQ4qNbOx1LpMHxxlSou4FNx309D3jOWtsbOATc5oV1SACx1pK8PodLnvuSlel5/OnSPiy9a7RKXsQhHhW9MSYauBx41f21AS4CFroXWQBc6ck6JLDsOVzKLW/+xB/e30iPyBYsv/tc7rqoNyGNNWlSxCmeHrr5KzATODZOsB1w2Fpb5f46B+ji4TokALhclrfX7GTuikxcFh6+YgA3jYzVEDIRP3DGRW+MmQjkW2vXGWPGHLu6lkVtHfefAcwAiImJOdMY4ge27z9CYlIqP2YXcF7vSJ6cEk/XCA0hE/EXnuzRjwYmGWMmAM2AVtTs4bcxxjRx79VHA3tru7O1dj4wHyAhIaHWHwbi36qqXbzy9Q6e+2wLzZo04pmrB3H1OdEaXyDiZ8646K219wH3Abj36P9krb3RGPMBcDU1Z95MB5Z4Iaf4mYy9RcxM2kjaniIuG9iRxybH0UFDyET8ki/Oo58FvGuMeRz4D/CaD9YhDimrrOYfn2fx8pfbaBMWyks3DmF8fJTTsUTkJLxS9NbaL4Av3Je3A8O88X3Fv6zbWcDMhSls23+UqUOieXBif9qEaQiZiL/TX8bKzzpaXsUzn2xmwffZdG7dnAW3DuOCPu2djiUip0hFLyf11Zb93Jecyt7CUm4a0Y17x/WjZVM9bUQCiV6xUqvCkkoeW5bBwnU59Gjfgvd/PZKhsRFOxxKRM6Cil//ycVouDy5Jp+BoBb8Z05PfjdUQMpFApqKX/5NfXMbDS9JZkbaPAVGteOPmocR10XwakUCnohestSxcl8PjyzZRWlnNvZf1Zcb5PTSfRiRIqOgbuN0FJdy/KJWvtx4goVtb5k4dRK8OLZ2OJSJepKJvoFwuy1vfZ/P0J5sxwKOTB/LL4d1opCFkIkFHRd8AZeUfITEphbU7D3F+n/Y8OSWO6LYaQiYSrFT0DUhltYv5X23n+c+20jy0MX++ZjBXDemiIWQiQU5F30Ck7Slk5sIUMnKLmBDfiUcmxdE+vKnTsUSkHqjog1xZZTXPr9rK/K+2E9EilJd/OYRxcRpCJtKQqOiD2E/ZBcxamML2A0e5NiGa2RMG0DosxOlYIlLPVPRB6Eh5FU9/nMlb3+8kum1z/nXbcM7tHel0LBFxiIo+yKzenM/s5FRyi8q4ZXQsf7q0Ly00hEykQVMDBIlDRyt47KMMkv+zh14dWrLwjlGc062t07FExA+o6AOctZblqft4eGkah0sq+Z+LenHXRb1o2kRDyESkhoo+gOUXlfHA4jQ+zcgjvktr3rp1OAM6t3I6loj4GRV9ALLW8sHaHB5blkFFlYv7xvfjtnO700RDyESkFir6ALO7oIT7klP5JusAw7pHMPeqeHq01xAyEambij5AVLssC77L5plPNtO4keHxK+O4YViMhpCJyM9S0QeArXnFzExK4T+7DjOmb3uenBJP5zbNnY4lIgFCRe/HKqpcvPzlNv7xeRYtmjbmr784i8lnddYQMhE5LSp6P5WSc5iZC1PI3FfMFYM78/AVA4hsqSFkInL6VPR+pqyymudWbuGVr7fTPrwpr9yUwCUDOjodS0QCmIrej/yw/SCJSSlkHyzh+mFdSRzfn9bNNYRMRDyjovcDxWWVzF2RydtrdhETEca/bx/OqF4aQiYi3qGid9jnmXnMXpRGXlEZt5/bnT9c2oewUD0sIuI9ahSHFByt4NEP01m8YS+9O7TkxTtHcXaMhpCJiPep6OuZtZYPU3KZszSd4rJK7h7bm99c2FNDyETEZ1T09WhfYc0Qss825TE4ujXzrh5Ov04aQiYivnXGRW+M6Qq8BXQCXMB8a+3zxpgI4D0gFsgGrrXWHvI8auCy1vLuT7t5ctkmKl0uZk/oz63ndqexxheISD3wZI++CvijtXa9MSYcWGeMWQncDKyy1s41xiQCicAsz6MGpp0Hj5KYlMr32w8yokcEc68aRGxkC6djiUgDcsZFb63NBXLdl4uNMZuALsBkYIx7sQXAFzTAoq92Wd74dgfPfrqZkEaNeHJKPNcN7aohZCJS77xyjN4YEwucDawBOrp/CGCtzTXGdKjjPjOAGQAxMTHeiOE3Nu+rGUK2cfdhxvbrwONT4ohqrSFkIuIMj4veGNMSSAJ+b60tOtWBW9ba+cB8gISEBOtpDn9QUeXixS+yeGF1FuHNQvjb9WdzxaAoDSETEUd5VPTGmBBqSv5ta22y++o8Y0yUe28+Csj3NGQg2LD7MLMWprA5r5jJZ3Xm4SsGEtEi1OlYIiIenXVjgNeATdbavxx301JgOjDX/XmJRwn9XGlFNX/+dDOvf7uDDuHNeG16AmP7awiZiPgPT/boRwPTgFRjzAb3dfdTU/DvG2NuA3YB13gW0X99t+0AiUmp7Coo4YbhMSSO70erZhpCJiL+xZOzbr4B6jr4PPZMv28gKCqr5Knlm3jnx910axfGO78awcie7ZyOJSJSK/1l7Gn6LCOP2YtT2V9czozze3DPxX1oHqrxBSLiv1T0p+jgkXLmfJjBhxv30q9TOPOnJTC4axunY4mI/CwV/c+w1rJkw14e+TCdI+VV/OGSPtxxQU9CmzRyOpqIyClR0Z/E3sOlPLA4jc8z8zmraxuevnoQfTqGOx1LROS0qOhr4XJZ/v3jLuauyKTaZXlw4gBuHhWrIWQiEpBU9CfYceAoiUkprNlRwOhe7XhqyiBi2oU5HUtE5Iyp6N2qql289s0O/rJyC6FNGjFvajzXJnTV+AIRCXgqemBTbhGzklJIySnkkgEdefzKODq2auZ0LBERr2jQRV9eVc0Ln2fx4hfbaBMWwgs3DGFCfCftxYtIUGmwRb9u5yFmJaWQlX+Eq87uwoMTB9BWQ8hEJAg1uKIvqajimU828+Z32US1asYbtwzlwr61jswXEQkKDarov9l6gMTkFHIOlTJtRDdmjutLuIaQiUiQaxBFX1hayRPLMnh/bQ7dI1vw3owRDO+hIWQi0jAEfdF/kr6PBxencfBoBXeO6cndY3vTLERDyESk4Qjaot9fXM6cpeksS82lf1QrXps+lPjo1k7HEhGpd0FX9NZaktfv4dGPMiitqObey/oy4/wehDTWEDIRaZiCquj3HC7l/uRUvtyynyExNUPIenXQEDIRadiCouhdLsu/1uxk3opMLDDnigFMG6khZCIiEARFv23/ERKTUvgp+xDn9Y7kySnxdI3QEDIRkWMCuujf/2k3DyxJo1mTRjxz9SCuPida4wtERE4Q0EXfvX0LxvbrwCOTB9IhXEPIRERqE9BFPzQ2gqGxEU7HEBHxazrnUEQkyKnoRUSCnIpeRCTIqehFRIKcil5EJMip6EVEgpyKXkQkyKnoRUSCnLHWOp0BY8x+YOcZ3j0SOODFON6iXKdHuU6fv2ZTrtPjSa5u1tr2P7eQXxS9J4wxa621CU7nOJFynR7lOn3+mk25Tk995NKhGxGRIKeiFxEJcsFQ9POdDlAH5To9ynX6/DWbcp0en+cK+GP0IiJycsGwRy8iIicRMEVvjBlnjNlsjMkyxiTWcntTY8x77tvXGGNi6yFTV2PMamPMJmNMujHm7lqWGWOMKTTGbHB/POTrXO71ZhtjUt3rXFvL7cYY8zf39koxxgyph0x9j9sOG4wxRcaY35+wTL1tL2PM68aYfGNM2nHXRRhjVhpjtro/t63jvtPdy2w1xkz3caZnjDGZ7sdpkTGmTR33Pelj7qNsc4wxe457vCbUcd+Tvn59kOu94zJlG2M21HFfn2yzurrBseeXtdbvP4DGwDagBxAKbAQGnLDMb4CX3ZevA96rh1xRwBD35XBgSy25xgAfObDNsoHIk9w+AVgBGGAEsMaBx3QfNecBO7K9gPOBIUDacdc9DSS6LycC82q5XwSw3f25rftyWx9muhRo4r48r7ZMp/KY+yjbHOBPp/BYn/T16+1cJ9z+Z+Ch+txmdXWDU8+vQNmjHwZkWWu3W2srgHeByScsMxlY4L68EBhrfPwfyFprc621692Xi4FNQBdfrtOLJgNv2Ro/AG2MMVH1uP6xwDZr7Zn+oZzHrLVfAQUnXH3882gBcGUtd70MWGmtLbDWHgJWAuN8lcla+6m1tsr95Q9AtDfWdbrq2F6n4lRevz7J5e6Aa4F3vLW+U8xUVzc48vwKlKLvAuw+7usc/rtQ/28Z94uiEGhXL+kA96Gis4E1tdw80hiz0RizwhgzsJ4iWeBTY8w6Y8yMWm4/lW3qS9dR94vPie11TEdrbS7UvFiBDrUs4+S2u5Wa38Rq83OPua/c5T6s9HodhyKc3F7nAXnW2q113O7zbXZCNzjy/AqUoq9tz/zE04VOZRmfMMa0BJKA31tri064eT01hycGA38HFtdHJmC0tXYIMB74rTEm3xNmAAACcklEQVTm/BNud3J7hQKTgA9qudmp7XU6HNl2xpjZQBXwdh2L/Nxj7gsvAT2Bs4Bcag6TnMix5xpwPSffm/fpNvuZbqjzbrVc59H2CpSizwG6Hvd1NLC3rmWMMU2A1pzZr5mnxRgTQs0D+ba1NvnE2621RdbaI+7Ly4EQY0ykr3NZa/e6P+cDi6j59fl4p7JNfWU8sN5am3fiDU5tr+PkHTuE5f6cX8sy9b7t3G/ITQRutO4DuSc6hcfc66y1edbaamutC3iljnU68lxz98BVwHt1LePLbVZHNzjy/AqUov8J6G2M6e7eG7wOWHrCMkuBY+9OXw18XtcLwlvcx/9eAzZZa/9SxzKdjr1XYIwZRs02P+jjXC2MMeHHLlPzZl7aCYstBW4yNUYAhcd+pawHde5lObG9TnD882g6sKSWZT4BLjXGtHUfqrjUfZ1PGGPGAbOASdbakjqWOZXH3BfZjn9fZ0od6zyV168vXAxkWmtzarvRl9vsJN3gzPPL2+82++qDmrNEtlDz7v1s93WPUvPkB2hGzaGALOBHoEc9ZDqXml+pUoAN7o8JwB3AHe5l7gLSqTnT4AdgVD3k6uFe30b3uo9tr+NzGeAF9/ZMBRLq6XEMo6a4Wx93nSPbi5ofNrlAJTV7UbdR877OKmCr+3OEe9kE4NXj7nur+7mWBdzi40xZ1ByzPfYcO3Z2WWdg+cke83rYXv90P39SqCmxqBOzub/+r9evL3O5r3/z2PPquGXrZZudpBsceX7pL2NFRIJcoBy6ERGRM6SiFxEJcip6EZEgp6IXEQlyKnoRkSCnohcRCXIqehGRIKeiFxEJcv8Pye3TH8lSsHMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x = np.linspace(0,20,500)\n",
    "y = 5*x + 7\n",
    "plt.plot(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "下面我生成一些随机的点，来作为我们的训练数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.rand(256)\n",
    "noise = np.random.randn(256) / 4\n",
    "y = x * 5 + 7 + noise\n",
    "df = pd.DataFrame()\n",
    "df['x'] = x\n",
    "df['y'] = y"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在图上显示下我们生成的数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAFgCAYAAACFYaNMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8nGd56P3f/SyzaUa75N2xndg4OyQuJJCmZmkbWj4p0AA1hUNTaAIE0kNPIZwu9LyhS1JaaGnTYp9CoW9KgNdQoFBSVuNSbLI4JIFE2LGS2PKixRpJM5rtWe73j2dmPDOakcayNkvX99N87IxmeRTcy7eu51qU1hohhBALz1jsCxBCiJVKArAQQiwSCcBCCLFIJAALIcQikQAshBCLRAKwEEIsEgnAQgixSCQACyHEIpEALIQQi8Ra7Atoxk033aQffPDBxb4MIYRolmrmSRfECXhkZGSxL0EIIebcBRGAhRBiOZIALIQQi0QCsBBCLBIJwEIIsUgkAAshxCKRACyEEItEArAQQiwSCcBCCLFIJAALIcQiuSBakYUQYj7s6xti9/5+jiczbOiIcfuNW9i5vXfBPl9OwEKIFWlf3xAf+upPGUrlaI/aDKVyfOirP2Vf39CCXYMEYCHEirR7fz+2qYiFLJRSeL5maCLH7fc/yq49B2cdiLXWTT9XArAQYkU6nswQtU0AUjmHk2M5fK3xtZ71adj3NYMT+aafLwFYCLEibeiIkXU8AIZTeXytcTyN62lOj+couB679/c3/X6erzk1keOZoVTTr5EALIRYkW6/cQuOp8kUXLKOh+trNGAZ4HqaM5MFjgxONPVerudzcizLkwNjvPeBx5q+BgnAQogVaef2Xu6++XJ6ExG0Diao24bCMk0MI5inXvBmzucWXJ+TYzkOHB3hfZ9/nGTGafoaJAALIVasndt7eeC26+iKh7AMhWEoNEEeGA0ha/oQmXM8To1n+c+fnuIP/u0nZB2P3kS46c+XACyEWPG29iaIR0wczyfn+DieTzxisrU30fA1mYLLqfEcDzx0jD/7jz5cX7Olu4W/2/Wipj9XArAQYsW7fksnEzkP01CELYVpKCZyHtdv6az7/FTO4dR4jvu+d4RPfD+4UXfV+jb+5k0vZH1nrOnPlQAshFjxDvSP0psIETINfA0h06A3EeJA/+iU545nHE6OZfmzrz/N3kdPAHDjtm7+8tevorc1wprWSNOfK63IQogV7/DgBDnHp+D5hEyD7niYRMRiIJmpet6ZdJ6TY1n+5Ks/5dCxMQB+7YVrec/LL6E9FqLnHPK/IAFYCLHC7esbIp338LXGVArX05wcz9LlhtjcHQeC7rbhdJ7nRyb531/6Cc8MpwF4+w2bePOLN9IVD9MeC53zZ0sAFkIsK+c6YGf3/n5aQibJjIODxlCgFCQzDvfcuAWtg+62n52e4K4vPsnpiRyGgt//pRfw6ivX0JMIEw/PLpRKDlgIsWzMZsDOkaEUqZyLaSgMBb4Gz4ewZfDz23o4OZ7j0edHee8Dj3F6IkfEMvjT117Br1y5htWtkVkHX5AALIRYRmoH7MRCFrappm0pLrg+KLBNg7BlErVNbFOhtebkWJbv/2yI//WFx5nIubRFbf76jVdzwyU9rG2PEg2Z53W9koIQQlwQmkktNHszrZJtKrJOMEhHKdA6yPlahuKrPz7BX3/rML6GNW0R7v31K9nSE2d1awTLPP/zqwRgIcSSV0ot2KaqSi3cDeUg3MzNtMr3KwVzx9O0hEwcT1PwfGxDEQ/Z2KbJR755GIBLeuPc8/orWd8RozcRLrcqn695S0EopT6llBpSSv2k4rGPKKX6lFJPKKX+TSnVPl+fL4RYPppJLeze309ni41CoQFVjG7JjMPtN24pP682T9wSNknlPRIRi609cbriIbKOz+mJHADXbmznY2+8mou6WljVOnfBF+Y3B/xp4Kaax74FXKG1vgo4DPzvefx8IcQyUTm7tyRqm1WphePJDF0tYda2R7CMYMC6bSgSEasqVVEbzLvjEXoTISbzLmcm80zkPDKFYEzlK7f38ufFk29PIoxScxd8YR4DsNZ6PzBa89g3tdZu8V8PAuvn6/OFEMtH5ezekqzjsb4jNuU5iYjNlp4469qj+BpSObdqw0W9YN4eDRENmaxqizKeDaaZveHa9fzBr17K2vYoHS3nXuPbjMWsgvht4BuNvqiUuk0p9YhS6pHh4eEFvCwhxFJTObtX6+BXx9NVqYXK50xkCwwks7i+ZnVruKocbUNHjJF0nv7hNH2nJzg6lOL0eJaJrMsTA+MAvOsXtnDHyy9hbVuURMSet+9Lncv+onN+c6U2AV/TWl9R8/gfAjuA1+smLmDHjh36kUcemZdrFEJcGEo3zgaSGdY3qIIoPefQsSRKwapEhNZoEEAzBZfeRITrt3Tyt999Bq01fk30sQzFXTdt55cvX82qtjBha9ZlZk3lKha8CkIp9TbgNcArmwm+QggBQbXDTCvjS8+54d7v0h61q3K2pZzxgX5IhA3Gst6U17/t+k3cdMVqVrdFsOegzGwmC5qCUErdBNwF3Ky1blyYJ4QQ52G6nPGx0UnyrsasOaMq4LHjSda2Rxck+ML8lqE9ABwAXqCUGlBKvR34eyABfEsp9WOl1Cfm6/OFECtXo5zxO27YzKpEhJzjU9o2pAj2wIUtxdBEDnMOy8xmMm8pCK31rjoPf3K+Pk8IIUp2bu/lbqjKGb/9ZZvYuipOR0uIUu5TAaYBCkVHS4gNnS0Lep3SCSeEuCBVdrMlwhZaa9IFr6pNuZQzzjkeJ8Yy3PONPr5TLEcrnXNtQ9HeEsI2zaqqioUgAVgIccGpbE02FRwZSqN1kNc9NZ7l0LEkd+y8mDtftY1sweP5M5P8n3//KQeLGy5++fJV3Li1h6/8+CSnxrMNqyrmmwRgIcQFp7KbrX84jVLg+uATjJH0fM19+46ydVWCNW0R/uDfnuSpUykAfvMlG3nHDZtZ0x7lN6+7aFG/DwnAQohFca6D0ysdT2ZoL9b3Fjwfr1TQq8HzNJ7WOFrze1/4MfGwzXA6jwLe+4pLuOXaDaxui8y4cn4hSAAWQiy4etPNfn/v4/TEw6Ty7owBeUNHjKFUjljIImQaTHpByZkGnIruiqzjk3WC4NsatfjBkRGuXt/Oxq7mNxfPp3nthJsr0gknxPJy08e+z3OjGTxfEzINYsWVQJahuKQ3TtbxcDzN3TdfXjcIVwZw1/N5fjQLBDfWGkW0iKWI2iYZxyceNtm2qnU+875N1bIt/hlcCLGi7Osb4shwGt8/O7d3JF0AgtRBM5ssdm7v5e6bL6c3EcHXsKEjiqkaB18A19cksy6u55Nz/KbWFc03SUEIIRbU7v392IYRzOxVKthC4QU30VpCZ8+EteMma1WWmfm+5suPneCDX3qCgjc1DJd2vSmCXwueTyxkkSm47N7fv+DVD+XrWpRPFUKsWMeTGVa1htEafK3R6PLP6z2JcPl5teMmG/F8zcBYhoeeGy0H35BpVAU3yzDQZ+/TESq2Gs8U5OebnICFEHNuugqH0g20te0RhlN5Cp6PZQBKYRrBMsxSDnimxgjH8xkYzfDn3+jjW08NAhCzTRzPw4dg3oNS5QCvCU7B3fEg0J+ZzDOZ97jh3u+ecyXGXJATsBBiTs20Gr40p8E0FJu7W9jYGWNNe4z3vvwSehMRxrMOvYlIwxtwJXnX45mhFB/44hNng2/IpCtus3VVgtWtYVCK1oiJZQYr5wHaohaJiMVIOsdQqkAsZDa9wn6uyQlYCDGnKpskgCm51npzGkonzzsbvGftifq3XnoRPYkIH/zSExweTAOwujWCbUJbNIRSip5EBIBMwaOzJcT6jhjXb+nkQP8oA8kMk3mPnnio/LzFyAlLABZCzKnKJomS2lxrM7N9S2prhk9PZPmjL/8EX8OZyQKGgve9ahv/+tDztEZCVTOAu+NhxrMO/3XXK8qPlYJ8aWbwdNc53yQACyHmVGWTRMl0N9Rm6oirPFF7vsbzNGcmC/g6aDv+0Gsu41WXreLrT5zk6PAkng5qi3sSYUxDNfzcc73O+SA5YCHEnKrdzXZkKMVzZzKMZQpT8qsz5Yvh7BJN1/cZzxQ4PpYNSsoU/PUbruamK1bz9MkJRiYLuH5QUeF4PgPJLBNZp+GNvGb2zM03OQELIeZUKcd7zzee5rkzWWzDYH17hILn86Gv/pS7i8+BmfPFEJxUT45nyRc8Tqfy5c8xFXzmh88RD1vs3t9PW9SmJWQxki5WVpiKrpZQ1Wm69rR9yzXryjnhxZiIJgFYCDHndm7vZff+fjb5uupH/Mrguq9viEPHkvjFlEF3PExr1K7Kw2qteeO16/jQvz9FKudWfYbvw5Mnxnn/3sfxtWZNWxQVUuUlnFrr8op5qD9/Yu+hEzNWW8wnCcBCiHkx3c24UjBUBHW5rqc5OR7Mc8i7HpmCx8vu+Q69iTCxkFUOvqVZD5YBlhk0VyQzDiHTIOt40+ZzmzltLzTJAQsh5sV0izFLwXB1WwRQpf/j5FiG4XSBWMggapv0nU7z30fPALBzWw+WAbYBtmmiUBgqqO91PH/GfG4pl1xpsTvhJAALIebF7TduYTzrcGQoRd/pCY4MpRgv3hQrBcNExGZtewTLUPha42roiQfr5PtHJssBvDVi8eHXXVGcHRGUmXm+Ju96FDyNqzW3XLMO21AcGUozkMzSEqoOttP9hbBYJAALIeaNAtBBPhZ9dkZjvWCoAa0hX/A5NZ6nYqwvEzmX3/n0w/TGQ/g6OPEWPL/8HEvBvxx8njOTBdZ3RNnaGy/f9KvtwFvMqodaEoCFEPNi9/5+WqNBW/Cla9rYuipBa9Rm9/7+cjAcSec4diZDpuDh+cHrkjU320p+Npjm5EQ+OCn71RPP2mMh0nmXVM4lFrLqjrSsHGHZbLvzfJMALISYF9PlXEvBcCzj4BOcfs2673JWcPNNVU06V8XHxrIOjqdxfb/u55Xs3N7L7TduYX1HjOPJDLv39y/qPGAJwEKIeTFTznXn9l58HdxUswyFV+c9VMU/hgJDqXLaQSmI2Ca2aWAQTFGzjOqQVpvjbabxYyFJABZCzItmcq5aa3xNVUqhFJQitoFtqvKWi8qZvrZB1TxhjUZrSESsaT+vshStmc0b800CsBBiXsyUcx1J5WgJW5QWWCiKgZXianlPlwesm0ZxmwWakGXg6+BEXHB98k6Qdti+OsFHbrl62hzvUitFk0YMIcR5mW6YTr2pZ1prTk/k+PP/eJpkJuhUK6UZdHHGQ2eLTVdLmDOTeUbSBVxPY9mwthhcx1wf0wDbDFISnoZXX7F6xilrS2EATyUJwEKIWavX3ls776H0vN37+zk2OklPPIzjaX56agKAF21oR6MZSeWZyLm0hE2648GM3u54pLx6vj0WYiCZwdfQEbMouJqC5xMyDRIRiwP9ow3nCZfcfuMWPvTVn5IpuERts+nNG/NFArAQYtama+8tff3w4ATpvEdHzKIlZPPUqRSFYs3Zm3as532/tI1ViQhKqYYzesezDt/4nzcCZ+f4Vs791Vo3lUaYbhj8YpAALISYUaM0Q6N5D0cGJ8on45zj4/k+I6kCI8op33Bb1xblrldfyhPHx3jf/sc5nswwkXXwfL98AoapKYLzTSOcyzD4+SY34YQQ05qudCsRtnhmKE3f6Qn6h9Okcg5ZJ2gPLp2M825QYOYWqx0UFPe1aZ44Plb13rGQyVCqwEg617CSYcq84cEUz52ZJDmZX9Sa3tlQWuuZn7XIduzYoR955JHFvgwhVqRdew5OOXFmCi4h02A4nWcs45QXXgb5WZu86+MW24Xd6t4I1ndEsE2D1a1RgCnvPZzKkSl4tEXthimCfX1D3PtgH4eH0timYlUijGUaOJ5e9O62IjXzU+QELISYQaPSrf6RSdqiNuvao9imEXSqmYqwqci7fjnPWylkgm0aeD5VQ3kqdcfDtEVt/uuuV/DAbdfVDaY7t/fSHguxqSvG1t4ErdHQotf0zobkgIUQ02qUc4UgENcOQT8ylKYtajGSLlAzsoFExGZ1a7R8qt2wP8azI2lSObeqomFzd3zG62pm+edSJwFYiBVipuWXjTQq3drcFSsPQU/lHIZTeXJuMFQnU/CmBN94SPHoH/9S1WPXb+nkoedGi23GUPB8htMF3vzizhmva6nV9M6GpCCEWAHOZwZCvY62W65Zh1KK585k+NnpCY6dyQQph+LIyXQ+OCErIGQEGywcX035vAP9o/QmQoTMoLstZBr0JkIc6B+d8bqW4njJcyUnYCFWgPNdx1NZulXZfLG+PcKx0WwwzczXWKZBvuKumwYcDaZSdLbYUz7veDJDV0u4quzsQq3pnQ0JwEKsAHOZL60N5qaRBQ0uoGtLHiCIwio43dZ+3nKq6Z0NSUEIsQLM5TqeysoFrTWmoXA1FZPKggosRRDkI7aJaSgGJ/JTPm85pBHOx7wFYKXUp5RSQ0qpn1Q89gal1E+VUr5Sasd8fbYQotpsA92+viF27TnIDfd+l117DrKvb6gczLXWpHLB+5SsaYvga40BmIbC9zVaB/84vj/l85biloqFNG+NGEqpG4E08C9a6yuKj10K+MBu4Pe11k11V0gjhhDnr1QF0Wy+tDLXW1n9cMs16/j/Hh0g73rlUjMFrGkLYyhVHqgTMk1G0nkKno9pKDZ1xnjwfb+wcN/w4mqqEWPecsBa6/1KqU01jz0NVA3REEIsjHPNlza6cfffR89ww9ZuPvfQcTTBSMj/+YqtvPPll2Aaqhy4LVOxubuFM5N5RicdhtN5du05eMHdKJtPSzYHrJS6TSn1iFLqkeHh4cW+HCFWnHpdamHL4KmT4zxQDL6bumLsfedLeVcx+EJ1WuH0RI7RSYeOmM2atuiirwBaapZsANZa79Fa79Ba7+jp6VnsyxFixam9ced5PsdHs6SKNb6lGcDJyQKGUf1T7c7tvTxw23Vs7U2wviNKT3Hc5IXYLjyflmwAFkIsrsobd47r8dxohlQ+WBkftQ22dLeQyrvTnmiX2gqgpUYCsBArXL1KBzibSmiL2PSPTJIpBCfflrDJpq4WQpY544l2LsvflqP5LEN7ADgAvEApNaCUertS6nVKqQHgeuDrSqn/nK/PF2KlahRQGz13uhbltR1RJvJueTlmPGyysSOKZZ4NHdOdaFd6ne9M5rMKYleDL/3bfH2mECtdszvaSs+983OPkSl4hC2D7niY1qhdblFORG3e89lDnBrPYSj4wC9v53s/G2IknceqyCpMd6JdDu3C80lakYVYRpqd+VAK1JMFF8tQuJ7m5HgWCE65RwZTvP3TDzOWdQhbBh9+7eW8/kXr2b46cc5LLS/0duH5JAFYiGWk2ZkPpUBtGwZ5zy/3EZ8az9IetRnNBA0WrRGLj77xhbxiey+GoeREO8ckAAuxjDQ73OZ4MoOpgh1tutjJpoGCpxlKFwBY3Rrh7978QnZc1FnVPCUn2rkjAViIZaTR8PTaFMGGjhiPHUtiGgpDqSnrgy7uaeHWl27io9880tQA99kOe1/ppAxNiGWk2eE2t9+4Bcf30VqD9qsCQchUvHvnxez5r2ebGuB+PsPeVzrZiizECnXTx75P/0gaxwvSDwAx2+Syta3YplF3E3JvIsIDt11X9T6NtibXe+4KIluRhRCN3XbjFsAoB99E2KSzxeY9L7/knDrYmn3uudQnrxQSgIVYgY4Mpfj77x0t534TYZPL1rbxp6+9MthWfA4dbM08V9IU9clNOCEuQOdz0+vR50Z5178eYiiVxzQUf/irl/K26zeVp5lB8zfzmn3u+e6kW67kBCzEBeZ8TpPfeXqQ3/r0wwyl8kRtk4++8WpufWl18IVz21TRzHNlKE99cgIW4gIzm9Ok1pq9jw7wh1/+CQXXpyNm87e/8SJu3NZ41Ou51PvO9NzzXb65XEkAFmIJqU0tXL+lkwP9o1WphnPdcOz7mt37j/KR//wZvoZ17VE+8ZZruHJ9+0J8S8C5pTRWEilDE2KJqN3BdnIsy2jGwQCiIZNExCJkmbSETAqeXz5NTmQdBlM5tIZrNnZU5YNdz+cvvtHHJ3/wLAAvWJVg91uvYVN3fFG+vxXUwry4O+GEEOemMrWQyjkkM07wBQWupzkzWaCrJUTMNsojHl3P58RYDoB17ZGq6Wcv2dLJXV98kq8+fhKAn9vUwT/+5rV0J8KL8v1JC/NUEoCFWCIqUwvDqXy5PldrgpU/PqRyLrZp8OFfu4Ld+/s5dCyJZShWt0VIRILXZgou933vGf7pByY/eGYEgF++fDV//cariIftKZ8rbcSLR6oghFgiKutpC55PqTBBVfyad33Wd8TKO9d6EmEu6Y2Xgy+AZSh+PDBWDr5vvW4jf7/rhQ2Dr9TnLh4JwEIsEZXbI2xDlQOvoUCj8bTGNFTVjavaJoic43F0eBLH0yjg935xK//PzVdgWyb1VKY9ZGnmwpMALMQSUVlPGwtbWIZBR8wiZBo4no+hFHfsvLgqPVAZtNM5h/7hNK4fJC/aYjY/fOYM+w8PN/xMqc9dXBKAhVgiKnOxW3sT3LHzYravbqMrHubFm7rY/ZZrufNV26peUwralqF47kyG4uo2VrWGWd8eZTidnzalIEszF5fchBNiCai3y23voRMNu88qnZ7I0T8yiQZsQ9GTCNMVDyodZmrSkPrcxSV1wEIsAbMZ6ai15u+/9wwf/eZhNLCxM0bO9eiNh6s2WExkC5yeyNOTCNetciidvI8MTlDwNCHLYGtvom4TiFRHNE3qgIW4UBwZSpHJuzi+JmQa9CTCxMNW3ZGOu/f38/yZNJ6GwYk8AJevbeWf3raD3/v841WBfCLrcGIsh2U03pJc+vVDX/0pbcUmkGdH0jz03Ci9iRBdLeFptyuL2ZMcsBCLbF/fEKlcEHxNQ+H6mpNjOc5M5uuOdDw9niGd98rB97I1CT77jutY0xatuimntWYwFTRprG6LTFvlUFsNkcq5GAomsq5UR8wjCcBCLLLd+/vpiAU1utovLcjUjE46U0Y6GkpzcjzPRM4FghI11/VoK76+djKZ1kGHXGWdcL0qh9pqiFIdcuWuOKmOmHuSghBiEVRWPAyn8qxuDbO2LcpIOk/B8wmZBlE7OB/t2nOQ48kMg+NZfKAUE00VBOujZzLs6xuqSimUfl/KLVeqV+VQO60sZBrl65judeL8yAlYiAVW232moDzPYUtPnHXtUTxfM55zuf3+R3l2JE3EMnD8s8HXMhS2aWCaBrZhNEwN1KYkMgW3bpVD7fMSEQtfQ2vUmvZ14vxIABZigdXmW1e3RQAYTOWYyBYYSGZxfY3SGl9rhlN5+kcmq97DMhWaYE7EqtZww9RAs4PVa5+3uTvO777iEjZ1xWccyC5mT1IQQiyw2nm+iYjNunbN6Yk8pyfyWKZiVSLCyfEsaF1urjBUMefrg1dRLWEait5EpOHnNTuFrN7z7pzdtyiaJAFYiPMwm0li9bZDWKbBNRs7ysFZKcWp8SyFs/fA2NzVwmTBJZlxWN8Rbdg4IdPNLhySghBilmY7SWy6vGypNXhwIkfBO9skFbGCO26ur+lNhBlIZjkylMY2VFVqoPaanjuT5vb7H+XaD39TVsEvQRKAhZil2U4Smy4v+44bNjE4kWcoFdT42qbCNiAetoIJaVDsVIuzviNKxvGr3rvymtJ5lzNpB19rco4voyaXIElBCDFL57qbrVK9fGvOcfn8IwOk80GNb8QyuGpdG+9++SXs3N7Lrj0HcXw97TLO2qHuSoGBKq8wklXwS4sEYCFm6Vw2/c6Ulx3LFHj7Zx7h0eeTANxyzXr+4vVXYlsG+/qG2LXnIA89N0rEMuiOh2ktBtnagF95TQXPxzQU2qdczyvNFEuLpCCEmKVma2xnyhWfTGZ4wycOlIPvu3dezF/ecjb4ll4bNoOT7MnxLBPZYF9cbcCvHeru+RofTXdxOpo0UywtEoCFmKVma2ynyxU/fWqc13/iAEeG0hgK/s/Nl/OBm7ZjGMaU1/a2RlDFIVsj6XzdgF871N1Qiq6WEImIJc0US5CMoxRint1w73fLpWUTWafcbqy1JmyZTBY8DAXtUZsXrG6tSk9UvhYglXMYmsiR9zQv3tQ5Y4nZClsFv5TIOEohloJSXtb1NCfHsxgofD9osHALHgpY3Rqms87Yx9o8cyJilxsvGs0JriSr4Jc2SUEIMc9KednBVA4FuL5PRYkvlgld8frjIpvNM4sLkwRgIeZZKS/r+xrHO9tabJvBT6mOB/3D6fKNtcpKhWbzzOLCNG8pCKXUp4DXAENa6yuKj3UCnwc2Ac8Bb9RaJ+frGoRYKn5+Ww9tsRDDxQaLsGXgFUebKSinJyAYtFNZqSBphOVrPk/AnwZuqnnsg8B3tNZbge8U/12IZS1XcHnHZx4uB99YyERpH2UoLENhKKA423cwlZMUwwoybydgrfV+pdSmmod/DdhZ/P1ngH3AXfN1DUKci/kYYpOcLHDrpx/ix8fHAXj5th4yBZdHjo2VmyqUCrrW8q6P0qpuikEG7CxPC10FsUprfQpAa31KKSV/gsSSUGp4KLgeqZzL6fEch44luWPnxdz5qm2zes+BZIa3fvIhni3O8n3dC9fx9Klxnj2TKeaDgxREImKTiNjlLcj1gm/tynpZkLk8LNmbcEqp25RSjyilHhkeHl7syxHL3O79/RRcjzOTBVxPYxkKX2vu23d0VsNrnjo1zuv/4Yc8OzKJZSh+8dJevvbkSfoG0xRcHwUUPM1AMsNEtjBtdcNsh/6IpW+hA/CgUmoNQPHXhn+ytdZ7tNY7tNY7enp6FuwCxcp0PJkJNgGjMAyFUqq4odg/50D338+M8KbdBxlK5YnaJu/ZeTH7j4zgFMsfNOATDFf3NZyeyE9b3VC7MBNkpsNysdAB+KvA24q/fxvwlQX+fHEBKw2lueHe7875bNsNHbEgB1vRv6Q1hE3jnALd1x4/ya2ffphUzqUzFuKzv/MSfvRcEtcPTr0Kyp/h6yAYA9PmdEszgivJTIflYT7L0B4guOHWrZQaAP4EuAeRXwNjAAAgAElEQVT4glLq7cAx4A3z9flieZnvPOjtN27h0LEkntaYKgi+WkNbiz0l0O3rG+LeB/vKe9o2d8X44Ksv5dmRST789afwNazviPIvv/1itvTEOZ7MEDYNstoPIm5FE0YpIE/3vdx+4xY+9NWfkim4DbdgiAvTfFZB7GrwpVfO12eK5asyDwr1Z+Gej53be7lj58Xct+8ojucTNg3aWmxs05yy7uf9ex8nmXHQWuNr6BtMc+unHy7H1UvXJPjMrS+mtzXY07ahI4bn+xRSBdya2SuGEex/s4o53Xrfy87tvdxd/G8gMx2WF5kFIS4I5zP8vFl3vmobV61vnzbQ7d7fTyrnArqqnbgy+O595/W0hM9ea+kE25MIMZoukC++0DJgXXuU1qiN1nra70WaMZYnCcDignAuw8/PVaMa29Ljf/SVn5QfP57M4Po+rl//vXzPrwq+UH2CtYwM41mHWMikp2KTseR0V6YlW4YmRKX5GkrTaFj6x799uO7jibCFZTT+f5vnRrN1P6MU4Nd3xHjHDZsJWaYM2BEyD1hcOOZjtu2uPQennKwzBZfhVJ6eRHjK47ahGJzIkcy6Dd/z+i1dVafo0s3Dyhtot1yzjgP9o9N+L9L9dkGTecBieZmPPGij3PJkwWNjndrboVQOd5oziwKeHUmXqxoa3Tw80D867Txf6X5bGSQFIVa0RjW2LSFzyuNn0nlG0oWgYUPVP+JoYHAiTypXKJ9eZ9NEId1vK4MEYLGiNcotv3J7DwPJLE+fmqB/OM2xM2lOTeTxNfTEw3THQ2zsjAaTzGpoYCzjcmQoNesmCul+WxkkAItlq5nOuXoDz2+5Zh2PHhuns8UmZCoyBY/xXBBEN3XF+NK7X8rFPQks08CsF4EJgnDB9Wd981C631YGyQGLZamUQ3U8j/GMw6nxbMPpZrW55V17DmKbitZImIKryTgFIJjdkHc8PrD3Ca7f0sneQycwDVWe8QBBWkIXfw2ZatZNFNL9tjJIABbL0u79/Tiex5m0g1JgmwaeH0w3u2p9+7QB8HgyQ1vE4ngyw3ix2qE0x6ErHizO3HvoBLdcs47/ePIUfYPp8mtLwbctarF1VSswu5uH0v22MkgAFhe8euVax5MZxjNB8DWK029MBa6vZ2xfXtsW5YkTY+ScoNvCKibqwpZZviFWqmR48H2/wMe/fZj79h3F8zVhyyARsQhZ5nmfVqX7bfmTACwuaI3KtRJhi1PjWWzz7G0OrYNdbNPdyBqayHFyPFsOvt3xEKOTBUxD0R0Pl59XeUOsmRZmIeqRACwuaI3qbLXWWEaQdihNN/PRJCJTp5uV9A+neesnH+LEWBYFXNQVw/M18bBFLGTSWlEvXHtDTE6rYjYkAIslbaZusHqNFK7nM5DMEzIV6YKPhyZqmyQidsPUwGPPJ7n1Mw8zlnEIWwbvuGEzh46NcTyZYV17lOF0Xm6IiTknAVgsWc10gyXCFs8MpfG0JmQatIRMRjMOlqG4qKuFM5N5RicdorbB5u543dTAd54e5D2ffYys49Easbjj5Rfzrz86Xv7crOOhANtQjGcdSTGIOSMBWCxZM80A/vi3D/PMcBrH0yjA9z0yBQ9Dweq2KEopuuMRYiGL3kSkbuvv5x8+xh/820/wfM2q1jD/72+/hD8pBv3KzwXoaAnz4Pvqtw/L3AYxG9KIIZas6brB9vUNcd++o0BwMgXwiit+LEORiNhTXlNJa83ffecId33xSTxfc0lvnK/c8TK2rU6ccxdao4lqc7kySSxPMwZgpdR7lFIdC3ExQlSarhts9/5+XN/HNBSWaRCxTcKmgaGYsnWi9oaZ72v++Cs/4a+/dRiAay/q4Evvfimr26Izfm49MrdBzFYzJ+DVwMNKqS8opW5SSjU1Zk2I8zVdG29pz1plrFXFNjTLMBq2/uZdj3fe/yj3HzwGwC9euorP/s5LaI3YTX1uPTK3QczWjAFYa/1HwFbgk8BvAUeUUn+ulLp4nq9NrHD15jSUVrdv6IjRFrOD8jKt0Wg8rbEtgzt2Xlz3NRM5h9/8vz/im08NAvDml2xk91uvJWyZTX9uPTK3QcxW0wPZlVJXA7cCNwHfA64DvqW1/sD8XV5ABrKLWrWzHvKej2UYU2Y9lG6OPTuSZjzrlgPl7/3iNt77ikuYix/oGg1dny5oi2VvbgayK6XuBN4GjAD/BLxfa+0opQzgCDDvAViIWlWzEoz63WelwOj7PiPpAq4fHDZufekm7nzl1vm5FumEE+egmTK0buD1WuvnKx/UWvtKqdfMz2UJMbOZus927+/H9TwGJwp4WqMUrEqE6TudWvBrEaKeGQOw1vpD03zt6bm9HCHmzs8GJ0hOOmjAVIpNXTGiIbk5JpYOacQQy9L9B59ndNIBwDYVm7paiNjBJmK5OSaWCgnAYsk5n64yrTUf/dZh/u67zwBB8F3TFiFsGQynciQzDuNZh117DkqeViw66YQTS8r5dJV5vuauLz5RDr7Xbenk47/xIta1xzg9niWZcehssVndGpFuNbEkNF2GtpikDG3l2LXnIEOpXHn+AkCm4GIbio6WcMNTcc7xeNf9j/K9nw0D8KtXruFjb3ohoeI09UbvW5oRIbMcxBxrqgxNTsBiSanXVeZ6PkeG0w1PxcnJPG/afaAcfFvCJmfSeX74zMi071s5V0JmOYjFIDlgsaRs6IhNOakOpvLYhlF+zPM1QxM5br//US5bk+DkeI7BiTwAnS02a9uC+b2VoyvrvW/lXInppq4JMV/kBCyWlEZzGFa1BuuAUjmHk2M5fK1xPZ8nBsbLwbcnEWJde6zuQJx67zuRdUhO5nnouVFOjWVJ5ZzydcgsB7EQJACLJaXeHIZtvXGs4m634VQepYKJZp4ORlAqgsWZqxKRqveqDKK17xsyDTTg+JqIZeD4mpNjuXIQllkOYiFICkIsObVdZaUcbabgknc9dDHwApiG4qLOKCfGcmQdr26Kod777tpzkILnEwtZdMfDnBzPoglSG6ahZOWQWBASgMWimanyoPLribCF1hrfB7/4dQW0hk1Ojufwfc1AMktHzKY7Hp5xb1vlLrnSss2RdJ6c69ObiEgVhFgQEoDFgioF1SNDKVI5txwwa/e91e6DyxRcTo3nqoIvQDLrYipY3xGl4PmMTjq4ns/WVa3TBtHam3KtURvLVA1XFwkxHyQAiwVTGVQzeRdfa4bTBZKTBXyCdMI933iandt7qyoTJrIFBpLZctohZCosQ5F1gnBsmwat0RDAtPvfKt1+45ZyWkM2HYvFIgFYLJjKoOr4wSJNz9eURpm7nubwUJp9fUPlFMF4tsDx0SyV7UJaQ29rhBNjWQwFXkUzUWVt73TpDRkhKZYC6YQTC+aGe79Le9RGKUX/cJpMwaPen74NHVHWd8Q4OZbhWDJL7R/RsGVgFRdxFjyfkGmwpSdOKudwejyHpzWGUuX0xkg6TzLjkIhYbO1NSKAVC0E64cTSUrm6Jx626gZfgOPJLJeujjMwlisHX1MFKQpDBQN38q5HImLha2iNnk1TuL5GaY2vNWcmCwxO5DgzWcDXmkzelS43saRIABYLptQMMZLOMZLOT/vcfzl4DK+4wcIAIrbJ+vYoGzpiGIbCNAw2d8f53VdcwqauOKcn8limYl17FK1UEKxRnJksYBD8u+Nr2VgslpRFyQErpX4X+B2CY/r/1Vr/zWJch1hYpbzrnZ97rHxDrRHX18TDFnfsvJgHHj5etW+tdknmnVSnN0bS+eAkrMDXwbZkrSFUbOaQLjexVCx4AFZKXUEQfF8MFIAHlVJf11ofWehrEQtv5/ZeWotlZYZSFBpEYttUfOH267hsbRuXrmmd8WZZZVlZTyIctCujyzfpFIqeRNDOLF1uYqlYjBPwpcBBrXUGQCn1feB1wF8uwrWIOXIu4xw3dMQYSeXRBC3EpWWZJZah+IvXXclla9uA5vatVZaVxcMWXXGb0UmHSEhR8IIhPfGwVZ4tIeVmYilY8CoIpdSlwFeA64Es8B3gEa31e2uedxtwG8DGjRuvff7552vfSiwR57qWfV/fEO/f+zjJjINC4/qUb8hFLIO/uuUqXvPCdVNeM1OALz2n9qTc6HEh5lFTVRCLUoamlHo7cAeQBp4Cslrr9zV6vpShLW2Nhp2HTIP2WKhu0NzXN8Q933iaw0NpSgfgq9a18YV3Xk+kZm7vuQZ4IZaApVuGprX+pNb6Gq31jcAoIPnfC1ijIeqHhxoPUb92UwdtsVA5+PbEw5yZzHPrPz88pUSssoGj3qhJIS5UixKAlVK9xV83Aq8HHliM6xDV9vUNsWvPQW6497vs2nOw6VrZyvreksGJfMOgOTie45ZPHOBHz44CwVFhIlfAMlTdOt3ptlkIcSFbrDrgLyqlngL+HbhDa51cpOsQReezlqfuEHXfZ1Wx6qAkapv0D6d43T/+Nz87nQLAUkHFA1pxcjyH6+kpp9t6AV4qGcRysFgpiJ/XWl+mtb5aa/2dxbgGUa2ZH/MbnZDrDVHf2nN2iHrJ6GSeM5PBRgulYFVrGGUojNI/BDW8tafbRlsypJJBXOhkGI8AqufjllQGwtrxkLXjI+sNUf/9vY9zYiwbdLRpTXF4GW1Rm5Cp6ImHSefcctOEUsFsh3qD1GVwjliOJAALoP4yzMpA2MziyspSsXjIxPGC+jLX0+Uys46Yzedvu44/+epTDKVyVU0TWuuG2yiaqQUW4kIjsyAEMPOP+Y1uhB0ZSrFrz0F2/Om3uP3+R3l2JI2p4GeDacazLjnXLwdf21Rs7mph2+rW8ueZhmJNWxitNQVPk3N8To1luffBPhmYI5Y9CcACqJ/HvfvmywG46WPf58RYlp+cnODIYIqJbLC48sxknlQumDBWGrA+lMpzPJmtO+lsVSLEcHEIT+XnTRbHUhoEw9aVgiNDad6/93EJwmJZkxSEKGuUxx3LOJgKXA0512dgLEOPGyaZceiI2eUB66ahqtINtUYmHV60oWPK5+3ac5DHjicxCG7GASitSeWqUxxCLDdyAhYN7d7fTzrvYiqFbZqETAOlwPMhU/CIh02640GpWcg00JqGwRcg5/gkJ/NTTrVPnkiSc3zyxRtwecdDKXB9X2p9xbImAVg0dDyZwStWKEAwED3YRhFUMmxb1Vquz+1JhPE8f8p7VPZjhi0Dx9dV9cUf//Zh0vnq1/lAwfGxDENqfcWyJgFYNLShI4ZpKLQOdrflXY+c4+P5wUaLyht3lmLKjF9DVf9+dWtkSn3xP/3gWcw6fwp9IBGxpNZXLGsSgEVDt9+4hXjYwvV9Cp5fnttgKMo30+6++XLClsHRkUw5/VCqlfCLKYmwqdjQEaO1WGdcWV88WfCwDFUell7prdddJPlfsaxJABYN7dzey1/dcjUhK/hjogjGRa7viNEWtdm9v5/Jgsczw5NogkYKS4FlGURsA7tY0dDREioHX6iuL24JmUGgrpnKp4C9h05IFYRY1iQAi2nt3N5LVzzMFWtbuWJdG1tXJWiN2kRtk6dPjfPeBw5RcH1MQxEywDQNDKVQKEylMIBkxmlYX/yOGzbj67ND2UtZi95EWCaeiWVPytDEjGq75LTWHE9mGM+6AGxfnaAlZPLjgTHsir/StQ6WaUZDJr2JSN024jtftQ2Aj307mEiqFPTGw/S2RtBa162COJftG0IsZRKAxYwq1/1ELIPnRzOk80H1w3WbO/mnt+3gkeeS3H7/o3i+xiwuwfTRJCI2m7vjPHDbdQ3f/85XbeNA/+i0rdAlM82kEOJCIikIMaNS11pnLMTR4cly8P2VK1fzL29/MfGIzc7tvdyx82IMpXB9jWUqulpChCyzqUqGZieeyXB2sZzICVg05ar1beRcn5wb1Oy+7aUXsXNrD2/71MNVqYDdb7l2VlPLmp14NtPUNiEuJBKAxYyeH5nkf3zqIZ4fzaCAD9y0nUtXJ+qnAm6+fNp0w3SamXg209Q2IS4kkoIQ03pyYJxf/8QPeX40g20qPvKGq3jXzosXLRUgw9nFciIBWDS0//Awb9j9Q0bSBRRwcXec7pZg9sNi7WlrNLVNbsCJC5GkIERdXzo0wPv3PlGuarioK4bj++WKg8pUQCrnMJzKk3M9WkIW+/qG6g5pn6uSMRnOLpYLOQGLKf5x31H+1xcex/M1lqHY0hOnJWxXpRlKqYCRdI4TySwFz8dQiljILA/bOZ9Fn0KsBHICXmH29Q1x74N99I9MArC5K8YHX30pO7f3orXm7q89xT//93NAsMGiNx7i5FgQYEOmQXc8xEAyU65auPNzjxXnPRh0x8O0Ru3yqqLSe0y3xkiIlUwC8DLRzI/6+/qGeP/ex0lmnPKksmeGJ/n9vY/zF6+7gi8/foqvP3EKgMvWJOgfSnNiPI8iCKSurzkxlmNrbxwIUgGtUZuNnTGUOjv6rJQL1iAlY0JMQ9UOQVmKduzYoR955JHFvowlq7I7LGqbZB0Px9NTbk7t2nOQx44lg/U/xYDp+xqNj2WYZIqzfRNhk5CpmMi55U3GAJah0FoTsgy64mE2dMQYyxQoeH5VWVim4NKbiABMKRkrfW22pWpCXCDUzE+RHPCy0GxJ2PFkBtf3qTisorVPwaMcfLtabAqez3jOw0dhG2fn+vpaozU4ni7ndIfTeSay9YftSMmYENOTFMQ8WciBMc12hyXCFif8YNVPPatbw/QkIvSdnsBQwYQywzCwTIVGk3d8LFMVVxOdze3ahqKjJVy3g62Z7jYhVioJwPPgfAfGnGvw3tAR47kzaSaybvlmWWvUYlNXvOo9h9N5jDqbKyD4eSlsBXW9IdPAKa4X8tHgg+bsss3SHjgIAv141uHB99VPKUjJmBCNSQpiHpxPl9hsSreu39LJUKpQLAWDguczlCpw/ZbOqmtqi9p1W3ZtAyxTMVLcctEdD+PrIBCvbYugjCBox0ImXdMMVxdCnBsJwPPgfLrEZhO8D/SP0hMPETKNcuDsiYc40D865ZpKg89LTAWdLWEcTzNZ8OgfTpN3PaK2gWEoBsZyhEyD333FJfzDm68hZJmS0xVijkgAngcbOmLlbcElzZ4UZxO8jyczdMfDbOmJs311K1t64nTHw1Wv2dAR48RYlhNjWSBIOdgG2KZBMuNgFm+25VyPM5NB6/GatgiXrk7Qkwiz99AJAGkDFmIOSQ54HlQOMK8sC2vmpDibaV8zvcbzfMJWEGgBQqbC9TSaoKysVJa2tj1Ka9TmyGCKgq/rNlA8cNt1EnCFmCNyAp4H5zMwZjalW9O9Jltweee/HmLf4WEgaIzoiYfYtirO1t44bjnXGy3ndl3fx6tJVUgDhRBzT07A86T27v++viF27Tk4Y2XDTIPJG1VI1HvN1RvaeMsnH+LR55MA/Po167jn9VdhW2f/3t215+CU07NlGFPKyOVmmxBzTzrhFkCznWpz+T4DyQy3/vPDHBlKA/CunRfz/l/ahmFU/9Dz8W8f5r59R3F9n7Bp0Bazi+kJaCtuP57t9QqxgjXVCScn4AVQWdkAsx9K0+z7PH1qglv/+WFOT+QwFPzxay7j1pdtnnJ6vn5LJ3sPnaAjZpPKueRdn9FJhzt2XsxV69vZvb+fI0MpCq5fVYkhQViIuSE54AUwV8PLm3mfHx4d4Tf2HOT0RI6IZfA3b3phOfjW1hfft+8ojufRk4iwpSfOpWtaWd8R5UD/KDu393L7jVuIhSx6EmHWtEVlnKQQc0wC8AI4n7K0c3mfrz1+kt/+54cZzzq0RW0++Vs/x80vXMe+viHu/NxjnBjLcHo8RzrvEgtZeL5mvFgZUVIZ0GUDsRDzSwLwAjjfoTSlG3iHBycYSGYZSeemvM+nfvAsd37uMXKuz+rWCJ99x0t42SXd5ZNvpuBhGcFIyZNjOVI5h7BlkPeq50JUBvTFWjskxEohOeAF0OzK9Xoqb7ytaYsyks4zOungeJqtvQl+54ZN/OjZUf7x+0cB2Nob55Nv+zk2dgVBtHSKDVsGrqcxDIWPZjiVJxGxcDO6Yb2ybCAWYn5JAF4gzQylqVdiVnvjrScRoSVs0ZuI8JnffjEf/NITfKnYpXbtxnb2/I8ddFUMyylNSuuOhzk5ngUfUJqc69Nlhblj50YO9I/W/Yvh+i2d3LfvKJ6vCVsGiYhFyDKl9ViIObIoZWhKqfcB7wA08CRwq9Y61+j5F3oZWjMqT7qu5zM4kafg+fg6yBNFQyY9iTCJiM1EtsDJsSyGYZB3gxTCCze0Y5uKU+O5qvrgyjrfiazDSDpP3vWJhUw+/hsvaviXQul6HM9jPOOQ93wsw+COnRdz56u2LeB/GSEuSEtzILtSah1wJ7BDa30FYAK/sdDXsdSUTrqerzk1nsfTmlIzmg9MFjwGklmGJnIcH83i+JSDrwKOnB5ndLIwZYJaZf45EbFY3RZhbXt02uBbeT3d8QgX9ya4bE1buUJCCDE3FisFYQFRpZQDxICTi3Qd82I2w9hLqYJnRyZRChx36k8mrq8ZTOWrHjOL830nHY3raVRITZnd0Ez+ufaaDw9OsKYtWvUcuQEnxNxa8ACstT6hlPor4BiQBb6ptf5m7fOUUrcBtwFs3LhxYS/yPDQaxn7LwBgH+kcbBuXSDa+C52MaimYSQ7ahMA2FVzwJj6Tz5XkOlcFypvxzvWtO5z1G0nl6irvdQG7ACTHXFiMF0QH8GrAZWAu0KKXeUvs8rfUerfUOrfWOnp6ehb7MWdu9vx/H8zg9nuNng6mg7jbncN++o9MOWS+lCkyl8P2Zw69tgGkolFLlZFOhoqTsXIJlvXrfzhabZKb+rjchxNxYjDrgVwHPaq2HtdYO8CXgpYtwHfPiyFCKkVQB19eYxbrb0YyDW9wc3KihoTRBbXN3C55unME3FFgGoBSoYFGmaShMFQTk2QTLevW+XS1hEhFLZv8KMY8WIwd8DLhOKRUjSEG8Elg2JQ4F1wd1du17aQNx7Zm2Xj61lCrY1zfEvQ/20Xc6NeV1u35uA6taI8U24mCATmcihOtpulpCjGedc15+2ajed2tvQtbHCzGPFiMH/COl1F7gEOACjwF7Fvo65ottKrIO+L5GKdDF02xttV8pRdDoht3LLunijZ84wGMD40DwHt1xm/965gx333w5u99y7ZxtGz6fAfJCiNmTcZRzbNeegzw7kiaVO7uhOAjKPus7olUB7pZr1rH30Ikp4yX/8NXb+fcnT/G1J04BQdpBARHbLG87nuuTaekvAlkfL8ScaKoOWALwHGs0s/eWa9ZN6Tjbvb9/yo/+E7kCyUmHiZwLBP8rWgaYpoHWwcm6LWbzyB/94qzK3YQQC0IC8GJp5jS5r2+I2+9/FF9rQqZBdzxM2DY4diZDzq0ekGMoMJXCMg0Kngco2qMW6bxHZ4tNV0t4SqCXoCzEopIAvFSVZ/NO5PC1RqFwfR+lVHltfFvEYiLnVt2EMwi64kqtyQXPR6FY2x4hEbEZTuVIZpwpqQ6pXhBiwclGjLk0mx/3G72mVHfbFrUZSuXRpTBb/MtwdWuYWMjE9TV518cvtiX7BKfhqB0EX1MFgfvYaCZoyPA1Cs5784YQYmHIPOAm1NsmMdNmiOleczyZwfV8RicL5TK1ks6YjVOsGe5JhDGUwjYNwlbwRNNQdMfDhEwD1/NxffB1kKLwddCWnMqdHbIu7cNCLF0SgJswm80Q9V5TcD3e9a+PcmIsy/OjWRz/7MCdksmCSzrvcWYyTyJis7Y9Uh6kbhqKrpYQrcXxkl7xtYYK6owVYBowXDEvQtqHhVi6JAXRhNKgnEr1TpaVKYfhVJ7VrWfn8k5kHYZS+eJpFaoXCwVsIzjNdrXYjE46xEIW8bCFaaiqsrXSZLNSnbGhFJahaIuHSGYdcq6H1lrqeYVY4iQAN6GZzRC1A21GUnlOjOUARWvUZiSdn9KMUck2VBBIi9srPN9nIJkFYEt3C3/8q9vZub23vK14IJkhHrZoCZt0x88OzDEMRabgzaojTgixsCQAN6GZTrHazRWr2yIMJLMMpnIkIhZ51y+nCbw6gdhQwaqgWMjixFgOyzC4pDdO1vGYLJw9L1dONju77+3sdYUsk3tef5UEXSEuAJIDbkJpUM50g2lqB9okIjbr2iNoDeNZh4gVTC0rxd6gtvdsrYqvfQylGE4X8HxNa2TmfHMz1yWEWLrkBNykmWbq1ktTWKbBNRs7+PBrr+C3PvUQ6UKQUlBQvvlmm4odG9s5dHwc1w8aMEwDklmHaMiiNWpPW8nQzK45IcTSJCfgOVJv9XzB9bmoM8qv/O1/MTAWBN+odXbYethUJMIWB59NUnB9IpZJxDIwDQMDxUg6qGaQSgYhlicJwHOkNh1gKcg6Lp97ZKDYsQbd8RCehjVtYa5c18aa9ijpvBcEZBWsHHJ9jedrNJqC58sgdCGWMUlBzKFSOuDbT53mA3ufYDRztiHCUBCzTcYyDuMZh+54hOFUvjRXvVxOhnE2L6yB3kREKhmEWKYkAM+xvOvx4a89XRV8Q2bQKXFmskDYMsi5QVVDaf9bqY3Y9zUojeNr1rXH5IaaEMucpCDm0GTe4WPfPMzzo9U3zBwvmPZQ8HwSEQvLMMgUXOzS/AYVpCcsU+H50BKyJPgKsQLICXiOjGUKfPhrT/HFQyfKjxmcLTtzfI1lKJIZB9sM2oUt08DTPh2xoLVYppcJsbJIAK5Q2sXWPzIJwOauGB989aUzBsPB8Rx3ffEJ9h0eBqAlZJIteNimga91ecSk62u646GqYFtvULsEXyFWBpkHXLSvb4j3732cZMbBKN4F8zW0x2z+6parq4Livr4h7vnG0/SPTKK1xjQNck5Qw/u6F63jYP8IIdNgJF0oryVyPR9Xay5b01Z+n0zBpTcRkcWXQiw/Mg/4XOze308q5xZXu4Pr+/gaRicL3PONp8sB+OPfPszHv8XWD6EAAA3HSURBVHuEyqUVTrGB4tVXrOZPX3s57/jMowylcmzpiZef8/SpCcJmdco9apscGZxg156DssFCiBVoRdyE29c3xK49B7nh3u+ya8/BunN8jyczuL6P1hrH88uDc3wNR4bT7OsbYl/fEPftO0rNxqCy/uE0LWG7blOGaSjaYtUT1UbSeVJ575zmDAshlo9lH4CbHaa+oSOGZRg4pR7h4g8QSoFtGOze38/u/f04XoPoCzwzPMm+vqG6Mxru2HkxtmlWBeVkxqGzxT6nOcNCiOVj2acgaqeUNVrTc/uNW3j/3sfJpQvBA6Vh58Cq1jADyQye1piKKUPUIYjXpeBZasioTSVUjpJc3xFjPOvQ1RKueo5ssBBi5Vj2AbjZYeo7t/fykVuu5t2fPUSm4AXbJVQwUOfEWI6obRC2TJwGB2DTUKxKhKcNnrVBedeegzPOGRZCLF/LPgWxoSNG1qneP9EoyO3c3ss/vPkaNnbGWNUaRhkKT2u0r/G0ZmQyOB1bNf/VbFOxrj2KZRrnFDzr5Ypl7oMQK8eyD8DnGuRK+dtMwcPXGktBJGSSKQRH36htcMXadrpiFmHLYE1bmBesSmCZ6pyDp8zzFWJlW/YpiJ3be7kbqnKvM5V67dzeSyJisao1zMmxXHkjRSJsEg9bfPk9LwPO7oA7nyYKmecrxMq17AMwnHuQ83xNV0uIvtNpCsWqh46YTWvEYm372RSDBE8hxPlY9gG4clNxM40Ojufz+PExTk/ky8G3Ox4iEbbwNJKfFULMmWWdA262Brgk53j84Mgw77z/UYZSeUyluKgrRtgyWN0WlfysEGJOLesTcL0a4JF0jjs/9xitUbvqRDyZd/nWU4P80Zd/QjrvEguZ/Nlrr+DVV64hUrFsUwgh5sqyDsC1NcCpnMNIqoAGNnbGyifiu/IuI5MF/vTrT+F4ms6WEB/59av4+W09hGprzoQQYo4s6wBcu6l4OJUHBWHTKLf+pnIOf/GNPk6MZdHA+o4oH33j1VyzsQPLlOArhJg/yzrClGqAR9I5jg6lmCx4uP9/e/ceI1dZxnH8+8zMXqbbXXZ72dLLlrbQK9VKqYpoaqGKWCM1sRpIEDUodwgSLxj8wwDBiAETEhIhakATapUINAYSREAUKFBsuRTohZba7W172e62e5+Zxz/OtCzrXmZ3Z+bM7Pw+zaYznbNnnqcz++yZ95z3eZNOVXkUd6c7kaS1o4fGdPFtqIszYVwZP1i3mW/97lU1xRGRnBrTBXjFgnrWLJ3O0bYeupNOxILmOs3tPTS3dbG/pfPU2m0NdXEOtHSwZV8rx9p72HX4hDqTiUhOjekCDPDyzqPMqIuzcGoNDXXjiBik3Nnf2kVrZwKAuZOrONASHAWXRSMkks6Rtm66E0l1JhORnBnzBXhPczvx9FUM4ytjTKyqIMWHHc0unF9PTbyMFEHjHTMjEjEiGMc7E+pMJiI5M6ZPwgFUV8TY0XSCRCqVXv6dU83WZ9TGee9AK0fauolZsBKG9eoD3JVIqTOZiOTMmD4Cfv69Jg6d6AqaqDt0JT5cILOqPEosakyoKscsWDAzmXJS7jhB97NoxDTzTURyJu8F2Mzmm9nmXl+tZnZzLp7rgRd2Mr4iRu24MhK9mqiXRaG+poLqyjLMjCnVlcHQgwU9gHuSKSJmXL/iTM18E5GcyfsQhLtvBT4BYGZRYC/wWC6e64MjwarFR9qCKx1iEWNabSWNzR1U9WqCXhMvA5wDrV1MHF+h5eFFJC/CHgNeCbzv7ruzudNUymk63oUB+1u7ACiPRpheV4kB4ytidPQkP7ISRSwaYenMOi0RLyJ5E/YY8KXA2mzuMJly9rZ0cO/ft7KvpROA8liEhro4BiRS8L3Pzf5Ik/ZDxztpbO5ge9PxAVdN7iuTlZZFRAZj7v2sMJmPJzYrB/YBZ7v7wX4evwq4CmDmzJnn7t499EFydyJFY3M7dz35Ls+8GxTEs6fWUBYxDrd10TCh6tTQwsk2ldsPtnK8K8mEqjImVlXQ0ZOkJ+mDdj472WWtLGrEy6IZfY+IlBTLaKMQC/Bq4Hp3v2iobZctW+YbN24cdJvOniS7Dp/gZ49v4fXdzQCsXjKNH35pPtNr40Qi/f9/nFwYM5lyDh0PegBHzZg9qYqnbl4+6Pf0HsJo705QX12pIQwRgQwLcJhDEJeRpeGHtq4EW/a1ctPazaeK73c/O4ufrlrAjLqBiy8EEzUSyRT7jnWSSAWXnqXc2dZ0YsBhhd6TO07ScvIiMlyhFGAzGwd8EfjraPfV0tHD67uPcsMj/2F70wkiBj+6aB7XrTiL00+LYzb4L6KGunEcbO3CDCJmnPxTFrUBpyEPZ6VlEZGBhFKA3b3d3Se6e8to9nO0rZsXdxziprWb2d/SSUUswp1fW8xlnz6DydUVGe3j6uVz6EmlcHfcnVTKSeFMqa4Y8IhWy8mLSDaEfRXEiLg7Tcc7eXrLAW5Z9wbHOnqoqYxxzzeW8JWPTWNCVXnG+1qxoJ65k8cTiRhJd2JRY9ppcWLRyIBHtFpOXkSyIezrgIctlXIOHu/kiU17+dXT20imnCk1Fdy9ZglLZ9ZSXVk29E76uPXLC/u9qmGwI1qtiCwio1VUBTiRTLG/pYOHX9rNb/+9C4AzJ1fxy69/nEXTaj5yVcJwrFhQz+0EU5cbm9s1E05E8qJoCnB3IsXe5g7ue3Y7j23aC8A5M2u5Y/VizqofP+qFM3VEKyL5VhQFOOWw6/AJ7nryPf657RAAF8yfzG2rFtEwMU5FTKsWi0jxKYoC3NWT5MePvskbjcFFE2vOnc6NF85lWm2cMi2cKSJFqigK8J7mdtrSxffq5XO44vxZnF5TSXSQCRYiIoWuKApwVyJYzeInF8/nq0umMaW6ctDZbf052fthT3M7DTrJJiIFILReEMMRnzbPH3riGS6YX8/k6oohZ7f1peY5IpJnBd8LImMNdXFWLpxCfU3lsIsvBJeXlUWNceUxzIK/B5tqLCKSD0VRgKsqYkwan9nU4v6oeY6IFKKiKMCjPdmm5jkiUoiKogCPlprniEghKokCrOY5IlKIiuIytGzQVGMRKTQlcQQsIlKIVIBFREKiAiwiEhIVYBGRkKgAi4iERAVYRCQkKsAiIiFRARYRCYkKsIhISFSARURCUhQN2c3sELB7FLuYBBzOUjiFRHkVF+VVXEaT12F3v3iojYqiAI+WmW1092Vhx5Ftyqu4KK/iko+8NAQhIhISFWARkZCUSgF+MOwAckR5FRflVVxynldJjAGLiBSiUjkCFhEpOCrAIiIhGVMF2MwuNrOtZrbDzG7t5/EKM1uXfvwVM5uV/yiHL4O8bjGzd8zsTTP7h5mdEUacwzVUXr22W2NmbmYFf6lTJjmZ2TfTr9cWM3sk3zGOVAbvw5lm9pyZbUq/F1eFEedwmNnvzazJzN4e4HEzs/vSOb9pZkuzGoC7j4kvIAq8D8wByoE3gEV9trkO+E369qXAurDjzlJeFwDj0revHSt5pberBl4ANgDLwo47C6/VXGATUJe+Xx923FnM7UHg2vTtRcAHYcedQV7LgaXA2wM8vgp4CjDgPOCVbD7/WDoC/hSww913uns38CdgdZ9tVgMPp28/Cqw0M8tjjCMxZF7u/py7t6fvbgBm5DnGkcjk9QK4A7gb6MxncCOUSU7fB+5392YAd2/Kc4wjlUluDtSkb58G7MtjfCPi7i8ARwfZZDXwBw9sAGrNbGq2nn8sFeDpwJ5e9xvT/9bvNu6eAFqAiXmJbuQyyau3Kwl+Yxe6IfMys3OABnf/Wz4DG4VMXqt5wDwze9HMNpjZkNNVC0Qmuf0cuNzMGoEngRvzE1pODffnb1jG0rL0/R3J9r3GLpNtCk3GMZvZ5cAy4PM5jSg7Bs3LzCLAr4Hv5CugLMjktYoRDEOsIPik8i8zW+zux3Ic22hlkttlwEPufo+ZfQb4Yzq3VO7Dy5mc1oyxdATcCDT0uj+D//8IdGobM4sRfEwa7ONHIcgkL8zsC8BtwCXu3pWn2EZjqLyqgcXA82b2AcH42/oCPxGX6XvwCXfvcfddwFaCglzoMsntSuDPAO7+MlBJ0NCmmGX08zdSY6kAvwbMNbPZZlZOcJJtfZ9t1gPfTt9eAzzr6ZH2AjZkXumP6g8QFN9iGVMcNC93b3H3Se4+y91nEYxtX+LuG8MJNyOZvAcfJzhpiplNIhiS2JnXKEcmk9z+C6wEMLOFBAX4UF6jzL71wBXpqyHOA1rcfX/W9h72Wcgsn9FcBWwjOFt7W/rfbif4wYXgDfEXYAfwKjAn7JizlNczwEFgc/prfdgxZyOvPts+T4FfBZHha2XAvcA7wFvApWHHnMXcFgEvElwhsRm4KOyYM8hpLbAf6CE42r0SuAa4ptfrdX8657ey/R7UVGQRkZCMpSEIEZGiogIsIhISFWARkZCoAIuIhEQFWEQkJCrAIiIhUQEWEQmJCrCUHDP7ZLq3a6WZVaX78i4OOy4pPZqIISXJzO4kmBkZBxrd/RchhyQlSAVYSlK6n8FrBH2Gz3f3ZMghSQnSEISUqgnAeIKua5UhxyIlSkfAUpLMbD3Bqg6zganufkPIIUkJGksN2UUyYmZXAAl3f8TMosBLZnahuz8bdmxSWnQELCISEo0Bi4iERAVYRCQkKsAiIiFRARYRCYkKsIhISFSARURCogIsIhKS/wFf5dlHpaMKxQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 360x360 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.lmplot(x='x', y='y', data=df);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们随机生成了一些点，下面将使用PyTorch建立一个线性的模型来对其进行拟合，这就是所说的训练的过程，由于只有一层线性模型，所以我们就直接使用了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "model=Linear(1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中参数(1, 1)代表输入输出的特征(feature)数量都是1. `Linear` 模型的表达式是 $y=w \\cdot x+b$，其中 $w$ 代表权重， $b$ 代表偏置"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "损失函数我们使用均方损失函数：`MSELoss`，这个后面会详细介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "criterion = MSELoss()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [

    "优化器我们选择最常见的优化方法 `SGD`，就是每一次迭代计算 `mini-batch` 的梯度，然后对参数进行更新，学习率 0.01 ，优化器本章后面也会进行介绍"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "optim = SGD(model.parameters(), lr = 0.01)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练3000次"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "epochs = 3000"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "准备训练数据: `x_train`, `y_train` 的形状是 (256, 1)， 代表 `mini-batch` 大小为256， `feature` 为1. `astype('float32')` 是为了下一步可以直接转换为 `torch.float`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train = x.reshape(-1, 1).astype('float32')\n",
    "y_train = y.reshape(-1, 1).astype('float32')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "开始训练了"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch 0, loss 105.8649\n",
      "epoch 100, loss 0.7534\n",
      "epoch 200, loss 0.1216\n",
      "epoch 300, loss 0.1029\n",
      "epoch 400, loss 0.0915\n",
      "epoch 500, loss 0.0828\n",
      "epoch 600, loss 0.0763\n",
      "epoch 700, loss 0.0713\n",
      "epoch 800, loss 0.0675\n",
      "epoch 900, loss 0.0647\n",
      "epoch 1000, loss 0.0625\n",
      "epoch 1100, loss 0.0608\n",
      "epoch 1200, loss 0.0596\n",
      "epoch 1300, loss 0.0586\n",
      "epoch 1400, loss 0.0579\n",
      "epoch 1500, loss 0.0574\n",
      "epoch 1600, loss 0.0570\n",
      "epoch 1700, loss 0.0566\n",
      "epoch 1800, loss 0.0564\n",
      "epoch 1900, loss 0.0562\n",
      "epoch 2000, loss 0.0561\n",
      "epoch 2100, loss 0.0560\n",
      "epoch 2200, loss 0.0559\n",
      "epoch 2300, loss 0.0558\n",
      "epoch 2400, loss 0.0558\n",
      "epoch 2500, loss 0.0558\n",
      "epoch 2600, loss 0.0557\n",
      "epoch 2700, loss 0.0557\n",
      "epoch 2800, loss 0.0557\n",
      "epoch 2900, loss 0.0557\n"
     ]
    }
   ],
   "source": [
    "for i in range(epochs):\n",
    "    # 整理输入和输出的数据，这里输入和输出一定要是torch的Tensor类型\n",
    "    inputs = torch.from_numpy(x_train)\n",
    "    labels = torch.from_numpy(y_train)\n",
    "    #使用模型进行预测\n",
    "    outputs = model(inputs)\n",
    "    #梯度置0，否则会累加\n",
    "    optim.zero_grad()\n",
    "    # 计算损失\n",
    "    loss = criterion(outputs, labels)\n",
    "    # 反向传播\n",
    "    loss.backward()\n",
    "    # 使用优化器默认方法优化\n",
    "    optim.step()\n",
    "    if (i%100==0):\n",
    "        #每 100次打印一下损失函数，看看效果\n",
    "        print('epoch {}, loss {:1.4f}'.format(i,loss.data.item()))       "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "训练完成了，看一下训练的成果是多少。用 `model.parameters()` 提取模型参数。 $w$， $b$ 是我们所需要训练的模型参数\n",
    "我们期望的数据 $w=5$，$b=7$ 可以做一下对比"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.994358062744141 7.0252156257629395\n"
     ]
    }
   ],
   "source": [
    "[w, b] = model.parameters()\n",
    "print (w.item(),b.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "再次可视化一下我们的模型，看看我们训练的数据，如果你不喜欢seaborn，可以直接使用matplot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzs3Xl8XFd5wP3fnU2zSTOSZrSOJCuSvG9xLK9ZyYKzkDQhQKG0hCUphdKWNoX2fUspHxpoS15Km0LBIUBYUwg4TiB74iSOHVuSt3iRZVmWJY3WGUmz7zP3/UOZq32zdvt8Px8+rkb3zj0S9Jmj5zznOZIsywiCIAhLn2qhByAIgiDMDhHQBUEQLhMioAuCIFwmREAXBEG4TIiALgiCcJkQAV0QBOEyIQK6IAjCZUIEdEEQhMuECOiCIAiXCc18Psxms8nLli2bz0cKgiAseUeOHHHLsmyf7Lp5DejLli2jrq5uPh8pCIKw5EmS1DKV60TKRRAE4TIhArogCMJlQgR0QRCEy8S85tDHEo/HcTqdRCKRhR7KkqfX63E4HGi12oUeiiAIC2DBA7rT6SQzM5Nly5YhSdJCD2fJkmWZ3t5enE4n5eXlCz0cQRAWwIIH9EgkIoL5LJAkidzcXFwu10IPRRCuaE6vk9qOWlxBF3aTneqiahwWx7w8e1Hk0EUwnx3i9ygIC8vpdbK3YS+heIh8cz6heIi9DXtxep3z8vwFn6ELgiBcLmo7arHqrWRlZAGQlZFFb6iXx2oeoyK7Ys5n7CKgj/Av//IvmM1mHn744TG//8wzz7B8+XJWr149zyMTBGGxcwVd5Jvzla8bext58fyLBONBJEkikojQ4e/gnhX3zElQXxQpl+lwep3sqd/D7rrd7KnfM29/yqQ988wznDlzZl6fKQjC0mA32QnEAgC4Q25eanqJlJyi0FxIPBnnjOsMyVSS2o7aOXn+kgroc5WfeuSRR1ixYgW33HILDQ0NADz++ONUV1ezYcMGPvjBDxIKhTh48CDPPvssf//3f8/GjRtpamoa8zpBEK5M1UXVeCIefFEf53rPEY6HUavUFGUWYdQaMelMdAQ6cAXnpnhhSQX0ofkplaQiKyMLq946o0+7I0eO8NRTT3Hs2DF+97vfUVs78F733XcftbW1nDhxglWrVvHEE0+wY8cO7r77br71rW9x/PhxKioqxrxOEIQrk8Pi4J4V92DUGmnztmHUGEmlUjT1N3HWfZZ4Iq5Uv8yFJZVDH5mfAjDrzHQHui/5Pffv38+9996L0WgE4O677wbg1KlT/NM//RMej4dAIMD73//+Me+f6nWCIFwZHBYHDosDd8jNy00v0xvqRaPSEEvEON59HEemg+qi6jl59pKaoQ/NT6UFYoEZf9qNVe73wAMP8D//8z+cPHmSr371q+PuZJ3qdYIgXDmcXifHOo/h9DlRS2riyTjBeBC1Ss3Gwo1zVuWypAL60PxUSk7hi/rwRDwz+rS7/vrr2bNnD+FwGL/fz3PPPQeA3++nsLCQeDzOL37xC+X6zMxM/H6/8vV41wmCsPRdShFGeq0vkohwTeE1GLQGAvEAK20r+fj6j2PVW+dsvEsqoA/NT3UHujFqjTMu/9m0aRMf+chH2LhxIx/84Ae57rrrAPj617/O1q1bufXWW1m5cqVy/R//8R/zrW99i6uvvpqmpqZxrxMEYWm71CKM9FqfSWei1ddKUk5izbCCBHqNfs7y5wCSLMtz9uYjbd68WR55wEV9fT2rVq2atzFc7sTvUxBmx576PYTiIWWTEIAv6sOoNXLvqnvHvW933W7UKjX7mvfRGejErDMjyzKukIu7qu7iExs/Me1JqCRJR2RZ3jzZdUtqUVQQBGGmptprpcHdgCfqwR/1Y8mwUJVTRY4xZ9IiDLvJzpstb5JvzifXkEtHoANPxEOuMZc8c96c9nWZNKBLkvQj4C6gR5blte+99i3gA0AMaAI+KcuyZ85GKQiCMAvSaRSr3kq+OZ9ALMDehr2jUrdOr5MLngtoJA05hhzCiTCH2w+z2r6aEkvJmO+b/pCQJInm/mauyr6KzIxMStWl5BpyqS6qJikn5/Tnm0oO/SfArhGvvQKslWV5PXAO+MdZHpcgCMKsm+peltqOWtba15KSU4QTYfQaPaF4iOfOPUeDu2HYAunIXLtRa8SkNeGL+OiP9JOhzmBr8Vb02rnNn8MUArosy28BfSNee1mW5cR7Xx4C5qc3pCAIwgy4gi7MOvOw18w686idm66gizJrGVsdW8nQZOD0OekKdJFjyGGVfdWwBdL0h0QsEeOw8zAH2w5i0BgIx8PsKNnBxvytXOwx0h+eWUXeVMxGDv1TwP+N901Jkh4CHgIoLS2dhccJgiBcmvRelqELnWPtZUlfZzPasBltvON8B5PWhFVvVWb2gJJmUUtqajtqSckp+sJ99Ef78YX9/Gy/zNk2DWDm/z67ec77os+obFGSpP8XSADjFmDLsrxbluXNsixvttvn9s8NQRCEiUx1L8vI67r8XSTlJFU5Vco16Zm93WTnpOskKTlFm6+NRCpBzL8JV/NXOduWDcAntpexdVnFnP98lxzQJUn6BAOLpX8iz2ft4xJgNg/8SdfR0cH9998/4bXf+c53pt3Q64033uCuu+665PEJwpVqqntZRl5nN9lZY1+DzWRTrknP7KuLqukOdNMV6CIRKeT4iQdpa7sRAKPBx7//SYyv3bN2Xn6+S0q5SJK0C/gycIMsy1dEe8FkMolarZ7WPUVFRTz99NMTXvOd73yHj3/840ovGUEQ5la618p0rksvfPqiPsw6M4FYAE/Eww1lN+CwONhRfAvfftaGLA+G1F3bDpJtTuGNzt8Zv1MpW/wVcCNgkyTJCXyVgaqWDOCV9/qgHJJl+bNzOM45dfHiRXbt2sXWrVs5duwYy5cv56c//SmrV6/mU5/6FC+//DJ/+Zd/SXV1NZ///OdxuVwYjUYef/xxVq5cSXNzMx/72MdIJBLs2rVr2PveddddnDp1imQyyZe//GVeeuklJEniwQcfRJZlOjo6uOmmm7DZbOzbt4+XX36Zr371q0SjUSoqKvjxj3+M2WzmxRdf5G/+5m+w2Wxs2rRpAX9bgnB5GVmX7sh0cKL7BHWddUiyxKaiTdxeeTv3rLiH2o5aZcZ+Q9kNFGYW88kf17CvoUB5v5uuPkOOtYtgLEihefWcV7YMNWlAl2X5o2O8PCc9Yr/23GnOdPhm9T1XF2Xx1Q+smfS6hoYGnnjiCXbu3MmnPvUpvve97wGg1+t5++23Abj55pv5/ve/T1VVFYcPH+Zzn/scr7/+On/913/NX/zFX/Bnf/ZnfPe73x3z/Xfv3k1zczPHjh1Do9HQ19dHTk4O3/72t9m3bx82mw23282//uu/8uqrr2Iymfj3f/93vv3tb/OlL32JBx98kNdff53Kyko+8pGPzN4vSBCuYCPr0ls8Lfzo6I/I0GRQailFRuZA2wG6A908sPGBYTtEH3/rAo88/7zy9Ye3mPBof0kimUCnzqfMXoZapZ7zypahxE7R95SUlLBz504APv7xj/Pf//3fAErwDAQCHDx4kA996EPKPdFoFIADBw7w29/+FoA//dM/5ctf/vKo93/11Vf57Gc/i0Yz8CvPyckZdc2hQ4c4c+aMMo5YLMb27ds5e/Ys5eXlVFVVKePbvXv3rPzcgrCUjTW7dvqdk+4CTRt5BmhXsIuYHENKSZh0JrwRL33hPvb59uGP+fnCli/Q5jbw0ccPKe+x7aocfv7prWjUKpzeyintQp0riyqgT2UmPVdGttBNf20ymQBIpVJYrVaOHz8+pftHkmV5Stfceuut/OpXvxr2+vHjxye9VxCuJE6vkxfPv8hrza+Rb85nnX0dbd42njr1FDtLdlJmLRt3F+hQI89Y8Ea8qGQViWQCb8RLY18jOpUOnaTD2e/n2m+eGHb/Nz4iY9B38aNjP1QC+ER9Xubakuq2OJdaW1t55513APjVr37FtddeO+z7WVlZlJeX85vf/AYYCL4nTgz8l7tz506eeuopgHFb6N522218//vfJ5EY2I/V1zewV2toO95t27Zx4MABzp8/D0AoFOLcuXNKnr6pqUkZnyBcqdJpklOuUxSaC5Ua8HO958g15NIV7JryiWYjz1iw6C2kpBQatYbOQCcZ6gxk1Fy88GEOHL1Tue7Wze9y/Zb/4/+r+Srvdr07q0dizoQI6O9ZtWoVTz75JOvXr6evr4+/+Iu/GHXNL37xC5544gk2bNjAmjVr2Lt3LwD/9V//xXe/+12qq6vxer1jvv9nPvMZSktLWb9+PRs2bOCXv/wlAA899BC33347N910E3a7nZ/85Cd89KMfZf369Wzbto2zZ8+i1+vZvXs3d955J9deey1lZWVz94sQhEUunSaJJ+MYtAblrM6m/iaseiveyOD/D461C3SokfXmBaYCdJIOnUpHf6Sf9o7NvPvupwmH8wDIL3yDNeu+R0FOiHAiTLY+m3e73521IzFnSrTPZXg1ylK3GH6fgjATk3VD3F23m3xzPofbDxNNRDFqjciyTF1nHVdZr8JqsLLdsR2YWrvbsfLwvz56lt8cHFznKrJ3ksp6gjyTnQxNBittKznaeRSDxoA75ObvdvwdAD3BHmo7atlUsGlWc+iifa4gCEvOVLohptMkVTlVHHYeBiAlpyjOLKY33MtK20pScmpYrfhEhtabN7uD3PToG8BAMNdqorx/20F0mhQvN/lJyUnlw8KoNdIf7ifXkAuAO+TmrZa3yDZkT9jJcS6JgA4sW7bsspidC8JSN7LqZGjPlHRQrC6qVoJ+dXE1J3tO0hPo4abym9iYvxGn3zmsVnxkW9yxZv+hWIJbv/0W7Z6wcu1f3eEj2xyhsU+HN+pljX0NTZ4m4qk4yVQSg8bAhcgFriu9jpSc4kT3CSRJYkPehlH9Xq6ogD6VChBhcqIDg7DUjaw6gYE8+NBDJdLb8ms7agnGgqy1r0W2y8iyjNPvHDfNMdbs/5mzeznTdDXPv9uvXPffH72aet/PcEU9NHX6sOgtbC7aTI4hh3fa3kFGptXbSnFWMR9e/WHicpzuQDfReJTrSq4b1h5g5Njn2oIHdL1eT29vL7m5uSKoz4Asy/T29qLX6xd6KIJwyUZ2Q3QH3ZzoOUEkEWFP/R4lWKf/MzRIp7fkj5fmGDn7P9OaxS/etgIDwfyBHcv46gdW0+5r5/dvNqFRacg2ZBNJRDjsHDjcYqtj67B8fHrGD5ClzyIYDw575lidHOfSggd0h8OB0+nE5Rp/JVqYGr1ej8MhWtMLS1c6nQIQiUfY37YfWZa5vux6pSwwHaydXieP1TyGO+Qm35RPVU6VMjseK82Rnv23ueE/nhss8FNre/ijnee4c20ekiQNHG6Rt5YzrjNEEhGlt/kp1ynuW3Wfct/IGX8kEeFA2wEApQ5+Kjn82bTgAV2r1VJePn/NawRBWLyGplPebnkbq97KhvwN2IyDaYz0jHhvw15cQRcF5gKiySiH2w+ztXgrOcYc6l317KnfMyxXbtTY+bufqUkkBzMBZRVPYMuSCCfyefSdR3l4+8PK4RaZukwa+xrpj/STpc/CmmEd9iExcsZfnj0Qx9r97eg1+jFz+HNtwQO6IAjCUOl0SvqQ5rqOulGHNKeDaUFmgVK6CNDY10hBrIALngsYtUY6A52803aIR57tw90/2ECr4qoXkDLOolFpcGStAMAT9fCV17/ChsINdPo7iSajeKNeLBkWCswFo84SHSvfX2YtQ6/R89Dmh+b4tzQ2EdAFQVh0JjukOR1Mq3KqeO3Ca/hiPuKJODE5Rr4xn5W2lZxxnaG9ayUnmwZ3fRcWHCUr920aXA2sMqxiuW05AA3uBjLUGXjjXowaI8+ee5Y19jU4shx4Ih6a+pt42PHwsDFO9fSj+SR2igqCsOiMdUizSlJxqucU1UXVw7bsS0iE4iHa/e30BHo433ueuoteXjzwAU42DTS00xtaURd+EVteDavtqzFnmOkN9+KL+Khpr6Er2EWbrw2j2kg0GWWNfQ3BeBBv1IvVYGVnyU6c/uFb+qd6+tF8EjN0QRAWHSWPnTGQx/ZEPFj0lmF57L0Ne2nsa0Sv0SPLMjajjYqsa3il5kZ6hrzXxnVP0eI/RZbKSkpOYdaZ2VCwgYOtBzkcO4xWpUWlUhGMBTFYDbT72nFkOfBGveyqHDjfICWnRpUfpvP9L5x/gbdb3kaWZDYXTrqZc06JgC4IwqIz8pBmGNzGD4PB9JG3HqHN14ZBnYnz4oe5ELAo71FW/huuq3RwuL2LaCKKPcuu3F9uLaepr4n+SD/+iB+VpKIyuxKbwUZvuBdPxIPVYFXea6JUSiwZ49qyayctm5wPIqALgrAgJurZMrR8ceSRb2kOi4PbKm/jO68009KxQnl94/IGNMaDnHadZt/FRhKpBJm6TBJygqLMIgDCiTBqlZoySxnLrMto9baiklS0eFvQqXVTbiEwlZ2t80nk0AVBmHfpGu5QPDRm69mRhzSH4iEy1Bk83/g8e+r34PQ6efl0F198UqcE89J8N/fd+DrmrKNIKomdpTu5vux6llmXkSSJQWVALakJxAL0hntRoaLAXEC+OZ8VthWYdCbCiTAalYaHtz9MiaVkwoOkYSA1ZNaZh702WYfHuSRm6IIgzLvJZrZDZ++SJNET6GFZ9jLMOjMX3SG++OTgQROZeolVq/4Pd6SNMz064nKcHH0OO0t3YjPa2FGyg+b+Zs66zxJKhJBkiZ0lOyk0FdIX6SMUD5Gpy6TUUkqOIYdyaznVjmqqmXxxc7FVuoiALgjCnJgopeIKulCr1Jx2ncYb8WLRW6jIriAYC47agbnv4j76w/3YDMU8uldNf3AweL70N9djMnj5yfGjuMPZxBIxGvoa0Kl1w8YyVn34nvo9tHnb6AoMVLgEYgHUkhqr3orT65xSymQqqaH5JFIugiDMuslSKipJxVstbxFNRLHqrUQTUd5qeQtPxMNjNY9xpOMIp12n6Qv3EUvEaW65lm/+zk5/cGCX5yeuT/L/3N/OioJMajtqKc8u5+bym7m96na2Fm3ForfQ2NeojGesWXN1UTVqlZoCcwF6tZ58Uz7ZhmyKM4unfPLQyNTQROmZ+SBm6IIgzLrJUioygx1WJQb+DcVDHO8+jkFjGNjOn4jy2yPtnDp/h/K+N6yS+eBWGX/Mj1E7EKDTO0r9UT+WDAu5+lx6w710+bsmXNRMB+PHah4jISfI1ecq/WB8Ud+UFzaH9lNfaCKgC4IwymSnBk1msja4sixzXcl1HOs+xhnXGWRkkAfO9Mw35XOhJ86hd9+n3KvX93Nr9VFuqbgB/5AAPdaO0sb+RorMRYQSoXH7oqc5LA4qsivYUbIDlaQac6xLiQjogiAMM5VTgyYz1mJhi6eFdn87u+t209TfhFFjJJFMsNq+Gr1Gz9utbxMPaKk9to1USq3cZyv9bwotOkos140K0Hvq97DWPtAZMb2jNBwP0+5v559v+OcpjXexLWzOhAjogiAMMxu11SMXC1s8LRxoO8DOkp1Kq9lfn/415dZybEYboXgYd/uHCAYGO6/mFP6KmOY4CZWG6uL7qMqtGnU26FR2lE53rAu9sDkTkwZ0SZJ+BNwF9MiyvPa91z4E/AuwCtgiy3Ld+O8gCMJSMpVTgyYztA1ud6Cbdn87O0t2Ki1my7PLKc4qJhAPcOS8lfrm7cq9KvPzXFPVQ1yOE4k7KLWUEk6Ex6ztnmxH6aWMdSHa3s6WqczQfwL8D/DTIa+dAu4DfjAHYxIEYQHNNAUxMv9+R9UdPN/4/KgPCZN8NYePr1O+riqQyS76Oa3ei3iimVj1VspsZZh1Zjr9nWwp3jLqWbM1u15MC5szMWnZoizLbwF9I16rl2W5Yc5GJQjCgplJF8HxyhVVkkrpjtgfhC/8WMXrRweD+b9+JMED7/Ni1Wdxe+XtVGRXUGopJVOXSZu3jTZfGw3uBmWXaNpiKxtcaCKHLgjCMDNJQYyXfw/FQ/SGPHzvRTPtfYNh55H7szGaugdm83o7D256kJqOGrIN2XQEOmjsa8Tpc3LbVbexyr5qzAXay2V2PRvmPKBLkvQQ8BBAaWnpXD9OEIRZcKlBcrz8+3N1ag6dy1Reu2ZFI/9w606qHaNn/YWZhdR21CptcYfm3he6+dViN+cBXZbl3cBugM2bN8tz/TxBEObWRDXqI/Pv77bA469rgIFgvr4sxqduVBGM26npqKEws3BUYB76YbK7bveMF2ivJCLlIghXqEvZPDRZjXp6kdLtU/Gfvx9cVDVmJPm7D/jJy5xeKeTlVCM+HyZdFJUk6VfAO8AKSZKckiR9WpKkeyVJcgLbgT9IkvTSXA9UEITZM1mvlfHueazmMY52HuV0z2n6Qn1kZWRh1Vup7agFIFtfwJOvVg0L5j/9zAr+5gNd2MzTbzO7GI95W8wmnaHLsvzRcb61Z5bHIgjCPJnu5qH0B8BFz0USqQSt3laOdBzhtqtuo9JWSZe/m4d/c4Knjwx+IDz20av5wIaBAyV6609e0kz7cqoRnw8i5SIIV6Dpbh6q7aglmUriiXhQS2qy9dn4Y35evvAyDe05vHysGBgI5tetTHLtmi4SuhhOb/WwVEz6Oel68eU5y9lTv2fCtI+oYpk60T5XEK5A6dz0UBPNmF1BF52BTsosZaTkFLFUDCleSsOZv+LlY3kAlNv0fPGeFnZt8lOQOTyNM1a9+JaiLdR01Ewr7SNMTMzQBeEKNN0dlnaTnUPOQxRmFqImixffuR5ZHmygtf9LN1HX/RKhuGXcNM7Imfae+j2L6jzOy4EI6IJwBZpubrq6qJrnG1/g5bpKXP15yuv3be/guuU2SnKMvHBhemmc6aR9ZtrO90ohArogXKYmC4LTyU0/fyLKKwfvUb5ed9VFbl2fRK1SKxUn0y0xnOr1s9HO90ohcuiCcBm6lLLEsbzT1Muyf/gD33j+LACbysw8+qcxPrBJS4mlZFhQnW6J4VSvH1qRo5JUo0olhUFihi4IS8zImbcj04HT7xw2E59pT/NOb5jt33x92Gt1/3QLNnPGuPdMN40z1etno53vlUIEdEFYQoamH9QqNX849weOdR9jY/5Gdjh2KDPxQDTASvtK5T530E1DX4MyQx8vBx1LpLj/+wd51+lVXvvd53awqTR7SuObbonhVK4Xu0WnTgR0QVhC0jPvWDJGbXstPaEe7EY7rpCL2o5athZvxaq30uptVYKgO+jmcPthVJKKEkuJEvRH5qC/+UI9P3jzgvL11/9oLX+6rWzUGOZ7gfJyOlForokcuiAsIa6gC7POTGNfIyadiWQqSaY2k2QqiUlnorGvEbPOTLY+W8lPN/Q1oJJUpOQUy3OXj8pBv3y6i2X/8AclmH9gQxHN37xj3GA+G7n56RA9z6dOzNAFYQlJpx+8ES9WvRWj1kggFsCsM2PQGOiP9BOIBVhhW6Hk0p1eJyWWEpbnLleOaTPrzBxqvsAXn/yD8t5Wo4a3vvQ+svTacZ8/G+eNXgqxW3RqREAXhCXC6XXSG+rltebX8Mf8xBIxsvXZdPo7KckqIRgPolVrlXTE0CAYioeU4BuNw9d+K+MPL1fe+6/u8JGh78MXXUOWfvzAKRYoFzeRchGEJSCd6jBoDdxy1S0UmAo42n2UpJwcSEfoBtIRa+xrRqUj0uWB3oiPn+2Hh3+uwh8emIV/8sYUj30yRVW+eUqlgENbBriDbt5pe4dnG56lqb9JbNlfBCRZnr8zJzZv3izX1dXN2/ME4XLxwyM/5JTrFPFkHIveQlVOFf6on3Z/OxXZFcMWJ8datPz9iV7+7fnBgLt2mZsHb7SikgbndCk5Rb2rnhW2FeMueKY/WJKpJKddp1FLapKpJGvy1qBWqdlStGVUCaVIlcycJElHZFnePNl1YoYuCIuc0+vktebXUKHCqrcSTUQ57DyMSWeiIruChzY/xL2r7lWC+dBFy8buKNd+84QSzFfkZ3L267v49A1Zo5pztXhauOC5MOGCZ3qBst3fTjwZx2qwsq1kG+XZ5SRTSR4/9rhotrWARA5dEBaZkTNsd8hNvjkfSZJQSSqMWiMAJ3tOsta+dlj72d5QL1a9FU8wzld+pSIlD57juf9LN1GSM3DvWKWAp3pOsTZv7aQLng6Lg4rsCnaU7Bg2w+8IdJBIJkSzrQUkArogLCJj9S3Z17yPqwuupqm/CYB4Mk6zp5k2bxs9/h62lWyjzFpGIBbg1Quv09F6N60uq/KeO9YfBV09knoDMBDQx9qlWZFdQZl1eKnieAueY232EQumC08EdEFYRMYqC8wz59EeaGdr8VaOdB3h3a53MWqNFJgKsBqsnHGdIVOXyfELNg4d+RPlvTZUtrH+qnZC8RBJOW/MmfbIdrZT3ZE51gxfLakpNBcOu07s6JxfIqALwhy41N2UY81y1+Wt49ULr6Ir0pGpy2Rt3lpScopQPES2IZs2l56v/Xqwpa1Wf4E7t1zElKEnFI8QjAWpLq4edX7nWD1hajpqgMl3ZI41w39w04PUdNTgi/rEjs4FIgK6IMyymbR7HSuVodfoubn8ZoxaI23eNhwWBytyVnCis4VfvDK8M+E/3ufl7bYaznv6CUQD5Bpz2eHYgV6jV3LvY42xxdPCC+dfIEefQ6u3lRx9DsttyydtrjXye4WZheL8zwUkArogzLKZ7KYcr2/J0A8DXyTED1+10NY7OCu/pfo4N1Uto7m/maLMIlQqFctzliMjc9J1knAizAMbHxhzjO6QmzOuM2gkDUiwPm+90sZ2usFY7OhcWCKgC8Ism+7i4MjUR7qWe6xZbm1DMb88NJg6uXmDi7D2ZaKo2d/ahgoVK20rWWlfSWNfo9IioMBcMCzQDh1jui9MunWAqE5ZukRAF4RZNp12r2OlZ2o6akalZ1463cWf/+yI8vXVy5LsuqYLlUqiJ7CGZdnLMOvM7D27l9Ou02wr3sZ2x3ZgYMPQyA+ToWNMB/1wIowlwwKI6pSlSgR0QZhl02n3Oiz1EXTT2NdId7CbVm8rX9jyBVp7o3xs91nleqtRw/4vvY/M9xpSTD24AAAgAElEQVRo7anfg1FrVD48CjIL8IQ9NPY1YjMNNOIa68Nk6BizMrLoD/eTklOsLV477j3C4id2igrCLJtOu9d0O9x0z/JoMkq+KZ9OXx/v/8/aYcH8r+/08eCu83ij3aPuT6vKqSKZStId7J7wWLehY7RmWEmkEqy2rybHmDPp0XHC4iVm6IIwB6a6OJhOfQzmsY3sP1lGS9dO5ZpP3phiUzmAGV80NSy3PTK9YzPaWJO3hnZ/+6SVJkPHmM7ji+qUpW3SgC5J0o+Au4AeWZbXvvdaDvB/wDLgIvBhWZb7526YgnB5Sqc+uoPdBDzrOFxfpXwvO+c4WfaXiWp24g5WYTPZRuW2x9zgo1LzhS1fmPWj4ITFbyopl58Au0a89g/Aa7IsVwGvvfe1IAjT5LA4WGm5ldfeuU8J5rmZUW7d/gfyit7CkmEhmoxyuP0w7qB7VG57aOrkrOss73a/iz/qVw62EK4sk87QZVl+S5KkZSNevge48b3/+0ngDeDLszguQVhwtc5anq5/mnZfO8VZxdy/6n6qHbOXV+4Pxtj6jdeIJVPKa6UVj+NLXORcr4VcYy5GzcBmIIPWwImeE1TlVI1aXE3PrDv8HTgsDmWmPtXNTMLl41IXRfNlWe4EeO/fvEmuF4QlpdZZy6PvPIo/6qfUUoo/6ufRdx6l1jnxARBTkUzJPPDjGq7++itKMF9d9Ro55V8hlHLij/rpC/eBDFcXXk2GJoNoIkokERk3QA+tllFJqlHnhgpXhjlfFJUk6SHgIYDS0tK5fpwgzIqn658m15BLrjEXQPn36fqnZzRL/8GbTXzzhcHKlQevL6Al+QNavC2UGEuQkIgkI6TkFOc955FUEnctvwudWodRaxx3ti06HQpw6QG9W5KkQlmWOyVJKgR6xrtQluXdwG4YOLHoEp8nCJfc8OpStPvaKbUMn4BY9VZava3jju3F8y9S11mHJEtsKtrE7ZW3K+M72OTmY48fVq7fUZHLTz+1hefO7aWpIYEaNRnqDMKJMIlkYmCTj96CN+rljYtvsDxnOZ/Y+IlxxzudzUzC5etSA/qzwCeAf3vv372zNiJBGMNMGl5diuKsYjwRjzIzB/BEPBRnFY85tiePP8m5vnPEEjEa+xp56cJL/PTET3low5f5ym+kYdfX/dMt2MwZwODM2h12E0vF6Av3YdFbUEVVZKgy0Kg0ZBuyyTPnTfhzTmczk3D5mkrZ4q8YWAC1SZLkBL7KQCD/tSRJnwZagQ/N5SAFYSYNry7F/avu59F3HgUGZuaeiIfecC93VN4x7ISg6qLqgb8awi7iiTjvdr+LXqMnV59P+8WP85Vzg8F8z+d2cHVp9rDn2E12IokIbd42OgOdBGIBVJIKtaRmpW0l7yt/HznGnElTJ2O1sxW15FeeqVS5fHScb908y2MRhHHNV454aFpnU+EmWjwttHpbKc4q5o7KO2jxtYz6KyEQDRBLxGj2NKPX6Il578Dfv015z2tWNPLbT/7NmM9zZDp44fwLxFNxtJKWaCKKLMlsK9rGjctuxGay4Yv6ppQ6EbXkgtgpKiwJ85EjHpnWMelMmHVmJa2zp37PmH8ltHpb0Wl09HqKifV+cvAN9XVk5+3FGdeyp75sVM7f6XVS01HDWvtaOgIduHQucow5FGcWs7FwI2adWdmGP17qZD7XFYTFTwR0YUmYjxzxZGmd8f5KkBL5PPPmisEXVT4iln8klOyj1yNj0Vv4/bnfc6bnDJ/Y+InBVrhDnleeXQ6AL+ojHA8rfWAmSp3M97qCsPiJgC4sCfORIx4vYNe76tlTv4ejXUfRa/RsyNuAzWQjGoev/06FNzQYzCPWf0JSdxFNRJCQUEkqMrWZXPRcxBvxkn8+n89c85kJnxeMBbl31b2Tjne+1xWExU8EdGHJuNQc8WQlhWljpXVaPC1c8FygxFJCdVE1b7W8xRstb9LdfitnnVbluq/9URlRzUG8kY/x2/rfUu+ux6gzkmfIQ6vVkqnLJJaMUddZx2f4zLjPm04aSdSeCyOJgC5c1oaWFOYacpGROdB2gO5ANw9sfGBYUB8rrXOq5xRr89aSlZFFVkYWltQunq0dDMBbqoLcUx3HE7mITtJRlFXENUXXEE1G0al0SCoJtaRGp9bRG+lFkqUJnzedNJKoPRdGEgFduCyMtziYLim0GW3KIckqSYU77B6VmhgrrVORXUGZtYw2N/zHcypgIHiaDF6+fn8mWo0BMAAQjofxRDxo1VpyDbl0BjpRS2rKrGX4Y35SqRSbijZN+LzppJFE7bkwkgjowpI30eKgK+giloiRbRis/zZoDPSF+3AFXaPea2Ra5+fH9vC3P1WTTA3OrLdf/QJluWa0msGe5enc9z0r7uGF8y/Q3N+MJcOCUWMkHA+TlJNsLtzM7ZW3T/i86RC158JIIqALi9pUyvLGWhzsDfXyWM1j+CI+2nxtJFIJJd8cToTRaXQTpiaSKZlPP1nLGw065bXqNYcptvvoCfZRZF4+7Pp0qsNhcfDgNQ9ye+XtU8rbz5SoPReGEgFdWLSmWpZ3zn0OT9SDL+rDoreQq8/lXN854sk4O0t24g67ebf7XdbJ6zBnmOmP9FOZXTnuEWsjG2hdU9VBSfFpYokYoMNhduCL+vBFfeOmOhwWB7sqd5FrzMUVHEj5CMJcEwFdWLQmK8tLV6/8/tzvMelMVOVUEU1EebnpZfLN+TgsDvLMedxReQdvt75Nq6+Vq6xXsbNk55iz5bEaaN23o5No0khWxuDG6KnUiosacWEhiIAuLFoTleWlA2ZjXyMrbStp6m/ijPsMa2xriCQjdAW6uKn8JgBsJht3r7yb7kA3D21+aNRz/JE4X3nmFM8c71BeSzfQ2l23+5JqxUWNuLAQREAXFq2JyvLSATOejJNnzsOoNdLsaeZc/zlyDDno1LphaY6xyvkSyRT/V9fGt18+R28wBoxuoHWppYGiRlxYCCKgC4vWRGV5zzc+T745H4veQiQRIUufxfr89fRH+lmRu4JTrlPDctwX+y+SZ85jd93ugWAcXcn397lo6PZTvSybH3+ymvUO67TGMBFRIy4shEs9gk4Q5tzQA5C7A90YtUYlB50OmFU5VQRjQULxEMF4EK1ai1ql5sGrH1TuC8fDyMgYtUbkRD7/+4rEF5+6gC8a5X//ZBO//vPtYwbzycYwkeqiajyRgYXalJxSmmyNtxArCLNBkuX5O0Ro8+bNcl1d3bw9T7h8DV10jCQinOw5SU+gh5vKbxq14Lmnfg8uf5i367N4+yzoNHDjmgg3r1XzobWT90yZyRhFJ0RhNkiSdESW5c2TXSdSLsKiMtUgOHRTTTAW5IayG8a8NpZI8ftjYQ6ezSISh50rYNtyN87QWfY2ONGombNAK2rEhfkmZujCojF01j00X30ppX6yLPPS6W7+7YV6LvaGqCqM86GtarQ6N4fbD6OSVFj0Ftbmrb3kZwjCfBEzdGHJmUrd+VRm76favfzrH85w6EIfVXlmHv1wOd2JfZj0Vk72NKCSVKTkFMtzl49bTijSJcJSJBZFhUXDFXRh1pmHvWbWmXEFXcrsPRQPoVapebPlTb70ypd4/MjjOL1OALp9ER7+zQk+8NjbnGzv5f1Xe/jz9/exrSJLWdh0ep1Y9Ba2OrYqZY3pZ6QNfVa+OZ9QPMTehr3KcwRhsRIzdGHRmErdeSwZo7a9FpPORL45n9Ou0wSjcQL9W/nlYRexRIrNVV7evwHsZvOwHZrpjUCheGjUM1SSSjn8uam/ieLMYrEpSFhyxAxdWDQmKvVLz94b+xox6UwYtUaMGhPNnXZ+/GoJP9zfxQ3L7Xz5nih/VC2Tn5mJSlKRlZGFVW+ltqN23Gdc7L9IV6BLmZG7Q25O95zGHXIrYxs5ixeExUjM0IUFMV6Oemg7WEmSyFBn8Hzj8zT1NxFJRPBGvFj1Vnr6zdScLaXfn0VhdpyVVx3i6pVlHO08xuaizcNm4EN3aI7Vcja90zR9T74pH0/EQ2Nfo5KWEZuChKVABHRh3qSD+Dn3OZr6m1ibt5Yya9moxlXpBdChFS+RRIQDbQeIRM283ZaP11uBRhNgx+rzaDOPkW0YaIKVoc1gf9t+bii9AZtp7GA8spxwd93uYbn7qpwqDrUfosvfRUpOiYMjhCVDpFyEeTF0odET9aBRaahtr+WV869wsO0gjX2NvHD+BeX6oRUvfeE+2rwuLrSu48SpD+D1lZCXV8d1m16nM/UMwViADfkbUEkqNuRvQJZlTvScmPIOzXTuPs1msrHGvga7yT6t3aGCsNDEDF2YF0MDtC/qQ61Sc6H/wsCCpEpFQk5wsPUgG/M3Uu2oVppbdfndPF3n4mL7JqJxHaasU6wpbyLPokOWNahUKgrMBUpqxGa0sS5vHa9ceAV30E1WRhZl1jKeb3weSZKQkEjJqWFpnrH6tahVar6w5QsiiAtLipihC/NiaEmiJEm82vQqp3tOc77/PP2hfuSUjD/m5z8P/SdOrxO7yc7ThxM88nQeDRfXYDFFKCn/JasqainJySQrI4tdlbuoLqomFA8pz3GH3JzsOckq+ypuKr+JYDxIvbseb8TLgbYD7G/dj1qlHlaKeKn9WgRhsZnRDF2SpL8GHgQk4HFZlr8zK6MSLjvptEYsGaPT10lHoIOknEQraQkkAkRTUapyqugOdfPEO3X8+I3Bo9+WFfSwaeUpTve4yTGUodfo8UQ8ABSZi+gL9ymdFU90n0CSJDbkbaCxf3BR85DzEMuylwHQ1N/Edsd2YLAUUWzTFy4HlzxDlyRpLQPBfAuwAbhLkqSq2RqYcHlJlwue6D5BLBXDqDWCBBq1Bp1Kh16rxx8y0nTmb/nxG1oAckwa7rvhABXlb6PXZHDbVbehklT0h/uHpW6GdlaMxqNcV3IdNpMNb8SLXqPHoDHQG+7FoDGg1+jxRryAKEUULj8zmaGvAg7JshwCkCTpTeBe4D9mY2DC4jbdrfHptMYjbz2CL+qj0FyIFJAwao1oMBHp/nv6U4MHS7z6t9dTmZeJ07tmWLWLVqPlVM8prBlWjFqjcvRbNQOLnnvq9ygpmHSvdIBcQy7hRFh5HUQponD5mUlAPwU8IklSLhAG7gBGdd6SJOkh4CGA0tLSGTxOWCwmOi8TGDfQOywObqu8jQxtBqFYiBOd79LdeSuJyDblvW/e2MzXbv8jHJZM5Z57VtzDC+df4O2Wt5ElmS3FW9hVuWvMD5ChC5wV2RW81fIWkiSxzbGN0+7TyLLM9WXXK9UvohRRuJzMqNuiJEmfBj4PBIAzQFiW5S+Od73otrh4zKT5VHoWPHTzTvrg5GgyOmG3RKfXyZPHn+TNs3HONQ+WEsqG17hhnZsvbPkC1Y7hJYbT7cI49GdTSSpkZGRZHrfKRRAWu3nptijL8hPAE+898BuA6F60BMz0RPrxzsvc37qf60qvm7AHSr8/k8df2Dh4o6aD8orfcEfVrRRlXUNNRw2FmYXDxjHdA5fFAqdwpZpplUueLMs9kiSVAvcB22dnWMJcmmqAHG8WP14TLUmWxuyW2B3opj8YY8s3XiWeHPyL8LZt+zHrY0jSOpr6m7AZbUrflaHjEAcuC8LUzHRj0W/fy6HHgc/Lstw/C2MS5thUAuREs/jxDk7eVLRpoDQxEaOxrxFv1ItGpeXk2ev4xtOvKO/92VvjlNiDHGzzY9BaUUkDxVaNfY1sdWwdFajFgcuCMDUzTblcN1sDEebPVALkRLP46qJqdGodb7e8TSAWwJxhpsxShiRJPH3qaZp9zWRoMtCF76S3Z/B/In///hV8/qbK93qn5CtVKEatkb5QHye6T/BGyxvkG/O5uuBqJZc+3geIWNAUhOHEEXRXoPEWGbcUbcHpd9LgbmDfxX3km/JxZDmoyqnCZrKRklOcdZ3FnGFWDmdOV5Gssa2hpqOGuo46MhLriLj/XHleqT3EX+3ScP+agX7k6UXVWDLGYedhfFEfdR116DV6Cs2F5JvziSajPLz9YSWoixOEhCuZOIJOGNdYLWSX5yynpqOGZCpJs6cZJGjztaHX6OkL97G1eCs6jY6+SB8Oi4OsjCxOu04P24kZjOiQu/6HiPIkmdzSb2HMNPBq8zK2OYb3TrHqrVQXV/O9mu8hyzIlWSVU5lSSpc+iN9TL0/VPKwFdLHQKwuREQL9CjQyQe+r3YNVbOd1zGrPOzMqclZxynaIr2EVFdgUnek5QlVNFtj5bWfhM9yZPpVScrL+DWLRAeb+Kyt/RFz9KghRxWYNeox9WSZP+QDnVfYruYDcGjYG+yMAW/iz9wKEUrd7Wef+9CMJSJgK6AAwulHqjXrL12fiTfvRqPU19TcTiMewmOw9vf5jajlol/27RW3j7tJ2WjsGODxb786hMB/HLWiRJQoUKDRo25G1Ap9EN653S6e/kKc9T2Ew2VLKKWCJGTXsNAAatgeKs4oX6dQjCkiS6LQrA4EKpJcNCT6CHht4GZGRW2VdxVc5VGLQGYLAnyzvnw/zylZ1KMDdlnkHO+xydqd+QZ8gjEo8QiofI1GZy21W3YTPZRvVOebr+aXINuVxTeA2+mI/+SD++mI99Lfto87Zx/6r7F+R3IQhLlZihC8BgJUmBuYC6jjpScgq1pCZHn0NKTrE2by21HbWst93Kt363TLlPqwlTsOwHZGhldOo19IX7aPW1kmfKozK3kmtLrh335KB2XzulltKB1812XAEXyBBLxFhtX01hZuF8/xoEYUkTAV0Ahi+UmrQmJEkiMyMTm9FGVU4VJm0O//xrCX/4TeWeh27rxp9qJJZcP9A9EZBlmc5AJ9sd24kmo+g0unGPcSvOKsYT8eAKucg35lOaVYo/6segNbC+YP24O0FBVL0IwlhEQBcUQxdK071aZBl+vl+ipklSrvvuxzZx5/pC9tTv4bmG7mEz6XAijN1kJyWnRlXSpDsjpt2/6n4efedReoI92I12/FE//pifW666ZcKdoDNtXSAIlysR0IVR0umXuiYdew4bldfv3GDglo1e2oPPsafejiPTgUatoT/cT44hh3AiTDAWZLV9NXaTfdJSw2pHNQ9vf5hv7P8Gnf5OCjILuOWqW6jKrcIX9Y27E3S6vV0E4UohArowSr8/c1ievCg7xbc+ks/xnlpC8cFZcU1HDXdX3c2z556lM9CJ3WRntX01apV6wkOZh6p2VPPYHY8N2+g0WWtb0dtFEMYmArqgON15kbsfO0UyNZheOfAP76PYalDq1EfOiuNynH++4Z9nlM8ea6PTyPTMUKK3iyCMTQR0gWRK5qOP76em2c/A8bDwyfcFsFndyFIv4Bg1K3aH3JzrPUebtw1gWBB3ep3sqd8zrQA/nZ2goreLIIxN9HK5TDm9Tl48/yJ1nXVIssSmok3cXnn7qKD5/Teb+LcXzipfv29dgIKCE3gjXrRqLWvta/nMNZ8ZdqiFO+TmsPMwKklFlj6LdXnrlAMngGkdRjGTn09UuQhXCtHL5TI01SCWPhXoXN85cg25yMgcaDtAd6CbBzY+gMPi4OB5Nx/74WHlnlyri9VV+2gPdaEJlmE32QnHw7zW/Bq7KncNmxWf6z2HSlKRklOsyFkxbFESmJcFS9HbRRBGEwF9iZhOqV5tRy2usAub0abUh6skFe6wm1caa/na0yeUayVk/uR9pwikumjq60MtqWn1tmLQGtCoNBg1Rh6reYyK7Ap0ah3heJg2bxsOi4MVOSuUTUNDFyXFgqUgLAwR0KdhIf/Mn06p3jn3Oepd9aglNRISSJBMSjRf+CDPRHTKdX97Z4ILwX20BftZn7eeo51Hyc7IBgma+5uxZFhIpVK4Q252lOxQ0ic3LrsRg9Yw7qKkWLAUhIUhAvoUzcZmlpl8IJxzn8MT9eCL+rDoLVTlVJFjyBk183V6nTT1N4EE8VSc7mA3kb7bifmvV665ZYOLezblAioazsfJNeTSG+llvX09rf5WEskEKVJY9BZScgqrfuBUoXSQDsVDeCIeYOxFSbFgKQgLQzTnmqKhM+R0cEuffzkV6Q+EUDxEvjmfUDzE3oa9OL2Tn6udDtLpdrXRRJTDzsO0eFpGzXxrO2pZm7eWPEMebd25+Fq+qQTz7OzzFFV8A2fiKd5pewd30I0lw4KMrIzDE/GQJElFdgWheIiknKQqZ7CbollnRpZl7llxD0atke5AN0atUflgS5cgjvU9QRDmlpihT9FMN7PMZHdjOkifcZ0hkohg0BgIx8Occp3ivlX3jRqnXipjf22F8pqk8pNX+r8st5fiiQ7s6IwmoxxuP0xVdhWHOg7hDrnZWLCRFTkraOxvxOl1UpRVxCrbKiVPDoPpk4kWJcWCpSAsDBHQp2imm1lm8oHgCroos5aRqcuksa+R/kj/wCEQGdZhgTMYTfDd54vwhgY3Bt265RBZphAXPTbyzHkYtAZaPC3AQM/xtkAbWpWWiuwKYskYdrOdHaU7lAXQaDKKL+oT6RNBWAJEQJ+i8TazLM9ZPqVNNFP9QBgrz56+12ayKbNlX9Q3rMPh3/3mBL872k56Y9AfXxukqtDP/rY23CEZk9ZESk6hklTcVnEbvZFePGEPKVJsyN/ASvtKVNJgBi4lpwjGgtPawSkIwsISG4umYWSwdWQ6qOmomdImmvEOZh567USHN4/3nAPnUnz5tyeV53xqZzmfuj5LGackSUhIvHnxTfqj/eToc3BYBg5+1ql1yodCetNQWvoD495V987Db1YQhImIjUVzYLxzOKeSF59Kv5Lx8uxOv3PUvfm6aq795mA9+cqCTJ75/E7coc5RM3yArkAX5/vPk63PJhKP8MbFN1ies5xPbPwEMFCZ0hvqpTPQSXegG41aw4NXPzjbv0JBEOaQCOgzMN28+GSLhRO9X/revmCMLY+8SiLVqFxz4B/ehyz18rN3f8S+5n3kmfNYl7dOqaTJUGdQnl1OcWYxjX2NA+eGGrLJM+cp49lStIXHjz5OUk6SZ86jyFxETUcNhZmFIsUiCEuECOgzMBtd/4amcZr6m4gkIpRnl496v2RK5pM/qeWtc4Nncv7801u5tsqmpGoa+xrJN+ejklTUttey1bEVq97K/tb93F55O6oMlZKDT8mpYR88Tr+TG5bdMCrtInqMC8LSMaOALknSF4HPADJwEvikLMuR2RjYUjCdrn9jLXYCwzYrRRIRDrQdAKDMWqa8n7NzFRVPPq+819+/fwWfv6lS+Tqdqokn42Trs5GkgYXRxr5GthZvRZKlST94GtwNeKIe/FE/loz3Ni4ZR29cEgRh8brkgC5JUjHwV8BqWZbDkiT9Gvhj4CezNLZFb6p9vIcudqpVat5seZOfHP8JkXgEi95CZW4lVTlVysz8rPssNe01tPToaWu5F+gEoHqZmY9c10df+HX21J9UKmrSqRpLhoVwIoxRa0Sv0eOJeAjEAmwq2jThzk6n18kFzwU0kkY5eehw+2FW21dTYimZv1+oIAgzMtOUiwYwSJIUB4xAx8yHtLRMZRNNegYdS8aoba8lJadwBV20+dvI0mbRF+mj1dPKzVfdjEln4oK7n7NnPz7kHVLcc+1h7Jl6osnyUa0H0qmfqpwqDrcPdFCUZRmtWjusrW1tRy1nXWfpi/SRrc9WdrnWdtSy1j6wcSmcCKPX6Ac2LvWM3rgkCMLidckBXZbldkmSHgVagTDwsizLL8/ayBaZ8fqwTKU/S3oGfbj9MCadibO9ZwnEAqglNRmaDHxRH7IsU9N+jIPHt+MNDAbz27eexGh00eBuQNJUsqFgAzC8oiad+rHqrVQXVXPSdZLuQDc3l9/Mrspdw8bT4e/AYXEoM/W9DXvxR/2ssq8iM2Ng45In4sGit4zauCQIwuI2k5RLNnAPUA54gN9IkvRxWZZ/PuK6h4CHAEpLS2cw1IWT7i/uCruIJWLoNDrO9JzhtorblPrwiRp2pWfQ6V4sXb4uNCoNNoONlJwilojh9d7MC/UblHu2rmpiecnAAmhK1uOP+oklYsPGNbQCJp36CcaC3FB2w5gfLOOVRbZ6Wwc2Lhlt2IyjNy4JgrA0zCTlcgvQLMuyC0CSpN8BO4BhAV2W5d3AbhjYWDSD5y2YF8+/yLm+c9iMNrIN2UQSEc71naPd387O0p2T1qGnZ9BatZZwPExcjiMjsyxzGZ3uPPo6B9MaVcUeVlfWolGpGchiQSQRITMjE51msPUtDF/YnErqZ7yyyBx9zoQ5dkEQloaZBPRWYJskSUYGUi43A0t3G+gE6jrryDXkKjNWo9ZIriGXus463l/5/mHXpmfNI1MxW4q2kKHO4LXm18gz5uENZHDy5GeV+ySVj83r/8DGwlV4IlpcYRcpOYWERG+4lzX2NWRmZM6or8p4ZZbLbcupLqoWW/wFYYmbSQ79sCRJTwNHgQRwjPdm4pcbSZaQGf7HhYyMUW0cM0CqJNWo3uk1HTXcs+Ieri+9lQ9+7wT+sFq5J8fxXQx6L3rtVWRmZJJvzmc5yznacZRALIA5w0y2IRub0UYoHiIYC15S0J2ozFJ0SBSEpU/0cpmCx488zoG2A+Qacgda1ybC9IZ7WWVbhVlnHtZjpbm/mXZ/O7FkjHxTPlU5VdhMNrwRH8/UZFLXNBjItTlPkpl1nkJzIZW5lWhUGqKJKAk5waaCTUiSRE+gh2XZy0b1dnH6nZd0UIY4XFkQlh7Ry2UW3V55O92BbtxhN62eVvqifahQkWvIZUP+Bpx+J92BbqURVjQRpcBcoPQc10Zv4Lk6q/J+11R6sdhexRtNEornkyKFLMvEkjGOdB5hq2Mr+eZ89l3cR3+4n6KsIuVQjd5QL48fe5wbym64pJOTxExcEC5fV2RAn+4s1WFx8MDGB3jh/AvsC+yjMqeSdXnr0Gv0SirFYXGwp34PRq2RzmAnp1yn6PWYaG8ZLEHMs8R5+AMy8ZTMaVcBGrIHGp4AAA+ySURBVJWGYCyIJEl0+Dvwx/3oNXo25G1AJamIJ+PoVDp+f+73FJgLsGRY8Ea8JJKJSzooQxCEy9sVF9Av9WxQh8WBzWjjzuV3DsuZ94Z6eazmMbL12ey7uG9gUdQXxNn0OWAwvbJuza/ZVrqc5n4NK+0rqcqpoi/UR6mllN7wQJdDlaTi7uV3K/1WJFnifP95NCoNq2yrCCfCHO0+yqaCTcPGNp2TkwRBuHxdcQF9OkfBjZzJN7gbWGVfpXzfHXRz2nWa/nA/HoMHGYnj9TtJRlYo15jyf4TR2EpZ7lYyMzLZ17yPht4GCjILWJ67HHfYTSKVoDKnksqcyuG13xIkUgkyMzKVRdkMdQbRRHTYOKfbEEwQhMvTFRfQp9rydqyZ/AXPBYxao9JzpbGvEbWkJiknae9czbkLdyv3Z1hfwpJ9iFxTLlXZ2/FEPJx2nUar1g5syQ976A32siZvDcWZxcr2/KFVKOFEmNKsUnKMOcruzTsr7+RY1zFxLJwgCKNccQF9qi1vx5rJr7Wv5VTPKXKNuQMfAsFuvL4Cjpy+U7nPZGpDnfsDNCqJSCpBgamAi76LOL1O8kx5/P/t3Xls2+d5wPHvQ4kiRdISdUeWZFuO7MR2lcaxldpF0tpNZ8dGlyyDt6Zd0AUIsjUDgmFBMWzoNhRbgg1FuwUDsiNAs2VXltWoa69dsqyuUztpHMm341PyIVtHTN3UxUPiuz94mLJliZIpXno+gAGKvx/J5zHlxy9fvu/zayhtYEP1Blr7W7kxeoPO4U5efPjF2KeD+GZf5Y5yapbUTGmn6/V7cRe6cVgdumZcKTXFoivoiba8nW4kv9y9nPFguJthy7U29n8U37jKsH3TAS4PnaJzxI8jrwjvqJdD1w5RaC3EnmcnMBlgyDcEwOa6zbGe5PHFOH4VSvRTwq2j8URXtCilFhfL7Kfklmjfk+gI12F1TFsgoyP5qN7RXg5cPcClgXb+aq/wbz+/P3bswTXvUV7/HY57DjFpJlldspr+8X4CoQD2fHu4V3koSLWrmkpXJa394asNzTb3nWisSikFi3CEDomtxY4fyfuCPg5dP0RbeyMd3TcL+Vc2eNnQ4KO134nnSgH+CT91xXWMBEbYsmILVwavUGwrZk3FGm6M3KB9qJ0qVxWDvkG8fm9Cc9+6blwplahFWdATEd/BcPexNo6f/43YMeeSs9SvOAjO5ZQ7tlPuKGdwPNzcaseqHbzb9i5uu5v+8X6G/OEplgpnBf5Jf2wTkcPq0LlvpVRS5WxBb+loYfe53XR6O6kpqmHXml001TbN6Tn8ATd/8GYBEN5xa7P62L7pY7pGLzPqn+DUp6dY6V5J73gv53vP47A66B3rpdhejG/CR6m9lGH/MGPBMYwxlBSWsKp0lU6bKKUWRE72cmnpaOF7H32PssIy3HY3g75B+sb7+Nbmb92xqMevOS8qqOD7/+2kZzgYO/5Y0y9xu3w4rA68Pi+nPaeZNJPY8mw0lDYw5BsiEArgsDporGzktOc0IsK68nV0jnTe8YITSik1m5zq5TLXrfq7z+2mrLAMq8VKa38rY8ExJswEb5x4Y0pBjz5vc0czR7qPUGIvxdO9navdBUC4mP/dbz3EA8sm+cP/+3eKTBXGGPLz8qlZUoNn1MNwcDh2pSCAk56TtA20sa5iHVcGrnDyxklqimp4adNLc/6EoJRSc5HxBX0+W/U7vZ247W4u9F3AbrXjtDrxTfg42nWUjqGO2KXj9l7Yy2RokqNdRxkaXEtr97bYcyyrbuMbjzjZ2VgNwNb6reFdob4Bim3FfKn+S3xw/QMqnZVsrtsce9zWFVs513MOZ4GTR5Y/Eltu2NzVTPWSah2dK6UWTMYX9Lls1Y+qKarhxI0TFFoLseXZAAiGglS6KmOPiz7vwbZ2Lp77/dhj8ws+Zc19PwUJsv+KKzZFsqNhB4HJwJRWuXmSR7WresprjwRGGPANUFdcpw20lFIplfHr0HtGe3AVuKbc5ypw0TPac8fH7Fqzi96xXnwTPkKhEJ5RD1cHr2Kz2Hiv7T06hjpo7+/jT98q5n+bb17Hs3zZq7hr/55Caz6+CR9VripaulqA6deEP//Q8+RZ8vD6vYRMKLYUsdReOueYlVLqbmX8CD3Rrfrxmmqb+Orar3Lw2kGue6/jm/Sxvmo99yy5h0kT4pkfHOaq5+bIesPaDzk/vJeQxYbdYmc4MIxFLDRWNE4pwtOtCa9eUn3bpdtaulrmHLNSSt2tjC/oiW7Vv9XXG7+Os8BJa38rgmARC6cv38PF9rWxcx5dN0DTqn7OePoonyinf7wfp9VJKBRi273bsFvtU7sfTuNOG3/mE7NSSt2NjC/otcW1PLz04dvWlM82Fx2dInn50Mv0D1bQ/Mmm2LH7l4b4tc3dfGX1Tlq6WhgPjmMRC10jXdSX1McuXjHfIhy/KUkbaCmlUiXjC3rHUAfNXc08UPUAn6/7/JxWjFhMGe9+8Ks3fxbDy08bjHhxWCtuG13HL4+8252cumVfKZVqGV/Qp1vlEr1K0L0l9067Lt0/MclTr/2Ss93e2H0vbB/m/urCGac/tAgrpbJZ1q1y6R3r5YznDL1jvVS5qhgLjrH3wl46hjoAeOWnZ7nvT96NFfO//PVGPvjjz7L6Hpt2LFRK5bSMH6FHV7kEJgO09rdyvOs4IQmxqnQVFrHERu4/+OgI//T+ydjjNqycZNv6T7G7AkATT615Kk0ZKKVUamR8QW9a2sSbJ97kYv9FygrLCIaChEyIwbFBesd6mQiU88oed+z8clc+OzcfxRv0cPJGgIK+As54zvDsg8/qqFwpldMyfsqltriWSlclJYUl+Cf9FNuKaShtoLRwKd/9cTGv7LmZws9e+gK/u+Ma14Yvkid5lBaWkid5tA208U7bO2nMQimlFl7Gj9A7hjo42n0UCxbchW7qi1fyo4+ddPXUxc65796D7HqoAbttiGNdxyixl8TWjzusDkImxLGuY7AhXVkopdTCm3dBF5H7gLfj7loJ/Jkx5tW7jioi2kDLnm9HEM5fK+XYhZtXDHK5m/nyg0M0VjRit9rZe2EvI4ERCq2FU2NFMJK6NsFKKZUO8y7oxpgLwIMAIpIHdAJ7khQXcHPJ4grXel79SXns/iLnME0PvM9nq9ZSXzK1RbDL5qJvvA8RwZ5vxzfho2+8j0eXPZrM0JRSKuMka8rlMeCSMaY9Sc8HhJcsVrmqOOK5OU++deN+7LZxlrtXsNy9fMr5vqCPEf8IXr+XvvE+SmwllDnLWF26mscbHp/xtebac10ppTJNsgr608BbSXqumOiSxS+uKWLzqhCFBeD1b4jNj8c3wOod7eXQ9UOUOcvYUr+F057TeEY8rKtYx46GHTMW5/n0XFdKqUxz16tcRKQAeAL44R2O/46IHBGRIz09c2sf27S0iUHfIKNBLzbrzfa0TUubYseirWtPek5ijGFZ0TIuDVwiOBnufy7IrEU5fjdqdG272+6Otc5VSqlskIxlizuAY8aYG9MdNMa8bozZaIzZWFExt/ax0/Ugj46abz3mm/DRWNnIxb6L+Cf8uO1uLFjYf2V/bBfpncyn57pSSmWaZEy5fI0FmG6Jmqm/SvyxPef28Iv2X+AscMamZEQkdpGKmUbp8+m5rpRSmeauRugi4gB+BfhRcsKZv6alTXhGPIRMCGMMY8ExRgOjt12k4k6PjZ++iZ/aUUqpbHFXBd0YM2aMKTPGDCUroPmqLa5la/1WDIYB3wC2PBufq/kcdqt91pH2TFM7SimVLTJ+p+hcTHch50QvUqGtc5VS2S7je7nMhY60lVKLWU6N0EFH2kqpxStrCrru5FRKqZllxZRLdCfnWHBs2qsUKaWUypKCrjs5lVJqdllR0HUnp1JKzS4rCnp0J2c83cmplFJTZUVB152cSik1u6wo6Lq+XCmlZpc1yxZ1fblSSs0sK0boSimlZqcFXSmlcoQWdKWUyhFa0JVSKkdoQVdKqRwhxpjUvZhID9A+j4eWA71JDicbLMa8NefFYzHmPd+clxtjZt1JmdKCPl8icsQYszHdcaTaYsxbc148FmPeC52zTrkopVSO0IKulFI5IlsK+uvpDiBNFmPemvPisRjzXtCcs2IOXSml1OyyZYSulFJqFhlV0EXkcRG5ICJtIvJH0xy3icjbkeMfi8iK1EeZXAnk/JKInBWRUyKyX0SWpyPOZJst77jzdomIEZGsXw2RSM4i8puR9/uMiPxHqmNMtgR+v5eJyAEROR75Hd+ZjjiTSUTeEBGPiHxyh+MiIn8b+Ts5JSIPJe3FjTEZ8QfIAy4BK4EC4CSw9pZzfg/4h8jtp4G30x13CnLeCjgit1/I9pwTzTty3hLgIHAY2JjuuFPwXq8CjgMlkZ8r0x13CnJ+HXghcnstcDXdcSch7y8ADwGf3OH4TuAdQIBNwMfJeu1MGqE/DLQZYy4bYwLAfwJP3nLOk8Cbkdu7gcdERFIYY7LNmrMx5oAxZizy42EgF3oIJ/JeA/wF8F3Al8rgFkgiOT8PvGaMGQAwxnhSHGOyJZKzAYoit4uBrhTGtyCMMQeB/hlOeRL4FxN2GHCLSHUyXjuTCnoNcD3u547IfdOeY4yZAIaAspREtzASyTnec4T/Z892s+YtIuuBOmPMT1IZ2AJK5L1eDawWkQ9F5LCIPJ6y6BZGIjl/B3hGRDqA/wFeTE1oaTXXf/cJy6QLXEw30r51CU4i52SThPMRkWeAjcAXFzSi1JgxbxGxAH8DPJuqgFIgkfc6n/C0yxbCn8QOichnjDGDCxzbQkkk568B/2yM+b6IbAb+NZJzaOHDS5sFq2OZNELvAOrifq7l9o9fsXNEJJ/wR7SZPtpkukRyRkS+DHwbeMIY409RbAtptryXAJ8B3heRq4TnGfdl+Rejif5+7zXGBI0xV4ALhAt8tkok5+eA/wIwxnwE2An3O8llCf27n49MKugtwCoRqReRAsJfeu675Zx9wG9Hbu8Cfm4i3zJkqVlzjkw9/CPhYp7tc6pRM+ZtjBkyxpQbY1YYY1YQ/u7gCWPMkfSEmxSJ/H7/mPCX4IhIOeEpmMspjTK5Esn5GvAYgIisIVzQe1IaZertA74RWe2yCRgyxnQn5ZnT/Y3wNN/+XiT8zfi3I/f9OeF/zBB+s38ItAHNwMp0x5yCnH8G3ABORP7sS3fMqcj7lnPfJ8tXuST4Xgvw18BZ4DTwdLpjTkHOa4EPCa+AOQFsS3fMScj5LaAbCBIejT8HfBP4Ztz7/Frk7+R0Mn+3daeoUkrliEyaclFKKXUXtKArpVSO0IKulFI5Qgu6UkrlCC3oSimVI7SgK6VUjtCCrpRSOUILulJK5Yj/BwTAIBITxp5rAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "predicted = model.forward(torch.from_numpy(x_train)).data.numpy()\n",
    "plt.plot(x_train, y_train, 'go', label = 'data', alpha = 0.3)\n",
    "plt.plot(x_train, predicted, label = 'predicted', alpha = 1)\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上就是一个使用PyTorch做线性回归的简单样例了，下面我们会对上面的内容做详细的介绍\n",
    "##  2.2.3 损失函数(Loss Function)\n",
    "损失函数（loss function）是用来估量模型的预测值(我们例子中的output)与真实值（例子中的y_train）的不一致程度，它是一个非负实值函数，损失函数越小，模型的鲁棒性就越好。\n",
    "我们训练模型的过程，就是通过不断的迭代计算，使用梯度下降的优化算法，使得损失函数越来越小。损失函数越小就表示算法达到意义上的最优。\n",
    "\n",
    "这里有一个重点：因为PyTorch是使用mini-batch来进行计算的，所以损失函数的计算出来的结果已经对mini-batch取了平均\n",
    "\n",
    "常见（PyTorch内置）的损失函数有以下几个：\n",
    "### nn.L1Loss:\n",
    "输入x和目标y之间差的绝对值，要求 x 和 y 的维度要一样（可以是向量或者矩阵），得到的 loss 维度也是对应一样的\n",
    "\n",
    "$ loss(x,y)=1/n\\sum|x_i-y_i| $\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.NLLLoss:\n",
    "用于多分类的负对数似然损失函数\n",
    "\n",
    "$ loss(x, class) = -x[class]$\n",
    "\n",
    "NLLLoss中如果传递了weights参数，会对损失进行加权，公式就变成了\n",
    "\n",
    "$ loss(x, class) = -weights[class] * x[class] $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.MSELoss:\n",
    "均方损失函数 ，输入x和目标y之间均方差\n",
    "\n",
    "$ loss(x,y)=1/n\\sum(x_i-y_i)^2 $"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.CrossEntropyLoss:\n",
    "多分类用的交叉熵损失函数，LogSoftMax和NLLLoss集成到一个类中，会调用nn.NLLLoss函数，我们可以理解为CrossEntropyLoss()=log_softmax() + NLLLoss()\n",
    "\n",
    "\n",
    " $ \\begin{aligned} loss(x, class) &= -\\text{log}\\frac{exp(x[class])}{\\sum_j exp(x[j]))}\\ &= -x[class] + log(\\sum_j exp(x[j])) \\end{aligned}  $\n",
    " \n",
    " 因为使用了NLLLoss，所以也可以传入weight参数，这时loss的计算公式变为：\n",
    " \n",
    " $ loss(x, class) = weights[class] * (-x[class] + log(\\sum_j exp(x[j]))) $\n",
    " \n",
    " 所以一般多分类的情况会使用这个损失函数"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### nn.BCELoss:\n",
    "计算 x 与 y 之间的二进制交叉熵。\n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_i(t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $ \n",
    "\n",
    "与NLLLoss类似，也可以添加权重参数： \n",
    "\n",
    "$ loss(o,t)=-\\frac{1}{n}\\sum_iweights[i]* (t[i]* log(o[i])+(1-t[i])* log(1-o[i])) $\n",
    "\n",
    "用的时候需要在该层前面加上 Sigmoid 函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.4 梯度下降\n",
    "在介绍损失函数的时候我们已经说了，梯度下降是一个使损失函数越来越小的优化算法，在无求解机器学习算法的模型参数，即约束优化问题时，梯度下降（Gradient Descent）是最常采用的方法之一。所以梯度下降是我们目前所说的机器学习的核心，了解了它的含义，也就了解了机器学习算法的含义。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 梯度\n",
    "在微积分里面，对多元函数的参数求∂偏导数，把求得的各个参数的偏导数以向量的形式写出来，就是梯度。\n",
    "例如函数f(x,y), 分别对x，y求偏导数，求得的梯度向量就是(∂f/∂x, ∂f/∂y)T，简称grad f(x,y)或者▽f(x,y)。\n",
    "\n",
    "几何上讲，梯度就是函数变化增加最快的地方，沿着梯度向量的方向，更加容易找到函数的最大值。反过来说，沿着梯度向量相反的方向梯度减少最快，也就是更加容易找到函数的最小值。\n",
    "\n",
    "我们需要最小化损失函数，可以通过梯度下降法来一步步的迭代求解，得到最小化的损失函数，和模型参数值。\n",
    "### 梯度下降法直观解释\n",
    "梯度下降法就好比下山，我们并不知道下山的路，于是决定走一步算一步，每走到一个位置的时候，求解当前位置的梯度，沿着梯度的负方向，也就是当前最陡峭的位置向下走一步，然后继续求解当前位置梯度，向这一步所在位置沿着最陡峭最易下山的位置走一步。这样一步步的走下去，一直走到觉得我们已经到了山脚。\n",
    "\n",
    "如下图所示，（此图摘自百度百科）\n",
    "![](1.png)\n",
    "\n",
    "这样走下去，有可能我们不能走到山脚，而是到了某一个局部的山峰低处（局部最优解）。\n",
    "\n",
    "这个问题在以前的机器学习中可能会遇到，因为机器学习中的特征比较少，所以导致很可能陷入到一个局部最优解中出不来，但是到了深度学习，动辄百万甚至上亿的特征，出现这种情况的概率几乎为0，所以我们可以不用考虑这个问题。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mini-batch的梯度下降法\n",
    "对整个训练集进行梯度下降法的时候，我们必须处理整个训练数据集，然后才能进行一步梯度下降，即每一步梯度下降法需要对整个训练集进行一次处理，如果训练数据集很大的时候处理速度会很慢，而且也不可能一次的载入到内存或者显存中，所以我们会把大数据集分成小数据集，一部分一部分的训练，这个训练子集即称为Mini-batch。\n",
    "在PyTorch中就是使用这种方法进行的训练，可以看看上一章中关于dataloader的介绍里面的batch_size就是我们一个Mini-batch的大小。\n",
    "\n",
    "为了介绍的更简洁，使用 吴恩达老师的 [deeplearning.ai](https://www.deeplearning.ai/deep-learning-specialization/) 课程板书。\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于普通的梯度下降法，一个epoch只能进行一次梯度下降；而对于Mini-batch梯度下降法，一个epoch可以进行Mini-batch的个数次梯度下降。\n",
    "![](2.png)\n",
    "普通的batch梯度下降法和Mini-batch梯度下降法代价函数的变化趋势，如下图所示：\n",
    "![](3.png)\n",
    "- 如果训练样本的大小比较小时，能够一次性的读取到内存中，那我们就不需要使用Mini-batch，\n",
    "- 如果训练样本的大小比较大时，一次读入不到内存或者现存中，那我们必须要使用 Mini-batch来分批的计算\n",
    "- Mini-batch size的计算规则如下，在内存允许的最大情况下使用2的N次方个size\n",
    "![](4.png)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`torch.optim`是一个实现了各种优化算法的库。大部分常用优化算法都有实现，我们直接调用即可。\n",
    "### torch.optim.SGD\n",
    "随机梯度下降算法，带有动量（momentum）的算法作为一个可选参数可以进行设置，样例如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "#lr参数为学习率，对于SGD来说一般选择0.1 0.01.0.001，如何设置会在后面实战的章节中详细说明\n",
    "##如果设置了momentum，就是带有动量的SGD，可以不设置\n",
    "optimizer = torch.optim.SGD(model.parameters(), lr=0.1, momentum=0.9)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.RMSprop\n",
    "除了以上的带有动量Momentum梯度下降法外，RMSprop（root mean square prop）也是一种可以加快梯度下降的算法，利用RMSprop算法，可以减小某些维度梯度更新波动较大的情况，使其梯度下降的速度变得更快"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "#我们的课程基本不会使用到RMSprop所以这里只给一个实例\n",
    "optimizer = torch.optim.RMSprop(model.parameters(), lr=0.01, alpha=0.99)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### torch.optim.Adam\n",
    "Adam 优化算法的基本思想就是将 Momentum 和 RMSprop 结合起来形成的一种适用于不同深度学习结构的优化算法"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这里的lr，betas，还有eps都是用默认值即可，所以Adam是一个使用起来最简单的优化方法\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.001, betas=(0.9, 0.999), eps=1e-08)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.5 方差/偏差\n",
    "- 偏差度量了学习算法的期望预测与真实结果的偏离程序，即刻画了学习算法本身的拟合能力\n",
    "- 方差度量了同样大小的训练集的变动所导致的学习性能的变化，即模型的泛化能力\n",
    "![](5.png)\n",
    "\n",
    "从图中我们可以看出\n",
    "- 高偏差（high bias）的情况，一般称为欠拟合（underfitting），即我们的模型并没有很好的去适配现有的数据，拟合度不够。\n",
    "- 高方差（high variance）的情况一般称作过拟合（overfitting），即模型对于训练数据拟合度太高了，失去了泛化的能力。\n",
    "\n",
    "如何解决这两种情况呢？\n",
    "\n",
    "欠拟合：\n",
    "- 增加网络结构，如增加隐藏层数目；\n",
    "- 训练更长时间；\n",
    "- 寻找合适的网络架构，使用更大的NN结构；\n",
    "\n",
    "过拟合 ：\n",
    "- 使用更多的数据；\n",
    "- 正则化（ regularization）；\n",
    "- 寻找合适的网络结构；\n",
    "\n",
    "例如我们上面的例子，可以计算出我们的偏差:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.005641937255859375 -0.025215625762939453\n"
     ]
    }
   ],
   "source": [
    "print (5-w.data.item(),7-b.data.item())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.2.6 正则化\n",
    "利用正则化来解决High variance 的问题，正则化是在 Cost function 中加入一项正则化项，惩罚模型的复杂度，这里我们简单的介绍一下正则化的概念"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L1正则化\n",
    "损失函数基础上加上权重参数的绝对值\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} \\left|w_j\\right|$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### L2正则化\n",
    "损失函数基础上加上权重参数的平方和\n",
    "\n",
    "$ L=E_{in}+\\lambda{\\sum_j} w^2_j$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "需要说明的是：l1 相比于 l2 会更容易获得稀疏解\n",
    "\n",
    "[知乎](https://www.zhihu.com/question/37096933/answer/70507353)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}