Pandas.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Pandas简介\n",
    "\n",
    "Python作为一门通用语言，并不像Stata、R等语言一样自带数据管理功能，因而通常需要额外的包，这其中Pandas由于强大的性能以及易用性成为了Python中最流行的数据管理包。在这里我们将对Pandas的基本功能做简要介绍。Pandas的官方文档地址为：https://pandas.pydata.org/docs/\n",
    "\n",
    "实际上Pandas是建立在NumPy基础之上的，接下来会看到Pandas的很多操作与NumPy非常类似。为了使用Panda，我们需要首先导入："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Pandas支持从多种数据源读入数据，包括csv文件、Stata文件、SPSS文件、SAS文件、数据库（SQL语言）、Excel文件等等。比如，如下代码导入了一个.csv文件："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>lon</th>\n",
       "      <th>lat</th>\n",
       "      <th>province</th>\n",
       "      <th>city</th>\n",
       "      <th>name</th>\n",
       "      <th>address</th>\n",
       "      <th>brand</th>\n",
       "      <th>category</th>\n",
       "      <th>birth</th>\n",
       "      <th>all_day</th>\n",
       "      <th>wifi</th>\n",
       "      <th>store_view</th>\n",
       "      <th>basketball</th>\n",
       "      <th>self_service</th>\n",
       "      <th>attractions</th>\n",
       "      <th>train</th>\n",
       "      <th>breakfast</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>15487</td>\n",
       "      <td>116.280940</td>\n",
       "      <td>39.951740</td>\n",
       "      <td>北京市</td>\n",
       "      <td>北京市</td>\n",
       "      <td>板井路</td>\n",
       "      <td>远大路远大居住区二期世纪金源大酒店一层东南角</td>\n",
       "      <td>KFC</td>\n",
       "      <td>餐饮</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>15205</td>\n",
       "      <td>116.362701</td>\n",
       "      <td>39.794070</td>\n",
       "      <td>北京市</td>\n",
       "      <td>北京市</td>\n",
       "      <td>大兴新宫DT</td>\n",
       "      <td>西红门路丙13号一层加二层</td>\n",
       "      <td>KFC</td>\n",
       "      <td>餐饮</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>15184</td>\n",
       "      <td>116.209496</td>\n",
       "      <td>39.752680</td>\n",
       "      <td>北京市</td>\n",
       "      <td>北京市</td>\n",
       "      <td>长阳</td>\n",
       "      <td>长阳镇起步区五号地商业综合体中粮万科半岛商业广场首层L1017A及二层L2030A??</td>\n",
       "      <td>KFC</td>\n",
       "      <td>餐饮</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>15186</td>\n",
       "      <td>116.157956</td>\n",
       "      <td>39.723521</td>\n",
       "      <td>北京市</td>\n",
       "      <td>北京市</td>\n",
       "      <td>良乡长虹东路</td>\n",
       "      <td>拱辰街道东羊庄村18号一层</td>\n",
       "      <td>KFC</td>\n",
       "      <td>餐饮</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>15183</td>\n",
       "      <td>116.220176</td>\n",
       "      <td>40.222182</td>\n",
       "      <td>北京市</td>\n",
       "      <td>北京市</td>\n",
       "      <td>昌平京客隆</td>\n",
       "      <td>西关路20号三号楼京客隆超市一层二层西侧3号商铺</td>\n",
       "      <td>KFC</td>\n",
       "      <td>餐饮</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      id         lon        lat province city    name  \\\n",
       "0  15487  116.280940  39.951740      北京市  北京市     板井路   \n",
       "1  15205  116.362701  39.794070      北京市  北京市  大兴新宫DT   \n",
       "2  15184  116.209496  39.752680      北京市  北京市      长阳   \n",
       "3  15186  116.157956  39.723521      北京市  北京市  良乡长虹东路   \n",
       "4  15183  116.220176  40.222182      北京市  北京市   昌平京客隆   \n",
       "\n",
       "                                       address brand category  birth  all_day  \\\n",
       "0                       远大路远大居住区二期世纪金源大酒店一层东南角   KFC       餐饮   True    False   \n",
       "1                                西红门路丙13号一层加二层   KFC       餐饮   True    False   \n",
       "2  长阳镇起步区五号地商业综合体中粮万科半岛商业广场首层L1017A及二层L2030A??   KFC       餐饮  False    False   \n",
       "3                                拱辰街道东羊庄村18号一层   KFC       餐饮   True    False   \n",
       "4                     西关路20号三号楼京客隆超市一层二层西侧3号商铺   KFC       餐饮   True    False   \n",
       "\n",
       "    wifi  store_view  basketball  self_service  attractions  train  breakfast  \n",
       "0  False        True        True          True        False  False       True  \n",
       "1  False        True       False          True        False  False       True  \n",
       "2  False       False       False          True        False  False      False  \n",
       "3  False        True        True          True        False  False       True  \n",
       "4  False        True        True          True        False  False       True  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "kfc = pd.read_csv(\"csv/kfc.csv\")\n",
    "kfc.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 序列\n",
    "\n",
    "Pandas最基础的对象是**序列（Series）**，而一个序列可以简单的看做是带有**索引（Index）**的一个NumPy数组，比如我们可以使用如下方法创建一个简单的序列："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1\n",
       "1    2\n",
       "2    3\n",
       "dtype: int64"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "a = pd.Series([1, 2, 3])\n",
    "a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者可以使用NumPy进行创建："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0.382985\n",
       "1    0.603967\n",
       "2    0.958308\n",
       "3    0.040463\n",
       "4    0.315148\n",
       "5    0.125481\n",
       "6    0.488112\n",
       "7    0.449575\n",
       "8    0.814861\n",
       "9    0.304866\n",
       "dtype: float64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "b = pd.Series(np.random.random(10))\n",
    "b"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意以上可以看到，Series对象在数组的基础上海多了一列「行号」，这个「行号」其实是Pandas的「索引」，索引是Series和array的关键区别。我们将在稍后介绍索引的使用方法。\n",
    "\n",
    "除了索引之外，Series很多行为都跟array没有本质区别，比如我们可以对其进行计算："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0   -0.959760\n",
      "1   -0.504236\n",
      "2   -0.042586\n",
      "3   -3.207357\n",
      "4   -1.154711\n",
      "5   -2.075602\n",
      "6   -0.717211\n",
      "7   -0.799452\n",
      "8   -0.204738\n",
      "9   -1.187882\n",
      "dtype: float64\n",
      "b的类型： <class 'pandas.core.series.Series'>\n",
      "b的和= 4.483766710801085\n",
      "b*log_b=\n",
      " 0   -0.367573\n",
      "1   -0.304542\n",
      "2   -0.040810\n",
      "3   -0.129781\n",
      "4   -0.363906\n",
      "5   -0.260448\n",
      "6   -0.350079\n",
      "7   -0.359414\n",
      "8   -0.166833\n",
      "9   -0.362145\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "log_b = np.log(b)\n",
    "print(log_b)\n",
    "print(\"b的类型：\", type(log_b))\n",
    "print(\"b的和=\", b.sum())\n",
    "blogb = b * log_b\n",
    "print(\"b*log_b=\\n\", blogb)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意到将一个Series对象进行计算之后，仍然是一个Series对象，且索引值保持了对应关系。\n",
    "\n",
    "在Pandas中，缺失值的处理也比较方便，可以使用numpy.nan（not a number）以及Python原生的None代表缺失值，一般而言两者等价，不需要额外关注使用哪种解决方案。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/lib/python3/dist-packages/pandas/core/series.py:856: RuntimeWarning: invalid value encountered in log\n",
      "  result = getattr(ufunc, method)(*inputs, **kwargs)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "0         NaN\n",
       "1   -2.263686\n",
       "2   -0.780213\n",
       "3         NaN\n",
       "4         NaN\n",
       "5         NaN\n",
       "6         NaN\n",
       "7         NaN\n",
       "8   -1.155624\n",
       "9         NaN\n",
       "dtype: float64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_b1 = np.log(b - 0.5)\n",
    "log_b1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到上面所有不可计算的值都被NaN代替了。可以使用isnull()函数以及notnull()函数判断是否为缺失值，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    False\n",
       "1     True\n",
       "2     True\n",
       "3    False\n",
       "4    False\n",
       "5    False\n",
       "6    False\n",
       "7    False\n",
       "8     True\n",
       "9    False\n",
       "dtype: bool"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.notnull(log_b1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果希望扔掉缺失值，可以用dropna()："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1   -2.263686\n",
       "2   -0.780213\n",
       "8   -1.155624\n",
       "dtype: float64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_b1.dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 字符串序列\n",
    "\n",
    "与NumPy的array不一样的是，Pandas中的Series的数据类型可以不仅仅是数值，而array一般为数值类型。比如我们可以创建一个字符串的Series："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0     Messi-10\n",
       "1       Xavi-6\n",
       "2    Iniesta-8\n",
       "3      Puyol-5\n",
       "dtype: object"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = pd.Series(['Messi-10', 'Xavi-6', 'Iniesta-8', 'Puyol-5'])\n",
    "s"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "NumPy中并没有针对字符串的向量化函数，Pandas则提供了常用的可以用于处理字符串的向量化函数，常见的比如len(), split(), index(), lower(), islower(), uppper(), isupper(), isdigit(), strip(),.....等Python字符串支持的函数，都有向量化的函数对应。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "字符串长度=\n",
      " 0    8\n",
      "1    6\n",
      "2    9\n",
      "3    7\n",
      "dtype: int64\n",
      "分割的字符串:\n",
      " 0     [Messi, 10]\n",
      "1       [Xavi, 6]\n",
      "2    [Iniesta, 8]\n",
      "3      [Puyol, 5]\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "len_s = s.str.len()\n",
    "print(\"字符串长度=\\n\", len_s)\n",
    "split_s = s.str.split('-')\n",
    "print(\"分割的字符串:\\n\", split_s)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现这些操作仍然是保留索引的。\n",
    "\n",
    "除此之外，还可以向量化地使用正则表达式。Pandas中支持如下正则表达式的方法：\n",
    "\n",
    "* match(): 对每个元素调用re.match()\n",
    "* extract(): 对每个元素调用re.match()，返回匹配的字符串组\n",
    "* findall(): 对每个元素调用re.findall()\n",
    "* replace(): 用正则表达式替换字符串\n",
    "* contains(): 对每个元素调用re.search()\n",
    "* count(): 对每个元素计算符合正则模式的字符串数量\n",
    "* split(), rsplit(): 分割字符串\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    [10]\n",
      "1     [6]\n",
      "2     [8]\n",
      "3     [5]\n",
      "dtype: object\n",
      "0      [Messi]\n",
      "1       [Xavi]\n",
      "2    [Iniesta]\n",
      "3      [Puyol]\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "code = s.str.findall(r'\\d+')\n",
    "name = s.str.findall(r'[a-zA-Z]+')\n",
    "print(code)\n",
    "print(name)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意以上使用split()、正则表达式得到的结果都是一个列表，可以使用str[]的方式获取其中的元素，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    10\n",
      "1     6\n",
      "2     8\n",
      "3     5\n",
      "dtype: int64\n",
      "0      Messi\n",
      "1       Xavi\n",
      "2    Iniesta\n",
      "3      Puyol\n",
      "dtype: object\n"
     ]
    }
   ],
   "source": [
    "code1 = code.str[0].astype('int_')\n",
    "print(code1)\n",
    "name1 = name.str[0]\n",
    "print(name1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意对于code，我们还额外使用了astype()函数将字符串数字转为了整型。\n",
    "\n",
    "此外还有很多关于字符串的方法，具体可以参考：https://pandas.pydata.org/docs/user_guide/text.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 时间序列\n",
    "\n",
    "这里标题中的时间序列指的是以时间为值的一个Pandas Series。时间在时间序列数据和面板数据中非常重要，而Python原生的Datetime类型以为性能较差且不能方便的向量化运算，因而在Pandas中借鉴了NumPy中的时间类型，比如在NumPy中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01-01\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array(['2020-01-01', '2020-01-02', '2020-01-03', '2020-01-04',\n",
       "       '2020-01-05'], dtype='datetime64[D]')"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "begin = np.array('2020-01-01', dtype=np.datetime64)\n",
    "print(begin)\n",
    "delta = np.arange(5)\n",
    "nextdays = begin + delta\n",
    "nextdays"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上可以看出，NumPy中日期时间类型的一个好处是可以方便的进行向量化操作。\n",
    "\n",
    "在Pandas中，引入了时间戳（timestamp），可以方便的将字符串转换为时间戳，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-03-13 00:00:00\n",
      "2020-01-13 00:00:00\n",
      "2020-03-13 00:00:00\n"
     ]
    }
   ],
   "source": [
    "import datetime\n",
    "\n",
    "dt = pd.Timestamp(datetime.datetime(2020, 3,\n",
    "                                    13))  ## 将Python的日期时间转换为pandas的日期时间\n",
    "print(dt)\n",
    "dt1 = pd.Timestamp('2020-1-13')  ## 将字符串转换为pandas的日期时间\n",
    "print(dt1)\n",
    "dt2 = pd.Timestamp(2020, 3, 13)\n",
    "print(dt2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意以上代表的是一个时间点（timestamp就是时间点的意思），而有的时候我们希望表示一个时间周期，比如2020-3-13我们可能希望表达的是这一整天，而非那一天的0点0分0秒，此时我们需要使用timespan，即时间间隔，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-01\n",
      "2020-01-30\n",
      "2020-01\n",
      "2020Q2\n",
      "2020Q4\n"
     ]
    }
   ],
   "source": [
    "dt = pd.Period('2020-01')\n",
    "print(dt)\n",
    "dt1 = pd.Period('2020-01-30', freq='D')\n",
    "print(dt1)\n",
    "dt2 = pd.Period('2020-01', freq='M')\n",
    "print(dt2)\n",
    "dt3 = pd.Period('2020-04', freq='Q')\n",
    "print(dt3)\n",
    "dt4 = pd.Period('2020q4', freq='Q')\n",
    "print(dt4)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "具体的日期时间转换函数可以查看：https://pandas.pydata.org/docs/user_guide/timeseries.html\n",
    "\n",
    "当然，这些日期时间都可以进行向量化计算，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dta=\n",
      " 0    2020-04\n",
      "1    2020-05\n",
      "2    2020-06\n",
      "3    2020-07\n",
      "4    2020-08\n",
      "5    2020-09\n",
      "6    2020-10\n",
      "7    2020-11\n",
      "8    2020-12\n",
      "9    2021-01\n",
      "dtype: period[M]\n",
      "-----------\n",
      "dta2=\n",
      " 0    2020-01-30\n",
      "1    2020-01-31\n",
      "2    2020-02-01\n",
      "3    2020-02-02\n",
      "4    2020-02-03\n",
      "5    2020-02-04\n",
      "6    2020-02-05\n",
      "7    2020-02-06\n",
      "8    2020-02-07\n",
      "9    2020-02-08\n",
      "dtype: period[D]\n"
     ]
    }
   ],
   "source": [
    "dt1 = pd.Period('2020-04', freq='M')\n",
    "a = pd.Series(np.arange(10))\n",
    "dta = dt1 + a\n",
    "dt2 = pd.Period('2020-01-30', freq='D')\n",
    "dta2 = dt2 + a\n",
    "print(\"dta=\\n\", dta)\n",
    "print(\"-----------\")\n",
    "print(\"dta2=\\n\", dta2)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外也可以使用shift()进行时间的迁移，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dta向前迁移1月\n",
      " 0        NaT\n",
      "1    2020-04\n",
      "2    2020-05\n",
      "3    2020-06\n",
      "4    2020-07\n",
      "5    2020-08\n",
      "6    2020-09\n",
      "7    2020-10\n",
      "8    2020-11\n",
      "9    2020-12\n",
      "dtype: period[M]\n",
      "-----------\n",
      "dta向后迁移1天\n",
      " 0    2020-01-31\n",
      "1    2020-02-01\n",
      "2    2020-02-02\n",
      "3    2020-02-03\n",
      "4    2020-02-04\n",
      "5    2020-02-05\n",
      "6    2020-02-06\n",
      "7    2020-02-07\n",
      "8    2020-02-08\n",
      "9           NaT\n",
      "dtype: period[D]\n"
     ]
    }
   ],
   "source": [
    "print(\"dta向前迁移1月\\n\", dta.shift(1))\n",
    "print(\"-----------\")\n",
    "print(\"dta向后迁移1天\\n\", dta2.shift(-1))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Map函数\n",
    "\n",
    "对于序列，如果需要计算比较复杂的函数，而凭借NumPy自带的函数又无法轻易完成，此时可以使用序列的Map方法。该方法的使用与我们之前讲过的Python中的map()函数使用类似，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0    一\n",
      "1    二\n",
      "2    五\n",
      "dtype: object\n",
      "0    1\n",
      "1    2\n",
      "2    5\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "str_series = pd.Series(['一', '二', '五'])\n",
    "print(str_series)\n",
    "\n",
    "\n",
    "def trans_cn_num(s):\n",
    "    dic = {\n",
    "        '一': 1,\n",
    "        '二': 2,\n",
    "        '三': 3,\n",
    "        '四': 4,\n",
    "        '五': 5,\n",
    "        '六': 6,\n",
    "        '七': 7,\n",
    "        '八': 8,\n",
    "        '九': 9,\n",
    "        '十': 10,\n",
    "    }\n",
    "    if s in dic:\n",
    "        return dic[s]\n",
    "    else:\n",
    "        return None\n",
    "\n",
    "\n",
    "num_series = str_series.map(trans_cn_num)\n",
    "print(num_series)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "序列的map()方法接受一个函数，并将该函数作用于序列的每一个成员上，然后返回一个对应的序列。当然，如果NumPy和Pandas的函数可以方便计算所需结果，更快更好的方式是使用这些NumPy、Pandas自带的函数。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 索引"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上可以看到，Pandas的数据与NumPy的数据最大差别在于索引，可以简单认为Pandas的Series是带有索引的array。\n",
    "\n",
    "索引在Pandas中扮演着至关重要的作用，而且更加方便的在于，我们可以用很多种类型的数据（比如字符串、元组、时间戳、时间间隔等）作为索引，而不仅仅是行号。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "不指定索引：\n",
      " 0    1.0\n",
      "1    2.0\n",
      "2    3.0\n",
      "dtype: float64\n",
      "---------\n",
      "指定索引：\n",
      " 1    1.0\n",
      "5    2.0\n",
      "3    3.0\n",
      "dtype: float64\n",
      "---------\n",
      "指定字符串索引：\n",
      " 小明    1.0\n",
      "小强    2.0\n",
      "小红    3.0\n",
      "dtype: float64\n",
      "---------\n",
      "指定字符串索引：\n",
      " 2020-01    1.0\n",
      "2020-02    2.0\n",
      "2020-03    3.0\n",
      "Freq: M, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "a = pd.Series([1., 2, 3])\n",
    "\n",
    "print(\"不指定索引：\\n\", a)\n",
    "print('---------')\n",
    "b = pd.Series([1., 2, 3], index=[1, 5, 3])\n",
    "print(\"指定索引：\\n\", b)\n",
    "print('---------')\n",
    "c = pd.Series([1., 2, 3], index=['小明', '小强', '小红'])\n",
    "print(\"指定字符串索引：\\n\", c)\n",
    "print('---------')\n",
    "dt1 = pd.Period('2020-01', freq='M')\n",
    "dt = dt1 + np.arange(3)\n",
    "d = pd.Series([1., 2, 3], index=dt)\n",
    "print(\"指定字符串索引：\\n\", d)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "既然带有索引，我们可以很方便的使用索引进行切片的操作，实际上，之前NumPy讲过的切片、掩码操作在这里都是可以使用的，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a的前两个元素：\n",
      " 0    1.0\n",
      "1    2.0\n",
      "dtype: float64\n",
      "b的索引为5的元素： 2.0\n",
      "b的前两个元素：\n",
      " 1    1.0\n",
      "5    2.0\n",
      "dtype: float64\n",
      "c中小红的元素： 3.0\n",
      "d中2月份的元素： 2.0\n",
      "d中3月份的元素： 3.0\n"
     ]
    }
   ],
   "source": [
    "print(\"a的前两个元素：\\n\", a[0:2])\n",
    "print(\"b的索引为5的元素：\", b[5])\n",
    "print(\"b的前两个元素：\\n\", b[0:2])\n",
    "print(\"c中小红的元素：\", c['小红'])\n",
    "print(\"d中2月份的元素：\", d['2020-02'])\n",
    "print(\"d中3月份的元素：\", d[pd.Period('2020-03', freq='M')])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "值得注意的是b的两行比较具有迷惑性：当我们使用b[5]时，似乎选取的是索引值为5元素，而当我们使用b[0:2]时，似乎选取的又是前两行，行为非常迷惑。\n",
    "\n",
    "实际上，当我们使用b[]选取单个元素时，的确是选取的索引值，也被称为**显式索引**，而当我们使用切片操作时，则默认使用**隐式索引**，也就是按照顺序取行号。\n",
    "\n",
    "这会导致一些困惑，为了避免这个问题，建议在写代码时，要么避免使用整数作为索引，要么最好还是使用索引器使得代码更加直观。常用索引器有：\n",
    "\n",
    "* loc[]：**显式索引**\n",
    "* iloc[]：**隐式索引**\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "显式索引：\n",
      " 1    1.0\n",
      "5    2.0\n",
      "3    3.0\n",
      "dtype: float64\n",
      "隐式索引：\n",
      " 5    2.0\n",
      "3    3.0\n",
      "dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(\"显式索引：\\n\", b.loc[1:3])\n",
    "print(\"隐式索引：\\n\", b.iloc[1:3])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外掩码也可以使用，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2020-02    2.0\n",
      "2020-03    3.0\n",
      "Freq: M, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print(d[d > 1])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于有了索引，我们除了可以将Series看做带有index的array之外，其实还可以将其看做带有顺序的词典，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.0\n"
     ]
    }
   ],
   "source": [
    "print(c['小红'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实我们也可以使用字典创建Series，比如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "小明    99\n",
       "小红    88\n",
       "小青    66\n",
       "dtype: int64"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score = {'小明': 99, '小红': 88, '小青': 66}\n",
    "score_pd = pd.Series(score)\n",
    "score_pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其实我们还可以使用「score_pd.小红」的方式取值，不过值得注意的是，「score_pd.小红」的写法虽然方便，但是容易与Series对象的属性、方法产生歧义，所以请尽量避免使用这种写法。\n",
    "\n",
    "而对于时间索引，前面我们已经介绍了使用shift()函数进行时间的推移，如果时间作为索引，我们还可以使用tshift()做时间索引的推移，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "d=\n",
      " 2020-01    1.0\n",
      "2020-02    2.0\n",
      "2020-03    3.0\n",
      "Freq: M, dtype: float64\n",
      "e=\n",
      " 2020-02    1.0\n",
      "2020-03    2.0\n",
      "2020-04    3.0\n",
      "Freq: M, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "print('d=\\n', d)\n",
    "e = d.tshift(1)\n",
    "print('e=\\n', e)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 索引对象\n",
    "\n",
    "Pandas中索引本身就是一个对象，比如我们可以通过："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['小明', '小红', '小青'], dtype='object')"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_pd.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "查看索引的取值，并使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'小明'"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_pd.index[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "来对索引进行取值。也可以为索引取名："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "name\n",
       "小明    99\n",
       "小红    88\n",
       "小青    66\n",
       "dtype: int64"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "score_pd.index.names = ['name']\n",
    "score_pd"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意这个索引对象也是可迭代的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "小明 -> 99\n",
      "小红 -> 88\n",
      "小青 -> 66\n"
     ]
    }
   ],
   "source": [
    "for n in score_pd.index:\n",
    "    print(n, '->', score_pd[n])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，针对时间有专门的时间索引对象，比如时间戳索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "DatetimeIndex(['2020-01-31 15:00:00', '2020-02-29 15:00:00',\n",
       "               '2020-03-31 15:00:00', '2020-04-30 15:00:00',\n",
       "               '2020-05-31 15:00:00'],\n",
       "              dtype='datetime64[ns]', freq='M')"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts = pd.date_range('2020-1-1 15:00:00', periods=5, freq='M')\n",
    "ts"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "时间戳索引还可以进行运算，比如计算间隔天数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "TimedeltaIndex(['395 days', '423 days', '453 days', '482 days', '512 days'], dtype='timedelta64[ns]', freq=None)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ts1 = pd.date_range('2019-1-1 15:00:00', periods=5, freq='D')\n",
    "ts - ts1\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意上面的提示为TimedeltaIndex，即表示了一个时间差（time delta）。\n",
    "\n",
    "也可以使用pd.period_range()函数创建时间间隔的索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PeriodIndex(['2000', '2001', '2002', '2003', '2004', '2005', '2006', '2007',\n",
       "             '2008', '2009', '2010'],\n",
       "            dtype='period[A-DEC]', freq='A-DEC')"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.period_range('2000', '2010', freq='Y')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 层级索引\n",
    "\n",
    "有事数据的索引不仅仅是一维的，可能是更多维的，一个比较典型的例子是面板数据，同时包括了个体和时间两个维度，用这两个维度的信息才能唯一确定一个观测。此时我们需要使用层级索引。\n",
    "\n",
    "在Pandas中，为了创建多级索引，只需要在创建数据时设置index为多维就可以了，比如"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "SH  2000    0\n",
       "    2001    1\n",
       "BJ  2000    2\n",
       "    2001    3\n",
       "HZ  2000    4\n",
       "    2001    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.Series(np.arange(6),\n",
    "                 index=[['SH', 'SH', 'BJ', 'BJ', 'HZ', 'HZ'],\n",
    "                        [2000, 2001, 2000, 2001, 2000, 2001]])\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现上面的Series中索引已经变成了两级。我们也可以为索引加上名称，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "city  year\n",
       "SH    2000    0\n",
       "      2001    1\n",
       "BJ    2000    2\n",
       "      2001    3\n",
       "HZ    2000    4\n",
       "      2001    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index.names = ['city', 'year']\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以查看这个索引，是一个多级索引对象："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('SH', 2000),\n",
       "            ('SH', 2001),\n",
       "            ('BJ', 2000),\n",
       "            ('BJ', 2001),\n",
       "            ('HZ', 2000),\n",
       "            ('HZ', 2001)],\n",
       "           names=['city', 'year'])"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以查看其层次的个数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index.nlevels"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "也可以列出某一个维度的索引的所有值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['SH', 'SH', 'BJ', 'BJ', 'HZ', 'HZ'], dtype='object', name='city')"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index.get_level_values(0)  #data.index.get_level_values('city')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者索引的取值的唯一值："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Int64Index([2000, 2001], dtype='int64', name='year')"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.index.unique(1)  #data.index.unique('year')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有了多级索引之后，可以同样使用切片等操作，比如获取单个元素："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['SH', 2001]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者获取某一个城市的数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "year\n",
       "2000    0\n",
       "2001    1\n",
       "dtype: int64"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['SH']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者某一年份的数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "city\n",
       "SH    1\n",
       "BJ    3\n",
       "HZ    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[:,2001]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以及掩码："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "city  year\n",
       "BJ    2001    3\n",
       "HZ    2000    4\n",
       "      2001    5\n",
       "dtype: int64"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data>=3]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 数据框\n",
    "\n",
    "虽然序列对象非常方便，但是仍然非常基础，每个序列只是一个一维的向量，而我们的数据通常都是有很多维度的变量的。在序列的基础上，我们可以把多个序列放在一起，就成为了**数据框（DataFrames）**\n",
    "\n",
    "数据框可以用多个Series创建，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0\n",
       "Xavi      NaN    60.0"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "goals = pd.Series([35, 10, 3], index=['Messi', 'Suarez', 'Pique'])\n",
    "passes = pd.Series([55, 40, 60], index=['Messi', 'Suarez', 'Xavi'])\n",
    "\n",
    "data = pd.DataFrame({'goals': goals, 'passes': passes})\n",
    "data.index.name = 'name'\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到Pandas很智能的将两个Series的index取了并集，没有数据的自动填充为NaN。\n",
    "\n",
    "创建数据框有很多种方式，比如我们也可以通过字典创建，只不过现在我们需要使用一个字典列表："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Messi</td>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Pique</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Suarez</td>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Xavi</td>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     name  goals  passes\n",
       "0   Messi   35.0    55.0\n",
       "1   Pique    3.0     NaN\n",
       "2  Suarez   10.0    40.0\n",
       "3    Xavi    NaN    60.0"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players = [{\n",
    "    'name': 'Messi',\n",
    "    'goals': 35,\n",
    "    'passes': 55\n",
    "}, {\n",
    "    'name': 'Pique',\n",
    "    'goals': 3\n",
    "}, {\n",
    "    'name': 'Suarez',\n",
    "    'goals': 10,\n",
    "    'passes': 40\n",
    "}, {\n",
    "    'name': 'Xavi',\n",
    "    'passes': 60\n",
    "}]\n",
    "data = pd.DataFrame(players)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的name被列为了一列数据，我们可以使用set_index()方法设定name为index："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Index(['Messi', 'Pique', 'Suarez', 'Xavi'], dtype='object', name='name')\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0\n",
       "Xavi      NaN    60.0"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = data.set_index('name')\n",
    "print(data.index)\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了可以使用data.index查看行的索引外，还可以使用data.columns查看列的标题（索引）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['goals', 'passes'], dtype='object')"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.columns"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现其实列标题也是一个索引。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "DataFrame使用起来非常方便，因为DataFrame可以被看做多种数据结构的一种混合，比如：\n",
    "\n",
    "**DataFrame可以被看做特殊的字典**，只不过这里的字典是按列取的字典，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "name\n",
       "Messi     35.0\n",
       "Pique      3.0\n",
       "Suarez    10.0\n",
       "Xavi       NaN\n",
       "Name: goals, dtype: float64"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['goals']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里需要注意，对于Series，d[0]是按照**行**取的，而对于DataFrame，data['goals']是按列取，这点非常不同。\n",
    "\n",
    "**DataFrame还可以看做是一个NumPy的矩阵**，比如我们可以对其进行转置等操作："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>name</th>\n",
       "      <th>Messi</th>\n",
       "      <th>Pique</th>\n",
       "      <th>Suarez</th>\n",
       "      <th>Xavi</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>goals</th>\n",
       "      <td>35.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>10.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>passes</th>\n",
       "      <td>55.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>40.0</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "name    Messi  Pique  Suarez  Xavi\n",
       "goals    35.0    3.0    10.0   NaN\n",
       "passes   55.0    NaN    40.0  60.0"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.transpose()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这里行和列就被完全倒转过来了。这里也提示我们，DataFrame中不仅仅列有index，行同样可以有index。\n",
    "\n",
    "也因为此，DataFrame可以进行各种NumPy允许的运算，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>3.555348</td>\n",
       "      <td>4.007333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>1.098612</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>2.302585</td>\n",
       "      <td>3.688879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>4.094345</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           goals    passes\n",
       "name                      \n",
       "Messi   3.555348  4.007333\n",
       "Pique   1.098612       NaN\n",
       "Suarez  2.302585  3.688879\n",
       "Xavi         NaN  4.094345"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_data = np.log(data)\n",
    "log_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，两个数据库也可以进行运算："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>38.555348</td>\n",
       "      <td>59.007333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>4.098612</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>12.302585</td>\n",
       "      <td>43.688879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>64.094345</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            goals     passes\n",
       "name                        \n",
       "Messi   38.555348  59.007333\n",
       "Pique    4.098612        NaN\n",
       "Suarez  12.302585  43.688879\n",
       "Xavi          NaN  64.094345"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data+log_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，更经常的是直接在原来的DataFrame中添加一列："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Pique     3.0     NaN   1.098612\n",
       "Suarez   10.0    40.0   2.302585\n",
       "Xavi      NaN    60.0        NaN"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['log_goals'] = np.log(data['goals'])\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "经过运算之后的index仍然是保持不变的。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要删除数据，可以使用.drop()函数，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>3.555348</td>\n",
       "      <td>4.007333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>1.098612</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>2.302585</td>\n",
       "      <td>3.688879</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           goals    passes\n",
       "name                      \n",
       "Messi   3.555348  4.007333\n",
       "Pique   1.098612       NaN\n",
       "Suarez  2.302585  3.688879"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_data2 = log_data.drop('Xavi')\n",
    "log_data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要删除某一列，需要用axis=1选项："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           goals\n",
       "name            \n",
       "Messi   3.555348\n",
       "Pique   1.098612\n",
       "Suarez  2.302585\n",
       "Xavi         NaN"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_data2 = log_data.drop('passes', axis=1)\n",
    "log_data2.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这个操作并不会影响原来的数据："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>3.555348</td>\n",
       "      <td>4.007333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>1.098612</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>2.302585</td>\n",
       "      <td>3.688879</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>4.094345</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           goals    passes\n",
       "name                      \n",
       "Messi   3.555348  4.007333\n",
       "Pique   1.098612       NaN\n",
       "Suarez  2.302585  3.688879\n",
       "Xavi         NaN  4.094345"
      ]
     },
     "execution_count": 54,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "log_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果希望彻底删除一列数据，可以直接用del："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           goals\n",
       "name            \n",
       "Messi   3.555348\n",
       "Pique   1.098612\n",
       "Suarez  2.302585\n",
       "Xavi         NaN"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "del log_data['passes']\n",
    "log_data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "对于缺失数据，当然可以使用isnull()以及notnull()函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "      <td>False</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>True</td>\n",
       "      <td>False</td>\n",
       "      <td>True</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi   False   False      False\n",
       "Pique   False    True      False\n",
       "Suarez  False   False      False\n",
       "Xavi     True   False       True"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.isnull()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "值得注意的是，如果使用dropna()函数，会把**任何**有缺失值的**行**都删掉："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Suarez   10.0    40.0   2.302585"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_copy = data\n",
    "data_copy.dropna()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，也可以扔掉**任何**有缺失值的**列**："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "Empty DataFrame\n",
       "Columns: []\n",
       "Index: [Messi, Pique, Suarez, Xavi]"
      ]
     },
     "execution_count": 58,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_copy = data\n",
    "data_copy.dropna(axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，Pandas还提供了fillna()用来填充缺失值，比如把所有的缺失值都用0来填充："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>0.0</td>\n",
       "      <td>60.0</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Pique     3.0     0.0   1.098612\n",
       "Suarez   10.0    40.0   2.302585\n",
       "Xavi      0.0    60.0   0.000000"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data1 = data.fillna(0)\n",
    "data1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "fillna()还有其他用法，我们在这里不再赘述。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 选取数据\n",
    "\n",
    "DataFrame的数据选取方法与Series类似，由于DataFrame可以看做是一个Series的词典，因而我们可以使用类似的方法调用数据，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "35.0"
      ]
     },
     "execution_count": 60,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['goals']['Messi']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要选取多列，可以使用列表的形式传递"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0\n",
       "Xavi      NaN    60.0"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[['goals', 'passes']]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而，如果使用了切片，则默认是对**行**进行切片："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       goals  passes  log_goals\n",
       "name                           \n",
       "Messi   35.0    55.0   3.555348\n",
       "Pique    3.0     NaN   1.098612"
      ]
     },
     "execution_count": 62,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Messi':'Pique']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "同样，**掩码也是对行操作**："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Suarez   10.0    40.0   2.302585"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data[data['goals'] >= 10]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要对两个维度都是用显式或隐式索引，那么需要使用loc()、iloc()两个函数。其中loc()为显式索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.loc['Messi':'Suarez', 'goals':'passes']\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "iloc()为隐式索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       passes  log_goals\n",
       "name                    \n",
       "Messi    55.0   3.555348\n",
       "Pique     NaN   1.098612"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.iloc[:2, 1:]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据排序\n",
    "\n",
    "在数据框中，可以使用sort_values()方法进行排序，比如对goals进行升序排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Pique     3.0     NaN   1.098612\n",
       "Suarez   10.0    40.0   2.302585\n",
       "Messi    35.0    55.0   3.555348\n",
       "Xavi      NaN    60.0        NaN"
      ]
     },
     "execution_count": 66,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.sort_values('goals')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "降序排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Suarez   10.0    40.0   2.302585\n",
       "Pique     3.0     NaN   1.098612\n",
       "Xavi      NaN    60.0        NaN"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.sort_values('goals', ascending=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以及先对goals进行降序排序，再对passes进行升序排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>log_goals</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>3.555348</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>2.302585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>1.098612</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  log_goals\n",
       "name                            \n",
       "Messi    35.0    55.0   3.555348\n",
       "Suarez   10.0    40.0   2.302585\n",
       "Pique     3.0     NaN   1.098612\n",
       "Xavi      NaN    60.0        NaN"
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.sort_values(['goals', 'passes'], ascending=[False, True])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据框中的时间索引\n",
    "\n",
    "当然在数据框中也可以使用时间索引和多级索引。\n",
    "\n",
    "首先对于时间索引，处理方法与之前类似，比如我们可以从"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1993q1</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1993q2</td>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1993q3</td>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1993q4</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1994q1</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     time  HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "0  1993q1     0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1  1993q2     0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "2  1993q3     0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "3  1993q4     0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "4  1994q1     0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "\n",
       "     France   Germany     Italy  ...  Switzerland  UnitedKingdom  \\\n",
       "0 -0.015177 -0.019680 -0.023383  ...    -0.032865       0.015124   \n",
       "1 -0.014549 -0.015441 -0.018116  ...    -0.019818       0.014795   \n",
       "2 -0.016704 -0.012701 -0.016875  ...    -0.004587       0.029149   \n",
       "3 -0.007476 -0.011667 -0.004963  ...     0.013651       0.036581   \n",
       "4  0.003748  0.022950 -0.002249  ...     0.026644       0.030078   \n",
       "\n",
       "   UnitedStates  Singapore  Philippines  Indonesia  Malaysia  Thailand  \\\n",
       "0      0.022959   0.087145    -0.004381   0.064024  0.085938  0.080000   \n",
       "1      0.018936   0.118075     0.016636   0.066068  0.131189  0.080000   \n",
       "2      0.017990   0.111130     0.031504   0.057959  0.109666  0.080000   \n",
       "3      0.020683   0.125324     0.034007   0.062365  0.075801  0.080000   \n",
       "4      0.029918   0.130709     0.049344   0.049743  0.049147  0.112509   \n",
       "\n",
       "     Taiwan  China  \n",
       "0  0.064902  0.143  \n",
       "1  0.065123  0.141  \n",
       "2  0.067379  0.135  \n",
       "3  0.069164  0.135  \n",
       "4  0.069451  0.125  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw = pd.read_csv(\"csv/hcw.csv\")\n",
    "hcw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意上面的time变量还不是代表时间的对象，我们首先可以使用pd.to_datetime()函数将其变换为时间戳："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    1993-01-01\n",
       "1    1993-04-01\n",
       "2    1993-07-01\n",
       "3    1993-10-01\n",
       "4    1994-01-01\n",
       "        ...    \n",
       "56   2007-01-01\n",
       "57   2007-04-01\n",
       "58   2007-07-01\n",
       "59   2007-10-01\n",
       "60   2008-01-01\n",
       "Name: time, Length: 61, dtype: datetime64[ns]"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tstamp = pd.to_datetime(hcw['time'])\n",
    "tstamp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果不能自动识别时间格式，可以使用类似「format=\"%Y\"」的参数指明数据中日期时间的格式，该命令会将数字、字符串等转换为Pandas的时间。\n",
    "\n",
    "接下来我们可以使用这个时间作为数据的index："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1993-01-01</th>\n",
       "      <td>1993q1</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.1430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993-04-01</th>\n",
       "      <td>1993q2</td>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.1410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993-07-01</th>\n",
       "      <td>1993q3</td>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993-10-01</th>\n",
       "      <td>1993q4</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1994-01-01</th>\n",
       "      <td>1994q1</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.1250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007-01-01</th>\n",
       "      <td>2007q1</td>\n",
       "      <td>0.055</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046517</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.1110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007-04-01</th>\n",
       "      <td>2007q2</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046339</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.1167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007-07-01</th>\n",
       "      <td>2007q3</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>...</td>\n",
       "      <td>0.040958</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.1002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007-10-01</th>\n",
       "      <td>2007q4</td>\n",
       "      <td>0.069</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>...</td>\n",
       "      <td>0.037792</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.1017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008-01-01</th>\n",
       "      <td>2008q1</td>\n",
       "      <td>0.073</td>\n",
       "      <td>0.027523</td>\n",
       "      <td>0.017431</td>\n",
       "      <td>0.029217</td>\n",
       "      <td>-0.005200</td>\n",
       "      <td>0.023732</td>\n",
       "      <td>0.018386</td>\n",
       "      <td>0.009020</td>\n",
       "      <td>-0.005581</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028398</td>\n",
       "      <td>0.028492</td>\n",
       "      <td>0.006137</td>\n",
       "      <td>0.079339</td>\n",
       "      <td>0.047945</td>\n",
       "      <td>0.133400</td>\n",
       "      <td>0.167034</td>\n",
       "      <td>0.049144</td>\n",
       "      <td>0.058841</td>\n",
       "      <td>0.1238</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              time  HongKong  Australia   Austria    Canada   Denmark  \\\n",
       "time                                                                    \n",
       "1993-01-01  1993q1     0.062   0.040489 -0.013084  0.010064 -0.012292   \n",
       "1993-04-01  1993q2     0.059   0.037857 -0.007581  0.021264 -0.003093   \n",
       "1993-07-01  1993q3     0.058   0.022509  0.000543  0.018919 -0.007764   \n",
       "1993-10-01  1993q4     0.062   0.028747  0.001181  0.025317 -0.004049   \n",
       "1994-01-01  1994q1     0.079   0.033990  0.025511  0.043567  0.031094   \n",
       "...            ...       ...        ...       ...       ...       ...   \n",
       "2007-01-01  2007q1     0.055   0.058013  0.036198  0.030712  0.033134   \n",
       "2007-04-01  2007q2     0.062   0.059519  0.032570  0.039827 -0.007169   \n",
       "2007-07-01  2007q3     0.068   0.056649  0.031558  0.034742  0.013517   \n",
       "2007-10-01  2007q4     0.069   0.045825  0.019095  0.038128  0.023794   \n",
       "2008-01-01  2008q1     0.073   0.027523  0.017431  0.029217 -0.005200   \n",
       "\n",
       "             Finland    France   Germany     Italy  ...  Switzerland  \\\n",
       "time                                                ...                \n",
       "1993-01-01 -0.028357 -0.015177 -0.019680 -0.023383  ...    -0.032865   \n",
       "1993-04-01 -0.023397 -0.014549 -0.015441 -0.018116  ...    -0.019818   \n",
       "1993-07-01 -0.006018 -0.016704 -0.012701 -0.016875  ...    -0.004587   \n",
       "1993-10-01 -0.004774 -0.007476 -0.011667 -0.004963  ...     0.013651   \n",
       "1994-01-01  0.012886  0.003748  0.022950 -0.002249  ...     0.026644   \n",
       "...              ...       ...       ...       ...  ...          ...   \n",
       "2007-01-01  0.047340  0.032755  0.035225  0.025621  ...     0.046517   \n",
       "2007-04-01  0.046808  0.030355  0.023897  0.017251  ...     0.046339   \n",
       "2007-07-01  0.045647  0.036748  0.020773  0.023338  ...     0.040958   \n",
       "2007-10-01  0.044177  0.021745  0.005865  0.005081  ...     0.037792   \n",
       "2008-01-01  0.023732  0.018386  0.009020 -0.005581  ...     0.028398   \n",
       "\n",
       "            UnitedKingdom  UnitedStates  Singapore  Philippines  Indonesia  \\\n",
       "time                                                                         \n",
       "1993-01-01       0.015124      0.022959   0.087145    -0.004381   0.064024   \n",
       "1993-04-01       0.014795      0.018936   0.118075     0.016636   0.066068   \n",
       "1993-07-01       0.029149      0.017990   0.111130     0.031504   0.057959   \n",
       "1993-10-01       0.036581      0.020683   0.125324     0.034007   0.062365   \n",
       "1994-01-01       0.030078      0.029918   0.130709     0.049344   0.049743   \n",
       "...                   ...           ...        ...          ...        ...   \n",
       "2007-01-01       0.027946      0.017708   0.063092     0.065573   0.095297   \n",
       "2007-04-01       0.039619      0.018756   0.101377     0.081891   0.110900   \n",
       "2007-07-01       0.038682      0.028225   0.105562     0.064359   0.110100   \n",
       "2007-10-01       0.033362      0.009288   0.069739     0.065470   0.110700   \n",
       "2008-01-01       0.028492      0.006137   0.079339     0.047945   0.133400   \n",
       "\n",
       "            Malaysia  Thailand    Taiwan   China  \n",
       "time                                              \n",
       "1993-01-01  0.085938  0.080000  0.064902  0.1430  \n",
       "1993-04-01  0.131189  0.080000  0.065123  0.1410  \n",
       "1993-07-01  0.109666  0.080000  0.067379  0.1350  \n",
       "1993-10-01  0.075801  0.080000  0.069164  0.1350  \n",
       "1994-01-01  0.049147  0.112509  0.069451  0.1250  \n",
       "...              ...       ...       ...     ...  \n",
       "2007-01-01  0.039884  0.049846  0.041019  0.1110  \n",
       "2007-04-01  0.080276  0.051197  0.051073  0.1167  \n",
       "2007-07-01  0.093361  0.050126  0.066369  0.1002  \n",
       "2007-10-01  0.151739  0.079587  0.062929  0.1017  \n",
       "2008-01-01  0.167034  0.049144  0.058841  0.1238  \n",
       "\n",
       "[61 rows x 26 columns]"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new = hcw.set_index(tstamp)\n",
    "hcw_new"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来可以使用.to_period()函数转换为Period："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PeriodIndex(['1993Q1', '1993Q2', '1993Q3', '1993Q4', '1994Q1', '1994Q2',\n",
       "             '1994Q3', '1994Q4', '1995Q1', '1995Q2', '1995Q3', '1995Q4',\n",
       "             '1996Q1', '1996Q2', '1996Q3', '1996Q4', '1997Q1', '1997Q2',\n",
       "             '1997Q3', '1997Q4', '1998Q1', '1998Q2', '1998Q3', '1998Q4',\n",
       "             '1999Q1', '1999Q2', '1999Q3', '1999Q4', '2000Q1', '2000Q2',\n",
       "             '2000Q3', '2000Q4', '2001Q1', '2001Q2', '2001Q3', '2001Q4',\n",
       "             '2002Q1', '2002Q2', '2002Q3', '2002Q4', '2003Q1', '2003Q2',\n",
       "             '2003Q3', '2003Q4', '2004Q1', '2004Q2', '2004Q3', '2004Q4',\n",
       "             '2005Q1', '2005Q2', '2005Q3', '2005Q4', '2006Q1', '2006Q2',\n",
       "             '2006Q3', '2006Q4', '2007Q1', '2007Q2', '2007Q3', '2007Q4',\n",
       "             '2008Q1'],\n",
       "            dtype='period[Q-DEC]', name='time', freq='Q-DEC')"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new.index.to_period('Q')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要将Period作为index，继续使用set_index："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1993Q1</th>\n",
       "      <td>1993q1</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.1430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q2</th>\n",
       "      <td>1993q2</td>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.1410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q3</th>\n",
       "      <td>1993q3</td>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q4</th>\n",
       "      <td>1993q4</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1994Q1</th>\n",
       "      <td>1994q1</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.1250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q1</th>\n",
       "      <td>2007q1</td>\n",
       "      <td>0.055</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046517</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.1110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q2</th>\n",
       "      <td>2007q2</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046339</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.1167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q3</th>\n",
       "      <td>2007q3</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>...</td>\n",
       "      <td>0.040958</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.1002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q4</th>\n",
       "      <td>2007q4</td>\n",
       "      <td>0.069</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>...</td>\n",
       "      <td>0.037792</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.1017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008Q1</th>\n",
       "      <td>2008q1</td>\n",
       "      <td>0.073</td>\n",
       "      <td>0.027523</td>\n",
       "      <td>0.017431</td>\n",
       "      <td>0.029217</td>\n",
       "      <td>-0.005200</td>\n",
       "      <td>0.023732</td>\n",
       "      <td>0.018386</td>\n",
       "      <td>0.009020</td>\n",
       "      <td>-0.005581</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028398</td>\n",
       "      <td>0.028492</td>\n",
       "      <td>0.006137</td>\n",
       "      <td>0.079339</td>\n",
       "      <td>0.047945</td>\n",
       "      <td>0.133400</td>\n",
       "      <td>0.167034</td>\n",
       "      <td>0.049144</td>\n",
       "      <td>0.058841</td>\n",
       "      <td>0.1238</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          time  HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "time                                                                          \n",
       "1993Q1  1993q1     0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1993Q2  1993q2     0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "1993Q3  1993q3     0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "1993Q4  1993q4     0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "1994Q1  1994q1     0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "...        ...       ...        ...       ...       ...       ...       ...   \n",
       "2007Q1  2007q1     0.055   0.058013  0.036198  0.030712  0.033134  0.047340   \n",
       "2007Q2  2007q2     0.062   0.059519  0.032570  0.039827 -0.007169  0.046808   \n",
       "2007Q3  2007q3     0.068   0.056649  0.031558  0.034742  0.013517  0.045647   \n",
       "2007Q4  2007q4     0.069   0.045825  0.019095  0.038128  0.023794  0.044177   \n",
       "2008Q1  2008q1     0.073   0.027523  0.017431  0.029217 -0.005200  0.023732   \n",
       "\n",
       "          France   Germany     Italy  ...  Switzerland  UnitedKingdom  \\\n",
       "time                                  ...                               \n",
       "1993Q1 -0.015177 -0.019680 -0.023383  ...    -0.032865       0.015124   \n",
       "1993Q2 -0.014549 -0.015441 -0.018116  ...    -0.019818       0.014795   \n",
       "1993Q3 -0.016704 -0.012701 -0.016875  ...    -0.004587       0.029149   \n",
       "1993Q4 -0.007476 -0.011667 -0.004963  ...     0.013651       0.036581   \n",
       "1994Q1  0.003748  0.022950 -0.002249  ...     0.026644       0.030078   \n",
       "...          ...       ...       ...  ...          ...            ...   \n",
       "2007Q1  0.032755  0.035225  0.025621  ...     0.046517       0.027946   \n",
       "2007Q2  0.030355  0.023897  0.017251  ...     0.046339       0.039619   \n",
       "2007Q3  0.036748  0.020773  0.023338  ...     0.040958       0.038682   \n",
       "2007Q4  0.021745  0.005865  0.005081  ...     0.037792       0.033362   \n",
       "2008Q1  0.018386  0.009020 -0.005581  ...     0.028398       0.028492   \n",
       "\n",
       "        UnitedStates  Singapore  Philippines  Indonesia  Malaysia  Thailand  \\\n",
       "time                                                                          \n",
       "1993Q1      0.022959   0.087145    -0.004381   0.064024  0.085938  0.080000   \n",
       "1993Q2      0.018936   0.118075     0.016636   0.066068  0.131189  0.080000   \n",
       "1993Q3      0.017990   0.111130     0.031504   0.057959  0.109666  0.080000   \n",
       "1993Q4      0.020683   0.125324     0.034007   0.062365  0.075801  0.080000   \n",
       "1994Q1      0.029918   0.130709     0.049344   0.049743  0.049147  0.112509   \n",
       "...              ...        ...          ...        ...       ...       ...   \n",
       "2007Q1      0.017708   0.063092     0.065573   0.095297  0.039884  0.049846   \n",
       "2007Q2      0.018756   0.101377     0.081891   0.110900  0.080276  0.051197   \n",
       "2007Q3      0.028225   0.105562     0.064359   0.110100  0.093361  0.050126   \n",
       "2007Q4      0.009288   0.069739     0.065470   0.110700  0.151739  0.079587   \n",
       "2008Q1      0.006137   0.079339     0.047945   0.133400  0.167034  0.049144   \n",
       "\n",
       "          Taiwan   China  \n",
       "time                      \n",
       "1993Q1  0.064902  0.1430  \n",
       "1993Q2  0.065123  0.1410  \n",
       "1993Q3  0.067379  0.1350  \n",
       "1993Q4  0.069164  0.1350  \n",
       "1994Q1  0.069451  0.1250  \n",
       "...          ...     ...  \n",
       "2007Q1  0.041019  0.1110  \n",
       "2007Q2  0.051073  0.1167  \n",
       "2007Q3  0.066369  0.1002  \n",
       "2007Q4  0.062929  0.1017  \n",
       "2008Q1  0.058841  0.1238  \n",
       "\n",
       "[61 rows x 26 columns]"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_final = hcw.set_index(hcw_new.index.to_period('Q'))\n",
    "hcw_final"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "有了时间，就可以使用shift()以及tshift()函数了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "time\n",
       "1993Q2    0.062\n",
       "1993Q3    0.059\n",
       "1993Q4    0.058\n",
       "1994Q1    0.062\n",
       "1994Q2    0.079\n",
       "          ...  \n",
       "2007Q2    0.055\n",
       "2007Q3    0.062\n",
       "2007Q4    0.068\n",
       "2008Q1    0.069\n",
       "2008Q2    0.073\n",
       "Freq: Q-DEC, Name: HongKong, Length: 61, dtype: float64"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lag_hcw = hcw_final['HongKong'].tshift(1)\n",
    "lag_hcw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以直接将其并进数据中："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>...</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "      <th>lag_gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>time</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1993Q1</th>\n",
       "      <td>1993q1</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>...</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.1430</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q2</th>\n",
       "      <td>1993q2</td>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>...</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.1410</td>\n",
       "      <td>0.062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q3</th>\n",
       "      <td>1993q3</td>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>...</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.1350</td>\n",
       "      <td>0.059</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q4</th>\n",
       "      <td>1993q4</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>...</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.1350</td>\n",
       "      <td>0.058</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1994Q1</th>\n",
       "      <td>1994q1</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>...</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.1250</td>\n",
       "      <td>0.062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q1</th>\n",
       "      <td>2007q1</td>\n",
       "      <td>0.055</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>...</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.1110</td>\n",
       "      <td>0.066</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q2</th>\n",
       "      <td>2007q2</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>...</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.1167</td>\n",
       "      <td>0.055</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q3</th>\n",
       "      <td>2007q3</td>\n",
       "      <td>0.068</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>...</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.1002</td>\n",
       "      <td>0.062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q4</th>\n",
       "      <td>2007q4</td>\n",
       "      <td>0.069</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>...</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.1017</td>\n",
       "      <td>0.068</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008Q1</th>\n",
       "      <td>2008q1</td>\n",
       "      <td>0.073</td>\n",
       "      <td>0.027523</td>\n",
       "      <td>0.017431</td>\n",
       "      <td>0.029217</td>\n",
       "      <td>-0.005200</td>\n",
       "      <td>0.023732</td>\n",
       "      <td>0.018386</td>\n",
       "      <td>0.009020</td>\n",
       "      <td>-0.005581</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028492</td>\n",
       "      <td>0.006137</td>\n",
       "      <td>0.079339</td>\n",
       "      <td>0.047945</td>\n",
       "      <td>0.133400</td>\n",
       "      <td>0.167034</td>\n",
       "      <td>0.049144</td>\n",
       "      <td>0.058841</td>\n",
       "      <td>0.1238</td>\n",
       "      <td>0.069</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 27 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "          time  HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "time                                                                          \n",
       "1993Q1  1993q1     0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1993Q2  1993q2     0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "1993Q3  1993q3     0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "1993Q4  1993q4     0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "1994Q1  1994q1     0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "...        ...       ...        ...       ...       ...       ...       ...   \n",
       "2007Q1  2007q1     0.055   0.058013  0.036198  0.030712  0.033134  0.047340   \n",
       "2007Q2  2007q2     0.062   0.059519  0.032570  0.039827 -0.007169  0.046808   \n",
       "2007Q3  2007q3     0.068   0.056649  0.031558  0.034742  0.013517  0.045647   \n",
       "2007Q4  2007q4     0.069   0.045825  0.019095  0.038128  0.023794  0.044177   \n",
       "2008Q1  2008q1     0.073   0.027523  0.017431  0.029217 -0.005200  0.023732   \n",
       "\n",
       "          France   Germany     Italy  ...  UnitedKingdom  UnitedStates  \\\n",
       "time                                  ...                                \n",
       "1993Q1 -0.015177 -0.019680 -0.023383  ...       0.015124      0.022959   \n",
       "1993Q2 -0.014549 -0.015441 -0.018116  ...       0.014795      0.018936   \n",
       "1993Q3 -0.016704 -0.012701 -0.016875  ...       0.029149      0.017990   \n",
       "1993Q4 -0.007476 -0.011667 -0.004963  ...       0.036581      0.020683   \n",
       "1994Q1  0.003748  0.022950 -0.002249  ...       0.030078      0.029918   \n",
       "...          ...       ...       ...  ...            ...           ...   \n",
       "2007Q1  0.032755  0.035225  0.025621  ...       0.027946      0.017708   \n",
       "2007Q2  0.030355  0.023897  0.017251  ...       0.039619      0.018756   \n",
       "2007Q3  0.036748  0.020773  0.023338  ...       0.038682      0.028225   \n",
       "2007Q4  0.021745  0.005865  0.005081  ...       0.033362      0.009288   \n",
       "2008Q1  0.018386  0.009020 -0.005581  ...       0.028492      0.006137   \n",
       "\n",
       "        Singapore  Philippines  Indonesia  Malaysia  Thailand    Taiwan  \\\n",
       "time                                                                      \n",
       "1993Q1   0.087145    -0.004381   0.064024  0.085938  0.080000  0.064902   \n",
       "1993Q2   0.118075     0.016636   0.066068  0.131189  0.080000  0.065123   \n",
       "1993Q3   0.111130     0.031504   0.057959  0.109666  0.080000  0.067379   \n",
       "1993Q4   0.125324     0.034007   0.062365  0.075801  0.080000  0.069164   \n",
       "1994Q1   0.130709     0.049344   0.049743  0.049147  0.112509  0.069451   \n",
       "...           ...          ...        ...       ...       ...       ...   \n",
       "2007Q1   0.063092     0.065573   0.095297  0.039884  0.049846  0.041019   \n",
       "2007Q2   0.101377     0.081891   0.110900  0.080276  0.051197  0.051073   \n",
       "2007Q3   0.105562     0.064359   0.110100  0.093361  0.050126  0.066369   \n",
       "2007Q4   0.069739     0.065470   0.110700  0.151739  0.079587  0.062929   \n",
       "2008Q1   0.079339     0.047945   0.133400  0.167034  0.049144  0.058841   \n",
       "\n",
       "         China  lag_gdp  \n",
       "time                     \n",
       "1993Q1  0.1430      NaN  \n",
       "1993Q2  0.1410    0.062  \n",
       "1993Q3  0.1350    0.059  \n",
       "1993Q4  0.1350    0.058  \n",
       "1994Q1  0.1250    0.062  \n",
       "...        ...      ...  \n",
       "2007Q1  0.1110    0.066  \n",
       "2007Q2  0.1167    0.055  \n",
       "2007Q3  0.1002    0.062  \n",
       "2007Q4  0.1017    0.068  \n",
       "2008Q1  0.1238    0.069  \n",
       "\n",
       "[61 rows x 27 columns]"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_final['lag_gdp'] = lag_hcw\n",
    "hcw_final"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 多级索引\n",
    "\n",
    "当然，数据框中也支持多级索引，创建方法与Series的多级索引并无差异。\n",
    "\n",
    "这里我们主要介绍如何从文件中读入数据并建立多级索引。\n",
    "\n",
    "首先第一种情况，很多数据是以「宽格式」存储的，比如以下文件："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>time</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1993q1</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.143</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1993q2</td>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.141</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1993q3</td>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1993q4</td>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.135</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1994q1</td>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 26 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     time  HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "0  1993q1     0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1  1993q2     0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "2  1993q3     0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "3  1993q4     0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "4  1994q1     0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "\n",
       "     France   Germany     Italy  ...  Switzerland  UnitedKingdom  \\\n",
       "0 -0.015177 -0.019680 -0.023383  ...    -0.032865       0.015124   \n",
       "1 -0.014549 -0.015441 -0.018116  ...    -0.019818       0.014795   \n",
       "2 -0.016704 -0.012701 -0.016875  ...    -0.004587       0.029149   \n",
       "3 -0.007476 -0.011667 -0.004963  ...     0.013651       0.036581   \n",
       "4  0.003748  0.022950 -0.002249  ...     0.026644       0.030078   \n",
       "\n",
       "   UnitedStates  Singapore  Philippines  Indonesia  Malaysia  Thailand  \\\n",
       "0      0.022959   0.087145    -0.004381   0.064024  0.085938  0.080000   \n",
       "1      0.018936   0.118075     0.016636   0.066068  0.131189  0.080000   \n",
       "2      0.017990   0.111130     0.031504   0.057959  0.109666  0.080000   \n",
       "3      0.020683   0.125324     0.034007   0.062365  0.075801  0.080000   \n",
       "4      0.029918   0.130709     0.049344   0.049743  0.049147  0.112509   \n",
       "\n",
       "     Taiwan  China  \n",
       "0  0.064902  0.143  \n",
       "1  0.065123  0.141  \n",
       "2  0.067379  0.135  \n",
       "3  0.069164  0.135  \n",
       "4  0.069451  0.125  \n",
       "\n",
       "[5 rows x 26 columns]"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw = pd.read_csv(\"csv/hcw.csv\")\n",
    "hcw.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了将其转换为「长格式」的Series，可以使用DataFrame的stack()方法。不过在使用该方法之前，不妨先把时间转换为正确的格式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>Japan</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quarter</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1993Q1</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>0.012683</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.1430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q2</th>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>-0.005571</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.1410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q3</th>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>-0.017558</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q4</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>-0.010101</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1994Q1</th>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>-0.022503</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.1250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q1</th>\n",
       "      <td>0.055</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>0.023959</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046517</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.1110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q2</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>0.014193</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046339</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.1167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q3</th>\n",
       "      <td>0.068</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>0.012907</td>\n",
       "      <td>...</td>\n",
       "      <td>0.040958</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.1002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q4</th>\n",
       "      <td>0.069</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>-0.004768</td>\n",
       "      <td>...</td>\n",
       "      <td>0.037792</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.1017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008Q1</th>\n",
       "      <td>0.073</td>\n",
       "      <td>0.027523</td>\n",
       "      <td>0.017431</td>\n",
       "      <td>0.029217</td>\n",
       "      <td>-0.005200</td>\n",
       "      <td>0.023732</td>\n",
       "      <td>0.018386</td>\n",
       "      <td>0.009020</td>\n",
       "      <td>-0.005581</td>\n",
       "      <td>-0.013647</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028398</td>\n",
       "      <td>0.028492</td>\n",
       "      <td>0.006137</td>\n",
       "      <td>0.079339</td>\n",
       "      <td>0.047945</td>\n",
       "      <td>0.133400</td>\n",
       "      <td>0.167034</td>\n",
       "      <td>0.049144</td>\n",
       "      <td>0.058841</td>\n",
       "      <td>0.1238</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "quarter                                                                \n",
       "1993Q1      0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1993Q2      0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "1993Q3      0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "1993Q4      0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "1994Q1      0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "...           ...        ...       ...       ...       ...       ...   \n",
       "2007Q1      0.055   0.058013  0.036198  0.030712  0.033134  0.047340   \n",
       "2007Q2      0.062   0.059519  0.032570  0.039827 -0.007169  0.046808   \n",
       "2007Q3      0.068   0.056649  0.031558  0.034742  0.013517  0.045647   \n",
       "2007Q4      0.069   0.045825  0.019095  0.038128  0.023794  0.044177   \n",
       "2008Q1      0.073   0.027523  0.017431  0.029217 -0.005200  0.023732   \n",
       "\n",
       "           France   Germany     Italy     Japan  ...  Switzerland  \\\n",
       "quarter                                          ...                \n",
       "1993Q1  -0.015177 -0.019680 -0.023383  0.012683  ...    -0.032865   \n",
       "1993Q2  -0.014549 -0.015441 -0.018116 -0.005571  ...    -0.019818   \n",
       "1993Q3  -0.016704 -0.012701 -0.016875 -0.017558  ...    -0.004587   \n",
       "1993Q4  -0.007476 -0.011667 -0.004963 -0.010101  ...     0.013651   \n",
       "1994Q1   0.003748  0.022950 -0.002249 -0.022503  ...     0.026644   \n",
       "...           ...       ...       ...       ...  ...          ...   \n",
       "2007Q1   0.032755  0.035225  0.025621  0.023959  ...     0.046517   \n",
       "2007Q2   0.030355  0.023897  0.017251  0.014193  ...     0.046339   \n",
       "2007Q3   0.036748  0.020773  0.023338  0.012907  ...     0.040958   \n",
       "2007Q4   0.021745  0.005865  0.005081 -0.004768  ...     0.037792   \n",
       "2008Q1   0.018386  0.009020 -0.005581 -0.013647  ...     0.028398   \n",
       "\n",
       "         UnitedKingdom  UnitedStates  Singapore  Philippines  Indonesia  \\\n",
       "quarter                                                                   \n",
       "1993Q1        0.015124      0.022959   0.087145    -0.004381   0.064024   \n",
       "1993Q2        0.014795      0.018936   0.118075     0.016636   0.066068   \n",
       "1993Q3        0.029149      0.017990   0.111130     0.031504   0.057959   \n",
       "1993Q4        0.036581      0.020683   0.125324     0.034007   0.062365   \n",
       "1994Q1        0.030078      0.029918   0.130709     0.049344   0.049743   \n",
       "...                ...           ...        ...          ...        ...   \n",
       "2007Q1        0.027946      0.017708   0.063092     0.065573   0.095297   \n",
       "2007Q2        0.039619      0.018756   0.101377     0.081891   0.110900   \n",
       "2007Q3        0.038682      0.028225   0.105562     0.064359   0.110100   \n",
       "2007Q4        0.033362      0.009288   0.069739     0.065470   0.110700   \n",
       "2008Q1        0.028492      0.006137   0.079339     0.047945   0.133400   \n",
       "\n",
       "         Malaysia  Thailand    Taiwan   China  \n",
       "quarter                                        \n",
       "1993Q1   0.085938  0.080000  0.064902  0.1430  \n",
       "1993Q2   0.131189  0.080000  0.065123  0.1410  \n",
       "1993Q3   0.109666  0.080000  0.067379  0.1350  \n",
       "1993Q4   0.075801  0.080000  0.069164  0.1350  \n",
       "1994Q1   0.049147  0.112509  0.069451  0.1250  \n",
       "...           ...       ...       ...     ...  \n",
       "2007Q1   0.039884  0.049846  0.041019  0.1110  \n",
       "2007Q2   0.080276  0.051197  0.051073  0.1167  \n",
       "2007Q3   0.093361  0.050126  0.066369  0.1002  \n",
       "2007Q4   0.151739  0.079587  0.062929  0.1017  \n",
       "2008Q1   0.167034  0.049144  0.058841  0.1238  \n",
       "\n",
       "[61 rows x 25 columns]"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw['quarter'] = pd.to_datetime(hcw['time'])\n",
    "hcw = hcw.set_index(hcw['quarter'])\n",
    "hcw = hcw.set_index(hcw.index.to_period('Q'))\n",
    "del hcw['time']\n",
    "del hcw['quarter']\n",
    "hcw"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来，就可以使用stack()方法转换为长格式了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "quarter  city     \n",
       "1993Q1   HongKong     0.062000\n",
       "         Australia    0.040489\n",
       "         Austria     -0.013084\n",
       "         Canada       0.010064\n",
       "         Denmark     -0.012292\n",
       "dtype: float64"
      ]
     },
     "execution_count": 78,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new = hcw.stack()\n",
    "hcw_new.index.names = ['quarter', 'city']\n",
    "hcw_new.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "MultiIndex([('1993Q1',      'HongKong'),\n",
       "            ('1993Q1',     'Australia'),\n",
       "            ('1993Q1',       'Austria'),\n",
       "            ('1993Q1',        'Canada'),\n",
       "            ('1993Q1',       'Denmark'),\n",
       "            ('1993Q1',       'Finland'),\n",
       "            ('1993Q1',        'France'),\n",
       "            ('1993Q1',       'Germany'),\n",
       "            ('1993Q1',         'Italy'),\n",
       "            ('1993Q1',         'Japan'),\n",
       "            ...\n",
       "            ('2008Q1',   'Switzerland'),\n",
       "            ('2008Q1', 'UnitedKingdom'),\n",
       "            ('2008Q1',  'UnitedStates'),\n",
       "            ('2008Q1',     'Singapore'),\n",
       "            ('2008Q1',   'Philippines'),\n",
       "            ('2008Q1',     'Indonesia'),\n",
       "            ('2008Q1',      'Malaysia'),\n",
       "            ('2008Q1',      'Thailand'),\n",
       "            ('2008Q1',        'Taiwan'),\n",
       "            ('2008Q1',         'China')],\n",
       "           names=['quarter', 'city'], length=1525)"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new.index"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要将数据转换回宽的格式，可以用unstack()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>city</th>\n",
       "      <th>HongKong</th>\n",
       "      <th>Australia</th>\n",
       "      <th>Austria</th>\n",
       "      <th>Canada</th>\n",
       "      <th>Denmark</th>\n",
       "      <th>Finland</th>\n",
       "      <th>France</th>\n",
       "      <th>Germany</th>\n",
       "      <th>Italy</th>\n",
       "      <th>Japan</th>\n",
       "      <th>...</th>\n",
       "      <th>Switzerland</th>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <th>UnitedStates</th>\n",
       "      <th>Singapore</th>\n",
       "      <th>Philippines</th>\n",
       "      <th>Indonesia</th>\n",
       "      <th>Malaysia</th>\n",
       "      <th>Thailand</th>\n",
       "      <th>Taiwan</th>\n",
       "      <th>China</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>quarter</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1993Q1</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.040489</td>\n",
       "      <td>-0.013084</td>\n",
       "      <td>0.010064</td>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.023383</td>\n",
       "      <td>0.012683</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.032865</td>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.087145</td>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.1430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q2</th>\n",
       "      <td>0.059</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>-0.005571</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.1410</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q3</th>\n",
       "      <td>0.058</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>-0.017558</td>\n",
       "      <td>...</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1993Q4</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>-0.010101</td>\n",
       "      <td>...</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.1350</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1994Q1</th>\n",
       "      <td>0.079</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>-0.022503</td>\n",
       "      <td>...</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.1250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q1</th>\n",
       "      <td>0.055</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>0.023959</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046517</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.1110</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q2</th>\n",
       "      <td>0.062</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>0.014193</td>\n",
       "      <td>...</td>\n",
       "      <td>0.046339</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.1167</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q3</th>\n",
       "      <td>0.068</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>0.012907</td>\n",
       "      <td>...</td>\n",
       "      <td>0.040958</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.1002</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007Q4</th>\n",
       "      <td>0.069</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>-0.004768</td>\n",
       "      <td>...</td>\n",
       "      <td>0.037792</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.1017</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008Q1</th>\n",
       "      <td>0.073</td>\n",
       "      <td>0.027523</td>\n",
       "      <td>0.017431</td>\n",
       "      <td>0.029217</td>\n",
       "      <td>-0.005200</td>\n",
       "      <td>0.023732</td>\n",
       "      <td>0.018386</td>\n",
       "      <td>0.009020</td>\n",
       "      <td>-0.005581</td>\n",
       "      <td>-0.013647</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028398</td>\n",
       "      <td>0.028492</td>\n",
       "      <td>0.006137</td>\n",
       "      <td>0.079339</td>\n",
       "      <td>0.047945</td>\n",
       "      <td>0.133400</td>\n",
       "      <td>0.167034</td>\n",
       "      <td>0.049144</td>\n",
       "      <td>0.058841</td>\n",
       "      <td>0.1238</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>61 rows × 25 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "city     HongKong  Australia   Austria    Canada   Denmark   Finland  \\\n",
       "quarter                                                                \n",
       "1993Q1      0.062   0.040489 -0.013084  0.010064 -0.012292 -0.028357   \n",
       "1993Q2      0.059   0.037857 -0.007581  0.021264 -0.003093 -0.023397   \n",
       "1993Q3      0.058   0.022509  0.000543  0.018919 -0.007764 -0.006018   \n",
       "1993Q4      0.062   0.028747  0.001181  0.025317 -0.004049 -0.004774   \n",
       "1994Q1      0.079   0.033990  0.025511  0.043567  0.031094  0.012886   \n",
       "...           ...        ...       ...       ...       ...       ...   \n",
       "2007Q1      0.055   0.058013  0.036198  0.030712  0.033134  0.047340   \n",
       "2007Q2      0.062   0.059519  0.032570  0.039827 -0.007169  0.046808   \n",
       "2007Q3      0.068   0.056649  0.031558  0.034742  0.013517  0.045647   \n",
       "2007Q4      0.069   0.045825  0.019095  0.038128  0.023794  0.044177   \n",
       "2008Q1      0.073   0.027523  0.017431  0.029217 -0.005200  0.023732   \n",
       "\n",
       "city       France   Germany     Italy     Japan  ...  Switzerland  \\\n",
       "quarter                                          ...                \n",
       "1993Q1  -0.015177 -0.019680 -0.023383  0.012683  ...    -0.032865   \n",
       "1993Q2  -0.014549 -0.015441 -0.018116 -0.005571  ...    -0.019818   \n",
       "1993Q3  -0.016704 -0.012701 -0.016875 -0.017558  ...    -0.004587   \n",
       "1993Q4  -0.007476 -0.011667 -0.004963 -0.010101  ...     0.013651   \n",
       "1994Q1   0.003748  0.022950 -0.002249 -0.022503  ...     0.026644   \n",
       "...           ...       ...       ...       ...  ...          ...   \n",
       "2007Q1   0.032755  0.035225  0.025621  0.023959  ...     0.046517   \n",
       "2007Q2   0.030355  0.023897  0.017251  0.014193  ...     0.046339   \n",
       "2007Q3   0.036748  0.020773  0.023338  0.012907  ...     0.040958   \n",
       "2007Q4   0.021745  0.005865  0.005081 -0.004768  ...     0.037792   \n",
       "2008Q1   0.018386  0.009020 -0.005581 -0.013647  ...     0.028398   \n",
       "\n",
       "city     UnitedKingdom  UnitedStates  Singapore  Philippines  Indonesia  \\\n",
       "quarter                                                                   \n",
       "1993Q1        0.015124      0.022959   0.087145    -0.004381   0.064024   \n",
       "1993Q2        0.014795      0.018936   0.118075     0.016636   0.066068   \n",
       "1993Q3        0.029149      0.017990   0.111130     0.031504   0.057959   \n",
       "1993Q4        0.036581      0.020683   0.125324     0.034007   0.062365   \n",
       "1994Q1        0.030078      0.029918   0.130709     0.049344   0.049743   \n",
       "...                ...           ...        ...          ...        ...   \n",
       "2007Q1        0.027946      0.017708   0.063092     0.065573   0.095297   \n",
       "2007Q2        0.039619      0.018756   0.101377     0.081891   0.110900   \n",
       "2007Q3        0.038682      0.028225   0.105562     0.064359   0.110100   \n",
       "2007Q4        0.033362      0.009288   0.069739     0.065470   0.110700   \n",
       "2008Q1        0.028492      0.006137   0.079339     0.047945   0.133400   \n",
       "\n",
       "city     Malaysia  Thailand    Taiwan   China  \n",
       "quarter                                        \n",
       "1993Q1   0.085938  0.080000  0.064902  0.1430  \n",
       "1993Q2   0.131189  0.080000  0.065123  0.1410  \n",
       "1993Q3   0.109666  0.080000  0.067379  0.1350  \n",
       "1993Q4   0.075801  0.080000  0.069164  0.1350  \n",
       "1994Q1   0.049147  0.112509  0.069451  0.1250  \n",
       "...           ...       ...       ...     ...  \n",
       "2007Q1   0.039884  0.049846  0.041019  0.1110  \n",
       "2007Q2   0.080276  0.051197  0.051073  0.1167  \n",
       "2007Q3   0.093361  0.050126  0.066369  0.1002  \n",
       "2007Q4   0.151739  0.079587  0.062929  0.1017  \n",
       "2008Q1   0.167034  0.049144  0.058841  0.1238  \n",
       "\n",
       "[61 rows x 25 columns]"
      ]
     },
     "execution_count": 80,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new.unstack()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th>quarter</th>\n",
       "      <th>1993Q1</th>\n",
       "      <th>1993Q2</th>\n",
       "      <th>1993Q3</th>\n",
       "      <th>1993Q4</th>\n",
       "      <th>1994Q1</th>\n",
       "      <th>1994Q2</th>\n",
       "      <th>1994Q3</th>\n",
       "      <th>1994Q4</th>\n",
       "      <th>1995Q1</th>\n",
       "      <th>1995Q2</th>\n",
       "      <th>...</th>\n",
       "      <th>2005Q4</th>\n",
       "      <th>2006Q1</th>\n",
       "      <th>2006Q2</th>\n",
       "      <th>2006Q3</th>\n",
       "      <th>2006Q4</th>\n",
       "      <th>2007Q1</th>\n",
       "      <th>2007Q2</th>\n",
       "      <th>2007Q3</th>\n",
       "      <th>2007Q4</th>\n",
       "      <th>2008Q1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>city</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>HongKong</th>\n",
       "      <td>0.062000</td>\n",
       "      <td>0.059000</td>\n",
       "      <td>0.058000</td>\n",
       "      <td>0.062000</td>\n",
       "      <td>0.079000</td>\n",
       "      <td>0.068000</td>\n",
       "      <td>0.046000</td>\n",
       "      <td>0.052000</td>\n",
       "      <td>0.037000</td>\n",
       "      <td>0.029000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.069000</td>\n",
       "      <td>0.090000</td>\n",
       "      <td>0.062000</td>\n",
       "      <td>0.064000</td>\n",
       "      <td>0.066000</td>\n",
       "      <td>0.055000</td>\n",
       "      <td>0.062000</td>\n",
       "      <td>0.068000</td>\n",
       "      <td>0.069000</td>\n",
       "      <td>0.073000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Australia</th>\n",
       "      <td>0.040489</td>\n",
       "      <td>0.037857</td>\n",
       "      <td>0.022509</td>\n",
       "      <td>0.028747</td>\n",
       "      <td>0.033990</td>\n",
       "      <td>0.037919</td>\n",
       "      <td>0.052289</td>\n",
       "      <td>0.031071</td>\n",
       "      <td>0.008696</td>\n",
       "      <td>0.006774</td>\n",
       "      <td>...</td>\n",
       "      <td>0.054983</td>\n",
       "      <td>0.048067</td>\n",
       "      <td>0.026982</td>\n",
       "      <td>0.032731</td>\n",
       "      <td>0.038575</td>\n",
       "      <td>0.058013</td>\n",
       "      <td>0.059519</td>\n",
       "      <td>0.056649</td>\n",
       "      <td>0.045825</td>\n",
       "      <td>0.027523</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Austria</th>\n",
       "      <td>-0.013084</td>\n",
       "      <td>-0.007581</td>\n",
       "      <td>0.000543</td>\n",
       "      <td>0.001181</td>\n",
       "      <td>0.025511</td>\n",
       "      <td>0.019941</td>\n",
       "      <td>0.017088</td>\n",
       "      <td>0.023035</td>\n",
       "      <td>0.025293</td>\n",
       "      <td>0.021850</td>\n",
       "      <td>...</td>\n",
       "      <td>0.032616</td>\n",
       "      <td>0.038320</td>\n",
       "      <td>0.035104</td>\n",
       "      <td>0.037220</td>\n",
       "      <td>0.038982</td>\n",
       "      <td>0.036198</td>\n",
       "      <td>0.032570</td>\n",
       "      <td>0.031558</td>\n",
       "      <td>0.019095</td>\n",
       "      <td>0.017431</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Canada</th>\n",
       "      <td>0.010064</td>\n",
       "      <td>0.021264</td>\n",
       "      <td>0.018919</td>\n",
       "      <td>0.025317</td>\n",
       "      <td>0.043567</td>\n",
       "      <td>0.050225</td>\n",
       "      <td>0.065122</td>\n",
       "      <td>0.067331</td>\n",
       "      <td>0.050921</td>\n",
       "      <td>0.031525</td>\n",
       "      <td>...</td>\n",
       "      <td>0.050334</td>\n",
       "      <td>0.049476</td>\n",
       "      <td>0.041199</td>\n",
       "      <td>0.031677</td>\n",
       "      <td>0.020005</td>\n",
       "      <td>0.030712</td>\n",
       "      <td>0.039827</td>\n",
       "      <td>0.034742</td>\n",
       "      <td>0.038128</td>\n",
       "      <td>0.029217</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Denmark</th>\n",
       "      <td>-0.012292</td>\n",
       "      <td>-0.003093</td>\n",
       "      <td>-0.007764</td>\n",
       "      <td>-0.004049</td>\n",
       "      <td>0.031094</td>\n",
       "      <td>0.064280</td>\n",
       "      <td>0.045955</td>\n",
       "      <td>0.055166</td>\n",
       "      <td>0.048057</td>\n",
       "      <td>0.011954</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028752</td>\n",
       "      <td>0.049316</td>\n",
       "      <td>0.038801</td>\n",
       "      <td>0.041836</td>\n",
       "      <td>0.029809</td>\n",
       "      <td>0.033134</td>\n",
       "      <td>-0.007169</td>\n",
       "      <td>0.013517</td>\n",
       "      <td>0.023794</td>\n",
       "      <td>-0.005200</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Finland</th>\n",
       "      <td>-0.028357</td>\n",
       "      <td>-0.023397</td>\n",
       "      <td>-0.006018</td>\n",
       "      <td>-0.004774</td>\n",
       "      <td>0.012886</td>\n",
       "      <td>0.035090</td>\n",
       "      <td>0.035247</td>\n",
       "      <td>0.057251</td>\n",
       "      <td>0.068382</td>\n",
       "      <td>0.079265</td>\n",
       "      <td>...</td>\n",
       "      <td>0.027939</td>\n",
       "      <td>0.043949</td>\n",
       "      <td>0.050649</td>\n",
       "      <td>0.044582</td>\n",
       "      <td>0.046806</td>\n",
       "      <td>0.047340</td>\n",
       "      <td>0.046808</td>\n",
       "      <td>0.045647</td>\n",
       "      <td>0.044177</td>\n",
       "      <td>0.023732</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>France</th>\n",
       "      <td>-0.015177</td>\n",
       "      <td>-0.014549</td>\n",
       "      <td>-0.016704</td>\n",
       "      <td>-0.007476</td>\n",
       "      <td>0.003748</td>\n",
       "      <td>0.016165</td>\n",
       "      <td>0.023915</td>\n",
       "      <td>0.029711</td>\n",
       "      <td>0.027446</td>\n",
       "      <td>0.021708</td>\n",
       "      <td>...</td>\n",
       "      <td>0.023968</td>\n",
       "      <td>0.027028</td>\n",
       "      <td>0.033463</td>\n",
       "      <td>0.030261</td>\n",
       "      <td>0.034915</td>\n",
       "      <td>0.032755</td>\n",
       "      <td>0.030355</td>\n",
       "      <td>0.036748</td>\n",
       "      <td>0.021745</td>\n",
       "      <td>0.018386</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Germany</th>\n",
       "      <td>-0.019680</td>\n",
       "      <td>-0.015441</td>\n",
       "      <td>-0.012701</td>\n",
       "      <td>-0.011667</td>\n",
       "      <td>0.022950</td>\n",
       "      <td>0.021070</td>\n",
       "      <td>0.020662</td>\n",
       "      <td>0.028744</td>\n",
       "      <td>0.016826</td>\n",
       "      <td>0.028715</td>\n",
       "      <td>...</td>\n",
       "      <td>0.007969</td>\n",
       "      <td>0.008848</td>\n",
       "      <td>0.016911</td>\n",
       "      <td>0.023511</td>\n",
       "      <td>0.033108</td>\n",
       "      <td>0.035225</td>\n",
       "      <td>0.023897</td>\n",
       "      <td>0.020773</td>\n",
       "      <td>0.005865</td>\n",
       "      <td>0.009020</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Italy</th>\n",
       "      <td>-0.023383</td>\n",
       "      <td>-0.018116</td>\n",
       "      <td>-0.016875</td>\n",
       "      <td>-0.004963</td>\n",
       "      <td>-0.002249</td>\n",
       "      <td>0.011635</td>\n",
       "      <td>0.026412</td>\n",
       "      <td>0.034283</td>\n",
       "      <td>0.025394</td>\n",
       "      <td>0.022612</td>\n",
       "      <td>...</td>\n",
       "      <td>0.015251</td>\n",
       "      <td>0.017682</td>\n",
       "      <td>0.019596</td>\n",
       "      <td>0.011813</td>\n",
       "      <td>0.012561</td>\n",
       "      <td>0.025621</td>\n",
       "      <td>0.017251</td>\n",
       "      <td>0.023338</td>\n",
       "      <td>0.005081</td>\n",
       "      <td>-0.005581</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Japan</th>\n",
       "      <td>0.012683</td>\n",
       "      <td>-0.005571</td>\n",
       "      <td>-0.017558</td>\n",
       "      <td>-0.010101</td>\n",
       "      <td>-0.022503</td>\n",
       "      <td>-0.005157</td>\n",
       "      <td>0.014087</td>\n",
       "      <td>0.005427</td>\n",
       "      <td>0.003919</td>\n",
       "      <td>0.015349</td>\n",
       "      <td>...</td>\n",
       "      <td>0.018941</td>\n",
       "      <td>0.015203</td>\n",
       "      <td>0.010000</td>\n",
       "      <td>0.005606</td>\n",
       "      <td>0.015460</td>\n",
       "      <td>0.023959</td>\n",
       "      <td>0.014193</td>\n",
       "      <td>0.012907</td>\n",
       "      <td>-0.004768</td>\n",
       "      <td>-0.013647</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Korea</th>\n",
       "      <td>0.058586</td>\n",
       "      <td>0.069521</td>\n",
       "      <td>0.081646</td>\n",
       "      <td>0.085533</td>\n",
       "      <td>0.085922</td>\n",
       "      <td>0.088415</td>\n",
       "      <td>0.092796</td>\n",
       "      <td>0.117461</td>\n",
       "      <td>0.115388</td>\n",
       "      <td>0.121889</td>\n",
       "      <td>...</td>\n",
       "      <td>0.028250</td>\n",
       "      <td>0.033213</td>\n",
       "      <td>0.025254</td>\n",
       "      <td>0.019650</td>\n",
       "      <td>0.014680</td>\n",
       "      <td>0.030068</td>\n",
       "      <td>0.037898</td>\n",
       "      <td>0.039380</td>\n",
       "      <td>0.035359</td>\n",
       "      <td>0.028462</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Mexico</th>\n",
       "      <td>0.043746</td>\n",
       "      <td>0.012290</td>\n",
       "      <td>0.004462</td>\n",
       "      <td>0.015493</td>\n",
       "      <td>0.034448</td>\n",
       "      <td>0.063911</td>\n",
       "      <td>0.064322</td>\n",
       "      <td>0.066942</td>\n",
       "      <td>0.043418</td>\n",
       "      <td>-0.059943</td>\n",
       "      <td>...</td>\n",
       "      <td>0.038414</td>\n",
       "      <td>0.070147</td>\n",
       "      <td>0.100338</td>\n",
       "      <td>0.083649</td>\n",
       "      <td>0.058821</td>\n",
       "      <td>0.038222</td>\n",
       "      <td>0.022226</td>\n",
       "      <td>0.031162</td>\n",
       "      <td>0.062714</td>\n",
       "      <td>0.056493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Netherlands</th>\n",
       "      <td>-0.008440</td>\n",
       "      <td>0.006629</td>\n",
       "      <td>0.008677</td>\n",
       "      <td>0.004312</td>\n",
       "      <td>0.016939</td>\n",
       "      <td>0.018988</td>\n",
       "      <td>0.019583</td>\n",
       "      <td>0.032141</td>\n",
       "      <td>0.027249</td>\n",
       "      <td>0.028448</td>\n",
       "      <td>...</td>\n",
       "      <td>0.032864</td>\n",
       "      <td>0.040600</td>\n",
       "      <td>0.036386</td>\n",
       "      <td>0.034624</td>\n",
       "      <td>0.042949</td>\n",
       "      <td>0.033721</td>\n",
       "      <td>0.027531</td>\n",
       "      <td>0.039287</td>\n",
       "      <td>0.031527</td>\n",
       "      <td>0.033958</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>NewZealand</th>\n",
       "      <td>0.047244</td>\n",
       "      <td>0.038759</td>\n",
       "      <td>0.089918</td>\n",
       "      <td>0.069751</td>\n",
       "      <td>0.060199</td>\n",
       "      <td>0.062555</td>\n",
       "      <td>0.042925</td>\n",
       "      <td>0.047609</td>\n",
       "      <td>0.022150</td>\n",
       "      <td>0.023303</td>\n",
       "      <td>...</td>\n",
       "      <td>0.008914</td>\n",
       "      <td>0.018647</td>\n",
       "      <td>-0.009261</td>\n",
       "      <td>0.011501</td>\n",
       "      <td>0.036756</td>\n",
       "      <td>0.039462</td>\n",
       "      <td>0.058293</td>\n",
       "      <td>0.051147</td>\n",
       "      <td>0.045905</td>\n",
       "      <td>0.031215</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Norway</th>\n",
       "      <td>0.004947</td>\n",
       "      <td>0.020743</td>\n",
       "      <td>0.038871</td>\n",
       "      <td>0.053403</td>\n",
       "      <td>0.031002</td>\n",
       "      <td>0.051313</td>\n",
       "      <td>0.024324</td>\n",
       "      <td>0.027524</td>\n",
       "      <td>0.065272</td>\n",
       "      <td>0.039222</td>\n",
       "      <td>...</td>\n",
       "      <td>0.100206</td>\n",
       "      <td>0.131181</td>\n",
       "      <td>0.096714</td>\n",
       "      <td>0.071056</td>\n",
       "      <td>0.034303</td>\n",
       "      <td>0.008020</td>\n",
       "      <td>0.033556</td>\n",
       "      <td>0.039770</td>\n",
       "      <td>0.089855</td>\n",
       "      <td>0.091568</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Switzerland</th>\n",
       "      <td>-0.032865</td>\n",
       "      <td>-0.019818</td>\n",
       "      <td>-0.004587</td>\n",
       "      <td>0.013651</td>\n",
       "      <td>0.026644</td>\n",
       "      <td>0.010768</td>\n",
       "      <td>0.013645</td>\n",
       "      <td>0.013112</td>\n",
       "      <td>-0.004910</td>\n",
       "      <td>-0.000607</td>\n",
       "      <td>...</td>\n",
       "      <td>0.027786</td>\n",
       "      <td>0.039038</td>\n",
       "      <td>0.035385</td>\n",
       "      <td>0.042196</td>\n",
       "      <td>0.042473</td>\n",
       "      <td>0.046517</td>\n",
       "      <td>0.046339</td>\n",
       "      <td>0.040958</td>\n",
       "      <td>0.037792</td>\n",
       "      <td>0.028398</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UnitedKingdom</th>\n",
       "      <td>0.015124</td>\n",
       "      <td>0.014795</td>\n",
       "      <td>0.029149</td>\n",
       "      <td>0.036581</td>\n",
       "      <td>0.030078</td>\n",
       "      <td>0.040359</td>\n",
       "      <td>0.038045</td>\n",
       "      <td>0.045036</td>\n",
       "      <td>0.031924</td>\n",
       "      <td>0.035790</td>\n",
       "      <td>...</td>\n",
       "      <td>0.017171</td>\n",
       "      <td>0.029647</td>\n",
       "      <td>0.025653</td>\n",
       "      <td>0.040619</td>\n",
       "      <td>0.032137</td>\n",
       "      <td>0.027946</td>\n",
       "      <td>0.039619</td>\n",
       "      <td>0.038682</td>\n",
       "      <td>0.033362</td>\n",
       "      <td>0.028492</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>UnitedStates</th>\n",
       "      <td>0.022959</td>\n",
       "      <td>0.018936</td>\n",
       "      <td>0.017990</td>\n",
       "      <td>0.020683</td>\n",
       "      <td>0.029918</td>\n",
       "      <td>0.037840</td>\n",
       "      <td>0.035534</td>\n",
       "      <td>0.035411</td>\n",
       "      <td>0.026520</td>\n",
       "      <td>0.012290</td>\n",
       "      <td>...</td>\n",
       "      <td>0.023995</td>\n",
       "      <td>0.028241</td>\n",
       "      <td>0.026496</td>\n",
       "      <td>0.022376</td>\n",
       "      <td>0.032515</td>\n",
       "      <td>0.017708</td>\n",
       "      <td>0.018756</td>\n",
       "      <td>0.028225</td>\n",
       "      <td>0.009288</td>\n",
       "      <td>0.006137</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Singapore</th>\n",
       "      <td>0.087145</td>\n",
       "      <td>0.118075</td>\n",
       "      <td>0.111130</td>\n",
       "      <td>0.125324</td>\n",
       "      <td>0.130709</td>\n",
       "      <td>0.100987</td>\n",
       "      <td>0.115984</td>\n",
       "      <td>0.092047</td>\n",
       "      <td>0.066013</td>\n",
       "      <td>0.079020</td>\n",
       "      <td>...</td>\n",
       "      <td>0.080031</td>\n",
       "      <td>0.096518</td>\n",
       "      <td>0.076721</td>\n",
       "      <td>0.067747</td>\n",
       "      <td>0.063880</td>\n",
       "      <td>0.063092</td>\n",
       "      <td>0.101377</td>\n",
       "      <td>0.105562</td>\n",
       "      <td>0.069739</td>\n",
       "      <td>0.079339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Philippines</th>\n",
       "      <td>-0.004381</td>\n",
       "      <td>0.016636</td>\n",
       "      <td>0.031504</td>\n",
       "      <td>0.034007</td>\n",
       "      <td>0.049344</td>\n",
       "      <td>0.059130</td>\n",
       "      <td>0.062870</td>\n",
       "      <td>0.062091</td>\n",
       "      <td>0.052228</td>\n",
       "      <td>0.041678</td>\n",
       "      <td>...</td>\n",
       "      <td>0.045504</td>\n",
       "      <td>0.046006</td>\n",
       "      <td>0.036120</td>\n",
       "      <td>0.043091</td>\n",
       "      <td>0.048391</td>\n",
       "      <td>0.065573</td>\n",
       "      <td>0.081891</td>\n",
       "      <td>0.064359</td>\n",
       "      <td>0.065470</td>\n",
       "      <td>0.047945</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Indonesia</th>\n",
       "      <td>0.064024</td>\n",
       "      <td>0.066068</td>\n",
       "      <td>0.057959</td>\n",
       "      <td>0.062365</td>\n",
       "      <td>0.049743</td>\n",
       "      <td>0.071988</td>\n",
       "      <td>0.069357</td>\n",
       "      <td>0.070347</td>\n",
       "      <td>0.103778</td>\n",
       "      <td>0.083252</td>\n",
       "      <td>...</td>\n",
       "      <td>0.072814</td>\n",
       "      <td>0.053086</td>\n",
       "      <td>0.043391</td>\n",
       "      <td>0.054189</td>\n",
       "      <td>0.080885</td>\n",
       "      <td>0.095297</td>\n",
       "      <td>0.110900</td>\n",
       "      <td>0.110100</td>\n",
       "      <td>0.110700</td>\n",
       "      <td>0.133400</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Malaysia</th>\n",
       "      <td>0.085938</td>\n",
       "      <td>0.131189</td>\n",
       "      <td>0.109666</td>\n",
       "      <td>0.075801</td>\n",
       "      <td>0.049147</td>\n",
       "      <td>0.061173</td>\n",
       "      <td>0.101110</td>\n",
       "      <td>0.134978</td>\n",
       "      <td>0.130363</td>\n",
       "      <td>0.119140</td>\n",
       "      <td>...</td>\n",
       "      <td>0.063079</td>\n",
       "      <td>0.112993</td>\n",
       "      <td>0.116340</td>\n",
       "      <td>0.110228</td>\n",
       "      <td>0.100116</td>\n",
       "      <td>0.039884</td>\n",
       "      <td>0.080276</td>\n",
       "      <td>0.093361</td>\n",
       "      <td>0.151739</td>\n",
       "      <td>0.167034</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Thailand</th>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.112509</td>\n",
       "      <td>0.092613</td>\n",
       "      <td>0.050375</td>\n",
       "      <td>0.093952</td>\n",
       "      <td>0.106779</td>\n",
       "      <td>0.112031</td>\n",
       "      <td>...</td>\n",
       "      <td>0.035448</td>\n",
       "      <td>0.066224</td>\n",
       "      <td>0.054813</td>\n",
       "      <td>0.048648</td>\n",
       "      <td>0.042142</td>\n",
       "      <td>0.049846</td>\n",
       "      <td>0.051197</td>\n",
       "      <td>0.050126</td>\n",
       "      <td>0.079587</td>\n",
       "      <td>0.049144</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Taiwan</th>\n",
       "      <td>0.064902</td>\n",
       "      <td>0.065123</td>\n",
       "      <td>0.067379</td>\n",
       "      <td>0.069164</td>\n",
       "      <td>0.069451</td>\n",
       "      <td>0.070135</td>\n",
       "      <td>0.069298</td>\n",
       "      <td>0.076049</td>\n",
       "      <td>0.071104</td>\n",
       "      <td>0.069214</td>\n",
       "      <td>...</td>\n",
       "      <td>0.066583</td>\n",
       "      <td>0.049956</td>\n",
       "      <td>0.049499</td>\n",
       "      <td>0.052018</td>\n",
       "      <td>0.039880</td>\n",
       "      <td>0.041019</td>\n",
       "      <td>0.051073</td>\n",
       "      <td>0.066369</td>\n",
       "      <td>0.062929</td>\n",
       "      <td>0.058841</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>China</th>\n",
       "      <td>0.143000</td>\n",
       "      <td>0.141000</td>\n",
       "      <td>0.135000</td>\n",
       "      <td>0.135000</td>\n",
       "      <td>0.125000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.120000</td>\n",
       "      <td>0.126000</td>\n",
       "      <td>0.115000</td>\n",
       "      <td>0.106000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.104000</td>\n",
       "      <td>0.104000</td>\n",
       "      <td>0.110000</td>\n",
       "      <td>0.108000</td>\n",
       "      <td>0.111000</td>\n",
       "      <td>0.111000</td>\n",
       "      <td>0.116700</td>\n",
       "      <td>0.100200</td>\n",
       "      <td>0.101700</td>\n",
       "      <td>0.123800</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>25 rows × 61 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "quarter          1993Q1    1993Q2    1993Q3    1993Q4    1994Q1    1994Q2  \\\n",
       "city                                                                        \n",
       "HongKong       0.062000  0.059000  0.058000  0.062000  0.079000  0.068000   \n",
       "Australia      0.040489  0.037857  0.022509  0.028747  0.033990  0.037919   \n",
       "Austria       -0.013084 -0.007581  0.000543  0.001181  0.025511  0.019941   \n",
       "Canada         0.010064  0.021264  0.018919  0.025317  0.043567  0.050225   \n",
       "Denmark       -0.012292 -0.003093 -0.007764 -0.004049  0.031094  0.064280   \n",
       "Finland       -0.028357 -0.023397 -0.006018 -0.004774  0.012886  0.035090   \n",
       "France        -0.015177 -0.014549 -0.016704 -0.007476  0.003748  0.016165   \n",
       "Germany       -0.019680 -0.015441 -0.012701 -0.011667  0.022950  0.021070   \n",
       "Italy         -0.023383 -0.018116 -0.016875 -0.004963 -0.002249  0.011635   \n",
       "Japan          0.012683 -0.005571 -0.017558 -0.010101 -0.022503 -0.005157   \n",
       "Korea          0.058586  0.069521  0.081646  0.085533  0.085922  0.088415   \n",
       "Mexico         0.043746  0.012290  0.004462  0.015493  0.034448  0.063911   \n",
       "Netherlands   -0.008440  0.006629  0.008677  0.004312  0.016939  0.018988   \n",
       "NewZealand     0.047244  0.038759  0.089918  0.069751  0.060199  0.062555   \n",
       "Norway         0.004947  0.020743  0.038871  0.053403  0.031002  0.051313   \n",
       "Switzerland   -0.032865 -0.019818 -0.004587  0.013651  0.026644  0.010768   \n",
       "UnitedKingdom  0.015124  0.014795  0.029149  0.036581  0.030078  0.040359   \n",
       "UnitedStates   0.022959  0.018936  0.017990  0.020683  0.029918  0.037840   \n",
       "Singapore      0.087145  0.118075  0.111130  0.125324  0.130709  0.100987   \n",
       "Philippines   -0.004381  0.016636  0.031504  0.034007  0.049344  0.059130   \n",
       "Indonesia      0.064024  0.066068  0.057959  0.062365  0.049743  0.071988   \n",
       "Malaysia       0.085938  0.131189  0.109666  0.075801  0.049147  0.061173   \n",
       "Thailand       0.080000  0.080000  0.080000  0.080000  0.112509  0.092613   \n",
       "Taiwan         0.064902  0.065123  0.067379  0.069164  0.069451  0.070135   \n",
       "China          0.143000  0.141000  0.135000  0.135000  0.125000  0.120000   \n",
       "\n",
       "quarter          1994Q3    1994Q4    1995Q1    1995Q2  ...    2005Q4  \\\n",
       "city                                                   ...             \n",
       "HongKong       0.046000  0.052000  0.037000  0.029000  ...  0.069000   \n",
       "Australia      0.052289  0.031071  0.008696  0.006774  ...  0.054983   \n",
       "Austria        0.017088  0.023035  0.025293  0.021850  ...  0.032616   \n",
       "Canada         0.065122  0.067331  0.050921  0.031525  ...  0.050334   \n",
       "Denmark        0.045955  0.055166  0.048057  0.011954  ...  0.028752   \n",
       "Finland        0.035247  0.057251  0.068382  0.079265  ...  0.027939   \n",
       "France         0.023915  0.029711  0.027446  0.021708  ...  0.023968   \n",
       "Germany        0.020662  0.028744  0.016826  0.028715  ...  0.007969   \n",
       "Italy          0.026412  0.034283  0.025394  0.022612  ...  0.015251   \n",
       "Japan          0.014087  0.005427  0.003919  0.015349  ...  0.018941   \n",
       "Korea          0.092796  0.117461  0.115388  0.121889  ...  0.028250   \n",
       "Mexico         0.064322  0.066942  0.043418 -0.059943  ...  0.038414   \n",
       "Netherlands    0.019583  0.032141  0.027249  0.028448  ...  0.032864   \n",
       "NewZealand     0.042925  0.047609  0.022150  0.023303  ...  0.008914   \n",
       "Norway         0.024324  0.027524  0.065272  0.039222  ...  0.100206   \n",
       "Switzerland    0.013645  0.013112 -0.004910 -0.000607  ...  0.027786   \n",
       "UnitedKingdom  0.038045  0.045036  0.031924  0.035790  ...  0.017171   \n",
       "UnitedStates   0.035534  0.035411  0.026520  0.012290  ...  0.023995   \n",
       "Singapore      0.115984  0.092047  0.066013  0.079020  ...  0.080031   \n",
       "Philippines    0.062870  0.062091  0.052228  0.041678  ...  0.045504   \n",
       "Indonesia      0.069357  0.070347  0.103778  0.083252  ...  0.072814   \n",
       "Malaysia       0.101110  0.134978  0.130363  0.119140  ...  0.063079   \n",
       "Thailand       0.050375  0.093952  0.106779  0.112031  ...  0.035448   \n",
       "Taiwan         0.069298  0.076049  0.071104  0.069214  ...  0.066583   \n",
       "China          0.120000  0.126000  0.115000  0.106000  ...  0.104000   \n",
       "\n",
       "quarter          2006Q1    2006Q2    2006Q3    2006Q4    2007Q1    2007Q2  \\\n",
       "city                                                                        \n",
       "HongKong       0.090000  0.062000  0.064000  0.066000  0.055000  0.062000   \n",
       "Australia      0.048067  0.026982  0.032731  0.038575  0.058013  0.059519   \n",
       "Austria        0.038320  0.035104  0.037220  0.038982  0.036198  0.032570   \n",
       "Canada         0.049476  0.041199  0.031677  0.020005  0.030712  0.039827   \n",
       "Denmark        0.049316  0.038801  0.041836  0.029809  0.033134 -0.007169   \n",
       "Finland        0.043949  0.050649  0.044582  0.046806  0.047340  0.046808   \n",
       "France         0.027028  0.033463  0.030261  0.034915  0.032755  0.030355   \n",
       "Germany        0.008848  0.016911  0.023511  0.033108  0.035225  0.023897   \n",
       "Italy          0.017682  0.019596  0.011813  0.012561  0.025621  0.017251   \n",
       "Japan          0.015203  0.010000  0.005606  0.015460  0.023959  0.014193   \n",
       "Korea          0.033213  0.025254  0.019650  0.014680  0.030068  0.037898   \n",
       "Mexico         0.070147  0.100338  0.083649  0.058821  0.038222  0.022226   \n",
       "Netherlands    0.040600  0.036386  0.034624  0.042949  0.033721  0.027531   \n",
       "NewZealand     0.018647 -0.009261  0.011501  0.036756  0.039462  0.058293   \n",
       "Norway         0.131181  0.096714  0.071056  0.034303  0.008020  0.033556   \n",
       "Switzerland    0.039038  0.035385  0.042196  0.042473  0.046517  0.046339   \n",
       "UnitedKingdom  0.029647  0.025653  0.040619  0.032137  0.027946  0.039619   \n",
       "UnitedStates   0.028241  0.026496  0.022376  0.032515  0.017708  0.018756   \n",
       "Singapore      0.096518  0.076721  0.067747  0.063880  0.063092  0.101377   \n",
       "Philippines    0.046006  0.036120  0.043091  0.048391  0.065573  0.081891   \n",
       "Indonesia      0.053086  0.043391  0.054189  0.080885  0.095297  0.110900   \n",
       "Malaysia       0.112993  0.116340  0.110228  0.100116  0.039884  0.080276   \n",
       "Thailand       0.066224  0.054813  0.048648  0.042142  0.049846  0.051197   \n",
       "Taiwan         0.049956  0.049499  0.052018  0.039880  0.041019  0.051073   \n",
       "China          0.104000  0.110000  0.108000  0.111000  0.111000  0.116700   \n",
       "\n",
       "quarter          2007Q3    2007Q4    2008Q1  \n",
       "city                                         \n",
       "HongKong       0.068000  0.069000  0.073000  \n",
       "Australia      0.056649  0.045825  0.027523  \n",
       "Austria        0.031558  0.019095  0.017431  \n",
       "Canada         0.034742  0.038128  0.029217  \n",
       "Denmark        0.013517  0.023794 -0.005200  \n",
       "Finland        0.045647  0.044177  0.023732  \n",
       "France         0.036748  0.021745  0.018386  \n",
       "Germany        0.020773  0.005865  0.009020  \n",
       "Italy          0.023338  0.005081 -0.005581  \n",
       "Japan          0.012907 -0.004768 -0.013647  \n",
       "Korea          0.039380  0.035359  0.028462  \n",
       "Mexico         0.031162  0.062714  0.056493  \n",
       "Netherlands    0.039287  0.031527  0.033958  \n",
       "NewZealand     0.051147  0.045905  0.031215  \n",
       "Norway         0.039770  0.089855  0.091568  \n",
       "Switzerland    0.040958  0.037792  0.028398  \n",
       "UnitedKingdom  0.038682  0.033362  0.028492  \n",
       "UnitedStates   0.028225  0.009288  0.006137  \n",
       "Singapore      0.105562  0.069739  0.079339  \n",
       "Philippines    0.064359  0.065470  0.047945  \n",
       "Indonesia      0.110100  0.110700  0.133400  \n",
       "Malaysia       0.093361  0.151739  0.167034  \n",
       "Thailand       0.050126  0.079587  0.049144  \n",
       "Taiwan         0.066369  0.062929  0.058841  \n",
       "China          0.100200  0.101700  0.123800  \n",
       "\n",
       "[25 rows x 61 columns]"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "hcw_new.unstack(0)  ## 指定按照层级索引的哪个level为列\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，如果一个变量本来就是长格式，就方便很多，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>CityCode</th>\n",
       "      <th>City</th>\n",
       "      <th>Year</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>110000</td>\n",
       "      <td>北京市</td>\n",
       "      <td>2003</td>\n",
       "      <td>1148.81990</td>\n",
       "      <td>36631000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>120000</td>\n",
       "      <td>天津市</td>\n",
       "      <td>2003</td>\n",
       "      <td>926.00000</td>\n",
       "      <td>24476600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>130100</td>\n",
       "      <td>石家庄市</td>\n",
       "      <td>2003</td>\n",
       "      <td>910.51001</td>\n",
       "      <td>13779438.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>130200</td>\n",
       "      <td>唐山市</td>\n",
       "      <td>2003</td>\n",
       "      <td>706.28003</td>\n",
       "      <td>12953220.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>130300</td>\n",
       "      <td>秦皇岛市</td>\n",
       "      <td>2003</td>\n",
       "      <td>273.29001</td>\n",
       "      <td>3870301.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   CityCode  City  Year  population         gdp\n",
       "0    110000   北京市  2003  1148.81990  36631000.0\n",
       "1    120000   天津市  2003   926.00000  24476600.0\n",
       "2    130100  石家庄市  2003   910.51001  13779438.0\n",
       "3    130200   唐山市  2003   706.28003  12953220.0\n",
       "4    130300  秦皇岛市  2003   273.29001   3870301.0"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city = pd.read_csv('csv/city.csv')\n",
    "city.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以指定层级索引比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CityCode</th>\n",
       "      <th>Year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>110000</th>\n",
       "      <th>2003</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1148.81990</td>\n",
       "      <td>36631000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120000</th>\n",
       "      <th>2003</th>\n",
       "      <td>天津市</td>\n",
       "      <td>926.00000</td>\n",
       "      <td>24476600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130100</th>\n",
       "      <th>2003</th>\n",
       "      <td>石家庄市</td>\n",
       "      <td>910.51001</td>\n",
       "      <td>13779438.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130200</th>\n",
       "      <th>2003</th>\n",
       "      <td>唐山市</td>\n",
       "      <td>706.28003</td>\n",
       "      <td>12953220.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130300</th>\n",
       "      <th>2003</th>\n",
       "      <td>秦皇岛市</td>\n",
       "      <td>273.29001</td>\n",
       "      <td>3870301.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640300</th>\n",
       "      <th>2006</th>\n",
       "      <td>吴忠市</td>\n",
       "      <td>127.66000</td>\n",
       "      <td>1196379.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640400</th>\n",
       "      <th>2006</th>\n",
       "      <td>固原市</td>\n",
       "      <td>151.23000</td>\n",
       "      <td>519279.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640500</th>\n",
       "      <th>2006</th>\n",
       "      <td>中卫市</td>\n",
       "      <td>104.19000</td>\n",
       "      <td>750050.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650100</th>\n",
       "      <th>2006</th>\n",
       "      <td>乌鲁木齐市</td>\n",
       "      <td>201.84000</td>\n",
       "      <td>6543019.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650200</th>\n",
       "      <th>2006</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>32.32000</td>\n",
       "      <td>4732562.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3402 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                City  population         gdp\n",
       "CityCode Year                               \n",
       "110000   2003    北京市  1148.81990  36631000.0\n",
       "120000   2003    天津市   926.00000  24476600.0\n",
       "130100   2003   石家庄市   910.51001  13779438.0\n",
       "130200   2003    唐山市   706.28003  12953220.0\n",
       "130300   2003   秦皇岛市   273.29001   3870301.0\n",
       "...              ...         ...         ...\n",
       "640300   2006    吴忠市   127.66000   1196379.0\n",
       "640400   2006    固原市   151.23000    519279.0\n",
       "640500   2006    中卫市   104.19000    750050.0\n",
       "650100   2006  乌鲁木齐市   201.84000   6543019.0\n",
       "650200   2006  克拉玛依市    32.32000   4732562.0\n",
       "\n",
       "[3402 rows x 3 columns]"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city = city.set_index(['CityCode', 'Year'])\n",
    "city"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "不过这样做有两个问题，比如，如果使用切片操作可能会报错，所以一般需要先对index排序："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CityCode</th>\n",
       "      <th>Year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">110000</th>\n",
       "      <th>2000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1107.530000</td>\n",
       "      <td>24787600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2001</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1122.300000</td>\n",
       "      <td>28456500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1136.000000</td>\n",
       "      <td>32127100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1148.819900</td>\n",
       "      <td>36631000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2004</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1162.890000</td>\n",
       "      <td>42833100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">650200</th>\n",
       "      <th>2007</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>35.340000</td>\n",
       "      <td>5151297.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>38.619999</td>\n",
       "      <td>6612062.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2009</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>39.349998</td>\n",
       "      <td>4802909.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>37.509998</td>\n",
       "      <td>7113531.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011</th>\n",
       "      <td>克拉玛依市</td>\n",
       "      <td>37.799999</td>\n",
       "      <td>8016855.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3402 rows × 3 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                City   population         gdp\n",
       "CityCode Year                                \n",
       "110000   2000    北京市  1107.530000  24787600.0\n",
       "         2001    北京市  1122.300000  28456500.0\n",
       "         2002    北京市  1136.000000  32127100.0\n",
       "         2003    北京市  1148.819900  36631000.0\n",
       "         2004    北京市  1162.890000  42833100.0\n",
       "...              ...          ...         ...\n",
       "650200   2007  克拉玛依市    35.340000   5151297.0\n",
       "         2008  克拉玛依市    38.619999   6612062.0\n",
       "         2009  克拉玛依市    39.349998   4802909.0\n",
       "         2010  克拉玛依市    37.509998   7113531.0\n",
       "         2011  克拉玛依市    37.799999   8016855.0\n",
       "\n",
       "[3402 rows x 3 columns]"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city = city.sort_index()\n",
    "city"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然后就可以进行切片操作了："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1321.6300</td>\n",
       "      <td>45511500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2001</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1327.1400</td>\n",
       "      <td>49508400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1334.0000</td>\n",
       "      <td>54087600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1341.7700</td>\n",
       "      <td>62508100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2004</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1352.3900</td>\n",
       "      <td>74502700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1360.2600</td>\n",
       "      <td>91541800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1368.0800</td>\n",
       "      <td>103663700.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1378.8600</td>\n",
       "      <td>121888500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1391.0400</td>\n",
       "      <td>136981500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2009</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1400.7000</td>\n",
       "      <td>150464500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1412.3199</td>\n",
       "      <td>171659800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1419.4000</td>\n",
       "      <td>191956900.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     City  population          gdp\n",
       "Year                              \n",
       "2000  上海市   1321.6300   45511500.0\n",
       "2001  上海市   1327.1400   49508400.0\n",
       "2002  上海市   1334.0000   54087600.0\n",
       "2003  上海市   1341.7700   62508100.0\n",
       "2004  上海市   1352.3900   74502700.0\n",
       "2005  上海市   1360.2600   91541800.0\n",
       "2006  上海市   1368.0800  103663700.0\n",
       "2007  上海市   1378.8600  121888500.0\n",
       "2008  上海市   1391.0400  136981500.0\n",
       "2009  上海市   1400.7000  150464500.0\n",
       "2010  上海市   1412.3199  171659800.0\n",
       "2011  上海市   1419.4000  191956900.0"
      ]
     },
     "execution_count": 85,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city.loc[310000,:]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CityCode</th>\n",
       "      <th>Year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>110000</th>\n",
       "      <th>2011</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1277.9000</td>\n",
       "      <td>162519300.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">310000</th>\n",
       "      <th>2010</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1412.3199</td>\n",
       "      <td>171659800.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011</th>\n",
       "      <td>上海市</td>\n",
       "      <td>1419.4000</td>\n",
       "      <td>191956900.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              City  population          gdp\n",
       "CityCode Year                              \n",
       "110000   2011  北京市   1277.9000  162519300.0\n",
       "310000   2010  上海市   1412.3199  171659800.0\n",
       "         2011  上海市   1419.4000  191956900.0"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city[city['gdp'] > 160000000]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而值得注意的是，现在切片操作有三个维度，其中有两个维度是行的index，一个维度是列的index，默认情况下iloc和loc都认为第二个维度是列。此时需要使用**IndexSlice对象**，才能准确表示切片："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>CityCode</th>\n",
       "      <th>Year</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">110000</th>\n",
       "      <th>2001</th>\n",
       "      <td>1122.30000</td>\n",
       "      <td>28456500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>1136.00000</td>\n",
       "      <td>32127100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>1148.81990</td>\n",
       "      <td>36631000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">120000</th>\n",
       "      <th>2001</th>\n",
       "      <td>913.97998</td>\n",
       "      <td>18401000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>919.00000</td>\n",
       "      <td>20511600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"2\" valign=\"top\">650100</th>\n",
       "      <th>2002</th>\n",
       "      <td>176.00000</td>\n",
       "      <td>3544426.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>181.53000</td>\n",
       "      <td>4085834.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">650200</th>\n",
       "      <th>2001</th>\n",
       "      <td>28.43000</td>\n",
       "      <td>1675526.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <td>29.00000</td>\n",
       "      <td>1704788.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <td>30.60000</td>\n",
       "      <td>2162707.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>844 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               population         gdp\n",
       "CityCode Year                        \n",
       "110000   2001  1122.30000  28456500.0\n",
       "         2002  1136.00000  32127100.0\n",
       "         2003  1148.81990  36631000.0\n",
       "120000   2001   913.97998  18401000.0\n",
       "         2002   919.00000  20511600.0\n",
       "...                   ...         ...\n",
       "650100   2002   176.00000   3544426.0\n",
       "         2003   181.53000   4085834.0\n",
       "650200   2001    28.43000   1675526.0\n",
       "         2002    29.00000   1704788.0\n",
       "         2003    30.60000   2162707.0\n",
       "\n",
       "[844 rows x 2 columns]"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ids = pd.IndexSlice\n",
    "city.loc[ids[:, 2001:2003], ['population', 'gdp']]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "另一个问题是在这里很难产生滞后项，如果要做，一种做法是：先unstack成一个宽的格式，然后转为时间序列，产生滞后，再stack并合并进来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "      <th>lag_gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <th>CityCode</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2000</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1107.5300</td>\n",
       "      <td>24787600.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2001</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1122.3000</td>\n",
       "      <td>28456500.0</td>\n",
       "      <td>24787600.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2002</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1136.0000</td>\n",
       "      <td>32127100.0</td>\n",
       "      <td>28456500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2003</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1148.8199</td>\n",
       "      <td>36631000.0</td>\n",
       "      <td>32127100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2004</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1162.8900</td>\n",
       "      <td>42833100.0</td>\n",
       "      <td>36631000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2005</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1180.7000</td>\n",
       "      <td>68863101.0</td>\n",
       "      <td>42833100.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2006</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1197.6000</td>\n",
       "      <td>78702835.0</td>\n",
       "      <td>68863101.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2007</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1213.2600</td>\n",
       "      <td>93533200.0</td>\n",
       "      <td>78702835.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2008</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1299.8500</td>\n",
       "      <td>104880500.0</td>\n",
       "      <td>93533200.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2009</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1245.8300</td>\n",
       "      <td>121530000.0</td>\n",
       "      <td>104880500.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2010</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1257.8000</td>\n",
       "      <td>141135800.0</td>\n",
       "      <td>121530000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2011</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1277.9000</td>\n",
       "      <td>162519300.0</td>\n",
       "      <td>141135800.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "              City  population          gdp      lag_gdp\n",
       "Year CityCode                                           \n",
       "2000 110000    北京市   1107.5300   24787600.0          NaN\n",
       "2001 110000    北京市   1122.3000   28456500.0   24787600.0\n",
       "2002 110000    北京市   1136.0000   32127100.0   28456500.0\n",
       "2003 110000    北京市   1148.8199   36631000.0   32127100.0\n",
       "2004 110000    北京市   1162.8900   42833100.0   36631000.0\n",
       "2005 110000    北京市   1180.7000   68863101.0   42833100.0\n",
       "2006 110000    北京市   1197.6000   78702835.0   68863101.0\n",
       "2007 110000    北京市   1213.2600   93533200.0   78702835.0\n",
       "2008 110000    北京市   1299.8500  104880500.0   93533200.0\n",
       "2009 110000    北京市   1245.8300  121530000.0  104880500.0\n",
       "2010 110000    北京市   1257.8000  141135800.0  121530000.0\n",
       "2011 110000    北京市   1277.9000  162519300.0  141135800.0"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_unstack = city.unstack(0)\n",
    "city_unstack.index = pd.to_datetime(city_unstack.index,\n",
    "                                    format=\"%Y\").to_period('Y')\n",
    "city_unstack\n",
    "lag_gdp = city_unstack['gdp'].shift(1)\n",
    "lag_gdp = lag_gdp.stack()\n",
    "city_new = city_unstack.stack()\n",
    "city_new['lag_gdp'] = lag_gdp\n",
    "city_new.loc[ids[:, 110000], :]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 虚拟变量与数据集合并\n",
    "\n",
    "在Pandas中，专门提供了一个函数用于产生虚拟变量：pd.get_dummies()，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "      <th>lag_gdp</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <th>CityCode</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2000</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1107.53000</td>\n",
       "      <td>24787600.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120000</th>\n",
       "      <td>天津市</td>\n",
       "      <td>912.00000</td>\n",
       "      <td>16393600.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130100</th>\n",
       "      <td>石家庄市</td>\n",
       "      <td>889.79999</td>\n",
       "      <td>10031119.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130200</th>\n",
       "      <td>唐山市</td>\n",
       "      <td>699.78998</td>\n",
       "      <td>9150473.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130300</th>\n",
       "      <td>秦皇岛市</td>\n",
       "      <td>266.29001</td>\n",
       "      <td>2853937.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "               City  population         gdp  lag_gdp\n",
       "Year CityCode                                       \n",
       "2000 110000     北京市  1107.53000  24787600.0      NaN\n",
       "     120000     天津市   912.00000  16393600.0      NaN\n",
       "     130100    石家庄市   889.79999  10031119.0      NaN\n",
       "     130200     唐山市   699.78998   9150473.0      NaN\n",
       "     130300    秦皇岛市   266.29001   2853937.0      NaN"
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_new.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>city_七台河市</th>\n",
       "      <th>city_三亚市</th>\n",
       "      <th>city_三明市</th>\n",
       "      <th>city_三门峡市</th>\n",
       "      <th>city_上海市</th>\n",
       "      <th>city_上饶市</th>\n",
       "      <th>city_东莞市</th>\n",
       "      <th>city_东营市</th>\n",
       "      <th>city_中卫市</th>\n",
       "      <th>city_中山市</th>\n",
       "      <th>...</th>\n",
       "      <th>city_鸡西市</th>\n",
       "      <th>city_鹤壁市</th>\n",
       "      <th>city_鹤岗市</th>\n",
       "      <th>city_鹰潭市</th>\n",
       "      <th>city_黄冈市</th>\n",
       "      <th>city_黄山市</th>\n",
       "      <th>city_黄石市</th>\n",
       "      <th>city_黑河市</th>\n",
       "      <th>city_齐齐哈尔市</th>\n",
       "      <th>city_龙岩市</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <th>CityCode</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2000</th>\n",
       "      <th>110000</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120000</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130100</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130200</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130300</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2011</th>\n",
       "      <th>640300</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640400</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640500</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650100</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>650200</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>3402 rows × 288 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               city_七台河市  city_三亚市  city_三明市  city_三门峡市  city_上海市  city_上饶市  \\\n",
       "Year CityCode                                                                 \n",
       "2000 110000            0         0         0          0         0         0   \n",
       "     120000            0         0         0          0         0         0   \n",
       "     130100            0         0         0          0         0         0   \n",
       "     130200            0         0         0          0         0         0   \n",
       "     130300            0         0         0          0         0         0   \n",
       "...                  ...       ...       ...        ...       ...       ...   \n",
       "2011 640300            0         0         0          0         0         0   \n",
       "     640400            0         0         0          0         0         0   \n",
       "     640500            0         0         0          0         0         0   \n",
       "     650100            0         0         0          0         0         0   \n",
       "     650200            0         0         0          0         0         0   \n",
       "\n",
       "               city_东莞市  city_东营市  city_中卫市  city_中山市  ...  city_鸡西市  \\\n",
       "Year CityCode                                          ...             \n",
       "2000 110000           0         0         0         0  ...         0   \n",
       "     120000           0         0         0         0  ...         0   \n",
       "     130100           0         0         0         0  ...         0   \n",
       "     130200           0         0         0         0  ...         0   \n",
       "     130300           0         0         0         0  ...         0   \n",
       "...                 ...       ...       ...       ...  ...       ...   \n",
       "2011 640300           0         0         0         0  ...         0   \n",
       "     640400           0         0         0         0  ...         0   \n",
       "     640500           0         0         1         0  ...         0   \n",
       "     650100           0         0         0         0  ...         0   \n",
       "     650200           0         0         0         0  ...         0   \n",
       "\n",
       "               city_鹤壁市  city_鹤岗市  city_鹰潭市  city_黄冈市  city_黄山市  city_黄石市  \\\n",
       "Year CityCode                                                               \n",
       "2000 110000           0         0         0         0         0         0   \n",
       "     120000           0         0         0         0         0         0   \n",
       "     130100           0         0         0         0         0         0   \n",
       "     130200           0         0         0         0         0         0   \n",
       "     130300           0         0         0         0         0         0   \n",
       "...                 ...       ...       ...       ...       ...       ...   \n",
       "2011 640300           0         0         0         0         0         0   \n",
       "     640400           0         0         0         0         0         0   \n",
       "     640500           0         0         0         0         0         0   \n",
       "     650100           0         0         0         0         0         0   \n",
       "     650200           0         0         0         0         0         0   \n",
       "\n",
       "               city_黑河市  city_齐齐哈尔市  city_龙岩市  \n",
       "Year CityCode                                  \n",
       "2000 110000           0           0         0  \n",
       "     120000           0           0         0  \n",
       "     130100           0           0         0  \n",
       "     130200           0           0         0  \n",
       "     130300           0           0         0  \n",
       "...                 ...         ...       ...  \n",
       "2011 640300           0           0         0  \n",
       "     640400           0           0         0  \n",
       "     640500           0           0         0  \n",
       "     650100           0           0         0  \n",
       "     650200           0           0         0  \n",
       "\n",
       "[3402 rows x 288 columns]"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_dummy = pd.get_dummies(city_new['City'], prefix=\"city\")\n",
    "city_dummy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中prefix选项可以忽略，不过为了不与其他变量的虚拟变量混淆，最好加上。\n",
    "\n",
    "但是这里有个小问题，产生的虚拟变量是一个单独的数据框，如果需要加回到原来的数据框中，就需要对两个数据框进行合并。合并的方法有很多种，其中的一种简单的方法是："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>City</th>\n",
       "      <th>population</th>\n",
       "      <th>gdp</th>\n",
       "      <th>lag_gdp</th>\n",
       "      <th>city_七台河市</th>\n",
       "      <th>city_三亚市</th>\n",
       "      <th>city_三明市</th>\n",
       "      <th>city_三门峡市</th>\n",
       "      <th>city_上海市</th>\n",
       "      <th>city_上饶市</th>\n",
       "      <th>...</th>\n",
       "      <th>city_鸡西市</th>\n",
       "      <th>city_鹤壁市</th>\n",
       "      <th>city_鹤岗市</th>\n",
       "      <th>city_鹰潭市</th>\n",
       "      <th>city_黄冈市</th>\n",
       "      <th>city_黄山市</th>\n",
       "      <th>city_黄石市</th>\n",
       "      <th>city_黑河市</th>\n",
       "      <th>city_齐齐哈尔市</th>\n",
       "      <th>city_龙岩市</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Year</th>\n",
       "      <th>CityCode</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"5\" valign=\"top\">2000</th>\n",
       "      <th>110000</th>\n",
       "      <td>北京市</td>\n",
       "      <td>1107.53000</td>\n",
       "      <td>24787600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>120000</th>\n",
       "      <td>天津市</td>\n",
       "      <td>912.00000</td>\n",
       "      <td>16393600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130100</th>\n",
       "      <td>石家庄市</td>\n",
       "      <td>889.79999</td>\n",
       "      <td>10031119.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130200</th>\n",
       "      <td>唐山市</td>\n",
       "      <td>699.78998</td>\n",
       "      <td>9150473.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>130300</th>\n",
       "      <td>秦皇岛市</td>\n",
       "      <td>266.29001</td>\n",
       "      <td>2853937.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 292 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "               City  population         gdp  lag_gdp  city_七台河市  city_三亚市  \\\n",
       "Year CityCode                                                               \n",
       "2000 110000     北京市  1107.53000  24787600.0      NaN          0         0   \n",
       "     120000     天津市   912.00000  16393600.0      NaN          0         0   \n",
       "     130100    石家庄市   889.79999  10031119.0      NaN          0         0   \n",
       "     130200     唐山市   699.78998   9150473.0      NaN          0         0   \n",
       "     130300    秦皇岛市   266.29001   2853937.0      NaN          0         0   \n",
       "\n",
       "               city_三明市  city_三门峡市  city_上海市  city_上饶市  ...  city_鸡西市  \\\n",
       "Year CityCode                                           ...             \n",
       "2000 110000           0          0         0         0  ...         0   \n",
       "     120000           0          0         0         0  ...         0   \n",
       "     130100           0          0         0         0  ...         0   \n",
       "     130200           0          0         0         0  ...         0   \n",
       "     130300           0          0         0         0  ...         0   \n",
       "\n",
       "               city_鹤壁市  city_鹤岗市  city_鹰潭市  city_黄冈市  city_黄山市  city_黄石市  \\\n",
       "Year CityCode                                                               \n",
       "2000 110000           0         0         0         0         0         0   \n",
       "     120000           0         0         0         0         0         0   \n",
       "     130100           0         0         0         0         0         0   \n",
       "     130200           0         0         0         0         0         0   \n",
       "     130300           0         0         0         0         0         0   \n",
       "\n",
       "               city_黑河市  city_齐齐哈尔市  city_龙岩市  \n",
       "Year CityCode                                  \n",
       "2000 110000           0           0         0  \n",
       "     120000           0           0         0  \n",
       "     130100           0           0         0  \n",
       "     130200           0           0         0  \n",
       "     130300           0           0         0  \n",
       "\n",
       "[5 rows x 292 columns]"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "city_with_dummy = pd.concat([city_new, city_dummy], axis=1)\n",
    "city_with_dummy.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中axis=1代表按列合并。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## concat合并\n",
    "\n",
    "刚刚已经展示了pd.concat()的用法，注意我们使用axis=1来表明需要按列合并。实际上这个函数的工作方式与NumPy中的concatenate()很类似，只不过合并时会对行、列的索引进行对齐，因而只要有索引其实不必担心顺序的影响，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0\n",
       "Xavi      NaN    60.0"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players1 = [{\n",
    "    'name': 'Messi',\n",
    "    'goals': 35,\n",
    "    'passes': 55\n",
    "}, {\n",
    "    'name': 'Pique',\n",
    "    'goals': 3\n",
    "}, {\n",
    "    'name': 'Suarez',\n",
    "    'goals': 10,\n",
    "    'passes': 40\n",
    "}, {\n",
    "    'name': 'Xavi',\n",
    "    'passes': 60\n",
    "}]\n",
    "data1 = pd.DataFrame(players1)\n",
    "data1 = data1.set_index('name')\n",
    "data1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>attempts</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>70</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        attempts\n",
       "name            \n",
       "Messi         70\n",
       "Suarez        30\n",
       "Pique         10"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players2 = [{\n",
    "    'name': 'Messi',\n",
    "    'attempts': 70\n",
    "}, {\n",
    "    'name': 'Suarez',\n",
    "    'attempts': 30\n",
    "}, {\n",
    "    'name': 'Pique',\n",
    "    'attempts': 10\n",
    "}]\n",
    "data2 = pd.DataFrame(players2)\n",
    "data2 = data2.set_index('name')\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_1168/1663737842.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  data = pd.concat([data1, data2], axis=1)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>attempts</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>70.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>10.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes  attempts\n",
       "Messi    35.0    55.0      70.0\n",
       "Pique     3.0     NaN      10.0\n",
       "Suarez   10.0    40.0      30.0\n",
       "Xavi      NaN    60.0       NaN"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.concat([data1, data2], axis=1)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "除了按列合并之外，当然默认的是竖向按行合并。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        goals  passes\n",
       "name                 \n",
       "Messi    35.0    55.0\n",
       "Pique     3.0     NaN\n",
       "Suarez   10.0    40.0\n",
       "Xavi      NaN    60.0"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players1 = [{\n",
    "    'name': 'Messi',\n",
    "    'goals': 35,\n",
    "    'passes': 55\n",
    "}, {\n",
    "    'name': 'Pique',\n",
    "    'goals': 3\n",
    "}, {\n",
    "    'name': 'Suarez',\n",
    "    'goals': 10,\n",
    "    'passes': 40\n",
    "}, {\n",
    "    'name': 'Xavi',\n",
    "    'passes': 60\n",
    "}]\n",
    "data1 = pd.DataFrame(players1)\n",
    "data1 = data1.set_index('name')\n",
    "data1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>save</th>\n",
       "      <th>passes</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>ter Stegen</th>\n",
       "      <td>0.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iniesta</th>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>80.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            goals  save  passes\n",
       "name                           \n",
       "ter Stegen    0.0   5.0     NaN\n",
       "Iniesta       1.0   NaN    80.0\n",
       "Xavi          NaN   NaN    60.0"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "players2 = [{\n",
    "    'name': 'ter Stegen',\n",
    "    'goals': 0,\n",
    "    'save': 5\n",
    "}, {\n",
    "    'name': 'Iniesta',\n",
    "    'goals': 1,\n",
    "    'passes': 80\n",
    "}, {\n",
    "    'name': 'Xavi',\n",
    "    'passes': 60\n",
    "}]\n",
    "data2 = pd.DataFrame(players2)\n",
    "data2 = data2.set_index('name')\n",
    "data2.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_1168/3647958450.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  data = pd.concat([data1, data2])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>save</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ter Stegen</th>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iniesta</th>\n",
       "      <td>1.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            goals  passes  save\n",
       "name                           \n",
       "Messi        35.0    55.0   NaN\n",
       "Pique         3.0     NaN   NaN\n",
       "Suarez       10.0    40.0   NaN\n",
       "Xavi          NaN    60.0   NaN\n",
       "ter Stegen    0.0     NaN   5.0\n",
       "Iniesta       1.0    80.0   NaN\n",
       "Xavi          NaN    60.0   NaN"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.concat([data1, data2])\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到以上程序完成了纵向的合并，但是需要注意的是，里面有两个Xavi，作为索引这本应该是不允许的，但是Pandas并不会自动检查。\n",
    "\n",
    "为了解决这个问题，首先可以使用verigy_integrity选项强制检查是否有重复："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Indexes have overlapping values: Index(['Xavi'], dtype='object', name='name')\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_1168/632251793.py:2: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  data = pd.concat([data1, data2], verify_integrity=True)\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    data = pd.concat([data1, data2], verify_integrity=True)\n",
    "except Exception as e:\n",
    "    print(e)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此时会提示重复的index，需要动手去解决。\n",
    "\n",
    "或者，我们可以加入keys选项，将其合并为层级索引："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/tmp/ipykernel_1168/2516956762.py:1: FutureWarning: Sorting because non-concatenation axis is not aligned. A future version\n",
      "of pandas will change to not sort by default.\n",
      "\n",
      "To accept the future behavior, pass 'sort=False'.\n",
      "\n",
      "To retain the current behavior and silence the warning, pass 'sort=True'.\n",
      "\n",
      "  data = pd.concat([data1, data2], keys=['data1', 'data2'])\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>goals</th>\n",
       "      <th>passes</th>\n",
       "      <th>save</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>name</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th rowspan=\"4\" valign=\"top\">data1</th>\n",
       "      <th>Messi</th>\n",
       "      <td>35.0</td>\n",
       "      <td>55.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Pique</th>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Suarez</th>\n",
       "      <td>10.0</td>\n",
       "      <td>40.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th rowspan=\"3\" valign=\"top\">data2</th>\n",
       "      <th>ter Stegen</th>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>5.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Iniesta</th>\n",
       "      <td>1.0</td>\n",
       "      <td>80.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>Xavi</th>\n",
       "      <td>NaN</td>\n",
       "      <td>60.0</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                  goals  passes  save\n",
       "      name                           \n",
       "data1 Messi        35.0    55.0   NaN\n",
       "      Pique         3.0     NaN   NaN\n",
       "      Suarez       10.0    40.0   NaN\n",
       "      Xavi          NaN    60.0   NaN\n",
       "data2 ter Stegen    0.0     NaN   5.0\n",
       "      Iniesta       1.0    80.0   NaN\n",
       "      Xavi          NaN    60.0   NaN"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.concat([data1, data2], keys=['data1', 'data2'])\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到上面的数据被划分为了一个两级索引，就不存在重复的问题了。\n",
    "\n",
    "当然，选取何种方法处理需要根据应用背景来决定。\n",
    "\n",
    "## merge合并\n",
    "\n",
    "与Stata类似，Pandas中也有merge函数。\n",
    "\n",
    "与Stata一样，pd.merge()函数也有一对一（1:1）和多对一（m:1），最简单的是一对一合并，如果不做特殊处理，将会默认按照共同变量名进行merge，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>fo7</th>\n",
       "      <th>fp3</th>\n",
       "      <th>fp301</th>\n",
       "      <th>fp402</th>\n",
       "      <th>fp403</th>\n",
       "      <th>fp407</th>\n",
       "      <th>fp502</th>\n",
       "      <th>fp503</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>60000</td>\n",
       "      <td>2500</td>\n",
       "      <td>200</td>\n",
       "      <td>100.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>160</td>\n",
       "      <td>200.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100125</td>\n",
       "      <td>11</td>\n",
       "      <td>400000</td>\n",
       "      <td>2000</td>\n",
       "      <td>1500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>3900</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>...</td>\n",
       "      <td>4100.0</td>\n",
       "      <td>194140.0</td>\n",
       "      <td>26400.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>400000.0</td>\n",
       "      <td>400000.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>400000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100160</td>\n",
       "      <td>12</td>\n",
       "      <td>30000</td>\n",
       "      <td>1000</td>\n",
       "      <td>400</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>500</td>\n",
       "      <td>400.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>28100.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>30000.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100286</td>\n",
       "      <td>13</td>\n",
       "      <td>-8</td>\n",
       "      <td>600</td>\n",
       "      <td>500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>500</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>32700.0</td>\n",
       "      <td>14400.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>44000.0</td>\n",
       "      <td>44000.000000</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>100376</td>\n",
       "      <td>13</td>\n",
       "      <td>30000</td>\n",
       "      <td>500</td>\n",
       "      <td>500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>30.0</td>\n",
       "      <td>158330.0</td>\n",
       "      <td>1440.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "    fid14  provcd14     fo7   fp3  fp301  fp402  fp403  fp407   fp502  \\\n",
       "0  100051        11   60000  2500    200  100.0  100.0    160   200.0   \n",
       "1  100125        11  400000  2000   1500   50.0   70.0   3900  1000.0   \n",
       "2  100160        12   30000  1000    400    0.0    0.0    500   400.0   \n",
       "3  100286        13      -8   600    500    0.0    0.0    500   100.0   \n",
       "4  100376        13   30000   500    500    0.0    0.0      0     0.0   \n",
       "\n",
       "     fp503  ...   other       pce     trco  familysize  felse_1  fincome1  \\\n",
       "0   1000.0  ...   300.0   71100.0   7200.0           3        0   70000.0   \n",
       "1  50000.0  ...  4100.0  194140.0  26400.0           1        0  400000.0   \n",
       "2      0.0  ...  1000.0   28100.0   3000.0           1        0   30000.0   \n",
       "3      0.0  ...   300.0   32700.0  14400.0           1        0   44000.0   \n",
       "4      0.0  ...    30.0  158330.0   1440.0           1    40000       NaN   \n",
       "\n",
       "    fincome1_per  foperate_1  ftransfer_1   fwage_1  \n",
       "0   23333.330078         0.0          0.0   60000.0  \n",
       "1  400000.000000         0.0          0.0  400000.0  \n",
       "2   30000.000000         0.0          0.0   30000.0  \n",
       "3   44000.000000     24000.0          0.0   20000.0  \n",
       "4            NaN         NaN          0.0   60000.0  \n",
       "\n",
       "[5 rows x 38 columns]"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "family = pd.read_csv(\"csv/cfps_family_econ.csv\")\n",
    "family.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>qq403b</th>\n",
       "      <th>qq1001</th>\n",
       "      <th>qq1101</th>\n",
       "      <th>qq1102</th>\n",
       "      <th>pn401a</th>\n",
       "      <th>pw1r</th>\n",
       "      <th>pg01</th>\n",
       "      <th>pg02</th>\n",
       "      <th>pgc103a</th>\n",
       "      <th>pg501</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>30.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100453431</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>101129501</td>\n",
       "      <td>101129</td>\n",
       "      <td>13</td>\n",
       "      <td>1990</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>103671501</td>\n",
       "      <td>103671</td>\n",
       "      <td>21</td>\n",
       "      <td>1988</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 90 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         pid   fid14  provcd14  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501  100051        11         1969          -8          -8   \n",
       "1  100051502  100051        11         1966          -8          -8   \n",
       "2  100453431  100453        43         1981           0           0   \n",
       "3  101129501  101129        13         1990          -8          -8   \n",
       "4  103671501  103671        21         1988          -8          -8   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...  qq403b  qq1001  qq1101  qq1102  \\\n",
       "0            0   -8    3.0    1.0  ...    10.0     2.0     1.0     3.0   \n",
       "1            1   -8    3.0    1.0  ...    30.0     5.0     1.0     1.0   \n",
       "2            1   -8    1.0    4.0  ...     0.0     1.0     1.0     2.0   \n",
       "3            0   -8    1.0    3.0  ...     0.0     2.0     0.0    -8.0   \n",
       "4            0   -8    3.0    1.0  ...     0.0     7.0     0.0    -8.0   \n",
       "\n",
       "   pn401a  pw1r  pg01  pg02  pgc103a  pg501  \n",
       "0     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "1     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "2     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "3     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "4     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "\n",
       "[5 rows x 90 columns]"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "adult = pd.read_csv(\"csv/cfps_adult.csv\")\n",
    "adult.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "数据量： (36865, 126)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>110043107</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1994</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100453431</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>430111101</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1950</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 126 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         pid   fid14  provcd14  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501  100051        11         1969          -8          -8   \n",
       "1  100051502  100051        11         1966          -8          -8   \n",
       "2  110043107  100051        11         1994           1           0   \n",
       "3  100453431  100453        43         1981           0           0   \n",
       "4  430111101  100453        43         1950          -8          -8   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...  other      pce     trco  familysize  \\\n",
       "0            0   -8    3.0    1.0  ...  300.0  71100.0   7200.0           3   \n",
       "1            1   -8    3.0    1.0  ...  300.0  71100.0   7200.0           3   \n",
       "2            1   -8    3.0    2.0  ...  300.0  71100.0   7200.0           3   \n",
       "3            1   -8    1.0    4.0  ...  600.0      NaN  16800.0           5   \n",
       "4            1   -8    3.0    3.0  ...  600.0      NaN  16800.0           5   \n",
       "\n",
       "   felse_1  fincome1  fincome1_per  foperate_1  ftransfer_1  fwage_1  \n",
       "0        0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "1        0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "2        0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "3        0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "4        0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "\n",
       "[5 rows x 126 columns]"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged = pd.merge(adult, family)\n",
    "print(\"数据量：\", merged.shape)\n",
    "merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面两个数据集中有共同的列fid14和provcd14，所以只要fid14和provcd14相同就合并在一起了。但是注意，以上的合并中**只保留了能够匹配成功的**（对应于Stata中的_merge==3），即两个数据集中的**交集**。如果需要取并集，保留所有的观测，需要使用how='outer'选项："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "数据量： (37316, 126)\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>110043107.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1994.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100453431.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>430111101.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 126 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           pid   fid14  provcd14  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501.0  100051        11       1969.0        -8.0        -8.0   \n",
       "1  100051502.0  100051        11       1966.0        -8.0        -8.0   \n",
       "2  110043107.0  100051        11       1994.0         1.0         0.0   \n",
       "3  100453431.0  100453        43       1981.0         0.0         0.0   \n",
       "4  430111101.0  100453        43       1950.0        -8.0        -8.0   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...  other      pce     trco  familysize  \\\n",
       "0          0.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "1          1.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "2          1.0 -8.0    3.0    2.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "3          1.0 -8.0    1.0    4.0  ...  600.0      NaN  16800.0         5.0   \n",
       "4          1.0 -8.0    3.0    3.0  ...  600.0      NaN  16800.0         5.0   \n",
       "\n",
       "   felse_1  fincome1  fincome1_per  foperate_1  ftransfer_1  fwage_1  \n",
       "0      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "1      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "2      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "3      0.0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "4      0.0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "\n",
       "[5 rows x 126 columns]"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged = pd.merge(adult, family, how='outer')\n",
    "print(\"数据量：\", merged.shape)\n",
    "merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "how选项还可以用left（保留第一个数据框的观测）、right（保留第二个数据框的观测）。\n",
    "\n",
    "此外如果两个数据库中有重名的列，可以使用suffixed参数为重名的列名字加一个前缀。\n",
    "\n",
    "有的时候两个数据集可能有多个重名的列，我们可能需要手动使用on选项指定用来匹配的列，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 104,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14_x</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1969.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1966.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>110043107.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1994.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100453431.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1981.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>430111101.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 127 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           pid   fid14  provcd14_x  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501.0  100051        11.0       1969.0        -8.0        -8.0   \n",
       "1  100051502.0  100051        11.0       1966.0        -8.0        -8.0   \n",
       "2  110043107.0  100051        11.0       1994.0         1.0         0.0   \n",
       "3  100453431.0  100453        43.0       1981.0         0.0         0.0   \n",
       "4  430111101.0  100453        43.0       1950.0        -8.0        -8.0   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...  other      pce     trco  familysize  \\\n",
       "0          0.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "1          1.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "2          1.0 -8.0    3.0    2.0  ...  300.0  71100.0   7200.0         3.0   \n",
       "3          1.0 -8.0    1.0    4.0  ...  600.0      NaN  16800.0         5.0   \n",
       "4          1.0 -8.0    3.0    3.0  ...  600.0      NaN  16800.0         5.0   \n",
       "\n",
       "   felse_1  fincome1  fincome1_per  foperate_1  ftransfer_1  fwage_1  \n",
       "0      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "1      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "2      0.0   70000.0  23333.330078         0.0          0.0  60000.0  \n",
       "3      0.0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "4      0.0   84000.0  16800.000000         0.0      48000.0  36000.0  \n",
       "\n",
       "[5 rows x 127 columns]"
      ]
     },
     "execution_count": 104,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged = pd.merge(adult, family, on='fid14', how='outer')\n",
    "merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要按照多个列进行匹配，可以使用列名字的列表，比如 on=['city','year']。\n",
    "\n",
    "有时也会碰上用来匹配的变量在两个数据框中不同名的情况，此时可以使用left_on和right_on，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14_x</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "      <th>家庭代码</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501.0</td>\n",
       "      <td>100051.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1969.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>100051.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502.0</td>\n",
       "      <td>100051.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1966.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>100051.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>110043107.0</td>\n",
       "      <td>100051.0</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1994.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "      <td>100051.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>100453431.0</td>\n",
       "      <td>100453.0</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1981.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "      <td>100453.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>430111101.0</td>\n",
       "      <td>100453.0</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "      <td>100453.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 128 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "           pid     fid14  provcd14_x  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501.0  100051.0        11.0       1969.0        -8.0        -8.0   \n",
       "1  100051502.0  100051.0        11.0       1966.0        -8.0        -8.0   \n",
       "2  110043107.0  100051.0        11.0       1994.0         1.0         0.0   \n",
       "3  100453431.0  100453.0        43.0       1981.0         0.0         0.0   \n",
       "4  430111101.0  100453.0        43.0       1950.0        -8.0        -8.0   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...      pce     trco  familysize  felse_1  \\\n",
       "0          0.0 -8.0    3.0    1.0  ...  71100.0   7200.0         3.0      0.0   \n",
       "1          1.0 -8.0    3.0    1.0  ...  71100.0   7200.0         3.0      0.0   \n",
       "2          1.0 -8.0    3.0    2.0  ...  71100.0   7200.0         3.0      0.0   \n",
       "3          1.0 -8.0    1.0    4.0  ...      NaN  16800.0         5.0      0.0   \n",
       "4          1.0 -8.0    3.0    3.0  ...      NaN  16800.0         5.0      0.0   \n",
       "\n",
       "   fincome1  fincome1_per  foperate_1  ftransfer_1  fwage_1      家庭代码  \n",
       "0   70000.0  23333.330078         0.0          0.0  60000.0  100051.0  \n",
       "1   70000.0  23333.330078         0.0          0.0  60000.0  100051.0  \n",
       "2   70000.0  23333.330078         0.0          0.0  60000.0  100051.0  \n",
       "3   84000.0  16800.000000         0.0      48000.0  36000.0  100453.0  \n",
       "4   84000.0  16800.000000         0.0      48000.0  36000.0  100453.0  \n",
       "\n",
       "[5 rows x 128 columns]"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "family['家庭代码'] = family['fid14']\n",
    "del family['fid14']\n",
    "merged = pd.merge(adult, family, left_on='fid14', right_on='家庭代码', how='outer')\n",
    "merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果需要按照索引进行合并，那么需要设定left_index和right_index，如果第一个数据库要按照索引合并，那么就设定left_index=True，如果第二个数据框需要用索引合并，就设定right_index=True。\n",
    "\n",
    "left_on, right_on, left_index, right_index可以组合使用，非常方便，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 106,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>provcd14</th>\n",
       "      <th>fo7</th>\n",
       "      <th>fp3</th>\n",
       "      <th>fp301</th>\n",
       "      <th>fp402</th>\n",
       "      <th>fp403</th>\n",
       "      <th>fp407</th>\n",
       "      <th>fp502</th>\n",
       "      <th>fp503</th>\n",
       "      <th>fu201</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>家庭代码</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100051</th>\n",
       "      <td>11</td>\n",
       "      <td>60000</td>\n",
       "      <td>2500</td>\n",
       "      <td>200</td>\n",
       "      <td>100.0</td>\n",
       "      <td>100.0</td>\n",
       "      <td>160</td>\n",
       "      <td>200.0</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>1000</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100125</th>\n",
       "      <td>11</td>\n",
       "      <td>400000</td>\n",
       "      <td>2000</td>\n",
       "      <td>1500</td>\n",
       "      <td>50.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>3900</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>50000.0</td>\n",
       "      <td>40000</td>\n",
       "      <td>...</td>\n",
       "      <td>4100.0</td>\n",
       "      <td>194140.0</td>\n",
       "      <td>26400.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>400000.0</td>\n",
       "      <td>400000.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>400000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100160</th>\n",
       "      <td>12</td>\n",
       "      <td>30000</td>\n",
       "      <td>1000</td>\n",
       "      <td>400</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>500</td>\n",
       "      <td>400.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>1000.0</td>\n",
       "      <td>28100.0</td>\n",
       "      <td>3000.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>30000.0</td>\n",
       "      <td>30000.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>30000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100286</th>\n",
       "      <td>13</td>\n",
       "      <td>-8</td>\n",
       "      <td>600</td>\n",
       "      <td>500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>500</td>\n",
       "      <td>100.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>32700.0</td>\n",
       "      <td>14400.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>44000.0</td>\n",
       "      <td>44000.000000</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>20000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100376</th>\n",
       "      <td>13</td>\n",
       "      <td>30000</td>\n",
       "      <td>500</td>\n",
       "      <td>500</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1000</td>\n",
       "      <td>...</td>\n",
       "      <td>30.0</td>\n",
       "      <td>158330.0</td>\n",
       "      <td>1440.0</td>\n",
       "      <td>1</td>\n",
       "      <td>40000</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 37 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        provcd14     fo7   fp3  fp301  fp402  fp403  fp407   fp502    fp503  \\\n",
       "家庭代码                                                                          \n",
       "100051        11   60000  2500    200  100.0  100.0    160   200.0   1000.0   \n",
       "100125        11  400000  2000   1500   50.0   70.0   3900  1000.0  50000.0   \n",
       "100160        12   30000  1000    400    0.0    0.0    500   400.0      0.0   \n",
       "100286        13      -8   600    500    0.0    0.0    500   100.0      0.0   \n",
       "100376        13   30000   500    500    0.0    0.0      0     0.0      0.0   \n",
       "\n",
       "        fu201  ...   other       pce     trco  familysize  felse_1  fincome1  \\\n",
       "家庭代码           ...                                                             \n",
       "100051   1000  ...   300.0   71100.0   7200.0           3        0   70000.0   \n",
       "100125  40000  ...  4100.0  194140.0  26400.0           1        0  400000.0   \n",
       "100160      0  ...  1000.0   28100.0   3000.0           1        0   30000.0   \n",
       "100286     -8  ...   300.0   32700.0  14400.0           1        0   44000.0   \n",
       "100376   1000  ...    30.0  158330.0   1440.0           1    40000       NaN   \n",
       "\n",
       "         fincome1_per  foperate_1  ftransfer_1   fwage_1  \n",
       "家庭代码                                                      \n",
       "100051   23333.330078         0.0          0.0   60000.0  \n",
       "100125  400000.000000         0.0          0.0  400000.0  \n",
       "100160   30000.000000         0.0          0.0   30000.0  \n",
       "100286   44000.000000     24000.0          0.0   20000.0  \n",
       "100376            NaN         NaN          0.0   60000.0  \n",
       "\n",
       "[5 rows x 37 columns]"
      ]
     },
     "execution_count": 106,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "family1 = family.set_index('家庭代码')\n",
    "family1.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14_x</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>other</th>\n",
       "      <th>pce</th>\n",
       "      <th>trco</th>\n",
       "      <th>familysize</th>\n",
       "      <th>felse_1</th>\n",
       "      <th>fincome1</th>\n",
       "      <th>fincome1_per</th>\n",
       "      <th>foperate_1</th>\n",
       "      <th>ftransfer_1</th>\n",
       "      <th>fwage_1</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0.0</th>\n",
       "      <td>100051501.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1969.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1.0</th>\n",
       "      <td>100051502.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1966.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>85.0</th>\n",
       "      <td>110043107.0</td>\n",
       "      <td>100051</td>\n",
       "      <td>11.0</td>\n",
       "      <td>1994.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>300.0</td>\n",
       "      <td>71100.0</td>\n",
       "      <td>7200.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70000.0</td>\n",
       "      <td>23333.330078</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>60000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2.0</th>\n",
       "      <td>100453431.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1981.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21798.0</th>\n",
       "      <td>430111101.0</td>\n",
       "      <td>100453</td>\n",
       "      <td>43.0</td>\n",
       "      <td>1950.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>600.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>16800.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>84000.0</td>\n",
       "      <td>16800.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>36000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 127 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                 pid   fid14  provcd14_x  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0.0      100051501.0  100051        11.0       1969.0        -8.0        -8.0   \n",
       "1.0      100051502.0  100051        11.0       1966.0        -8.0        -8.0   \n",
       "85.0     110043107.0  100051        11.0       1994.0         1.0         0.0   \n",
       "2.0      100453431.0  100453        43.0       1981.0         0.0         0.0   \n",
       "21798.0  430111101.0  100453        43.0       1950.0        -8.0        -8.0   \n",
       "\n",
       "         cfps_gender  te4  qa301  qa302  ...  other      pce     trco  \\\n",
       "0.0              0.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0   \n",
       "1.0              1.0 -8.0    3.0    1.0  ...  300.0  71100.0   7200.0   \n",
       "85.0             1.0 -8.0    3.0    2.0  ...  300.0  71100.0   7200.0   \n",
       "2.0              1.0 -8.0    1.0    4.0  ...  600.0      NaN  16800.0   \n",
       "21798.0          1.0 -8.0    3.0    3.0  ...  600.0      NaN  16800.0   \n",
       "\n",
       "         familysize  felse_1  fincome1  fincome1_per  foperate_1  ftransfer_1  \\\n",
       "0.0             3.0      0.0   70000.0  23333.330078         0.0          0.0   \n",
       "1.0             3.0      0.0   70000.0  23333.330078         0.0          0.0   \n",
       "85.0            3.0      0.0   70000.0  23333.330078         0.0          0.0   \n",
       "2.0             5.0      0.0   84000.0  16800.000000         0.0      48000.0   \n",
       "21798.0         5.0      0.0   84000.0  16800.000000         0.0      48000.0   \n",
       "\n",
       "         fwage_1  \n",
       "0.0      60000.0  \n",
       "1.0      60000.0  \n",
       "85.0     60000.0  \n",
       "2.0      36000.0  \n",
       "21798.0  36000.0  \n",
       "\n",
       "[5 rows x 127 columns]"
      ]
     },
     "execution_count": 107,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged = pd.merge(adult,\n",
    "                  family1,\n",
    "                  left_on='fid14',\n",
    "                  right_index=True,\n",
    "                  how='outer')\n",
    "merged.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 描述性统计\n",
    "\n",
    "在Pandas中对数据进行描述性统计也是非常方便的。\n",
    "\n",
    "首先，由于Pandas的每一列其实都可以看做NumPy的数组，因而NumPy的所有描述性统计功能都可以使用，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>cfps_minzu</th>\n",
       "      <th>cfps_party</th>\n",
       "      <th>cfps_gender</th>\n",
       "      <th>te4</th>\n",
       "      <th>qa301</th>\n",
       "      <th>qa302</th>\n",
       "      <th>...</th>\n",
       "      <th>qq403b</th>\n",
       "      <th>qq1001</th>\n",
       "      <th>qq1101</th>\n",
       "      <th>qq1102</th>\n",
       "      <th>pn401a</th>\n",
       "      <th>pw1r</th>\n",
       "      <th>pg01</th>\n",
       "      <th>pg02</th>\n",
       "      <th>pgc103a</th>\n",
       "      <th>pg501</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>10.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>30.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100453431</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>-8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>101129501</td>\n",
       "      <td>101129</td>\n",
       "      <td>13</td>\n",
       "      <td>1990</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>103671501</td>\n",
       "      <td>103671</td>\n",
       "      <td>21</td>\n",
       "      <td>1988</td>\n",
       "      <td>-8</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "      <td>-8</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 90 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         pid   fid14  provcd14  cfps_birthy  cfps_minzu  cfps_party  \\\n",
       "0  100051501  100051        11         1969          -8          -8   \n",
       "1  100051502  100051        11         1966          -8          -8   \n",
       "2  100453431  100453        43         1981           0           0   \n",
       "3  101129501  101129        13         1990          -8          -8   \n",
       "4  103671501  103671        21         1988          -8          -8   \n",
       "\n",
       "   cfps_gender  te4  qa301  qa302  ...  qq403b  qq1001  qq1101  qq1102  \\\n",
       "0            0   -8    3.0    1.0  ...    10.0     2.0     1.0     3.0   \n",
       "1            1   -8    3.0    1.0  ...    30.0     5.0     1.0     1.0   \n",
       "2            1   -8    1.0    4.0  ...     0.0     1.0     1.0     2.0   \n",
       "3            0   -8    1.0    3.0  ...     0.0     2.0     0.0    -8.0   \n",
       "4            0   -8    3.0    1.0  ...     0.0     7.0     0.0    -8.0   \n",
       "\n",
       "   pn401a  pw1r  pg01  pg02  pgc103a  pg501  \n",
       "0     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "1     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "2     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "3     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "4     NaN   NaN   NaN   NaN      NaN    NaN  \n",
       "\n",
       "[5 rows x 90 columns]"
      ]
     },
     "execution_count": 108,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv('csv/cfps_adult.csv')\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "个人收入人数= 37086\n",
      "个人平均收入= 8934.145380521793\n",
      "个人收入中位数= 55.0\n",
      "个人收入标准差= 18818.378009097283\n",
      "个人收入最大值= 442000.0\n",
      "个人收入75%分位数= 10000.0\n"
     ]
    }
   ],
   "source": [
    "print(\"个人收入人数=\", np.sum(data['p_income'].notnull()))\n",
    "print(\"个人平均收入=\", np.nanmean(data['p_income']))\n",
    "print(\"个人收入中位数=\", np.nanmedian(data['p_income']))\n",
    "print(\"个人收入标准差=\", np.nanstd(data['p_income']))\n",
    "print(\"个人收入最大值=\", np.nanmax(data['p_income']))\n",
    "print(\"个人收入75%分位数=\", np.nanpercentile(data['p_income'], 75))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者也可以直接使用Pandas序列的描述性统计功能，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 110,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "个人收入人数= 37086\n",
      "个人平均收入= 8934.145380521793\n",
      "个人收入中位数= 55.0\n",
      "个人收入标准差= 18818.63172692324\n",
      "个人收入最大值= 442000.0\n",
      "个人收入75%分位数= 10000.0\n"
     ]
    }
   ],
   "source": [
    "print(\"个人收入人数=\", data['p_income'].count())\n",
    "print(\"个人平均收入=\", data['p_income'].mean())\n",
    "print(\"个人收入中位数=\", data['p_income'].median())\n",
    "print(\"个人收入标准差=\", data['p_income'].std())\n",
    "print(\"个人收入最大值=\", data['p_income'].max())\n",
    "print(\"个人收入75%分位数=\", data['p_income'].quantile(0.75))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "一个小的区别是使用Pandas计算分为数是使用quantile()方法，而非percentile()。使用Pandas自带的函数更加方便，比如可以不用担心所有的NaN的问题。当然，同样可以使用axis选项控制按行还是按列计算，在这里不再举例。\n",
    "\n",
    "如果需要像Stata那样计算描述性统计表，可以使用describe()方法："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>p_income</th>\n",
       "      <th>qq1101</th>\n",
       "      <th>qq1102</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>37086.000000</td>\n",
       "      <td>33476.000000</td>\n",
       "      <td>33476.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>8934.145381</td>\n",
       "      <td>0.233212</td>\n",
       "      <td>-4.405992</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>18818.631727</td>\n",
       "      <td>0.445316</td>\n",
       "      <td>11.649321</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-9.000000</td>\n",
       "      <td>-8.000000</td>\n",
       "      <td>-8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>55.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>10000.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>-8.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>442000.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>500.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            p_income        qq1101        qq1102\n",
       "count   37086.000000  33476.000000  33476.000000\n",
       "mean     8934.145381      0.233212     -4.405992\n",
       "std     18818.631727      0.445316     11.649321\n",
       "min        -9.000000     -8.000000     -8.000000\n",
       "25%         0.000000      0.000000     -8.000000\n",
       "50%        55.000000      0.000000     -8.000000\n",
       "75%     10000.000000      0.000000     -8.000000\n",
       "max    442000.000000      1.000000    500.000000"
      ]
     },
     "execution_count": 111,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata = data.loc[:, ['p_income', 'qq1101', 'qq1102']]\n",
    "subdata.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "值得注意的是，以上describe()函数的输出结果也是一个DataFrame："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "           p_income        qq1101        qq1102\n",
      "count  37086.000000  33476.000000  33476.000000\n",
      "mean    8934.145381      0.233212     -4.405992\n",
      "std    18818.631727      0.445316     11.649321\n"
     ]
    }
   ],
   "source": [
    "summary = subdata.describe()\n",
    "print(summary.loc[['count', 'mean', 'std'], :])\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 数据加总与转换\n",
    "\n",
    "以上虽然实现了描述性统计，然而很多时候我们可能需要完成更多的数据加总（aggregate）等任务，类比于Stata的egen、collapse等操作。为此我们可以使用多种函数组合使用，具有更灵活的处理能力。\n",
    "\n",
    "aggregate函数用于加总、累积一个数据。其实我们前面已经使用了Dataframe的sum()、median()等函数进行简单的累计，使用aggregate函数还可以进行更复杂的加总操作，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "      <th>cfps_gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>100051501</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969</td>\n",
       "      <td>22000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>100051502</td>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966</td>\n",
       "      <td>55000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>100453431</td>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981</td>\n",
       "      <td>26990.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>101129501</td>\n",
       "      <td>101129</td>\n",
       "      <td>13</td>\n",
       "      <td>1990</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>103671501</td>\n",
       "      <td>103671</td>\n",
       "      <td>21</td>\n",
       "      <td>1988</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         pid   fid14  provcd14  cfps_birthy  p_income  te4  cfps_gender\n",
       "0  100051501  100051        11         1969   22000.0   -8            0\n",
       "1  100051502  100051        11         1966   55000.0   -8            1\n",
       "2  100453431  100453        43         1981   26990.0   -8            1\n",
       "3  101129501  101129        13         1990       0.0   -8            0\n",
       "4  103671501  103671        21         1988   24000.0   -8            0"
      ]
     },
     "execution_count": 113,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = pd.read_csv('csv/cfps_adult.csv')\n",
    "subdata = data[[\n",
    "    'pid', 'fid14', 'provcd14', 'cfps_birthy', 'p_income', 'te4', 'cfps_gender'\n",
    "]]\n",
    "subdata.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 114,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pid            2.489152e+09\n",
       "fid14          2.781941e+06\n",
       "provcd14       6.500000e+01\n",
       "cfps_birthy    1.998000e+03\n",
       "p_income       4.420000e+05\n",
       "te4            7.000000e+00\n",
       "cfps_gender    1.000000e+00\n",
       "dtype: float64"
      ]
     },
     "execution_count": 114,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata.aggregate(np.max)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上我们传入了一个**函数**np.max，并将这个函数作用于以上数据集中的每一列，并计算加总的结果。这里传入的函数可以是任何接受Series作为输入并输出标量的函数。再比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 115,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "pid            234385600.0\n",
       "fid14             279897.0\n",
       "provcd14              20.0\n",
       "cfps_birthy           29.0\n",
       "p_income               NaN\n",
       "te4                    0.0\n",
       "cfps_gender            1.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 115,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata.aggregate(lambda x: np.percentile(x, 75) - np.percentile(x, 25))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上我们传入了一个使用lambda表达式定义的函数（因为np.percentile需要两个输入，而aggregate所需要的函数值能有一个输入），并计算了四分位差。\n",
    "\n",
    "此外，我们还可以使用transform函数对数据进行变换。aggregate()函数所需要的输入是给一个序列值计算出一个标量（加总），而transform()函数所需要的输入是给一个序列计算出一个对应的新的序列。比如，如果我们需要进行标准化，即对每一个序列的每一个元素都减去这个序列的均值除以标准差，输出为标准化之后的序列，那么可以使用transform()函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 116,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>p_income</th>\n",
       "      <th>cfps_birthy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.694304</td>\n",
       "      <td>0.014151</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2.447885</td>\n",
       "      <td>-0.157956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.959467</td>\n",
       "      <td>0.702579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.474750</td>\n",
       "      <td>1.218900</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.800582</td>\n",
       "      <td>1.104162</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37142</th>\n",
       "      <td>1.650803</td>\n",
       "      <td>0.989424</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37143</th>\n",
       "      <td>-0.474750</td>\n",
       "      <td>1.046793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37144</th>\n",
       "      <td>2.075914</td>\n",
       "      <td>0.759948</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37145</th>\n",
       "      <td>-0.474750</td>\n",
       "      <td>0.186258</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37146</th>\n",
       "      <td>-0.474750</td>\n",
       "      <td>0.243627</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>37147 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       p_income  cfps_birthy\n",
       "0      0.694304     0.014151\n",
       "1      2.447885    -0.157956\n",
       "2      0.959467     0.702579\n",
       "3     -0.474750     1.218900\n",
       "4      0.800582     1.104162\n",
       "...         ...          ...\n",
       "37142  1.650803     0.989424\n",
       "37143 -0.474750     1.046793\n",
       "37144  2.075914     0.759948\n",
       "37145 -0.474750     0.186258\n",
       "37146 -0.474750     0.243627\n",
       "\n",
       "[37147 rows x 2 columns]"
      ]
     },
     "execution_count": 116,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "std_sub_data = subdata[['p_income', 'cfps_birthy'\n",
    "                        ]].transform(lambda x: (x - x.mean()) / x.std())\n",
    "std_sub_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 117,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>p_income</th>\n",
       "      <th>cfps_birthy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>3.708600e+04</td>\n",
       "      <td>3.714700e+04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>4.598238e-17</td>\n",
       "      <td>-2.206590e-15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.000000e+00</td>\n",
       "      <td>1.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>-4.752282e-01</td>\n",
       "      <td>-3.370619e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>-4.747500e-01</td>\n",
       "      <td>-7.890146e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>-4.718274e-01</td>\n",
       "      <td>1.415113e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>5.663826e-02</td>\n",
       "      <td>8.746858e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>2.301261e+01</td>\n",
       "      <td>1.677852e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           p_income   cfps_birthy\n",
       "count  3.708600e+04  3.714700e+04\n",
       "mean   4.598238e-17 -2.206590e-15\n",
       "std    1.000000e+00  1.000000e+00\n",
       "min   -4.752282e-01 -3.370619e+00\n",
       "25%   -4.747500e-01 -7.890146e-01\n",
       "50%   -4.718274e-01  1.415113e-02\n",
       "75%    5.663826e-02  8.746858e-01\n",
       "max    2.301261e+01  1.677852e+00"
      ]
     },
     "execution_count": 117,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "std_sub_data.describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上即实现了简单的标准化：通过传入一个**函数**对数据的每一个元素进行相同的操作。\n",
    "\n",
    "transform函数只能对每一个元素做相同的操作，如果需要更复杂的函数操作，可以使用apply函数，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 118,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>p_income</th>\n",
       "      <th>cfps_birthy</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>11.173184</td>\n",
       "      <td>1969.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>27.975585</td>\n",
       "      <td>1966.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>13.624432</td>\n",
       "      <td>1981.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1990.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>12.072435</td>\n",
       "      <td>1988.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37142</th>\n",
       "      <td>20.140987</td>\n",
       "      <td>1986.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37143</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1987.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37144</th>\n",
       "      <td>24.217962</td>\n",
       "      <td>1982.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37145</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1972.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37146</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1973.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>37147 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        p_income  cfps_birthy\n",
       "0      11.173184       1969.0\n",
       "1      27.975585       1966.0\n",
       "2      13.624432       1981.0\n",
       "3       0.000000       1990.0\n",
       "4      12.072435       1988.0\n",
       "...          ...          ...\n",
       "37142  20.140987       1986.0\n",
       "37143   0.000000       1987.0\n",
       "37144  24.217962       1982.0\n",
       "37145   0.000000       1972.0\n",
       "37146   0.000000       1973.0\n",
       "\n",
       "[37147 rows x 2 columns]"
      ]
     },
     "execution_count": 118,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "def income_by_age(x):\n",
    "    x['p_income'] = x['p_income'] / (x['cfps_birthy'].mean())\n",
    "    return x\n",
    "\n",
    "\n",
    "subdata[['p_income', 'cfps_birthy']].apply(income_by_age, axis=1)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 分组操作\n",
    "\n",
    "以上我们介绍了一些数据加总的方法，很多时候我们也会进行分组加总（类似于Stata中的egen、collapse），而Pandas中的groupby函数可以完成分组的任务。\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<pandas.core.groupby.generic.DataFrameGroupBy object at 0x7f3312a630a0>"
      ]
     },
     "execution_count": 119,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "group_data = subdata.groupby('cfps_gender')\n",
    "group_data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现，经过groupby之后，数据变成了一个groupby对象，该对象是可以被迭代的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cfps_gender= 0 \n",
      "          pid   fid14  provcd14  cfps_birthy  p_income  te4  cfps_gender\n",
      "0  100051501  100051        11         1969   22000.0   -8            0 \n",
      "---------\n",
      "cfps_gender= 1 \n",
      "          pid   fid14  provcd14  cfps_birthy  p_income  te4  cfps_gender\n",
      "1  100051502  100051        11         1966   55000.0   -8            1 \n",
      "---------\n"
     ]
    }
   ],
   "source": [
    "for d in group_data:\n",
    "    print(\"cfps_gender=\", d[0], \"\\n\", d[1].head(1), '\\n---------')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来我们可以直接对这个可迭代的groupby对象使用任何数据框可以使用的函数，比如我们前面讲过的sum()、median()、describe()、aggregate()、transfrom()、apply()等等。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cfps_gender</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3.865905e+08</td>\n",
       "      <td>383395.996339</td>\n",
       "      <td>38.491493</td>\n",
       "      <td>1968.468878</td>\n",
       "      <td>5678.255069</td>\n",
       "      <td>-7.073821</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3.900587e+08</td>\n",
       "      <td>385529.590579</td>\n",
       "      <td>38.819650</td>\n",
       "      <td>1969.037739</td>\n",
       "      <td>12189.684539</td>\n",
       "      <td>-6.939004</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      pid          fid14   provcd14  cfps_birthy  \\\n",
       "cfps_gender                                                        \n",
       "0            3.865905e+08  383395.996339  38.491493  1968.468878   \n",
       "1            3.900587e+08  385529.590579  38.819650  1969.037739   \n",
       "\n",
       "                 p_income       te4  \n",
       "cfps_gender                          \n",
       "0             5678.255069 -7.073821  \n",
       "1            12189.684539 -6.939004  "
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata.groupby('cfps_gender').mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "就计算了均值，而："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"8\" halign=\"left\">p_income</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>cfps_gender</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>18542.0</td>\n",
       "      <td>5678.255069</td>\n",
       "      <td>14157.507271</td>\n",
       "      <td>-9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2400.0</td>\n",
       "      <td>408400.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>18544.0</td>\n",
       "      <td>12189.684539</td>\n",
       "      <td>22060.143953</td>\n",
       "      <td>-9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>70.0</td>\n",
       "      <td>20000.0</td>\n",
       "      <td>442000.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            p_income                                                       \\\n",
       "               count          mean           std  min  25%   50%      75%   \n",
       "cfps_gender                                                                 \n",
       "0            18542.0   5678.255069  14157.507271 -9.0  0.0   0.0   2400.0   \n",
       "1            18544.0  12189.684539  22060.143953 -9.0  0.0  70.0  20000.0   \n",
       "\n",
       "                       \n",
       "                  max  \n",
       "cfps_gender            \n",
       "0            408400.0  \n",
       "1            442000.0  "
      ]
     },
     "execution_count": 122,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata[['p_income', 'cfps_gender']].groupby('cfps_gender').describe()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "则分组进行了描述性统计。\n",
    "\n",
    "进一步，如果需要进行分组标准化（减去组均值、除以组标准差），同样可以使用transform()函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "      <th>cfps_gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>-0.458419</td>\n",
       "      <td>-0.468260</td>\n",
       "      <td>0.075089</td>\n",
       "      <td>-0.002684</td>\n",
       "      <td>-0.240189</td>\n",
       "      <td>-0.984493</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-0.458419</td>\n",
       "      <td>-0.468260</td>\n",
       "      <td>-0.088741</td>\n",
       "      <td>0.798360</td>\n",
       "      <td>-0.240189</td>\n",
       "      <td>1.012225</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>-9.564826</td>\n",
       "      <td>-4.334972</td>\n",
       "      <td>0.815022</td>\n",
       "      <td>0.989367</td>\n",
       "      <td>-0.391710</td>\n",
       "      <td>1.002377</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-0.538837</td>\n",
       "      <td>-0.367395</td>\n",
       "      <td>1.220641</td>\n",
       "      <td>-0.454098</td>\n",
       "      <td>-0.258791</td>\n",
       "      <td>-0.991016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>-2.346715</td>\n",
       "      <td>-1.264389</td>\n",
       "      <td>1.279710</td>\n",
       "      <td>0.904451</td>\n",
       "      <td>-0.217963</td>\n",
       "      <td>-0.985369</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37142</th>\n",
       "      <td>22.221765</td>\n",
       "      <td>16.192015</td>\n",
       "      <td>0.916968</td>\n",
       "      <td>2.579299</td>\n",
       "      <td>-0.279450</td>\n",
       "      <td>1.013021</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37143</th>\n",
       "      <td>18.206071</td>\n",
       "      <td>10.233358</td>\n",
       "      <td>1.031706</td>\n",
       "      <td>-0.646650</td>\n",
       "      <td>-0.216135</td>\n",
       "      <td>-0.976092</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37144</th>\n",
       "      <td>33.830734</td>\n",
       "      <td>13.647222</td>\n",
       "      <td>0.630327</td>\n",
       "      <td>1.799230</td>\n",
       "      <td>-0.374700</td>\n",
       "      <td>0.985271</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37145</th>\n",
       "      <td>2.625339</td>\n",
       "      <td>2.046261</td>\n",
       "      <td>-0.048449</td>\n",
       "      <td>-0.335092</td>\n",
       "      <td>-0.443733</td>\n",
       "      <td>0.716498</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>37146</th>\n",
       "      <td>2.625339</td>\n",
       "      <td>2.046261</td>\n",
       "      <td>0.026427</td>\n",
       "      <td>-0.335092</td>\n",
       "      <td>-0.443733</td>\n",
       "      <td>-1.313579</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>37147 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "             pid      fid14  cfps_birthy  p_income       te4  cfps_gender\n",
       "0      -0.458419  -0.468260     0.075089 -0.002684 -0.240189    -0.984493\n",
       "1      -0.458419  -0.468260    -0.088741  0.798360 -0.240189     1.012225\n",
       "2      -9.564826  -4.334972     0.815022  0.989367 -0.391710     1.002377\n",
       "3      -0.538837  -0.367395     1.220641 -0.454098 -0.258791    -0.991016\n",
       "4      -2.346715  -1.264389     1.279710  0.904451 -0.217963    -0.985369\n",
       "...          ...        ...          ...       ...       ...          ...\n",
       "37142  22.221765  16.192015     0.916968  2.579299 -0.279450     1.013021\n",
       "37143  18.206071  10.233358     1.031706 -0.646650 -0.216135    -0.976092\n",
       "37144  33.830734  13.647222     0.630327  1.799230 -0.374700     0.985271\n",
       "37145   2.625339   2.046261    -0.048449 -0.335092 -0.443733     0.716498\n",
       "37146   2.625339   2.046261     0.026427 -0.335092 -0.443733    -1.313579\n",
       "\n",
       "[37147 rows x 6 columns]"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata.groupby('provcd14').transform(lambda x: (x - x.mean()) / x.std())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，我们不仅可以使用列名作为分组的依据，还可以使用任何的列表、序列等作为分组依据，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0        1\n",
       "1        1\n",
       "2        4\n",
       "3        1\n",
       "4        2\n",
       "        ..\n",
       "37142    1\n",
       "37143    3\n",
       "37144    4\n",
       "37145    6\n",
       "37146    6\n",
       "Name: provcd14, Length: 37147, dtype: int64"
      ]
     },
     "execution_count": 124,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "area = subdata['provcd14'] // 10\n",
    "area"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pid</th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "      <th>cfps_gender</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>provcd14</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.422558e+08</td>\n",
       "      <td>152304.716274</td>\n",
       "      <td>13.206604</td>\n",
       "      <td>1969.836792</td>\n",
       "      <td>8139.107223</td>\n",
       "      <td>-7.272406</td>\n",
       "      <td>0.494575</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2.203936e+08</td>\n",
       "      <td>231271.094290</td>\n",
       "      <td>21.577402</td>\n",
       "      <td>1966.407318</td>\n",
       "      <td>8557.943385</td>\n",
       "      <td>-7.477282</td>\n",
       "      <td>0.491757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3.396687e+08</td>\n",
       "      <td>334396.574044</td>\n",
       "      <td>33.690670</td>\n",
       "      <td>1966.688907</td>\n",
       "      <td>13047.479389</td>\n",
       "      <td>-7.281394</td>\n",
       "      <td>0.495660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4.252296e+08</td>\n",
       "      <td>415540.418640</td>\n",
       "      <td>42.559301</td>\n",
       "      <td>1969.584989</td>\n",
       "      <td>9099.210584</td>\n",
       "      <td>-6.805672</td>\n",
       "      <td>0.503247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5.153870e+08</td>\n",
       "      <td>506601.308499</td>\n",
       "      <td>51.712023</td>\n",
       "      <td>1969.130939</td>\n",
       "      <td>5670.474407</td>\n",
       "      <td>-6.682802</td>\n",
       "      <td>0.504127</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>6.169692e+08</td>\n",
       "      <td>606183.958244</td>\n",
       "      <td>61.862241</td>\n",
       "      <td>1970.876160</td>\n",
       "      <td>6758.139265</td>\n",
       "      <td>-6.659350</td>\n",
       "      <td>0.508030</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                   pid          fid14   provcd14  cfps_birthy      p_income  \\\n",
       "provcd14                                                                      \n",
       "1         1.422558e+08  152304.716274  13.206604  1969.836792   8139.107223   \n",
       "2         2.203936e+08  231271.094290  21.577402  1966.407318   8557.943385   \n",
       "3         3.396687e+08  334396.574044  33.690670  1966.688907  13047.479389   \n",
       "4         4.252296e+08  415540.418640  42.559301  1969.584989   9099.210584   \n",
       "5         5.153870e+08  506601.308499  51.712023  1969.130939   5670.474407   \n",
       "6         6.169692e+08  606183.958244  61.862241  1970.876160   6758.139265   \n",
       "\n",
       "               te4  cfps_gender  \n",
       "provcd14                         \n",
       "1        -7.272406     0.494575  \n",
       "2        -7.477282     0.491757  \n",
       "3        -7.281394     0.495660  \n",
       "4        -6.805672     0.503247  \n",
       "5        -6.682802     0.504127  \n",
       "6        -6.659350     0.508030  "
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "subdata.groupby(area).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者，我们甚至可以使用一个函数作为分组依据，该函数通过处理**索引**，并根据返回值进行分组："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "      <th>cfps_gender</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>pid</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>100051501</th>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1969</td>\n",
       "      <td>22000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100051502</th>\n",
       "      <td>100051</td>\n",
       "      <td>11</td>\n",
       "      <td>1966</td>\n",
       "      <td>55000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>100453431</th>\n",
       "      <td>100453</td>\n",
       "      <td>43</td>\n",
       "      <td>1981</td>\n",
       "      <td>26990.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>101129501</th>\n",
       "      <td>101129</td>\n",
       "      <td>13</td>\n",
       "      <td>1990</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>103671501</th>\n",
       "      <td>103671</td>\n",
       "      <td>21</td>\n",
       "      <td>1988</td>\n",
       "      <td>24000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>997150431</th>\n",
       "      <td>997150</td>\n",
       "      <td>14</td>\n",
       "      <td>1986</td>\n",
       "      <td>40000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1554992501</th>\n",
       "      <td>1554992</td>\n",
       "      <td>33</td>\n",
       "      <td>1987</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1715282501</th>\n",
       "      <td>1715282</td>\n",
       "      <td>44</td>\n",
       "      <td>1982</td>\n",
       "      <td>48000.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2489151501</th>\n",
       "      <td>2489151</td>\n",
       "      <td>65</td>\n",
       "      <td>1972</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2489151502</th>\n",
       "      <td>2489151</td>\n",
       "      <td>65</td>\n",
       "      <td>1973</td>\n",
       "      <td>0.0</td>\n",
       "      <td>-8</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>37147 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              fid14  provcd14  cfps_birthy  p_income  te4  cfps_gender\n",
       "pid                                                                   \n",
       "100051501    100051        11         1969   22000.0   -8            0\n",
       "100051502    100051        11         1966   55000.0   -8            1\n",
       "100453431    100453        43         1981   26990.0   -8            1\n",
       "101129501    101129        13         1990       0.0   -8            0\n",
       "103671501    103671        21         1988   24000.0   -8            0\n",
       "...             ...       ...          ...       ...  ...          ...\n",
       "997150431    997150        14         1986   40000.0   -8            1\n",
       "1554992501  1554992        33         1987       0.0   -8            0\n",
       "1715282501  1715282        44         1982   48000.0   -8            1\n",
       "2489151501  2489151        65         1972       0.0   -8            1\n",
       "2489151502  2489151        65         1973       0.0   -8            0\n",
       "\n",
       "[37147 rows x 6 columns]"
      ]
     },
     "execution_count": 126,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indexed_subdata = subdata.set_index('pid').sort_index()\n",
    "indexed_subdata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fid14</th>\n",
       "      <th>provcd14</th>\n",
       "      <th>cfps_birthy</th>\n",
       "      <th>p_income</th>\n",
       "      <th>te4</th>\n",
       "      <th>cfps_gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>146881.596627</td>\n",
       "      <td>14.118060</td>\n",
       "      <td>1969.975638</td>\n",
       "      <td>8151.068908</td>\n",
       "      <td>-7.264230</td>\n",
       "      <td>0.496838</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>229751.514308</td>\n",
       "      <td>22.124926</td>\n",
       "      <td>1966.774620</td>\n",
       "      <td>9040.960229</td>\n",
       "      <td>-7.499112</td>\n",
       "      <td>0.495757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>334860.242952</td>\n",
       "      <td>33.745025</td>\n",
       "      <td>1966.342731</td>\n",
       "      <td>12621.950276</td>\n",
       "      <td>-7.276534</td>\n",
       "      <td>0.494057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>416622.870781</td>\n",
       "      <td>42.434426</td>\n",
       "      <td>1969.430183</td>\n",
       "      <td>9003.045639</td>\n",
       "      <td>-6.798939</td>\n",
       "      <td>0.499711</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>506085.535280</td>\n",
       "      <td>51.347268</td>\n",
       "      <td>1969.255696</td>\n",
       "      <td>5966.716163</td>\n",
       "      <td>-6.679274</td>\n",
       "      <td>0.506083</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>605246.060482</td>\n",
       "      <td>61.500981</td>\n",
       "      <td>1970.892061</td>\n",
       "      <td>6866.954028</td>\n",
       "      <td>-6.668153</td>\n",
       "      <td>0.507939</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>732754.535714</td>\n",
       "      <td>29.214286</td>\n",
       "      <td>1983.250000</td>\n",
       "      <td>17153.571429</td>\n",
       "      <td>-6.392857</td>\n",
       "      <td>0.535714</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>847693.727273</td>\n",
       "      <td>32.181818</td>\n",
       "      <td>1979.590909</td>\n",
       "      <td>15863.636364</td>\n",
       "      <td>-6.454545</td>\n",
       "      <td>0.681818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>946309.620690</td>\n",
       "      <td>33.034483</td>\n",
       "      <td>1980.103448</td>\n",
       "      <td>9202.068966</td>\n",
       "      <td>-7.137931</td>\n",
       "      <td>0.689655</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           fid14   provcd14  cfps_birthy      p_income       te4  cfps_gender\n",
       "1  146881.596627  14.118060  1969.975638   8151.068908 -7.264230     0.496838\n",
       "2  229751.514308  22.124926  1966.774620   9040.960229 -7.499112     0.495757\n",
       "3  334860.242952  33.745025  1966.342731  12621.950276 -7.276534     0.494057\n",
       "4  416622.870781  42.434426  1969.430183   9003.045639 -6.798939     0.499711\n",
       "5  506085.535280  51.347268  1969.255696   5966.716163 -6.679274     0.506083\n",
       "6  605246.060482  61.500981  1970.892061   6866.954028 -6.668153     0.507939\n",
       "7  732754.535714  29.214286  1983.250000  17153.571429 -6.392857     0.535714\n",
       "8  847693.727273  32.181818  1979.590909  15863.636364 -6.454545     0.681818\n",
       "9  946309.620690  33.034483  1980.103448   9202.068966 -7.137931     0.689655"
      ]
     },
     "execution_count": 127,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "indexed_subdata.groupby(lambda x: int(str(x)[0])).mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上代码中，我们先将pid设为index，然后取出index的最高位数字作为分组依据，并计算了均值。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 一个简单的例子：实验、A/B-test与t检验\n",
    "\n",
    "**实验**（**experiment**）是统计学中因果推断的经典方法，在自然科学中有大量的应用。比如为了测试一个新药的效果，通常的做法是将患者分为实验组和对照组，给予对照组安慰剂，给予实验组药物，经过一段时间后比对两个组别的疗效差异。\n",
    "\n",
    "根据相同的思想，在互联网的产品设计中，通常对于同一产品的设计有不同的方案，比如一个按钮的位置、颜色，一个广告的位置、呈现方式等等。不同的设计细节会导致用户行为的不同，而选择何种的方案最好有数据的支撑。为了比较不同的方案，一个常用的做法是**A/B-test**，即通过计算机随机控制不同用户分别进入不同的组（比如实验组和控制组），每个组的用户看到的是不同的设计方案，经过一段时间后比对这两个不同组的用户的行为差异。\n",
    "\n",
    "不管是在实验中还是在A/B-test中，都涉及到分组比较的问题，最简单的情况是两个组进行比较。两个组之间的差异可能是由于设计的不同造成的，也有可能是因为纯随机的误差造成的，因而严谨的回答两个组是否有差异的问题需要统计学上的**假设检验**（**hypothesis test**）。\n",
    "\n",
    "抛开技术细节，一般的假设检验有三个重要的步骤：\n",
    "\n",
    "1. 确定**原假设**（**null hypothesis**）和**备择假设**（**alternative hypothesis**）\n",
    "2. 确定检验统计量及其抽样分布\n",
    "3. 计算检验统计量，并根据抽样分布计算$p$-value，根据$p$-value进行决策（比如当$p<\\alpha=0.05$时代表在5%的显著性水平下显著，原假设不成立）。\n",
    "\n",
    "对于两个组的均值比较的问题，常用的检验是$t$检验，其原假设为：$$H_0:\\mu_1=\\mu_0$$在原假设的条件下可以构建检验统计量：$$\\frac{\\bar{x}_1-\\bar{x}_2}{\\sqrt{\\frac{s_1^2}{n_1}+\\frac{s_2^2}{n_2}}}\\sim t\\left(v\\right)$$其中：\n",
    "\n",
    "* $\\bar{x}_i$为第$i$组的均值，特别的，如果$x$为0/1型变量（是/否等），那么$\\bar{x}_i=\\hat{p}_i$，即第$i$组中$1$的比例。\n",
    "* $s_i^2$为第$i$组的标准差，特别的，如果$x$为0/1型变量，那么$s_i^2=\\hat{p}_i\\cdot \\left(1-\\hat{p}_i\\right)$\n",
    "* $n_i$为第$i$组的样本量\n",
    "* $v$为自由度\n",
    "\n",
    "为了进行这个检验，可以使用SciPy中的scipy.stats.ttest_ind()函数。\n",
    "\n",
    "接下来，我们以一个实际的例子展示A/B-test的用法。数据如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>user_id</th>\n",
       "      <th>test_option</th>\n",
       "      <th>sale_dt</th>\n",
       "      <th>category</th>\n",
       "      <th>orderid</th>\n",
       "      <th>sale_price</th>\n",
       "      <th>gender</th>\n",
       "      <th>age</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>19916</td>\n",
       "      <td>0</td>\n",
       "      <td>20200224</td>\n",
       "      <td>6</td>\n",
       "      <td>21000064456673</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>19916</td>\n",
       "      <td>0</td>\n",
       "      <td>20200224</td>\n",
       "      <td>6</td>\n",
       "      <td>21000064456673</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>19916</td>\n",
       "      <td>0</td>\n",
       "      <td>20200224</td>\n",
       "      <td>6</td>\n",
       "      <td>21000064456673</td>\n",
       "      <td>5000.0</td>\n",
       "      <td>M</td>\n",
       "      <td>25</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>19824</td>\n",
       "      <td>0</td>\n",
       "      <td>20200212</td>\n",
       "      <td>2</td>\n",
       "      <td>2000062515210</td>\n",
       "      <td>29970.0</td>\n",
       "      <td>M</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>19824</td>\n",
       "      <td>0</td>\n",
       "      <td>20200212</td>\n",
       "      <td>2</td>\n",
       "      <td>2000062515210</td>\n",
       "      <td>2500.0</td>\n",
       "      <td>M</td>\n",
       "      <td>60</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   user_id  test_option   sale_dt  category         orderid  sale_price  \\\n",
       "0    19916            0  20200224         6  21000064456673      2500.0   \n",
       "1    19916            0  20200224         6  21000064456673      5000.0   \n",
       "2    19916            0  20200224         6  21000064456673      5000.0   \n",
       "3    19824            0  20200212         2   2000062515210     29970.0   \n",
       "4    19824            0  20200212         2   2000062515210      2500.0   \n",
       "\n",
       "  gender  age  \n",
       "0      M   25  \n",
       "1      M   25  \n",
       "2      M   25  \n",
       "3      M   60  \n",
       "4      M   60  "
      ]
     },
     "execution_count": 128,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "data = pd.read_csv(\"csv/ba_sales_data.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "其中user_id为用户id，orderid为订单id，sale_price为某个货物的价格，test_option为分组（0为控制组，1为实验组）。\n",
    "\n",
    "我们的目的是为了检测两个分组的用户之间的支出是否有显著不同，为此首先要计算每个用户的总支出："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sale_price</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>79310.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>48690.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>368550.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>55150.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         sale_price\n",
       "user_id            \n",
       "2           79310.0\n",
       "3           48690.0\n",
       "15         368550.0\n",
       "16              0.0\n",
       "18          55150.0"
      ]
     },
     "execution_count": 129,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spending = data[['user_id', 'sale_price']].groupby('user_id').sum()\n",
    "spending.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "由于每个用户确定在某一个组，因而可以使用最大（小）值确定这个用户的分组情况："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 130,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>test_option</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         test_option\n",
       "user_id             \n",
       "2                  1\n",
       "3                  1\n",
       "15                 1\n",
       "16                 1\n",
       "18                 1\n",
       "23                 1\n",
       "25                 1\n",
       "26                 0\n",
       "30                 1\n",
       "32                 1"
      ]
     },
     "execution_count": 130,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "option = data[['user_id', 'test_option']].groupby('user_id').max()\n",
    "option.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "接下来将两个数据合并在一起："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>sale_price</th>\n",
       "      <th>test_option</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>user_id</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>79310.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>48690.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>368550.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>55150.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23</th>\n",
       "      <td>94360.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25</th>\n",
       "      <td>89740.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>26</th>\n",
       "      <td>67780.0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>30</th>\n",
       "      <td>33050.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32</th>\n",
       "      <td>22800.0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         sale_price  test_option\n",
       "user_id                         \n",
       "2           79310.0            1\n",
       "3           48690.0            1\n",
       "15         368550.0            1\n",
       "16              0.0            1\n",
       "18          55150.0            1\n",
       "23          94360.0            1\n",
       "25          89740.0            1\n",
       "26          67780.0            0\n",
       "30          33050.0            1\n",
       "32          22800.0            1"
      ]
     },
     "execution_count": 131,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged = pd.merge(spending, option, left_index=True, right_index=True)\n",
    "merged.head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以做一个分组的描述性统计查看一下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr th {\n",
       "        text-align: left;\n",
       "    }\n",
       "\n",
       "    .dataframe thead tr:last-of-type th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th colspan=\"8\" halign=\"left\">sale_price</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th></th>\n",
       "      <th>count</th>\n",
       "      <th>mean</th>\n",
       "      <th>std</th>\n",
       "      <th>min</th>\n",
       "      <th>25%</th>\n",
       "      <th>50%</th>\n",
       "      <th>75%</th>\n",
       "      <th>max</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>test_option</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>3490.0</td>\n",
       "      <td>131485.802292</td>\n",
       "      <td>854336.342632</td>\n",
       "      <td>-1696190.0</td>\n",
       "      <td>28180.0</td>\n",
       "      <td>64585.0</td>\n",
       "      <td>143652.5</td>\n",
       "      <td>49510000.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3468.0</td>\n",
       "      <td>133786.828143</td>\n",
       "      <td>202518.866672</td>\n",
       "      <td>-437930.0</td>\n",
       "      <td>32200.0</td>\n",
       "      <td>74480.0</td>\n",
       "      <td>163202.5</td>\n",
       "      <td>3387500.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            sale_price                                                    \\\n",
       "                 count           mean            std        min      25%   \n",
       "test_option                                                                \n",
       "0               3490.0  131485.802292  854336.342632 -1696190.0  28180.0   \n",
       "1               3468.0  133786.828143  202518.866672  -437930.0  32200.0   \n",
       "\n",
       "                                            \n",
       "                 50%       75%         max  \n",
       "test_option                                 \n",
       "0            64585.0  143652.5  49510000.0  \n",
       "1            74480.0  163202.5   3387500.0  "
      ]
     },
     "execution_count": 132,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "merged.groupby('test_option').describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意最小值有负值，这里需要额外注意一下是否需要进行额外的数据清洗。\n",
    "\n",
    "最后使用$t$检验检验两个组的差异是否显著（函数的手册可以查看 https://docs.scipy.org/doc/scipy/reference/tutorial/stats.html ）："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 133,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(0.1543595200695521, 0.8773307475193647)"
      ]
     },
     "execution_count": 133,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import scipy.stats\n",
    "\n",
    "t, pval = scipy.stats.ttest_ind(\n",
    "    merged[merged['test_option'] == 1]['sale_price'],\n",
    "    merged[merged['test_option'] == 0]['sale_price'])\n",
    "t, pval\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现，$p = 0.877 > 0.1 > 0.05$，因而两个组没有显著差异。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}