Regex.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 正则表达式\n",
    "\n",
    "## 正则表达式简介\n",
    "\n",
    "**正则表达式**（**regular expression**，简称**regex**）是一个特殊的字符串，用来代表其他字符串的某种模式。\n",
    "\n",
    "比如，在实际应用中，一个非常常见的任务是判断某个字符串是否是合法的Email/网页地址/ip地址等等，或者从一个字符串中挑出满足这些模式的字符串。这些字符串都有固定的模式，虽然人类可以很轻松地识别出这些模式，但是对于机器而言，仍然是非常复杂而困难的，正则表达式就是为了解决这一问题。\n",
    "\n",
    "比如，如果我们想要匹配一个邮件地址，那么下面的字符串可以方便地表示Email地址的模式：\n",
    "\n",
    "```python\n",
    "\"\"\"\n",
    "(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\n",
    "\"\"\"\n",
    "```\n",
    "\n",
    "以上看起来十分复杂，但实际上是有章可循的。下面我们就结合Python介绍正则表达式的用法。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python中的正则表达式\n",
    "\n",
    "在Python中自带了“re”包，可以方便的实现匹配。为了展示起见，在系统介绍正则表达式之前，我们先介绍几个有用的函数。\n",
    "\n",
    "### match()函数\n",
    "\n",
    "首先是re.match()函数，该函数从一个字符串的**开头**开始匹配一个字符串，如果能够匹配，返回一个对象，该对象包含了匹配结果；如果不能匹配，返回None。比如，我们可以使用上面的正则表达式来匹配邮件地址："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果： <re.Match object; span=(0, 21), match='si.jichun@outlook.com'>\n",
      "位置： (0, 21)\n"
     ]
    }
   ],
   "source": [
    "import re\n",
    "\n",
    "pattern = r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "res = re.match(pattern, \"si.jichun@outlook.com\")\n",
    "print(\"结果：\", res)\n",
    "if res != None:\n",
    "    print(\"位置：\", res.span())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如果按照以上规则，判断该字符串不是以Email地址开头，则返回None："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果： None\n"
     ]
    }
   ],
   "source": [
    "pattern = r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "res = re.match(pattern, \"si jichun@outlook.com\")\n",
    "print(\"结果：\", res)\n",
    "if res != None:\n",
    "    print(\"位置：\", res.span())\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者，即使包含邮件地址，但是不再开头，也会匹配失败："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果： None\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "res=re.match(pattern,\"邮件地址：si.jichun@outlook.com\")\n",
    "print(\"结果：\",res)\n",
    "if res!=None:\n",
    "    print(\"位置：\",res.span())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "当然，如果在后面则没有问题："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果： <re.Match object; span=(0, 21), match='si.jichun@outlook.com'>\n",
      "位置： (0, 21)\n",
      "开始位置： 0\n",
      "结束位置： 21\n",
      "匹配的字符串： si.jichun@outlook.com\n",
      "匹配的字符串： si.jichun@outlook.com\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "mystr=\"si.jichun@outlook.com我的邮件地址\"\n",
    "res=re.match(pattern,mystr)\n",
    "print(\"结果：\",res)\n",
    "if res!=None:\n",
    "    print(\"位置：\",res.span())\n",
    "    print(\"开始位置：\",res.start())\n",
    "    print(\"结束位置：\",res.end())\n",
    "    print(\"匹配的字符串：\",res.group())\n",
    "    print(\"匹配的字符串：\",mystr[res.start():res.end()])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意在上面，我们使用re.match()函数匹配字符串之后，该函数返回了一个**Match对象**，我们使用了该对象的四个方法：\n",
    "1. res.span()返回匹配成功字符串的起始和结束位置\n",
    "2. res.group()返回匹配成功的字符串\n",
    "3. res.start()返回匹配成功字符串的起始位置\n",
    "3. res.end()返回匹配成功字符串的结束位置\n",
    "\n",
    "### re.search()函数\n",
    "\n",
    "re.search()函数用以搜索一个字符串中第一个位置。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "结果： <re.Match object; span=(5, 26), match='si.jichun@outlook.com'>\n",
      "位置： (5, 26)\n",
      "匹配的字符串： si.jichun@outlook.com\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "mystr=\"邮件地址：si.jichun@outlook.com。\"\n",
    "res=re.search(pattern,mystr)\n",
    "print(\"结果：\",res)\n",
    "if res!=None:\n",
    "    print(\"位置：\",res.span())\n",
    "    print(\"匹配的字符串：\", res.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意match()函数一定是从第一个字符开始匹配，而search()函数则是找到第一个匹配，两者用处不同。不过两者的共同点是，都会返回一个Match对象，因而返回之后的处理都是一样的。\n",
    "\n",
    "### re.finditer()函数\n",
    "\n",
    "re.search()函数只是找到第一个匹配，而如果想要找到所有的匹配，需要使用re.finditer()函数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "si.jichun@outlook.com ;start: 8 end: 29\n",
      "zhiyuezen@126.com ;start: 40 end: 57\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "mystr=\"\"\"\n",
    "我的邮件地址：si.jichun@outlook.com。\n",
    "或者，你可以使用：zhiyuezen@126.com。\n",
    "\"\"\"\n",
    "res=re.finditer(pattern,mystr)\n",
    "for r in res:\n",
    "    print(r.group(),\";start:\",r.start(),\"end:\",r.end())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "值得注意的是，re.finditer()函数返回的是一个**迭代器**(**iterator**)，因而必须使用循环语句对其进行调用。\n",
    "\n",
    "### re.sub()函数\n",
    "\n",
    "该函数用于替换字符串。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "我的邮件地址：就不告诉你。\n",
      "或者，你可以使用：就不告诉你。\n",
      "\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "mystr=\"\"\"\n",
    "我的邮件地址：si.jichun@outlook.com。\n",
    "或者，你可以使用：zhiyuezen@126.com。\n",
    "\"\"\"\n",
    "res=re.sub(pattern,\"就不告诉你\",mystr)\n",
    "print(res)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "该函数还有一个版本，即re.subn()，该函数不仅返回替换后的字符串，还返回替换的次数："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "我的邮件地址：就不告诉你。\n",
      "或者，你可以使用：就不告诉你。\n",
      "\n",
      "替换了： 2 次。\n"
     ]
    }
   ],
   "source": [
    "pattern=r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\"\n",
    "mystr=\"\"\"\n",
    "我的邮件地址：si.jichun@outlook.com。\n",
    "或者，你可以使用：zhiyuezen@126.com。\n",
    "\"\"\"\n",
    "res=re.subn(pattern,\"就不告诉你\",mystr)\n",
    "print(res[0])\n",
    "print(\"替换了：\",res[1],\"次。\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后需要注意的是，以上我们展示的情况比较简单，都是每个正则表达式进行一次匹配。如果一个正则表达式需要多次匹配多个字符串，比较好的做法是先使用re.compile()函数将正则表达式编译，该函数返回一个**Regex对象**，使用该对象的compare(),search(),finditer(),sub(),subn()等函数与上面直接使用各个函数完全等价，但是效率更高："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "si.jichun@outlook.com\n",
      "zhiyuezen@126.com\n",
      "\n",
      "我的邮件地址：就不告诉你。\n",
      "或者，你可以使用：就不告诉你。\n",
      "\n",
      "替换了： 2 次。\n"
     ]
    }
   ],
   "source": [
    "pattern=re.compile(r\"(\\w+\\.)*\\w+@(\\w+\\.)+[A-Za-z]+\")\n",
    "mystr=\"\"\"\n",
    "我的邮件地址：si.jichun@outlook.com。\n",
    "或者，你可以使用：zhiyuezen@126.com。\n",
    "\"\"\"\n",
    "## 查找\n",
    "res=pattern.finditer(mystr)\n",
    "for r in res:\n",
    "    print(r.group())\n",
    "    \n",
    "## 替换\n",
    "res=pattern.subn(\"就不告诉你\",mystr)\n",
    "print(res[0])\n",
    "print(\"替换了：\",res[1],\"次。\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 正则表达式详细介绍\n",
    "\n",
    "在掌握了Python中正则表达式的相关函数之后，我们可以详细介绍正则表达式的规则了。\n",
    "\n",
    "### 字符、特殊字符、字符集\n",
    "\n",
    "正则表达式的规则比较复杂，我们从最简单的规则开始。最简单的正则表达式就是一个字符串，比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "34 ： 元\n",
      "69 ： 元\n"
     ]
    }
   ],
   "source": [
    "mystr1=\"\"\"\n",
    "2019年4月，人民银行对金融机构开展中期借贷便利操作共2000亿元，\n",
    "期限1年，利率为3.30%。\n",
    "期末中期借贷便利余额为35600亿元。\n",
    "\"\"\"\n",
    "\n",
    "res=re.finditer(\"元\",mystr1)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上我们将\"元\"作为一个正则表达式，并在字符串中寻找这个规则，最终输出符合该规则的字符串在整个字符串中的位置。需要注意的是，一般而言正则表达式严格区分大小写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "66 ： Tesla\n",
      "988 ： Tesla\n",
      "1193 ： Tesla\n",
      "1741 ： Tesla\n",
      "1865 ： Tesla\n",
      "2431 ： Tesla\n",
      "2627 ： Tesla\n",
      "2697 ： Tesla\n",
      "3419 ： Tesla\n",
      "3484 ： Tesla\n",
      "3658 ： Tesla\n",
      "3907 ： Tesla\n",
      "4339 ： Tesla\n",
      "----------------\n"
     ]
    }
   ],
   "source": [
    "mystr2=\"\"\"\n",
    "Ark Invest chief Catherine Wood is holding firm on her thesis on Tesla — which she believes could run to $4,000 a share or higher — telling CNBC's \"ETF Edge\" on Monday that her conviction in the company has actually increased since last year.\n",
    "\n",
    "\"It is our largest position ... and our conviction has increased in the last year or so,\" said Wood, who is founder, CEO and chief investment officer at Ark. \"It's down about 29% this year. And it's always been our largest position this year, and yet our fund is up 28%.\"\n",
    "\n",
    "The way Ark achieves that is through its unique model, Wood said. The company centers on innovation as a growth driver, offering a number of exchange-traded funds focused on cutting-edge areas of the market like artificial intelligence, the future of finance and genomics.\n",
    "\n",
    "Most of Ark's actively managed ETFs are outperforming the broader market this year, and therein lies the strategy for how Ark — whose top holding is, indeed, the periodically downtrodden stock of Tesla — makes its money.\n",
    "\n",
    "\"Last year, when genomics was under assault because of reimbursement concerns and pricing concerns and so forth, we were leaning into those [stocks] heavily,\" which helped offset Tesla's 2018 declines, Wood explained.\n",
    "\n",
    "But in this article, there is no CFO, so I added it . Maybe I can also add a CxO.\n",
    "\n",
    "\"Our second-largest position, Invitae NVTA  -- which we think is one of the most important molecular diagnostic companies out there riding down the cost curve of DNA sequencing -- [is] up 125%,\" she said. \"It got as low as $5 last year. Today it's $18. So leaning into that has really paid off.\"\n",
    "\n",
    "That rang true for the rest of Ark's investments in the genomics space last year, and this year, \"we're doing the same ... with Tesla,\" the CEO said. \"But our conviction level there is so high that it never left our top position.\"\n",
    "\n",
    "Ark's conviction in Tesla has held strong through a myriad of issues — including concerns around profitability, cash flow and execution — both because Wood believes the company is grossly undervalued by Wall Street, and because of Ark's nimble investing method.\n",
    "\n",
    "\"Disruptive innovation is characterized by controversy and volatility, and so we know we are going to get opportunities to buy stocks. We lie in wait for those opportunities,\" Wood said. \"Many people think, because of what we do – disruptive innovation – that we're momentum driven. Absolutely not. We lie in wait. So take Tesla alone. [...] If you take away the performance of the stock ... and just look at what we delivered in alpha because of our trading around controversy, we delivered 175 basis points just from Tesla. And we get opportunities like that throughout the portfolio.\"\n",
    "\n",
    "Tesla's plan to raise $2.7 billion worth of capital didn't phase Wood either, she said Friday in a phone call with CNBC.\n",
    "\n",
    "\"Our bear case has it going to $700 and our bull case is $4,000, but now we think that's too low,\" she said, explaining that Ark's five-year time horizons for each case already assumed capital raises would occur.\n",
    "\n",
    "\"We understand what they're doing,\" Wood continued. \"In our bear case, even if they just were an electric vehicle manufacturer and nothing else, we expected them to raise $10 billion in capital, and in our bull case, which means they're going to need funding to roll out this autonomous strategy, we thought they were going to raise $20 billion.\"\n",
    "\n",
    "That $4,000 target also bakes in what Tesla CEO Elon Musk predicted on an investor call Thursday: that Tesla's self-driving segment would help drive the business to $500 billion.\n",
    "\n",
    "\"That $4,000,000 forecast is higher than $500 billion in the next five years,\" she said.\n",
    "\n",
    "And if Tesla does indeed rally a whopping 1,500% to the $4,000 level, it'll be because analysts finally value the company properly, the CEO argued.\n",
    "\n",
    "\"The analysts following this stock don't know how to analyze it,\" she said, adding that rather than seeing Tesla as exclusively an auto stock or a tech stock, she and her firm see it as a tech-auto-battery-utility hybrid. \"I don't think research departments out there are set up to analyze this stock.\"\n",
    "\n",
    "\"It's something for everyone, and no one can pull it all together,\" she said.\n",
    "\n",
    "Ark, for one, has four analysts in the areas of robotics, energy storage, artificial intelligence and transportation as a service collaborating on covering Tesla, Wood said.\n",
    "\"\"\"\n",
    "\n",
    "res=re.finditer(\"Tesla\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())\n",
    "\n",
    "print(\"----------------\")\n",
    "\n",
    "res=re.finditer(\"tesla\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "匹配\"Tesla\"可以找到很多条，而匹配\"tesla\"却一条都找不到。\n",
    "\n",
    "此外有的时候我们可能需要匹配任意字符，比如：我们可能需要找到所有的\"C\\*O\"，即不管是CEO还是CTO、CFO，我们统统希望找到，那么可以使用\".\"这个符号作为通配符："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "362 ： CEO\n",
      "1266 ： CFO\n",
      "1310 ： CxO\n",
      "1753 ： CEO\n",
      "3425 ： CEO\n",
      "3787 ： CEO\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(\"C.O\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以看到，上面把字符串中所有的C\\*O都给找了出来。\n",
    "\n",
    "然而，我们发现其中混入了一个奇奇怪怪的东西：CxO。其实我们可能只是想提取出CEO\\CFO\\CTO，如何写这个正则表达式呢？可以使用方括号\\[\\]，方括号之中定义了一个“**字符集**”，只要在方括号之中的任何一个匹配就可以，但是不在方括号中的就无法匹配："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "362 ： CEO\n",
      "1266 ： CFO\n",
      "1753 ： CEO\n",
      "3425 ： CEO\n",
      "3787 ： CEO\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(\"C[ETF]O\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，如果我们希望匹配所有的大写字母，可以使用简写形式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "362 ： CEO\n",
      "1266 ： CFO\n",
      "1753 ： CEO\n",
      "3425 ： CEO\n",
      "3787 ： CEO\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(\"C[A-Z]O\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "常见的所有简写有\n",
    "\n",
    "* 所有大写字母：\\[A-Z\\]\n",
    "* 所有小写字母：\\[a-z\\]\n",
    "* 所有数字：\\[0-9\\]\n",
    "\n",
    "当然，也可以制定某个区间，比如可以使用\\[E-G\\]代表E、F、G三个大写字母，\\[1-3\\]代表1、2、3三个数字等。比如以下程序提取除了所有以1、2、3三个数字开头的美元："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1564 ： $1\n",
      "2719 ： $2\n",
      "3204 ： $1\n",
      "3366 ： $2\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[1-3]\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而这里有一个问题需要注意：我们在输入美元符号\"$\"时，在前面加了一个斜杠\"\\\\\"，这是为什么呢？\n",
    "\n",
    "可以考虑以下的问题：既然\".\"能够匹配所有字符，那么如果我们使用：\n",
    "```python\n",
    "pattern=\"...\"\n",
    "```\n",
    "那么以上的模式就变成了匹配任意三个字符，而不是三个点。比如在mystr中，出现了三个点，但是如果我们使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 ： Ark\n",
      "4 ：  In\n",
      "7 ： ves\n",
      "10 ： t c\n",
      "13 ： hie\n",
      "16 ： f C\n",
      "19 ： ath\n",
      "22 ： eri\n",
      "25 ： ne \n",
      "28 ： Woo\n",
      "31 ： d i\n",
      "......\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(\"...\",mystr2)\n",
    "counter=0\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())\n",
    "    counter+=1\n",
    "    if counter>10:\n",
    "        break\n",
    "print(\"......\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上并没有得到我们想要的结果，而是匹配了任意的三个字符，且是不重叠的三个字符。\n",
    "\n",
    "为了解决这一问题，我们需要给在正则表达式中有特殊含义的字符进行转义：在特殊字符，如\".\"前面加一个斜杠，即\"\\\\\\.\"，现在程序如下："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "273 ： ...\n",
      "1732 ： ...\n",
      "2445 ： ...\n",
      "2496 ： ...\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\.\\.\\.\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上程序得到了我们想要的结果：连续的三个点。\n",
    "\n",
    "除了\".\"这个字符之外，以下所有字符都需要转义：\n",
    "```python\n",
    "\"\"\"\n",
    "* . ? + $ ^ [ ] ( ) { } | \\ /\n",
    "\"\"\"\n",
    "```\n",
    "分别使用：\n",
    "```python\n",
    "\"\"\"\n",
    "\\* \\. \\? \\+ \\$ \\^ \\[ \\] \\( \\) \\{ \\} \\| \\\\ \\/\n",
    "\"\"\"\n",
    "```\n",
    "进行转义。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "既然有了字符集这个集合，那么就会有补集。比如我们可能需要匹配除了数字之外的其他字符，或者除了大写字母之外的其他字符。只需要在\\[\\]中加入一个\"^\"符号就可以表示“补集”的概念，比如如下程序我们匹配了除了CEO之外的所有C\\*O："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1266 ： CFO\n",
      "1310 ： CxO\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"C[^E]O\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "以上介绍了对特殊字符的转义，除了这些在正则表达式中比较特殊的字符之外，还有一些字符在文本中比较特殊，有特别的表示方法，比如：\n",
    "\n",
    "* 空白字符：\n",
    "    - \\\\f：换页符\n",
    "    - \\\\n：换行符\n",
    "    - \\\\r：回车符\n",
    "    - \\\\t：制表符\n",
    "    - \\\\v：垂直制表符\n",
    "\n",
    "这里需要注意的是\\\\r和\\\\n。在Windows中，一个文本文件的每一行都是以\"\\\\r\\\\n\"结尾的，即\"\\\\r\\\\n\"代表一行的结束，新一行的开始。而在Linux\\Unix中，只需要\"\\\\n\"。\n",
    "\n",
    "还有一些其他比较重要的特殊字符：\n",
    "\n",
    "* 元字符\n",
    "    - \\\\d：所有的数字，等价于\\[0-9\\]\n",
    "    - \\\\D：所有的非数字，等价于\\[^0-9\\]\n",
    "    - \\\\w：所有的字母、数字或者下划线，等价于\\[a-zA-Z0-9_\\]\n",
    "    - \\\\W：所有的非(字母、数字或者下划线)，等价于\\[^a-zA-Z0-9_\\]\n",
    "    - \\\\s：所有的空白字符，等价于\\[\\\\f\\\\n\\\\r\\\\t\\\\v\\]\n",
    "    - \\\\S：所有的非空白字符，等价于\\[^\\\\f\\\\n\\\\r\\\\t\\\\v\\]\n",
    "    \n",
    "现在思考这样一个问题：我们希望将上面一段文字中所有的「it」或者「It」都找出来，我们可能会写出如下的表达式：\n",
    "```python\n",
    "\"[Ii]t\"\n",
    "```\n",
    "\n",
    "然而，使用这个表达式很有可能会把带有「it」的单词也都找出来，为了展示起见，我们把在「it」前后一个字符也都打出来："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "245 ： \"It \n",
      "266 ： siti\n",
      "403 ： \"It'\n",
      "438 ：  it'\n",
      "470 ： siti\n",
      "554 ：  its\n",
      "1001 ：  its\n",
      "1281 ：  it \n",
      "1338 ： siti\n",
      "1348 ： vita\n",
      "1521 ： \"It \n",
      "1558 ：  it'\n",
      "1736 ： with\n",
      "1810 ：  it \n",
      "1835 ： siti\n",
      "1945 ： fita\n",
      "1951 ： lity\n",
      "2173 ： lity\n",
      "2221 ： niti\n",
      "2254 ： ait \n",
      "2275 ： niti\n",
      "2418 ： ait.\n",
      "2652 ： niti\n",
      "2743 ： pita\n",
      "2767 ： eith\n",
      "2807 ： with\n",
      "2837 ：  it \n",
      "3006 ： pita\n",
      "3221 ： pita\n",
      "3720 ：  it'\n",
      "3860 ：  it,\n",
      "3979 ：  it \n",
      "4009 ： lity\n",
      "4104 ： \"It'\n",
      "4153 ：  it \n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\".[Ii]t.\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "我们发现以上表达式不止找到了it，还找到了with之类的其他单词。为了实现这一目的，可以使用「\\\\b」，该符号用来表示匹配不能构成单词的字符，或者单词边界："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "res=re.finditer(r\"\\\\b[Ii]t\\\\b\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "注意由于「\\\\b」在Python中也是转义字符，代表退格键，因而在上面实际使用的是「\\\\\\\\b」。同理，\\\\B代表不希望匹配单词的边界："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "266 ： siti\n",
      "470 ： siti\n",
      "1338 ： siti\n",
      "1348 ： vita\n",
      "1736 ： with\n",
      "1835 ： siti\n",
      "1945 ： fita\n",
      "1951 ： lity\n",
      "2173 ： lity\n",
      "2221 ： niti\n",
      "2275 ： niti\n",
      "2652 ： niti\n",
      "2743 ： pita\n",
      "2767 ： eith\n",
      "2807 ： with\n",
      "3006 ： pita\n",
      "3221 ： pita\n",
      "4009 ： lity\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\".\\B[Ii]t\\B.\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，我们还可以使用「^」符号匹配字符串的开头、「$」符号匹配字符串的结尾，使用「(?m)」代表启用分行匹配模式。以上几个特性可以联合起来使用，比如我们如果想匹配出所有以引号开头的行，可以使用如下表达式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "245 ： \"It is our largest position ... and our conviction has increased in the last year or so,\" said Wood, who is founder, CEO and chief investment officer at Ark. \"It's down about 29% this year. And it's always been our largest position this year, and yet our fund is up 28%.\"\n",
      "1014 ： \"Last year, when genomics was under assault because of reimbursement concerns and pricing concerns and so forth, we were leaning into those [stocks] heavily,\" which helped offset Tesla's 2018 declines, Wood explained.\n",
      "1316 ： \"Our second-largest position, Invitae NVTA  -- which we think is one of the most important molecular diagnostic companies out there riding down the cost curve of DNA sequencing -- [is] up 125%,\" she said. \"It got as low as $5 last year. Today it's $18. So leaning into that has really paid off.\"\n",
      "2108 ： \"Disruptive innovation is characterized by controversy and volatility, and so we know we are going to get opportunities to buy stocks. We lie in wait for those opportunities,\" Wood said. \"Many people think, because of what we do – disruptive innovation – that we're momentum driven. Absolutely not. We lie in wait. So take Tesla alone. [...] If you take away the performance of the stock ... and just look at what we delivered in alpha because of our trading around controversy, we delivered 175 basis points just from Tesla. And we get opportunities like that throughout the portfolio.\"\n",
      "2819 ： \"Our bear case has it going to $700 and our bull case is $4,000, but now we think that's too low,\" she said, explaining that Ark's five-year time horizons for each case already assumed capital raises would occur.\n",
      "3033 ： \"We understand what they're doing,\" Wood continued. \"In our bear case, even if they just were an electric vehicle manufacturer and nothing else, we expected them to raise $10 billion in capital, and in our bull case, which means they're going to need funding to roll out this autonomous strategy, we thought they were going to raise $20 billion.\"\n",
      "3561 ： \"That $4,000,000 forecast is higher than $500 billion in the next five years,\" she said.\n",
      "3800 ： \"The analysts following this stock don't know how to analyze it,\" she said, adding that rather than seeing Tesla as exclusively an auto stock or a tech stock, she and her firm see it as a tech-auto-battery-utility hybrid. \"I don't think research departments out there are set up to analyze this stock.\"\n",
      "4104 ： \"It's something for everyone, and no one can pull it all together,\" she said.\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r'(?m)^\".+',mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 重复匹配\n",
    "\n",
    "思考这样一个问题。如果我们需要把字符串中所有的货币提取出来，比如在“mystr2”中，有：\n",
    "```python\n",
    "\"$4,000\"\n",
    "\"$500\"\n",
    "\"$2.7\"\n",
    "```\n",
    "等，可以分别用：\n",
    "\n",
    "```python\n",
    "\"$\\d,\\d\\d\\d\"\n",
    "\"$\\d\\d\\d\"\n",
    "\"$\\d.\\d\"\n",
    "```\n",
    "\n",
    "进行匹配。但是由于我们并不知道后面的数字的位数，我们不可能针对每一种情况都写出一个正则表达式。\n",
    "\n",
    "为了解决这一问题，可以使用重复匹配附进行操作。\n",
    "\n",
    "比如，如果需要匹配一个或者多个（至少一个），可以使用“+”字符，正则表达式“\\d+”即一个或者多个数字（至少一个数字），比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4\n",
      "1539 ： $5\n",
      "1564 ： $18\n",
      "2719 ： $2\n",
      "2850 ： $700\n",
      "2876 ： $4\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4\n",
      "3546 ： $500\n",
      "3567 ： $4\n",
      "3602 ： $500\n",
      "3707 ： $4\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$\\d+\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而注意到，以上的正则表达式虽然匹配了数字，但是由于在计数的时候有逗号“,”的存在，导致我们只保留了美元符号后面的数字，可以做如下修改："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4,000\n",
      "1539 ： $5\n",
      "1564 ： $18\n",
      "2719 ： $2\n",
      "2850 ： $700\n",
      "2876 ： $4,000,\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[\\d,]+\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如此我们就得到了所有美元后面的整数。但是注意到，上面虽然提取了整数位，但是没有提取小数位。如果我们这么写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2719 ： $2.7\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[\\d,]+\\.[\\d]+\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "显然我们只匹配了小数。实际上，我们需要的是小数点及其后面的数字可以出现可以不出现，即0次或者多次。此时，我们可以使用“\\*”表示匹配0次或者多次："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4,000\n",
      "1539 ： $5\n",
      "1564 ： $18.\n",
      "2719 ： $2.7\n",
      "2850 ： $700\n",
      "2876 ： $4,000,\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[\\d,]+[\\.]*[\\d]*\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面的仍然不完美，我们发现数字的最后一位出现了“,”、“.”之类，这是我们不需要的，因此我们必须限定最后一位必须为数字："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4,000\n",
      "1564 ： $18\n",
      "2719 ： $2.7\n",
      "2850 ： $700\n",
      "2876 ： $4,000\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[\\d,]+[\\.]*[\\d]*\\d\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "如此，我们就得到了一个比较完善的提取货币的正则表达式。\n",
    "\n",
    "当然，以上的正则表达式还是不完美的，比如，如果有以下字符串：\n",
    "\n",
    "```python\n",
    "str1=\"The range is $5...7\"\n",
    "```\n",
    "\n",
    "那么我们仍然会得到匹配："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "13 ： $5...7\n"
     ]
    }
   ],
   "source": [
    "str1=\"The range is $5...7\"\n",
    "res=re.finditer(r\"\\$[\\d,]+[\\.]*[\\d]*\\d\",str1)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "因此我们需要限定，“.”可以重复0次或者1次，但是不能重复多次。此时我们可以使用“?”字符，用以匹配0次或者1次："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--------------\n",
      "106 ： $4,000\n",
      "1564 ： $18\n",
      "2719 ： $2.7\n",
      "2850 ： $700\n",
      "2876 ： $4,000\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n"
     ]
    }
   ],
   "source": [
    "str1=\"The range is $5...7\"\n",
    "res=re.finditer(r\"\\$[\\d,]+[\\.]?[\\d]*\\d\",str1)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())\n",
    "print(\"--------------\")\n",
    "res=re.finditer(r\"\\$[\\d,]+[\\.]?[\\d]*\\d\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "可以发现使用新的正则表达式完美的避开了这个问题。\n",
    "\n",
    "那么有没有办法限定重复次数呢？此时可以使用大括号“{}”：\n",
    "\n",
    "* 固定重复n次，使用：{n}\n",
    "* 重复n-m次，使用：{n,m}\n",
    "* 重复至少n次，使用：{n,}\n",
    "\n",
    "比如，为了搜索三个点“...”，可以使用\"\\\\.{3}\"代替\"\\\\.\\\\.\\\\.\"："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "273 ： ...\n",
      "1732 ： ...\n",
      "2445 ： ...\n",
      "2496 ： ...\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\.{3}\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "或者，如果我们需要把所有至少是三位数的整数提取出来，我们可以使用："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4,000\n",
      "2850 ： $700\n",
      "2876 ： $4,000\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$[\\d,]{0,}\\d{3}\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外还需要注意的是，默认情况下，在使用“+”、“\\*”、“{}”时，默认使用“贪婪模式”，即对于每一行，都把能够匹配到的最大的字符串匹配出来。比如，如果我们希望提取出所有引号“\"\"”中间的文本，我们可能会使用如下的正则表达式："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "148 ： \"ETF Edge\"\n",
      "245 ： \"It is our largest position ... and our conviction has increased in the last year or so,\" said Wood, who is founder, CEO and chief investment officer at Ark. \"It's down about 29% this year. And it's always been our largest position this year, and yet our fund is up 28%.\"\n",
      "1014 ： \"Last year, when genomics was under assault because of reimbursement concerns and pricing concerns and so forth, we were leaning into those [stocks] heavily,\"\n",
      "1316 ： \"Our second-largest position, Invitae NVTA  -- which we think is one of the most important molecular diagnostic companies out there riding down the cost curve of DNA sequencing -- [is] up 125%,\" she said. \"It got as low as $5 last year. Today it's $18. So leaning into that has really paid off.\"\n",
      "1710 ： \"we're doing the same ... with Tesla,\" the CEO said. \"But our conviction level there is so high that it never left our top position.\"\n",
      "2108 ： \"Disruptive innovation is characterized by controversy and volatility, and so we know we are going to get opportunities to buy stocks. We lie in wait for those opportunities,\" Wood said. \"Many people think, because of what we do – disruptive innovation – that we're momentum driven. Absolutely not. We lie in wait. So take Tesla alone. [...] If you take away the performance of the stock ... and just look at what we delivered in alpha because of our trading around controversy, we delivered 175 basis points just from Tesla. And we get opportunities like that throughout the portfolio.\"\n",
      "2819 ： \"Our bear case has it going to $700 and our bull case is $4,000, but now we think that's too low,\"\n",
      "3033 ： \"We understand what they're doing,\" Wood continued. \"In our bear case, even if they just were an electric vehicle manufacturer and nothing else, we expected them to raise $10 billion in capital, and in our bull case, which means they're going to need funding to roll out this autonomous strategy, we thought they were going to raise $20 billion.\"\n",
      "3561 ： \"That $4,000,000 forecast is higher than $500 billion in the next five years,\"\n",
      "3800 ： \"The analysts following this stock don't know how to analyze it,\" she said, adding that rather than seeing Tesla as exclusively an auto stock or a tech stock, she and her firm see it as a tech-auto-battery-utility hybrid. \"I don't think research departments out there are set up to analyze this stock.\"\n",
      "4104 ： \"It's something for everyone, and no one can pull it all together,\"\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r'\".*\"',mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "然而实际情况是，该正则表达式把每一行中第一个引号和最后一个引号之间的内容全部匹配出来了，尽管两个引号中间还有引号。\n",
    "\n",
    "为了避免“贪婪模式”，可以在“+”、“\\*”、“{}”后面加一个问号“?”："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "148 ： \"ETF Edge\"\n",
      "245 ： \"It is our largest position ... and our conviction has increased in the last year or so,\"\n",
      "403 ： \"It's down about 29% this year. And it's always been our largest position this year, and yet our fund is up 28%.\"\n",
      "1014 ： \"Last year, when genomics was under assault because of reimbursement concerns and pricing concerns and so forth, we were leaning into those [stocks] heavily,\"\n",
      "1316 ： \"Our second-largest position, Invitae NVTA  -- which we think is one of the most important molecular diagnostic companies out there riding down the cost curve of DNA sequencing -- [is] up 125%,\"\n",
      "1521 ： \"It got as low as $5 last year. Today it's $18. So leaning into that has really paid off.\"\n",
      "1710 ： \"we're doing the same ... with Tesla,\"\n",
      "1763 ： \"But our conviction level there is so high that it never left our top position.\"\n",
      "2108 ： \"Disruptive innovation is characterized by controversy and volatility, and so we know we are going to get opportunities to buy stocks. We lie in wait for those opportunities,\"\n",
      "2295 ： \"Many people think, because of what we do – disruptive innovation – that we're momentum driven. Absolutely not. We lie in wait. So take Tesla alone. [...] If you take away the performance of the stock ... and just look at what we delivered in alpha because of our trading around controversy, we delivered 175 basis points just from Tesla. And we get opportunities like that throughout the portfolio.\"\n",
      "2819 ： \"Our bear case has it going to $700 and our bull case is $4,000, but now we think that's too low,\"\n",
      "3033 ： \"We understand what they're doing,\"\n",
      "3085 ： \"In our bear case, even if they just were an electric vehicle manufacturer and nothing else, we expected them to raise $10 billion in capital, and in our bull case, which means they're going to need funding to roll out this autonomous strategy, we thought they were going to raise $20 billion.\"\n",
      "3561 ： \"That $4,000,000 forecast is higher than $500 billion in the next five years,\"\n",
      "3800 ： \"The analysts following this stock don't know how to analyze it,\"\n",
      "4022 ： \"I don't think research departments out there are set up to analyze this stock.\"\n",
      "4104 ： \"It's something for everyone, and no one can pull it all together,\"\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r'\".*?\"',mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "这样就尽可能小的把双引号以内的内容尽量“小”地匹配了出来。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 子表达式\n",
    "\n",
    "以上我们使用重复匹配匹配了一段字符串中所有的数字，然而以上规则仍然不够完美。比如，以下字符串：\n",
    "```python\n",
    "\"I have $4,0.5\\% of yours\"\n",
    "```\n",
    "\n",
    "仍然是符合我们上面的规则的，但是显然这不是我们想要的："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "7 ： $4,0.5\n"
     ]
    }
   ],
   "source": [
    "str1=\"I have $4,0.5\\% of yours\"\n",
    "res=re.finditer(r\"\\$[\\d,]+[\\.]?[\\d]*\\d\",str1)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "为了规避以上情况，思考一下，逗号的引入是每三位加一个逗号，因而在美元符号后面，应该首先是至少一位、不超过三位的数字，紧接着跟着“,\\\\d\\\\d\\\\d”零次或者多次的组合。问题是，如何能够将“,\\\\d\\\\d\\\\d”组合起来看成一个整体呢？这时，我们需要使用子表达式，即以括号“()”里面的内容为一个表达式，并将跟表达式视为一个整体。\n",
    "\n",
    "比如，按照上面的想法，我们可以写成：\n",
    "```python\n",
    "\"\\$\\d{1,3}(,\\d{3})*\"\n",
    "```\n",
    "\n",
    "以上表达式代表着美元符号后面跟着1-3位的数字，是必须的；而“,\\\\d\\\\d\\\\d”这个组合可以出现0次或者多次。\n",
    "\n",
    "此外，小数点及其后面也是可能有可能没有的，我们可以将小数点后面的表达式写成一个子表达式，并令其至多出现一次，组合起来，就变成了：\n",
    "\n",
    "```python\n",
    "\"\\$\\d{1,3}(,\\d{3})*(\\.\\d+)?\"\n",
    "```\n",
    "\n",
    "比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "106 ： $4,000\n",
      "1539 ： $5\n",
      "1564 ： $18\n",
      "2719 ： $2.7\n",
      "2850 ： $700\n",
      "2876 ： $4,000\n",
      "3204 ： $10\n",
      "3366 ： $20\n",
      "3386 ： $4,000\n",
      "3546 ： $500\n",
      "3567 ： $4,000,000\n",
      "3602 ： $500\n",
      "3707 ： $4,000\n",
      "--------------\n",
      "0 ： $4\n",
      "--------------\n",
      "0 ： $4\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$\\d{1,3}(,\\d{3})*(\\.\\d+)?\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())\n",
    "print(\"--------------\")\n",
    "res=re.finditer(r\"\\$\\d{1,3}(,\\d{3})*(\\.\\d+)?\",\"$4,,000\")\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())\n",
    "print(\"--------------\")\n",
    "res=re.finditer(r\"\\$\\d{1,3}(,\\d{3})*(\\.\\d+)?\",\"$4,0.5\")\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "此外，我们还可以在子表达式里面使用“逻辑或”，即使用“|”代表某一种模式成立即可。\n",
    "\n",
    "比如，如果我们希望挑出最高位数字为1或者2的，我们可以这样写："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1564 ： $18\n",
      "2719 ： $2.7\n",
      "3204 ： $10\n",
      "3366 ： $20\n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"\\$(1|2)\\d{0,2}(,\\d{3})*(\\.\\d+)?\",mystr2)\n",
    "for r in res:\n",
    "    print(r.start(),\"：\",r.group())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在这里需要特别强调时Matching对象的group()函数。我们知道，直接使用不带参数的group()函数可以返回匹配的字符串，实际上group()是可以带参数的，其参数为整型变量i，代表第i个子表达式的值。比如，如果我们使用：\n",
    "```python\n",
    "\"(\\$\\d{1,3})(,\\d{3})*(\\.\\d+)?\"\n",
    "```\n",
    "\n",
    "注意这里面有三个括号，分别是最高的1-3位数字、中间的多个三位数字以及小数点后面的数字。如果我们使用group(1)，就可以获得最高的1-3位数字。此外，使用groups()函数可以获得一个包含所有子表达式值的元组。比如："
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "$4,000      |$4      | $4    | ,000 \n",
      "$5          |$5      | $5   \n",
      "$18         |$18     | $18  \n",
      "$2.7        |$2      | $2    | .7   \n",
      "$700        |$700    | $700 \n",
      "$4,000      |$4      | $4    | ,000 \n",
      "$10         |$10     | $10  \n",
      "$20         |$20     | $20  \n",
      "$4,000      |$4      | $4    | ,000 \n",
      "$500        |$500    | $500 \n",
      "$4,000,000  |$4      | $4    | ,000 \n",
      "$500        |$500    | $500 \n",
      "$4,000      |$4      | $4    | ,000 \n"
     ]
    }
   ],
   "source": [
    "res=re.finditer(r\"(\\$\\d{1,3})(,\\d{3})*(\\.\\d+)?\",mystr2)\n",
    "for r in res:\n",
    "    print(\"%-10s\" % r.group(), \" |\", end='')\n",
    "    print(\"%-5s\" % r.group(1), \" \", end='')\n",
    "    for g in r.groups():\n",
    "        if g!=None:\n",
    "            print(\" | %-5s\" % g,end='')\n",
    "    print(\"\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}