我需要从这样的页面获取电子邮件:http://bari.geometriapulia.net/index.php/albo-lista/userprofile/abbatantuono-giuseppe
为此,我使用以下代码:
from bs4 import BeautifulSoup
import urllib.request
import re
url = "http://bari.geometriapulia.net/index.php/albo-lista/userprofile/abbatantuono-giuseppe"
content = urllib.request.urlopen(url).read()
soup = BeautifulSoup(content, "lxml")
for link in soup.find_all("a", href=re.compile(r"^mailto:")):
if "@" in str(link.string):
print(link.string)
此代码没有找到我想要的电子邮件,即您可以在个人资料图片下看到的两个电子邮件,但它找到了位于页面底部的电子邮件(不是我感兴趣的)。
为了尝试理解为什么,我下载了整个 HTML 页面,以及哪里应该有电子邮件,您可以在邮件应该在的位置阅读“...”,并且在其下方的行中还有一个警告:
<td class="fieldCell" id="cbfv_84"><span class="cbMailRepl" id="cbMa92357">...</span><noscript>
This e-mail address is protected by spam bot, you must activate JavaScript in you browser in order to visualize it
</noscript>
</td>
</tr>
<tr class="sectiontableentry2 cbft_emailaddress" id="cbfr_97">
<td class="titleCell"><label for="cbfv_97" id="cblabcbfv_97">e-mail:</label></td>
<td class="fieldCell" id="cbfv_97"><span class="cbMailRepl" id="cbMa92358">...</span><noscript>
This e-mail address is protected by spam bot, you must activate JavaScript in you browser in order to visualize it
所以我检查了浏览器中是否启用了 JavaScript,正如您从屏幕截图中看到的那样: http://prntscr.com/dwgl7w
那么我怎样才能下载页面而不让反垃圾邮件机器人系统从 HTML 代码中“剪掉”邮件呢? 这可能吗?