以此为起点.. http://docs.python-guide.org/en/latest/scenarios/scrape/
from lxml import html
import requests
page = requests.get('http://econpy.pythonanywhere.com/ex/001.html')
tree = html.fromstring(page.text)
一切都按预期进行..但是,....
from lxml import html
import requests
page = requests.get('http://www.streetinsider.com/ipo_history.php?type=upcoming')
tree = html.fromstring(page.text)
出现此错误...
File "<string>", line unknown
XMLSyntaxError: line 1: Document is empty
使用pyquery......
from pyquery import PyQuery as pq
from lxml import etree,html
import requests
response = pq(url='http://www.streetinsider.com/ipo_history.php?type=upcoming')
doc = pq(response.content)
抛出此错误...
File "<string>", line unknown
XMLSyntaxError: line 1504: Unexpected end tag : h2
从网页获取表格的任何帮助。
某些网站会检测并阻止某些用户代理。 (类似于网络机器人。)www.streetinsider.com 背后的网络应用程序似乎检测到 python 请求的用户代理,并(被动)阻止其 HTTP 请求。
您可以使用 requests.get 函数调用的 headers={'User-Agent': ''} 参数来设置 user-aget。
page = requests.get('http://www.streetinsider.com/ipo_history.php', \
headers={'User-Agent': 'tester'}, \
params={'type':'upcoming'})