从几周前开始,我就无法再下载雅虎财经数据了:
$ wget -O GLD.USA_20170612.txt --no-check-certificate http://chart.finance.yahoo.com/table.csv?s=GLD&a=2&b=1&c=2017&d=11&e=30&f=2017&ignore=.csv
--2017-06-12 12:21:28-- http://gld.usa_20170612.txt/
Resolving gld.usa_20170612.txt (gld.usa_20170612.txt)... failed: No address associated with hostname.
wget: unable to resolve host address ‘gld.usa_20170612.txt’
--2017-06-12 12:21:28-- http://chart.finance.yahoo.com/table.csv?s=GLD&a=2&b=1&c=2017&d=11&e=30&f=2017
Resolving chart.finance.yahoo.com (chart.finance.yahoo.com)... 87.248.114.12, 87.248.116.11, 87.248.116.12, ...
Connecting to chart.finance.yahoo.com (chart.finance.yahoo.com)|87.248.114.12|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://chart.finance.yahoo.com/table.csv?s=GLD&a=2&b=1&c=2017&d=11&e=30&f=2017 [following]
--2017-06-12 12:21:28-- https://chart.finance.yahoo.com/table.csv?s=GLD&a=2&b=1&c=2017&d=11&e=30&f=2017
Connecting to chart.finance.yahoo.com (chart.finance.yahoo.com)|87.248.114.12|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2017-06-12 12:21:28 ERROR 404: Not Found.
他们改变了一些东西。我从网站上获取了一个新的 URL,但它仍然不起作用:
$ wget https://query1.finance.yahoo.com/v7/finance/download/GLD?period1=1494584558&period2=1497262958&interval=1d&events=history&crumb=GGHpj6ucgIy
--2017-06-12 12:27:24-- https://query1.finance.yahoo.com/v7/finance/download/GLD?period1=1494584558
Resolving query1.finance.yahoo.com (query1.finance.yahoo.com)... 87.248.116.11, 87.248.116.12, 87.248.114.11, ...
Connecting to query1.finance.yahoo.com (query1.finance.yahoo.com)|87.248.116.11|:443... connected.
HTTP request sent, awaiting response... 401 Unauthorized
Username/Password Authentication Failed.
我不明白为什么它提到用户名和密码。当我单击 https://uk.finance.yahoo.com/quote/GLD/history?p=GLD 上的“下载”按钮时,它不会要求输入用户名和密码。看来不用订阅也能下载数据了。
如果有人知道下载雅虎财务数据的正确 wget 实现,请在此处分享。
...
更新:
感谢当前的回复,我被暗示这里可能涉及“cookies”。在关键字中搜索cookie的类似问题时,我发现了以下线程:
不幸的是这对我来说太复杂了...... 我希望能为这项工作提供一点帮助。
为了处理 Yahoo cokies,我编写了以下代码:
#!/usr/bin/sh
symbol=$1
today=$(date +%Y%m%d)
tomorrow=$(date --date='1 days' +%Y%m%d)
first_date=$(date -d "$2" '+%s')
last_date=$(date -d "$today" '+%s')
wget --no-check-certificate --save-cookies=cookie.txt https://finance.yahoo.com/quote/$symbol/?p=$symbol -O crumb.store
crumb=$(grep 'root.*App' crumb.store | sed 's/,/\n/g' | grep CrumbStore | sed 's/"CrumbStore":{"crumb":"\(.*\)"}/\1/')
wget --no-check-certificate --load-cookies=cookie.txt "https://query1.finance.yahoo.com/v7/finance/download/$symbol?period1=$first_date&period2=$last_date&interval=1d&events=history&crumb=$crumb" -O $symbol.csv
rm cookie.txt crumb.store
使用示例:
$ sh stockdownload_yahoo.sh QQQ 20170508
支持的日期格式示例:
$ date -d 20170328 +%s
1490652000
$ date -d 2017-03-28 +%s
1490652000
$ date -d "Mar 28 2017" +%s
1490652000
现在仅适用于高级用户 - 黄金用户(每月 50 美元!)。 这很不幸。