使用Python请求访问经过身份验证的页面

Question

我正在尝试编写一个简单的抓取工具来获取我的互联网帐户的使用详细信息 - 我已经使用 Powershell 成功编写了它，但我想将其移动到 Python 以方便使用/部署。如果我打印 r.text（POST 到登录页面的结果），我只会再次获得登录页面表单详细信息。

我认为解决方案可能类似于使用prepare_request？如果我遗漏了一些非常明显的东西，我很抱歉，自从我接触 python 以来已经大约 5 年了 ^^

import requests
USERNAME = 'usernamehere'
PASSWORD = 'passwordhere'
loginURL = 'https://myaccount.amcom.com.au/ClientLogin.aspx'
secureURL = 'https://myaccount.amcom.com.au/FibreUsageDetails.aspx'

session = requests.session()
req_headers = {'Content-Type': 'application/x-www-form-urlencoded'}

formdata = {
    'ctl00$MemberToolsContent$txtUsername': USERNAME,
    'ctl00$MemberToolsContent$txtPassword': PASSWORD,
    'ctl00$MemberToolsContent$btnLogin' : 'Login'
}

session.get(loginURL)
r = session.post(loginURL, data=formdata, headers=req_headers, allow_redirects=False)
r2 = session.get(secureURL)

我在尝试中引用了这些线程：

HTTP POST 和 GET 使用 cookie 在 python 中进行身份验证 身份验证和 python 请求

Powershell脚本供参考：

$r=Invoke-WebRequest -Uri 'https://myaccount.amcom.com.au/ClientLogin.aspx' -UseDefaultCredentials -SessionVariable RequestForm
$r.Forms[0].Fields['ctl00$MemberToolsContent$txtUsername'] = "usernamehere"
$r.Forms[0].Fields['ctl00$MemberToolsContent$txtPassword'] = "passwordhere"
$r.Forms[0].Fields['ctl00$MemberToolsContent$btnLogin'] = "Login"

$response = Invoke-WebRequest -Uri 'https://myaccount.amcom.com.au/ClientLogin.aspx' -WebSession $RequestForm -Method POST -Body $r.Forms[0].Fields -ContentType 'application/x-www-form-urlencoded'
$response2 = Invoke-WebRequest -Uri 'https://myaccount.amcom.com.au/FibreUsageDetails.aspx' -WebSession $RequestForm

Answer 1

import requests
import re
from bs4 import BeautifulSoup

user="xyzmohsin"
passwd="abcpassword"

s=requests.Session()
headers={"User-Agent":"Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36"}
s.headers.update(headers)

login_url="https://myaccount.amcom.com.au/ClientLogin.aspx"
r=s.get(login_url)
soup=BeautifulSoup(r.content)
RadMasterScriptManager_TSM=soup.find(src=re.compile("RadMasterScriptManager_TSM"))['src'].split("=")[-1]
EVENTTARGET=soup.find(id="__EVENTTARGET")['value']
EVENTARGUMENT=soup.find(id="__EVENTARGUMENT")['value']
VIEWSTATE=soup.find(id="__VIEWSTATE")['value']
VIEWSTATEGENERATOR=soup.find(id="__VIEWSTATEGENERATOR")['value']


data={"RadMasterScriptManager_TSM":RadMasterScriptManager_TSM,
"__EVENTTARGET":EVENTTARGET,
"__EVENTARGUMENT":EVENTARGUMENT,
"__VIEWSTATE":VIEWSTATE,
"__VIEWSTATEGENERATOR":VIEWSTATEGENERATOR,
"ctl00_TopMenu_RadMenu_TopNav_ClientState":"",
"ctl00%24MemberToolsContent%24HiddenField_Redirect":"",
"ctl00%24MemberToolsContent%24txtUsername":user,
"ctl00%24MemberToolsContent%24txtPassword":passwd,
"ctl00%24MemberToolsContent%24btnLogin":"Login"}

headers={"Content-Type":"application/x-www-form-urlencoded",
"Host":"myaccount.amcom.com.au",
"Origin":"https://myaccount.amcom.com.au",
"Referer":"https://myaccount.amcom.com.au/ClientLogin.aspx"}

r=s.post(login_url,data=data,headers=headers)

我没有用户名和密码，因此无法测试最终发布请求中的标头。如果不起作用 - 请从最终发布请求的标头中删除

Host

、

Origin

和

Referer

。

希望有帮助:-)