我尝试在网站上使用 Scrapy 登录:https://starngage.com/plus/en-us/login.
但是 Scrapy 似乎找不到登录表单,即使当我检查登录表单元素时它就在那里
下面是我的代码:
import scrapy
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from scrapy.crawler import CrawlerProcess
from scrapy.http import FormRequest
class chnlids(scrapy.Spider):
name = "chnlids"
page_number = 0
def start_requests(self):
yield scrapy.Request(
url='https://starngage.com/plus/en-us/login',
callback=self.login
)
def login(self,response):
return scrapy.FormRequest.from_response(
response,
formdata={
'email': 'demo',
'password': 'demo'},
callback=self.parse)
def parse(self,response):
i = 0
if(self.page_number != 10):
self.page_number += 1
next_page = f'https://starngage.com/plus/en-us/influencer/ranking/youtube/all-countries?page={self.page_number}'
yield scrapy.Request(next_page, callback=self.parse,meta={'dont_redirect': True, 'handle_httpstatus_list': [302]})
for key in range(100):
yield{
"channel_id" : response.css("a.text-break::text")[i].get()
}
i += 1
我得到这个错误:
ValueError: No <form> element found in <200 https://starngage.com/plus/en-us/login>
我试图在开发工具中监控我的登录过程,这是我填写表格时手动登录的结果播放负载:
有人可以帮助我吗?无论答案是什么,我都非常感激和尊重