Scrapy Cant Found the Login Form Error: raise ValueError(f"No <form> element found in {response}")

问题描述 投票:0回答:0

我尝试在网站上使用 Scrapy 登录:https://starngage.com/plus/en-us/login.

但是 Scrapy 似乎找不到登录表单,即使当我检查登录表单元素时它就在那里

login form in the inspect element

下面是我的代码:

import scrapy
from scrapy.spiders import CrawlSpider, Rule
from scrapy.linkextractors import LinkExtractor
from scrapy.crawler import CrawlerProcess
from scrapy.http import FormRequest

class chnlids(scrapy.Spider):
    name = "chnlids"
    page_number = 0
    
    def start_requests(self):
        yield scrapy.Request(
            url='https://starngage.com/plus/en-us/login',
            callback=self.login
        )
    
    def login(self,response):
      return scrapy.FormRequest.from_response(
            response,
            formdata={
              'email': 'demo', 
              'password': 'demo'},
            callback=self.parse)

          
    def parse(self,response):
      i = 0
      if(self.page_number != 10):
        self.page_number += 1
        next_page = f'https://starngage.com/plus/en-us/influencer/ranking/youtube/all-countries?page={self.page_number}'
        yield scrapy.Request(next_page, callback=self.parse,meta={'dont_redirect': True, 'handle_httpstatus_list': [302]}) 
        for key in range(100):
          yield{
              "channel_id" : response.css("a.text-break::text")[i].get()
          }
        i += 1

我得到这个错误:

ValueError: No <form> element found in <200 https://starngage.com/plus/en-us/login>

我试图在开发工具中监控我的登录过程,这是我填写表格时手动登录的结果播放负载: Login PlayLoad Login Headers

有人可以帮助我吗?无论答案是什么,我都非常感激和尊重

python web-scraping scrapy web-crawler
© www.soinside.com 2019 - 2024. All rights reserved.