如何防止 Selenium 在新内容加载之前过快单击“显示更多”按钮?

问题描述 投票:0回答:1

我正在尝试使用 Selenium 从足球网站抓取数据,我需要多次单击“显示更多”按钮才能加载所有比赛。该按钮在加载过程中消失并被加载器动画取代,但我的脚本在新内容完全加载之前连续单击该按钮的速度太快。这会导致多次点击而不实际加载数据。

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException
import time

service = Service("C:\\Users\\sddd\\Downloads\\chromedriver-win64\\chromedriver.exe")
driver = webdriver.Chrome(service=service)
driver.get("https://oddspedia.com/football")

try:
    # Accept cookies
    cookie_button = WebDriverWait(driver, 5).until(
        EC.element_to_be_clickable((By.XPATH, '//button[@class="cookie-popup__btn"]'))
    )
    cookie_button.click()
    print("Cookies accepted.")

    while True:
        try:
            show_more_button = WebDriverWait(driver, 5).until(
                EC.visibility_of_element_located((By.XPATH, '//div[@class="ml-show-more-btn"]/button'))
            )
            show_more_button.click()
            print("Clicked 'Show more' button.")

            WebDriverWait(driver, 2).until(
                EC.presence_of_element_located((By.CLASS_NAME, 'loader'))
            )

            WebDriverWait(driver, 10).until(
                EC.invisibility_of_element_located((By.CLASS_NAME, 'loader'))
            )

            time.sleep(1)  # Optional: slight pause to ensure content is ready
            print("Loading completed, ready for next click.")

        except TimeoutException:
            print("No more 'Show more' button or page loading timeout.")
            break

except NoSuchElementException:
    print("Element not found.")
finally:
    driver.quit()

如何修改此脚本,使 Selenium 等待新内容加载,然后再次单击“显示更多”按钮?有没有更好的方法来确保页面在尝试再次单击之前已完成加载新数据?

python selenium-webdriver web-scraping webdriver
1个回答
0
投票

基于此:

按钮在加载过程中消失,并被加载动画取代,

单击按钮后,尝试等待按钮消失,而不是等待加载器动画。

© www.soinside.com 2019 - 2024. All rights reserved.