Selenium，如何定位并单击特定按钮

Question

我正在使用 selenium 尝试抓取该网站中的产品列表： https://www.zonacriativa.com.br/harry-potter

但是，我无法获取完整的产品列表。该页面列出了 116 个产品，但一次只显示少数产品。如果我想查看其他产品，我需要点击底部的“Carregar mais Produtos”（加载更多产品）按钮几次才能获得完整列表。

我找不到这个按钮，因为它没有 id 并且它的类是一个巨大的字符串。我尝试了几种方法，例如下面的示例，但它们似乎不起作用。有什么建议吗？

driver.find_element("xpath", "//button[text()='Carregar mais Produtos']").click()

driver.find_element("css selector", ".vtex-button__label.flex.items-center.justify-center.h-100.ph5").click()

driver.find_element(By.CLASS_NAME, "vtex-button.bw1.ba.fw5.v-mid.relative.pa0.lh-solid.br2.min-h-small.t-action--small.bg-action-primary.b--action-primary.c-on-action-primary.hover-bg-action-primary.hover-b--action-primary.hover-c-on-action-primary.pointer").click()

Answer 1

您尝试单击的元素最初位于可见屏幕之外，因此您无法单击它。另外，这个 XPath 至少对我来说没有找到该元素。
您需要做的就是向下滚动页面，直到该按钮变得可见且可单击，然后单击它。
以下代码单击该按钮 1 次：

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)

url = "https://www.zonacriativa.com.br/harry-potter"
driver.get(url)
while True:
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(@class,'buttonShowMore')]//button"))).click()
        break
    except:
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

上面的代码可以简单地修改为滚动并单击该按钮，直到我们到达未显示该按钮的最新页面：

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC

options = Options()
options.add_argument("start-maximized")

webdriver_service = Service('C:\webdrivers\chromedriver.exe')
driver = webdriver.Chrome(options=options, service=webdriver_service)
wait = WebDriverWait(driver, 5)

url = "https://www.zonacriativa.com.br/harry-potter"
driver.get(url)
while driver.find_elements(By.XPATH, "//div[contains(@class,'buttonShowMore')]//button"):
    try:
        wait.until(EC.element_to_be_clickable((By.XPATH, "//div[contains(@class,'buttonShowMore')]//button"))).click()
    except:
        driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")

Answer 2

我遇到了同样的问题，该按钮位于整个页面底部的某个位置，我需要向下滚动才能找到它。我使用了这些功能。

from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.webdriver import WebDriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

import time

def scroll_to_top_of_page(driver: WebDriver):
    driver.execute_script("window.scrollTo(0, 0);")
    print("Scrolled to top of page")

def scroll_down_until_find_element(driver: WebDriver, xpath: str):
    wait = WebDriverWait(driver, 5)
    print(f"Scrolling down until find element: {xpath}")
    while True:
        try:
            wait.until(EC.visibility_of_element_located((By.XPATH, xpath)))
            print(f"Found element: {xpath}")
            break
        except:
            driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
            print("Scrolling down...")
            time.sleep(0.5)

然后您可以搜索该按钮并单击它

button_xpath = "<button_xpath>"
scroll_to_top_of_page(driver)
scroll_down_until_find_element(driver, button_xpath)
button = driver.find_element(By.XPATH, button_xpath)
button.click()

这个答案的灵感来自于@Prophet 的答案。谢谢你。