如何通过 data-testid 找到按钮并在 Selenium 中单击它？

Question

作为研究项目的一部分，我正在尝试废弃 https://charts.spotify.com/charts/view/citytoptrack-barcelona-weekly/2024-02-29 的数据。但是，当使用 Selenium 访问该页面时，会弹出登录登陆页面。

因此，要登录并访问原始页面，我想到使用以下代码复制人工单击“登录”按钮：

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service

website = 'https://charts.spotify.com/charts/view/citytoptrack-barcelona-weekly/2024-02-29'
path = '/home/isaac/chromedriver/chromedriver'
service = Service(executable_path=path)
driver = webdriver.Chrome(service=service)
driver.get(website)
driver.find_element(By.XPATH, value='//button[@class="Button-qlcn5g-0 jdERo"]').click()

作为按钮的HTML代码：

<div class="ChartsHomeHeader__HeaderLinks-vilr39-2 hQLnwL">
  <a data-testid="charts-login" href="https://accounts.spotify.com/login?continue=https%3A%2F%2Fcharts.spotify.com/login" style="margin-left:12px" class="Button-qlcn5g-0 jdERo">
    <div class="ButtonInner-sc-14ud5tc-0 iMWZgy encore-bright-accent-set">Log in</div>
    <div class="ButtonFocus-sc-2hq6ey-0 iPAQn"></div>
  </a>
 </div>

但是，当尝试找到“登录”按钮时，程序崩溃并返回此错误：

Traceback (most recent call last):

  File "/home/isaac/scrapingTest.py", line 10, in <module>

    driver.find_element(By.XPATH, value='//button[@class="Button-qlcn5g-0 jdERo"]').click()

  File "/home/isaac/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 741, in find_element

    return self.execute(Command.FIND_ELEMENT, {"using": by, "value": value})["value"]

  File "/home/isaac/.local/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 347, in execute

    self.error_handler.check_response(response)

  File "/home/isaac/.local/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 229, in check_response

    raise exception_class(message, screen, stacktrace)

selenium.common.exceptions.NoSuchElementException: Message: no such element: Unable to locate element: {"method":"xpath","selector":"//button[@class="Button-qlcn5g-0 jdERo"]"}

  (Session info: chrome=122.0.6261.111); For documentation on this error, please visit: https://www.selenium.dev/documentation/webdriver/troubleshooting/errors#no-such-element-exception

Stacktrace:

#0 0x56474bb1fe93 <unknown>

#1 0x56474b817ce6 <unknown>

#2 0x56474b862e48 <unknown>

#3 0x56474b862f01 <unknown>

#4 0x56474b8a63f4 <unknown>

#5 0x56474b884edd <unknown>

#6 0x56474b8a3899 <unknown>

#7 0x56474b884c53 <unknown>

#8 0x56474b855db3 <unknown>

#9 0x56474b85677e <unknown>

#10 0x56474bae57cb <unknown>

#11 0x56474bae97e5 <unknown>

#12 0x56474bad30e1 <unknown>

#13 0x56474baea372 <unknown>

#14 0x56474bab71bf <unknown>

#15 0x56474bb0e488 <unknown>

#16 0x56474bb0e683 <unknown>

#17 0x56474bb1f044 <unknown>

#18 0x7f45c5294ac3 <unknown>

我在网上查了很久也没找到解决的办法。如何才能成功找到并点击登录按钮？

我是网络抓取新手，如果这是一个非常基本的问题，我很抱歉。

此外，有什么方法可以访问https://charts.spotify.com/charts/view/citytoptrack-barcelona-weekly/2024-02-29来抓取其正在记录的数据，而无需复制日志-人类与硒的行为？也许我正在走一条艰难的路...

提前致谢:)

Answer 1

将解决方案代码留在这里，以防有人需要！:

from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

website = 'https://charts.spotify.com/charts/view/citytoptrack-barcelona-weekly/2024-02-29'
path = '/home/isaac/chromedriver/chromedriver'
service = Service(executable_path=path)
driver = webdriver.Chrome(service=service)
driver.get(website)
element = WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//a[@class='Button-qlcn5g-0 jdERo']")))
element.click();