使用 Selenium 从多个页面抓取数据

Question

我正在尝试从 Yahoo Fantasy Football 抓取 ADP 数据，并且正在努力让我的抓取工具移过初始页面（包含 30 名球员）。

尝试单击下一页时，我不断收到 NoSuchElementException。这里的专家可以提供的任何帮助将不胜感激。

以下是我一直在使用的代码：

Player_Name = []
Position_Team = []
Draft_Position = []

url = 'https://football.fantasysports.yahoo.com/f1/draftanalysis?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuYmluZy5jb20v&guce_referrer_sig=AQAAAGyHhLSemb5ibj1nJvdsNbQg14lWsna6Y9rAs7DlZ8-d_QA5SkDx_7bjMHMhdGMkQMOLR9F8a56RJ3Xq2zeuUu7QCCivRyHB3FAKv9Oi54Nk01Aev3KFisakR7REUsv42FuWxEKthpGw-BScwgVqzUdRc0a6oATq71C7j85X1C16'

driver.get(url)
time.sleep(1)

for i in range(3):
    players = driver.find_elements(by='xpath', value='//tr[contains(@data-tst, "table-row")]')
delay = 20
      WebDriverWait(driver,delay).until(EC.presence_of_element_located((By.XPATH,'//tr[contains(@data-tst, "table-row")]')))
    for player in players:
        Player_Name.append(player.find_element(by='xpath', value ='.//div[contains(@data-tst, "player-name")]').text)
        Position_Team.append(player.find_element(by='xpath', value = './/div[contains(@class, "C($text-secondary)")]').text)
        Draft_Position.append(player.find_element(by='xpath', value ='.//div[contains(@class, "Ta(c)")]').text)

        next = driver.find_element(by = 'xpath', value = '//*[@id="yui_3_18_1_1_1724006094920_810"]')
        next.click()

Answer 1

试试这个：

next = driver.find_element(By.CSS_SELECTOR, value = 'button[role="presentation"]:last-of-type')

您的选择器似乎不起作用，因为雅虎似乎在页面加载时随机化按钮的 ID。