在尝试网络抓取PrizePicks CS2道具时陷入困境

Question

我正在尝试从Prizepicks 中抓取CS2 道具，但它卡在一段代码上，我不知道如何修复这部分。我尝试使用 api 东西，但它对我来说效果不太好，所以我尝试从 app.prizepicks 中提取它。任何建议将不胜感激，因为我真的不知道还能做什么。

下面是代码：

from path import Path
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
import pandas as pd
import undetected_chromedriver as uc

############################################################################

# Initialize the WebDriver
driver = uc.Chrome()
driver.maximize_window()  # Open the window in full screen mode

############################################################################

# Scraping PrizePicks
driver.get("https://app.prizepicks.com/")
time.sleep(5)

# Waiting and closes popup
WebDriverWait(driver, 15).until(EC.presence_of_all_elements_located((By.CLASS_NAME, "close")))
time.sleep(5)
driver.find_element(By.XPATH, "/html/body/div[3]/div[3]/div/div/button").click()
time.sleep(3)

# Creating tables for players
ppPlayers = []

# It will click CS2 tab, if you want to scrape a different sport, change 'CS2' to the sport of your liking. 
driver.find_element(By.XPATH, "//div[@class='name'][normalize-space()='CS2']").click()
time.sleep(3)

# Waits until stat container element is viewable
stat_container = WebDriverWait(driver, 10).until(EC.visibility_of_element_located((By.CLASS_NAME, "stat-container")))

# Finding all the stat elements within the stat-container. (In this case, it's Map 1-2 Kills, Headshots, etc.)
categories = driver.find_element(By.XPATH, "//div[@role='button'][normalize-space()='MAPS 1-2 Headshots']").click()
projectionsPP = (By.CSS_SELECTOR, ".projection")
wait = WebDriverWait(driver, 25)
element_wait = wait.until(EC.presence_of_all_elements_located(projectionsPP))

# Creating tables for players
ppPlayers = []



############################################################################

# Data Scraping Automation begins here

for projection in projectionsPP:

    names = projection.find_element_by_xpath('.//div[@class="name"]').text
    value = projection.find_element_by_xpath('.//div[@class="presale-score"]').get_attribute('innerHTML')
    proptype = projection.find_element(By.CLASS_NAME, "text").get_attribute('innerHTML')

    players = {
            'Name': names,
            'Value': value,
            'Prop': proptype.replace("<wbr>", "")
        }

    ppPlayers.append(players)

dfProps = pd.DataFrame(ppPlayers)
dfProps.to_csv('CS2 Props.csv')

print("PrizePicks Props Offered: ", '\n')
print(dfProps)
print('\n')

driver.quit()

我希望它从 CS2 选项卡中提取 prop 信息并打印到 Excel 工作表，但是一旦我运行调试器，它就会转到 CS2 选项卡并卡在这个特定部分。每次都会超时，无论我增加多少时间，它都会卡在那里。

projectionsPP = (By.CSS_SELECTOR, ".projection")
wait = WebDriverWait(driver, 25)
element_wait = wait.until(EC.presence_of_all_elements_located(projectionsPP))

任何帮助将不胜感激。另外，我还不知道如何有效地使用 Stack Overflow，所以如果我搞砸了一些基本的东西，请给我恩典，谢谢！

Answer 1

刚刚注册了一个帐户，因为我做了一个类似的刮刀。对于抓取奖品精选，我建议在使用像 selenium 这样的库之前始终寻找 API 端点。 Prizepicks 是 https://api.prizepicks.com/projections，但是，我更好奇您计划如何抓取 CS2 数据。我祝你好运。