有没有办法滚动到页面上的最终产品？我只能从给定的 url 获取前 20 个产品

Question

我正在尝试从给定的网址“https://www.swiggy.com/instamart/city/gurgaon/c/fresh-vegetables?custom_back=true”获取并打印所有列出的产品，但目前面临滚动问题。它仅从给定链接获取前 20 个产品，其余的我无法获取我已尝试使用给定代码

 last_height = driver.execute_script("return document.body.scrollHeight")
#
# while True:
#     driver.execute_script("window.scrollTo(0, document.body.scrollHeight);")
#     time.sleep(20)  # Wait for the page to load new content
#     new_height = driver.execute_script("return document.body.scrollHeight")
#     if new_height == last_height:
#         break
#     last_height = new_height

Answer 1

我试了一下，成功了。

def scroll_down_to_last_element():
    # Wait for the elements to be visible
    wait = WebDriverWait(driver, 30)

    # Initialize an empty list to store elements
    elements = []

    # Scroll down until no new elements are loaded within 10 seconds
    start_time = time.time()
    while True:
        # Scroll to the last loaded element
        elements = wait.until(EC.presence_of_all_elements_located(
            (
            By.XPATH, "//button[@class='novMV' and contains(text(), 'double tap to go back to filter list')]")))

        # Get the last element
        last_element = elements[-1]

        # Scroll to the last element
        driver.execute_script("arguments[0].scrollIntoView();", last_element)

        # Wait briefly to allow new elements to potentially load
        time.sleep(1)

        # Check if new elements have stopped loading or if 10 seconds have