无法从亚马逊抓取产品价格

Question

def scrapedPage(URL, user_agent):
    if (not os.environ.get('PYTHONHTTPSVERIFY', '') and getattr(ssl, '_create_unverified_context', None)):
        ssl._create_default_https_context = ssl._create_unverified_context

    header = {"User-Agent": user_agent}

    proxy = {'https': 'using-proxy'}

    page = requests.get(URL, headers=header, proxies=proxy)


    return page

我无法刮掉产品的价格，即 12,300 卢比

user_agent = random.choice(user_agent_list)

pricelink='https://www.amazon.in/gp/offer-listing/B07NXTH1BD/ref=dp_olp_afts?ie=UTF8&condition=all'

new_page = scrapedPage(pricelink, user_agent)

new_soup = BeautifulSoup(new_page.content, 'html.parser')

print(new_soup.prettify())


find_price = new_soup.find_all('span',attrs={'style':'text-decoration: inherit; white-space: nowrap'})

我尝试了很多方法，如 findAll()、select()、find() 等，但我无法获得相同的价格。请帮帮我。

它返回空列表

Answer 1

男性关注 yout 代码的变化：

new_soup = BeautifulSoup(new_page.content, 'lxml') # html.parser > lxml

find_price = new_soup.select('span.olpOfferPrice > span') # change selector
print(find_price[0].text)

输出

Rs. 12,300.00