无法使用请求模块解析网页结果

问题描述 投票:0回答:1

我创建了一个脚本,使用请求模块从这个网页中抓取产品名称。当我运行脚本时,我可以看到状态代码为 200,但脚本没有带来任何结果。如何使用 requests 模块从网页中获取结果?

from bs4 import BeautifulSoup
import requests

link = "https://branddb.wipo.int/en/advancedsearch/results?sort=score%20desc&strategy=concept&rows=30&asStructure=%7B%22_id%22:%2262a3%22,%22boolean%22:%22AND%22,%22bricks%22:%5B%7B%22_id%22:%2262a4%22,%22key%22:%22type%22,%22value%22:%5B%22AO%22,%22EMBLEM%22,%22GI%22,%22INN%22,%22TRADEMARK%22%5D%7D%5D%7D&_=1722527941041&fg=_void_&start=0"

headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
    'Accept-Encoding': 'gzip, deflate, br, zstd',
    'Accept-Language': 'en-US,en;q=0.9',
    'referer': 'https://branddb.wipo.int/',
    'origin': 'https://branddb.wipo.int',
}
res = requests.get(link,headers=headers)
print(res.status_code)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select("span.brandName"):
    print(item.get_text())
python python-3.x web-scraping python-requests
1个回答
0
投票

请求不足以加载需要 JavaScript 的网站。相反,使用像 PlaywrightSelenium 这样的库来下载 HTML,然后可以使用 Beautiful Soup 进行抓取。

© www.soinside.com 2019 - 2024. All rights reserved.