我创建了一个脚本,使用请求模块从这个网页中抓取产品名称。当我运行脚本时,我可以看到状态代码为 200,但脚本没有带来任何结果。如何使用 requests 模块从网页中获取结果?
from bs4 import BeautifulSoup
import requests
link = "https://branddb.wipo.int/en/advancedsearch/results?sort=score%20desc&strategy=concept&rows=30&asStructure=%7B%22_id%22:%2262a3%22,%22boolean%22:%22AND%22,%22bricks%22:%5B%7B%22_id%22:%2262a4%22,%22key%22:%22type%22,%22value%22:%5B%22AO%22,%22EMBLEM%22,%22GI%22,%22INN%22,%22TRADEMARK%22%5D%7D%5D%7D&_=1722527941041&fg=_void_&start=0"
headers = {
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/126.0.0.0 Safari/537.36",
'Accept-Encoding': 'gzip, deflate, br, zstd',
'Accept-Language': 'en-US,en;q=0.9',
'referer': 'https://branddb.wipo.int/',
'origin': 'https://branddb.wipo.int',
}
res = requests.get(link,headers=headers)
print(res.status_code)
soup = BeautifulSoup(res.text,"lxml")
for item in soup.select("span.brandName"):
print(item.get_text())
请求不足以加载需要 JavaScript 的网站。相反,使用像 Playwright 或 Selenium 这样的库来下载 HTML,然后可以使用 Beautiful Soup 进行抓取。