Python 3：urllib.error.HTTPError：HTTP 错误 403：禁止

Question

这个问题已经在这里被问过几次，我已经阅读并尝试了大部分答案，但我还没有更接近解决方案。

当我在浏览器中手动输入 URL 时（我尝试过 Firefox、Edge 和 Opera），一切正常。

一旦启动脚本，我就会收到：“urllib.error.HTTPError: HTTP Error 403: Forbidden”

本来我没有在请求中设置标头，但现在标头已经存在，我仍然得到 403。 行：“with urllib.request.urlopen(f"{MP_URL}/api/assets/asset/{el['id']}") as f:"

代码：

import json
import os
import urllib.request

opener = urllib.request.build_opener()
opener.addheaders = [('User-Agent', 'Mozilla 5.0')]
urllib.request.install_opener(opener)

MP_URL = "https://www.unrealengine.com/marketplace"
JSON_DIR = "./data"
OUT_FILE = "mp_vault_list.md"


def main():
    mp_jsons = []
    cost = 0.
    disc = 0.
    currency = ""
    count = 0
    for f in os.listdir(JSON_DIR):
        if not f.endswith(".json"):
            continue
        with open(os.path.join(JSON_DIR, f), encoding="utf8") as fd:
            j = json.load(fd)
            count += len(j['data']['elements'])
            mp_jsons.append(j)
    with open(OUT_FILE, "w+", encoding="utf8") as fd:
        fd.write(f"# Vault content\n\n")
        i = 0
        for j in mp_jsons:
            for el in j['data']['elements']:
                i += 1
                print(f"\rProcessing Vault {i}/{count}", end="")
                # The ID found in the json dump is completely valid, and we have to resolve it to the urlSlug if we
                # want a nice and valid URL
                with urllib.request.urlopen(f"{MP_URL}/api/assets/asset/{el['id']}") as f:
                    jj = json.load(f)
                    jjd = jj['data']['data']
                    fd.write(f"* [{jjd['title']}]({MP_URL}/en-US/product/{jjd['urlSlug']})\n")
                    cost += jjd['priceValue']
                    currency = jjd['currencyCode']
                    try:
                        disc += jjd['discountPriceValue']
                    except KeyError:
                        pass

        fd.write(f"\n# Vault stats\n\n")
        fd.write(f"* Value: {cost / 100}{currency}\n")
        fd.write(f"* Current cost: {disc / 100}{currency}\n")
        fd.write(f"* Size: {count}\n")


if __name__ == '__main__':
    main()

本来我没有在请求中设置标头，但现在标头已经存在，我仍然得到 403。

data文件夹中的JSON文件： {“status”：“确定”，“data”：{“elements”：[{“catalogItemId”：“c1889d04a19f42cd86341de462bdd64b”，“namespace”：“ue”，“categories”：null，“title”：“Tappy Chicken” ,"description":"Tappy Chicken 演示了如何使用蓝图创建 16 位风格的横向卷轴游戏！","seller":{"id":"o-aa83a0a9bc45e98c80c1b1c9d92e9e","noAi":false,"owner ":"311cffd7000046919e47765426990332","状态":"ACTIVE","已接受":[{"ns":"ue","custom":false}],"financeCheckExempted":true,"name":"Epic Games" ,"supportEmail":"[电子邮件受保护]"},"keyImages":[{"type":"缩略图","url":"https://cdn1.epicgames.com/ue/item/store_TappyChicken_thumbnail- 140x140-5f29f37cd10f728c6aaef67b811f2085.png","md5":"5f29f37cd10f728c6aaef67b811f2085","宽度":140,"高度":140,"大小":18856,"上传日期":"2014 -07-31T20:00:16.640Z"} ,{"type":"屏幕截图","url":"https://cdn1.epicgames.com/ue/item/store_TappyChicken_screenshot_3-1920x1080-0315ae3bfe61ea5a124dca24274620fc.png","md5":"0315ae3bfe61ea5a124dca242746 20fc","宽度": 1920,"高度":1080,"大小":225338,"上传日期":"2014-07-31T20:00:33.092Z"},{"type":"截图","url":"https:// cdn1.epicgames.com/ue/item/store_TappyChicken_screenshot_1-1920x1080-797505e1581ee9a86a55b3a88b40c28c.png","md5":"797505e1581ee9a86a55b3a88b40c28c","宽度":1920 ,"高度":1080,"尺寸":131210,"上传日期":" 2014-07-31T20:00:23.384Z"},{"type":"精选","url":"https://cdn1.epicgames.com/ue/item/store_TappyChicken_featured-476x246-0ff729b32eac8911749b350a02c26178.png", “md5”：“0ff729b32eac8911749b350a02c26178”，“宽度”：476，“高度”：246，“大小”：30643，“uploadedDate”：“2014-07-31T20：00：05.976Z”}，{“类型”：“屏幕截图","url":"https://cdn1.epicgames.com/ue/item/store_TappyChicken_screenshot_2-1920x1080-191ad0f829e90e1884bae1dad4fa5bcd.png","md5":"191ad0f829e90e1884bae1dad4fa5bcd","宽度" ：1920，“身高”：1080 ,"size":240490,"uploadedDate":"2014-07-31T20:00:45.565Z"},{"type":"截图","url":"https://cdn1.epicgames.com/ue /item/store_TappyChicken_screenshot_4-1920x1080-483f24f2a6672156176b16b17e29ae64.png","md5":"483f24f2a6672156176b16b17e29ae64","宽度":1920,"高度":1080,"大小":56107,"上传日期":"2014-07-31T20:00 :55.678Z"}],"缩略图":"https://cdn1.epicgames.com/ue/item/store_TappyChicken_thumbnail-140x140-5f29f37cd10f728c6aaef67b811f2085.png","learnThumbnail":"https://cdn1.epicgames.com/ ue/item/store_TappyChicken_featured-476x246-0ff729b32eac8911749b350a02c26178.png","捆绑包":false,"拥有":true,"可下载":true,"isCatalogItem":true,"commentRatingId":"c1889d04a19f42cd86341de46 2bdd64b","id":" 6af75ba1005e41d0bdacd277f6acf3a2","RatingId":"6af75ba1005e41d0bDACD277f6acf3a2","评级":{"targetId":"6af75ba1005e41d0bdacd277f6acf3a2","averageRating":0,"Rating5":0,"评级4":0,"评级3":0,"评级2 ":0,"评级1":0,"legacyRatingNum":2,"评级5百分比":0,"评级4百分比":0,"评级3百分比":0,"评级2百分比":0,"评级1百分比":0,"总计": 0},"已审核":false}]}} 在此处输入代码 在此输入代码

Answer 1

似乎我需要代理或其他什么，该网站阻止了我的请求。 :(