从 Finra API 提取债券广度数据不起作用 – 为什么？

Question

我发现了以下关于量化交易的示例，关于如何从 Finra API 提取数据，在这个特定示例中，它对我来说效果很好

import warnings
warnings.filterwarnings('ignore')

import pandas as pd
import requests

#EXAMPLE FOR FIXED INCIME
#assign dataset to request
groupName = 'fixedIncomeMarket'
datasetName = 'treasuryWeeklyAggregates'


#build url

url = f'https://api.finra.org/data/group/{groupName}/name/{datasetName}'

#create headers
headers = {
    'Content-Type': 'application/json',
    'Accept': 'application/json'
    }

#create custom filter
customFilter = {
    "limit" : 5000    
        }


#make POST request
request = requests.post(url,headers=headers,json=customFilter)

#format to dataframe
data = pd.DataFrame.from_dict(request.json())

data

但是当我对我尝试拉取的数据集尝试相同的操作（具有以下参数）时，它不起作用：

groupName = 'fixedIncomeMarket'
datasetName = 'corporate144AMarketBreadth'

我收到此错误消息：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
Cell In[4], line 33
     30 request = requests.post(url,headers=headers,json=customFilter)
     32 #format to dataframe
---> 33 data = pd.DataFrame.from_dict(request.json())
     35 data

File ~/PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pandas/core/frame.py:1760, in DataFrame.from_dict(cls, data, orient, dtype, columns)
   1754     raise ValueError(
   1755         f"Expected 'index', 'columns' or 'tight' for orient parameter. "
   1756         f"Got '{orient}' instead"
   1757     )
   1759 if orient != "tight":
-> 1760     return cls(data, index=index, columns=columns, dtype=dtype)
   1761 else:
   1762     realdata = data["data"]

File ~/PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pandas/core/frame.py:709, in DataFrame.__init__(self, data, index, columns, dtype, copy)
    703     mgr = self._init_mgr(
    704         data, axes={"index": index, "columns": columns}, dtype=dtype, copy=copy
    705     )
    707 elif isinstance(data, dict):
    708     # GH#38939 de facto copy defaults to False only in non-dict cases
--> 709     mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
    710 elif isinstance(data, ma.MaskedArray):
    711     from numpy.ma import mrecords

File ~/PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pandas/core/internals/construction.py:481, in dict_to_mgr(data, index, columns, dtype, typ, copy)
    477     else:
    478         # dtype check to exclude e.g. range objects, scalars
    479         arrays = [x.copy() if hasattr(x, "dtype") else x for x in arrays]
--> 481 return arrays_to_mgr(arrays, columns, index, dtype=dtype, typ=typ, consolidate=copy)

File ~/PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pandas/core/internals/construction.py:115, in arrays_to_mgr(arrays, columns, index, dtype, verify_integrity, typ, consolidate)
    112 if verify_integrity:
    113     # figure out the index, if necessary
    114     if index is None:
--> 115         index = _extract_index(arrays)
    116     else:
    117         index = ensure_index(index)

File ~/PycharmProjects/pythonProject/venv/lib/python3.8/site-packages/pandas/core/internals/construction.py:645, in _extract_index(data)
    642         raise ValueError("Per-column arrays must each be 1-dimensional")
    644 if not indexes and not raw_lengths:
--> 645     raise ValueError("If using all scalar values, you must pass an index")
    647 if have_series:
    648     index = union_indexes(indexes)

ValueError: If using all scalar values, you must pass an index 

有人可以帮我获取我想要的数据吗？

这是API手册：https://developer.finra.org/docs（可以搜索我要找的广度数据，得到具体参数）

Answer 1

如果我们尝试打印 request.json()，可能存在授权问题

#make POST request
request = requests.post(url,headers=headers,json=customFilter)
print(request.json())

我们将会看到

{'statusCode': 401, 'statusDescription': 'Failed to authenticate, check your authentication token', 'message': 'Unauthorized', 'requestId': '123', 'resource': '/data/group/{group}/name/{name}'}

您可以从 finra 获得免费的 client_id 和 client_secret 并像这样使用它们

import warnings

warnings.filterwarnings("ignore")

import pandas as pd
import requests


def get_token(api_key, api_secret):

    bearer_token = requests.post(
        "https://ews.fip.finra.org/fip/rest/ews/oauth2/access_token?grant_type=client_credentials",
        auth=(api_key, api_secret),
    )
    return bearer_token.json()["access_token"]


groupName = "fixedIncomeMarket"
datasetName = "corporate144AMarketBreadth"


client_id = "YOUR_CLIENT_ID"
client_secret = "YOUR_CLIENT_SECRET"
token = get_token(client_id, client_secret)

url = f"https://api.finra.org/data/group/{groupName}/name/{datasetName}"

headers = {"Content-Type": "application/json", "Accept": "application/json", "Authorization": f"Bearer {token}"}

customFilter = {"limit": 5000}


request = requests.get(url, headers=headers, json=customFilter)

data = pd.DataFrame.from_dict(request.json())
data