我目前正在尝试将json作为pandas数据帧处理。这里发生的事情是我得到了连续的json结构流。它们只是附加。这是一条完整的路线。我从中提取了一个.txt，想要通过熊猫分析它。
示例代码段：
{ “positionFlightMessage”：{ “messageUuid”： “95b3b6ca-5dd2-44b4-918a-baa51022d143”， “schemaVersion”： “1.0-RC1”， “时间戳”：1533134514， “FLIGHTNUMBER”： “DLH1601”， “位置”： { “航点”：{ “纬度”：44.14525， “经度”： - 1.31849}， “flightLevel”：340， “标题”：24.0}， “为messageSource”： “ADSB”， “flightUniqueId”：“AFR1601-1532928365-航空公司-0002" ， “airlineIcaoCode”： “AFR”， “atcCallsign”： “AFR89GA”， “燃料”：{}， “速度”：{ “地面速度”：442.0}， “高度”：{ “高度”：34000.0 }， “下一位置”：{ “航点”：{}}， “messageSubtype”： “ADSB”}} { “positionFlightMessage”：{ “messageUuid”： “884708c1-2fff-4ebf-b72c-bbc6ed2c3623”， “schemaVersion”： “1.0-RC1”， “时间戳”：1533134515， “FLIGHTNUMBER”： “DLH012”， “位置”：{ “航点”：{ “纬度”：37.34542， “经度”：143.79951}， “flightLevel”：320，”标题为 “：54.0}，” 为messageSource “：” ADSB”， “flightUniqueId”： “EVA12-1532928367航空公司-0096”， “airlineIcaoCode”： “DLH”， “atcCallsign”： “EVA012”， “燃料”：{} ， “速度”：{ “地面速度”：462.0}， “高度”：{ “高度”：32000.0}， “下一位置”：{ “航点”：{}}， “messageSubtype”：“ADSB “}} ...
正如你在这个光线中看到的那样，每个json都以{"positionFlightMessage":开头并以messageSubtype":"ADSB"结束
在json结束之后，下一个json会在它之后追加。
我需要的是一个表格，如下所示：
95b3b6ca-5dd2-44b4-918a-baa51022d143 1.0-RC1 1533134514 DLH1601 4.414.525 -131.849 340 24.0 ADSB AFR1601-1532928365-airline-0002 AFR AFR89GA 442.0 34000.0 ADSB
884708c1-2fff-4ebf-b72c-bbc6ed2c3623 1.0-RC1 1533134515 DLH012 3.734.542 14.379.951 320 54.0 ADSB EVA12-1532928367-airline-0096 DLH EVA012 462.0 32000.0 ADSB
我试图使用pandas读json但我得到一个错误。
import pandas as pd
df = pd.read_json("tD.txt",orient='columns')
df.head()
ValueError：尾随数据
tD.txt具有上面给出的片段，没有最后的（...）点
我认为问题是，每个json都是附加的。我可以在每次之后添加一个新行
messageSubtype":"ADSB"}}
然后阅读它，但也许你有一个解决方案，我可以直接转换大txt文件并轻松转换为df
尝试将json流输出如下：
注意起始'['和结尾']'。还要注意每个json输入之间的'，'。
data = [{
"positionFlightMessage": {
"messageUuid": "95b3b6ca-5dd2-44b4-918a-baa51022d143",
"schemaVersion": "1.0-RC1",
"timestamp": 1533134514,
"flightNumber": "DLH1601",
"position": {
"waypoint": {
"latitude": 44.14525,
"longitude": -1.31849
},
"flightLevel": 340,
"heading": 24.0
},
"messageSource": "ADSB",
"flightUniqueId": "AFR1601-1532928365-airline-0002",
"airlineIcaoCode": "AFR",
"atcCallsign": "AFR89GA",
"fuel": {},
"speed": {
"groundSpeed": 442.0
},
"altitude": {
"altitude": 34000.0
},
"nextPosition": {
"waypoint": {}
},
"messageSubtype": "ADSB"
}
}, {
"positionFlightMessage": {
"messageUuid": "884708c1-2fff-4ebf-b72c-bbc6ed2c3623",
"schemaVersion": "1.0-RC1",
"timestamp": 1533134515,
"flightNumber": "DLH012",
"position": {
"waypoint": {
"latitude": 37.34542,
"longitude": 143.79951
},
"flightLevel": 320,
"heading": 54.0
},
"messageSource": "ADSB",
"flightUniqueId": "EVA12-1532928367-airline-0096",
"airlineIcaoCode": "DLH",
"atcCallsign": "EVA012",
"fuel": {},
"speed": {
"groundSpeed": 462.0
},
"altitude": {
"altitude": 32000.0
},
"nextPosition": {
"waypoint": {}
},
"messageSubtype": "ADSB"
}
}]
现在你应该能够遍历json中的每个'list'元素并将它附加到pandas df。
print(len(data))
for i in range(0,len(data)):
#here is just show messageSource only. Up to you to find out the rest..
print(data[i]['positionFlightMessage']['messageSource'])
#instead of printing here you should append it to pandas df.
希望这会帮助你一点点。
现在这里是使用正则表达式的JSON解决方案。
s = '{"positionFlightMessage":{"messageUuid":"95b3b6ca-5dd2-44b4-918a-baa51022d143","schemaVersion":"1.0-RC1","timestamp":1533134514,"flightNumber":"DLH1601","position":{"waypoint":{"latitude":44.14525,"longitude":-1.31849},"flightLevel":340,"heading":24.0},"messageSource":"ADSB","flightUniqueId":"AFR1601-1532928365-airline-0002","airlineIcaoCode":"AFR","atcCallsign":"AFR89GA","fuel":{},"speed":{"groundSpeed":442.0},"altitude":{"altitude":34000.0},"nextPosition":{"waypoint":{}},"messageSubtype":"ADSB"}}{"positionFlightMessage":{"messageUuid":"884708c1-2fff-4ebf-b72c-bbc6ed2c3623","schemaVersion":"1.0-RC1","timestamp":1533134515,"flightNumber":"DLH012","position":{"waypoint":{"latitude":37.34542,"longitude":143.79951},"flightLevel":320,"heading":54.0},"messageSource":"ADSB","flightUniqueId":"EVA12-1532928367-airline-0096","airlineIcaoCode":"DLH","atcCallsign":"EVA012","fuel":{},"speed":{"groundSpeed":462.0},"altitude":{"altitude":32000.0},"nextPosition":{"waypoint":{}},"messageSubtype":"ADSB"}}'
import re
import json
replaced = json.loads('['+re.sub(r'{\"positionFlightMessage*', ',{\"positionFlightMessage', s)[1:] + ']')
dfTemp = pd.DataFrame(data=replaced)
df = pd.DataFrame()
counter = 0
def newDf(row):
global df,counter
counter += 1
temp = pd.DataFrame([row])
df = df.append(temp)
dfTemp['positionFlightMessage'] = dfTemp['positionFlightMessage'].apply(newDf)
print(df)
{"positionFlightMessage替换所有出现的,{"positionFlightMessage并丢弃第一个分隔符。