解析或分解数据框中的字典列表

Question

我有一个数据框，其中包含要解包的嵌套字典列表。

我需要从

priceHistory

以及

WaterConservation

和

EnergyEfficient

中列出的商品获取日期和价格。下面的示例只是一个更大的数据帧的两行，其中每个数据帧行的字典项数量不同。

df = pd.DataFrame(
    [[19, [{'priceChangeRate': 0, 'date': '2015-05-29', 'source': 'Public Record', 'postingIsRental': False, 'time': 1432857600000, 'sellerAgent': None, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 275, 'buyerAgent': None, 'event': 'Sold', 'price': 877205}], ['Low flow commode', 'Low flow fixtures', 'Water-Smart Landscaping'],''],
     [89, [{'priceChangeRate': 0.090909090909091, 'date': '2023-07-14', 'source': 'Public Record', 'postingIsRental': False, 'time': 1689292800000, 'sellerAgent': {'name': 'seller1', 'photo': {'url': 'https://sellerphoto1.jpg'}, 'profileUrl': '/profile/sellerprofile1/'}, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 308, 'buyerAgent': {'name': 'buyer1', 'photo': {'url': 'https://buyerphoto1.jpg'}, 'profileUrl': '/profile/buyerprofile1/'}, 'event': 'Sold', 'price': 1200000}, {'priceChangeRate': 0, 'date': '2015-08-20', 'source': 'Public Record', 'postingIsRental': False, 'time': 1440028800000, 'sellerAgent': None, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 50, 'buyerAgent': None, 'event': 'Sold', 'price': 195000}],'', ['Windows', 'Insulation', 'HVAC', 'Appliances', 'Lighting']]],
    columns=['id', 'priceHistory', 'WaterConservation', 'EnergyEfficient'])

我尝试了太多的东西，无法在这里列出，但这似乎是最有效的（只是为了得到

priceHistory

）（source）：

df = pd.concat(
    [
        df,
        df.pop("priceHistory").apply(
            lambda x: pd.Series({k: v for d in x for k, v in d.items()})
        ),
    ],
    axis=1,
)
print(df)

但我收到此错误： 类型错误：“float”对象不可迭代

Answer 1

您可以使用

pd.json_normalize

从

priceHistory

获取日期和价格信息。如果不需要，则删除

priceHistory

列，并将

data

和

price

连接到主 df。

例如：

import pandas as pd

df = pd.DataFrame(
    [[19, [{'priceChangeRate': 0, 'date': '2015-05-29', 'source': 'Public Record', 'postingIsRental': False, 'time': 1432857600000, 'sellerAgent': None, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 275, 'buyerAgent': None, 'event': 'Sold', 'price': 877205}], ['Low flow commode', 'Low flow fixtures', 'Water-Smart Landscaping'],''],
     [89, [{'priceChangeRate': 0.090909090909091, 'date': '2023-07-14', 'source': 'Public Record', 'postingIsRental': False, 'time': 1689292800000, 'sellerAgent': {'name': 'seller1', 'photo': {'url': 'https://sellerphoto1.jpg'}, 'profileUrl': '/profile/sellerprofile1/'}, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 308, 'buyerAgent': {'name': 'buyer1', 'photo': {'url': 'https://buyerphoto1.jpg'}, 'profileUrl': '/profile/buyerprofile1/'}, 'event': 'Sold', 'price': 1200000}, {'priceChangeRate': 0, 'date': '2015-08-20', 'source': 'Public Record', 'postingIsRental': False, 'time': 1440028800000, 'sellerAgent': None, 'showCountyLink': False, 'attributeSource': {'infoString2': 'Public Record', 'infoString3': None, 'infoString1': None}, 'pricePerSquareFoot': 50, 'buyerAgent': None, 'event': 'Sold', 'price': 195000}],'', ['Windows', 'Insulation', 'HVAC', 'Appliances', 'Lighting']]],
    columns=['id', 'priceHistory', 'WaterConservation', 'EnergyEfficient'])

price_history_df = pd.json_normalize(df['priceHistory'].explode().tolist(), sep='_')

df = df.drop('priceHistory', axis=1).join(price_history_df[['date', 'price']], how='left')

Answer 2

您可以使用：

s = df.pop('priceHistory').explode()
out = df.join(pd.json_normalize(s).set_index(s.index))

---

print (out)
   id                                  WaterConservation  \
0  19  [Low flow commode, Low flow fixtures, Water-Sm...   
1  89                                                      
1  89                                                      

                                     EnergyEfficient  priceChangeRate  \
0                                                            0.000000   
1  [Windows, Insulation, HVAC, Appliances, Lighting]         0.090909   
1  [Windows, Insulation, HVAC, Appliances, Lighting]         0.000000   

         date         source  postingIsRental           time  sellerAgent  \
0  2015-05-29  Public Record            False  1432857600000          NaN   
1  2023-07-14  Public Record            False  1689292800000          NaN   
1  2015-08-20  Public Record            False  1440028800000          NaN   

   showCountyLink  pricePerSquareFoot  buyerAgent event    price  \
0           False                 275         NaN  Sold   877205   
1           False                 308         NaN  Sold  1200000   
1           False                  50         NaN  Sold   195000   

  attributeSource.infoString2 attributeSource.infoString3  \
0               Public Record                        None   
1               Public Record                        None   
1               Public Record                        None   

  attributeSource.infoString1 sellerAgent.name     sellerAgent.photo.url  \
0                        None              NaN                       NaN   
1                        None          seller1  https://sellerphoto1.jpg   
1                        None              NaN                       NaN   

     sellerAgent.profileUrl buyerAgent.name     buyerAgent.photo.url  \
0                       NaN             NaN                      NaN   
1  /profile/sellerprofile1/          buyer1  https://buyerphoto1.jpg   
1                       NaN             NaN                      NaN   

     buyerAgent.profileUrl  
0                      NaN  
1  /profile/buyerprofile1/  
1                      NaN