将 Pandas Dataframe 从多索引列转换为没有重复的单个索引

Question

我正在比较两个数据框并在值级别显示数据框之间的更改值

当数据帧中的值不同时，我将得到预期的结果，但是当数据帧相等时，我将获得多索引数据帧并尝试转换为没有重复列的普通数据帧

import pandas as pd
import numpy as np
before_df = pd.DataFrame(data= {'col1':[1,2,3,4], 'col2':['A','B','C','D'], 'amount': [12, 21,31,51]})
after_df = pd.DataFrame(data= {'col1':[1,2,3,4], 'col2':['A','B','C','D'], 'amount': [12, 21,31,51]})

config_index = ['col1']


before_df = before_df.set_index(config_index)
after_df = after_df.set_index(config_index)

before_keys = before_df.index
after_keys = after_df.index

common_keys = np.intersect1d(before_keys, after_keys, assume_unique=True)
common_columns = np.intersect1d(before_df.columns, before_df.columns, assume_unique=True)

before_common = before_df.loc[common_keys, common_columns]
after_common = after_df.loc[common_keys, common_columns]

common_data = pd.concat([before_common.reset_index(), after_common.reset_index()], sort=True)

changed_keys = common_data.drop_duplicates(keep=False)
changed_keys = changed_keys[config_index]

changed_keys = changed_keys.set_index(config_index).index
change_df = pd.concat([before_common.loc[changed_keys], after_common.loc[changed_keys]], 
 axis='columns', keys=['old', 'new'])

#print(change_df)

change_df_swap = change_df.swaplevel(axis='columns')

changed_df_group = change_df_swap.groupby(level=0, axis='columns')


changed_data = changed_df_group.apply(lambda frame: frame.apply(lambda x: x.iloc[0] if x.iloc[0] == x.iloc[1]  else f'{x.iloc[0]} ---> {x.iloc[1]}', axis='columns'))

print(changed_data)

输出

Empty DataFrame
Columns: [(amount, old), (col2, old), (amount, new), (col2, new)]
Index: []

我期望输出为

Empty DataFrame
Columns: [amount, col2]
Index: []

Answer 1

您应该执行以下操作：展平列并删除重复项。为此，请修改代码的最后步骤，以确保具有预期结构的平坦、干净的数据框。

import pandas as pd
import numpy as np

before_df = pd.DataFrame(data={'col1': [1, 2, 3, 4], 'col2': ['A', 'B', 'C', 'D'], 'amount': [12, 21, 31, 51]})
after_df = pd.DataFrame(data={'col1': [1, 2, 3, 4], 'col2': ['A', 'B', 'C', 'D'], 'amount': [12, 21, 31, 51]})

config_index = ['col1']

before_df = before_df.set_index(config_index)
after_df = after_df.set_index(config_index)

before_keys = before_df.index
after_keys = after_df.index

common_keys = np.intersect1d(before_keys, after_keys, assume_unique=True)
common_columns = np.intersect1d(before_df.columns, before_df.columns, assume_unique=True)

before_common = before_df.loc[common_keys, common_columns]
after_common = after_df.loc[common_keys, common_columns]

common_data = pd.concat([before_common.reset_index(), after_common.reset_index()], sort=True)

changed_keys = common_data.drop_duplicates(keep=False)
changed_keys = changed_keys[config_index]

changed_keys = changed_keys.set_index(config_index).index
change_df = pd.concat([before_common.loc[changed_keys], after_common.loc[changed_keys]],
                      axis='columns', keys=['old', 'new'])

change_df_swap = change_df.swaplevel(axis='columns')
changed_df_group = change_df_swap.groupby(level=0, axis='columns')

changed_data = changed_df_group.apply(lambda frame: frame.apply(
    lambda x: x.iloc[0] if x.iloc[0] == x.iloc[1] else f'{x.iloc[0]} ---> {x.iloc[1]}',
    axis='columns'
))

changed_data.columns = changed_data.columns.get_level_values(0)
changed_data = changed_data.loc[:, ~changed_data.columns.duplicated()]

print(changed_data)

这将为您提供预期的输出

Empty DataFrame
Columns: [amount, col2]
Index: []