我正在嘗試過濾 dask 數據幀以僅包含由字典定義的特定時間段，其中鍵是 ISO 區域，值是時間戳列表。這是一個修改過的字典作為例子。iso_region_dict = {'MISO-E':[Timestamp('2016-05-17 22:15:00'),Timestamp('2016-10-21 13:45:00'),Timestamp('2016-12-26 02:45:00')], 'CAISO':[Timestamp('2016-08-24 10:15:00'),Timestamp('2016-07-03 14:30:00'),Timestamp('2016-04-22 12:45:00')]}我的 dask 數據框看起來像這樣（timeseries_ddf）： building_id time electricity_cooling_kwh electricity_heating_kwh total_site_electricity_kwh iso_zone0 2 2016-01-01 00:15:00 0.0 0.0 4.082225 MISO-E1 2 2016-05-17 22:15:00 0.0 0.0 5.627103 MISO-E2 2 2016-10-21 13:45:00 0.0 0.0 21.547435 MISO-E3 2 2016-12-26 02:45:00 0.0 0.0 4.082225 MISO-E4 2 2016-10-21 14:00:00 0.0 0.0 21.547435 MISO-E完整的數據框有數千個建筑 ID，“時間”列采用日期時間格式，范圍從 2016-1-1 到 2016-12-31，每個 building_id 的間隔為 15 分鐘。我想過濾此數據框以僅包含在 iso_region_dict 中為每個 building_id 定義的時間列中的時間戳。這是一個非常大的數據框，這就是我使用 dask 的原因。期望的輸出（timeseries_discharge_ddf）：building_id time electricity_cooling_kwh electricity_heating_kwh total_site_electricity_kwh iso_zone 0 2 2016-05-17 22:15:00 0.0 0.0 5.627103 MISO-E 1 2 2016-10-21 13:45:00 0.0 0.0 21.547435 MISO-E 2 2 2016-12-26 02:45:00 0.0 0.0 4.082225 MISO-E我用時間戳列表做了類似的事情，我只是通過以下方式過濾：timeseries_discharge_ddf = timeseries_ddf.map_partitions(lambda x: x[x.time.isin(discharge_timestamps)])我現在嘗試實現的附加步驟是這個過濾器，但是 discharge_timestamps 列表的變化取決于 iso_zone 是什么。
1 回答
BIG陽
TA貢獻1859條經驗 獲得超6個贊
我認為在這里使用 merge 或 join 會更容易。
數據
import pandas as pd
import dask.dataframe as dd
diz_df = {'building_id': {0: 2, 1: 2, 2: 2, 3: 2, 4: 2},
'time': {0: '2016-01-01 00:15:00',
1: '2016-05-17 22:15:00',
2: '2016-10-21 13:45:00',
3: '2016-12-26 02:45:00',
4: '2016-10-21 14:00:00'},
'electricity_cooling_kwh': {0: 0.0, 1: 0.0, 2: 0.0, 3: 0.0, 4: 0.0},
'electricity_heating_kwh': {0: 0.0, 1: 0.0, 2: 0.0, 3: 0.0, 4: 0.0},
'total_site_electricity_kwh': {0: 4.082225,
1: 5.627103,
2: 21.547435,
3: 4.082225,
4: 21.547435},
'iso_zone': {0: 'MISO-E', 1: 'MISO-E', 2: 'MISO-E', 3: 'MISO-E', 4: 'MISO-E'}}
diz_filter = {'iso_zone': {0: 'MISO-E',
1: 'MISO-E',
2: 'MISO-E',
3: 'CAISO',
4: 'CAISO',
5: 'CAISO'},
'time': {0: '2016-05-17 22:15:00',
1: '2016-10-21 13:45:00',
2: '2016-12-26 02:45:00',
3: '2016-08-24 10:15:00',
4: '2016-07-03 14:30:00',
5: '2016-04-22 12:45:00'}}
df = pd.DataFrame(diz_df)
df_filter = pd.DataFrame(diz_filter)
# converting to datetime
df["time"] = df["time"].astype("M8")
df_filter["time"] = df_filter["time"].astype("M8")
使用pandas
df_out = pd.merge(df, df_filter, on=["time", "iso_zone"])
使用dask
df = dd.from_pandas(df, npartitions=2)
# It doesn't matter if the second dataframe is pandas or dask
# df_filter = dd.from_pandas(df_filter, npartitions=2)
df_out = dd.merge(df, df_filter, on=["time", "iso_zone"])
添加回答
舉報
0/150
提交
取消