我想做的事根據 df1 中的聚類定義將輸入文件 df0 轉換為所需的輸出 df2我擁有的df0 = spark.createDataFrame([('A',0.05),('B',0.01),('C',0.75),('D',1.05),('E',0.00),('F',0.95),('G',0.34), ('H',0.13)],("items","quotient"))df1 = spark.createDataFrame([('C0',0.00,0.00),('C1',0.01,0.05),('C2',0.06,0.10), ('C3',0.11,0.30), ('C4',0.31,0.50), ('C5',0.51,99.99)],("cluster","from","to"))我想要的是df2 = spark.createDataFrame([('A',0.05,'C1'),('B',0.01,'C1'),('C',0.75,'C5'),('D',1.05,'C5'),('E',0.00,'C0'),('F',0.95,'C3'),('G',0.34,'C2'), ('H',0.13,'C4')],("items","quotient","cluster"))筆記編碼環境是 Palantir 中的 PySpark。為了簡化編碼，可以調整 DataFrame df1 的結構和內容：df1 告訴 df0 中的項目應該鏈接到哪個集群。提前非常感謝您的時間和反饋！
1 回答
白豬掌柜的
TA貢獻1893條經驗 獲得超10個贊
這是一個簡單的左連接問題。
df0.join(df1, df0['quotient'].between(df1['from'], df1['to']), "left") \
.select(*df0.columns, df1['cluster']).show()
+-----+--------+-------+
|items|quotient|cluster|
+-----+--------+-------+
| A| 0.05| C1|
| B| 0.01| C1|
| C| 0.75| C5|
| D| 1.05| C5|
| E| 0.0| C0|
| F| 0.95| C5|
| G| 0.34| C4|
| H| 0.13| C3|
+-----+--------+-------+
添加回答
舉報
0/150
提交
取消