其中一个 HDFS (01) 扫描严重倾斜,impala 只分配了 9 台主机。我能以任何方式解决这个问题吗?
Operator #Hosts Avg Time Max Time #Rows Est. #Rows Peak Mem Est. Peak Mem Detail
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE 1 329.041us 329.041us 11.59K -1 4.75 MB 0 UNPARTITIONED
02:HASH JOIN 19 11.944ms 65.503ms 11.59K -1 34.10 MB 2.00 GB RIGHT OUTER JOIN, PARTITIONED
|--04:EXCHANGE 19 303.506us 467.216us 34.21K -1 432.00 KB 0 HASH(state_vectors_data4.lat,state_vectors_data4.lastcontact,state_vectors_data4.icao24)
| 01:SCAN HDFS 9 615.482ms 1s916ms 34.21K -1 126.49 MB 1.38 GB opensky.state_vectors_data4
03:EXCHANGE 19 277.767us 406.210us 65.33K -1 1.14 MB 0 HASH(position_data4.lat,position_data4.maxtime,position_data4.icao24)
00:SCAN HDFS 19 1s358ms 1s822ms 65.33K -1 248.28 MB 1.80 GB opensky.position_data4