you tend to enable the feature. 52 Chapter 3. What’s Next Kyuubi, Release 1.9.0-SNAPSHOT Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example 0-SNAPSHOT Tips Turn on AQE by default can significantly improve the user experience. Other sub-features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

tes if you tend to enable the feature. 3.3. Deploying Kyuubi 51 Kyuubi, Release 1.8.0 Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example Release 1.8.0 Tips Turn on AQE by default can significantly improve the user experience. Other sub-features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

tes if you tend to enable the feature. 3.3. Deploying Kyuubi 51 Kyuubi, Release 1.8.0 Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example Release 1.8.0 Tips Turn on AQE by default can significantly improve the user experience. Other sub-features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

es if you tend to enable the feature. 52 Chapter 3. What’s Next Kyuubi, Release 1.8.1 Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example Release 1.8.1 Tips Turn on AQE by default can significantly improve the user experience. Other sub-features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

tes if you tend to enable the feature. 3.3. Deploying Kyuubi 51 Kyuubi, Release 1.8.0 Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example Release 1.8.0 Tips Turn on AQE by default can significantly improve the user experience. Other sub-features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

should also increase skewedPartitionThresholdInBytes if you tend to enable the feature. Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example hold=-1 Tips Turn on AQE by default can significantly improve the user experience. Other sub- features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

should also increase skewedPartitionThresholdInBytes if you tend to enable the feature. Hidden Features spark.sql.adaptive.enabled=true spark.sql.adaptive.skewJoin.enabled=true spark.sql.adaptive.skewJoin This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example enabled=true Tips Turn on AQE by default can significantly improve the user experience. Other sub- features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

should also increase skewedPartitionThresholdInBytes if you tend to enable the feature. Hidden Features DemoteBroadcastHashJoin Internally, Spark has an optimization rule that detects a join child with This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example hold=-1 Tips Turn on AQE by default can significantly improve the user experience. Other sub- features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

should also increase skewedPartitionThresholdInBytes if you tend to enable the feature. Hidden Features spark.sql.adaptive.enabled=true spark.sql.adaptive.skewJoin.enabled=true spark.sql.adaptive.skewJoin This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example enabled=true Tips Turn on AQE by default can significantly improve the user experience. Other sub- features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is

should also increase skewedPartitionThresholdInBytes if you tend to enable the feature. Hidden Features spark.sql.adaptive.enabled=true spark.sql.adaptive.skewJoin.enabled=true spark.sql.adaptive.skewJoin This optimization rule detects and converts a Join to an empty LocalRelation. Disabling the Hidden Features We can exclude some of the AQE additional rules if performance regression or bug occurs. For example enabled=true Tips Turn on AQE by default can significantly improve the user experience. Other sub- features are all enabled. advisoryPartitionSizeInBytes is targeting the HDFS block size minPartitionNum is