merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example

merging small files automatically Small files is a long time issue with Apache Spark. Kyuubi can merge small files by adding an extra shuffle. Currently, Kyuubi supports handle small files with datasource queries to remove data from tables. DELETE FROM foo WHERE id >= 1 and id < 2; Taking MERGE INTO as a example, MERGE INTO target_table t USING source_table s ON t.id = s.id WHEN MATCHED AND s.opType = 'delete' WITH ( 'connector' = 'hudi', 'path' = 's3://bucket-name/hudi/', 'table.type' = 'MERGE_ON_READ' -- this creates a MERGE_ON_READ table, by default is␣ ˓→COPY_ON_WRITE ); Taking Query Data as a example