HashDistributionSpec: + See https://iceberg.apache.org/docs/1.6.0/spark-writes/#writing-distribution-modes Using write.distribution-mode - should create the appropriate number of Iceberg files + Given a table that is created with: CREATE TABLE polaris.my_namespace.HashDistributionSpec ( id int, label String, partitionKey long, date Date, timestamp Timestamp ) USING iceberg TBLPROPERTIES ( 'format-version' = '2', 'write.distribution-mode' = 'hash' ) PARTITIONED BY (partitionKey); + And it has 20 rows over 5 data files when writing with 4 executor threads + When we add another 20 rows of the same data that is logically distributed over 5 partitions + Then there are now 5 more data files + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + Run completed in 26 seconds, 514 milliseconds. Total number of tests run: 30 Suites: completed 15, aborted 0 Tests: succeeded 30, failed 0, canceled 0, ignored 4, pending 0 All tests passed.