New Feature
- Supports UDAF falling back.
- Supports native round-robin partitioner.
- Supports native range partitioner.
- Supports native WindowGroupLimitExec introduced in Spark-3.5.
- Supports SHJ falling back to SMJ when built side is too big.
- Fully supports to Apache Celeborn shuffle service.
- Initial supports to Apache Uniffle shuffle service.
- Initial supports to Apache Paimon datasource.
Improvement
Improved memory management in AggExec/SortMergeJoinExec, reducing number of OOMs. Imptoved metric statistics.
Bug fixes
- Fixed inconsistent string to data casting.
- Fixed inconsistent bloom filter join when bloom filter is generated by Spark.
- Fixed incorrect sort ordering when writing tables with dynamic partitions.
- Fixed inconsistent sha2x functions.
- Fixed a lot of bugs those might lead to query failure, see
What's Changed.