[core] Support add partition for paimon #4323
Conversation
| @@ -273,6 +273,17 @@ default void invalidateTable(Identifier identifier) {} | |||
| void dropPartition(Identifier identifier, Map<String, String> partitions) | |||
| throws TableNotExistException, PartitionNotExistException; | |||
|
|
|||
| /** | |||
| * Create a partition for the specify table. | |||
| @@ -310,6 +310,27 @@ private Map<String, String> convertToProperties(Database database) { | |||
| return properties; | |||
| } | |||
|
|
|||
| * @throws PartitionNotExistException if the partition does not exist | ||
| */ | ||
| void createPartition(Identifier identifier, Map<String, String> partitions) | ||
| throws TableNotExistException, PartitionNotExistException; |
There was a problem hiding this comment.
When throw PartitionNotExistException?
| Identifier identifier = toIdentifier(tablePath); | ||
| catalog.createPartition(identifier, partitionSpec.getPartitionSpec()); | ||
| } else { | ||
| throw new UnsupportedOperationException(); |
| } catch (Catalog.TableNotExistException e) { | ||
| throw new CatalogException(e); | ||
| } catch (Catalog.PartitionNotExistException e) { | ||
| throw new RuntimeException(e); |
There was a problem hiding this comment.
Why throw RuntimeException?
| TableSchema tableSchema = getDataTableSchema(identifier); | ||
| if (!tableSchema.partitionKeys().isEmpty() | ||
| && new CoreOptions(tableSchema.options()).partitionedTableInMetastore()) { | ||
| try { |
There was a problem hiding this comment.
Add a warn log if tableSchema.partitionKeys().isEmpty() or tableSchema.options()).partitionedTableInMetastore() is false?
xuzifu666
changed the title
xuzifu666
changed the title
xuzifu666
changed the title
xuzifu666
changed the title
| catalog = ((CachingCatalog) catalog).wrapped(); | ||
| } | ||
|
|
||
| if (catalog instanceof HiveCatalog) { |
There was a problem hiding this comment.
Why need to check HiveCatalog?
| } | ||
| } | ||
| try { | ||
| if (catalog instanceof CachingCatalog) { |
There was a problem hiding this comment.
Why need to check CachingCatalog?
There was a problem hiding this comment.
Here catalog is CachingCatalog which wraped HiveCatalog, so handle it as CacheingCatalog firstly. @JingsongLi
|
|
||
| if (catalog instanceof HiveCatalog) { | ||
| Identifier identifier = toIdentifier(tablePath); | ||
| catalog.createPartition(identifier, partitionSpec.getPartitionSpec()); |
There was a problem hiding this comment.
Had move the logic to AbstractCatalog, so Catalog API not contains createPartition API.
| throws TableNotExistException { | ||
| TableSchema tableSchema = getDataTableSchema(identifier); | ||
| if (!tableSchema.partitionKeys().isEmpty() | ||
| && new CoreOptions(tableSchema.options()).partitionedTableInMetastore()) { |
There was a problem hiding this comment.
throw Unsupport exception if it is not partitioned table in metastore.
JingsongLi
changed the title
Yes, move to AbstractCatalog maybe better, would change as it. Thanks @JingsongLi |
Purpose
Currently paimon can not support add partition, due to add partition not need commit transaction，so make hms sync to achieve it.
Linked issue: close #xxx
Tests
API and Format
Documentation