Merge branch 'master' into doc-data-types

apache · Dec 2, 2024 · 1229c88 · 1229c88
2 parents 6cdef9a + a5cb687
commit 1229c88
Show file tree

Hide file tree

Showing 36 changed files with 1,089 additions and 173 deletions.
diff --git a/docs/content/flink/sql-alter.md b/docs/content/flink/sql-alter.md
@@ -78,6 +78,10 @@ If you use object storage, such as S3 or OSS, please use this syntax carefully,
 
 The following SQL adds two columns `c1` and `c2` to table `my_table`.
 
+{{< hint info >}}
+To add a column in a row type, see [Changing Column Type](#changing-column-type).
+{{< /hint >}}
+
 ```sql
 ALTER TABLE my_table ADD (c1 INT, c2 STRING);
 ```
@@ -99,6 +103,10 @@ otherwise this operation may fail, throws an exception like `The following colum
 ALTER TABLE my_table DROP (c1, c2);
 ```
 
+{{< hint info >}}
+To drop a column in a row type, see [Changing Column Type](#changing-column-type).
+{{< /hint >}}
+
 ## Dropping Partitions
 
 The following SQL drops the partitions of the paimon table.
@@ -185,6 +193,14 @@ The following SQL changes type of column `col_a` to `DOUBLE`.
 ALTER TABLE my_table MODIFY col_a DOUBLE;
 ```
 
+Paimon also supports changing columns of row type, array type, and map type.
+
+```sql
+-- col_a previously has type ARRAY<MAP<INT, ROW(f1 INT, f2 STRING)>>
+-- the following SQL changes f1 to BIGINT, drops f2, and adds f3
+ALTER TABLE my_table MODIFY col_a ARRAY<MAP<INT, ROW(f1 BIGINT, f3 DOUBLE)>>;
+```
+
 ## Adding watermark
 
 The following SQL adds a computed column `ts` from existing column `log_ts`, and a watermark with strategy `ts - INTERVAL '1' HOUR` on column `ts` which is marked as event time attribute of table `my_table`.

diff --git a/docs/content/flink/sql-ddl.md b/docs/content/flink/sql-ddl.md
@@ -203,8 +203,8 @@ Paimon will automatically collect the statistics of the data file for speeding u
 The statistics collector mode can be configured by `'metadata.stats-mode'`, by default is `'truncate(16)'`.
 You can configure the field level by setting `'fields.{field_name}.stats-mode'`.
 
-For the stats mode of `none`, we suggest that you configure `metadata.stats-dense-store` = `true`, which will
-significantly reduce the storage size of the manifest.
+For the stats mode of `none`, by default `metadata.stats-dense-store` is `true`, which will significantly reduce the
+storage size of the manifest. But the Paimon sdk in reading engine requires at least version 0.9.1 or 1.0.0 or higher.
 
 ### Field Default Value
 

diff --git a/docs/content/migration/iceberg-compatibility.md b/docs/content/migration/iceberg-compatibility.md
@@ -371,9 +371,26 @@ you also need to set some (or all) of the following table options when creating
       <td>String</td>
       <td>hadoop-conf-dir for Iceberg Hive catalog.</td>
     </tr>
+    <tr>
+      <td><h5>metadata.iceberg.manifest-compression</h5></td>
+      <td style="word-wrap: break-word;">gzip</td>
+      <td>String</td>
+      <td>Compression for Iceberg manifest files.</td>
+    </tr>
+    <tr>
+      <td><h5>metadata.iceberg.manifest-legacy-version</h5></td>
+      <td style="word-wrap: break-word;">false</td>
+      <td>Boolean</td>
+      <td>Should use the legacy manifest version to generate Iceberg's 1.4 manifest files.</td>
+    </tr>
     </tbody>
 </table>
 
+## AWS Athena
+
+AWS Athena may use old manifest reader to read Iceberg manifest by names, we should let Paimon producing legacy Iceberg
+manifest list file, you can enable: `'metadata.iceberg.manifest-legacy-version'`.
+
 ## Trino Iceberg
 
 In this example, we use Trino Iceberg connector to access Paimon table through Iceberg Hive catalog.