AWS is expanding Aurora PostgreSQL so users can analyze historical data stored in Amazon S3 directly from the database. The feature embeds DuckDB inside Aurora, allowing queries that combine live operational tables with data held in Apache Iceberg and Parquet formats without first copying it into the database. Applications continue to use existing PostgreSQL interfaces. The change follows AWS’s recent acquisition of the company behind DuckDB. During queries Aurora reads only the necessary data and columns from S3 and can cache frequently used data. Developers can see metrics such as rows scanned, data read from S3, and cache hits. AWS illustrated the capability with a financial example that joined seven days of recent transactions in Aurora with five years of history in a Parquet file on S3. External Iceberg catalogs can be linked via AWS Glue. There is no separate feature fee, though compute and S3 read costs may rise.
AWS is expanding Aurora PostgreSQL so users can analyze historical data stored in Amazon S3 directly from the database. The feature embeds DuckDB inside Aurora, allowing queries that combine live operational tables with data held in Apache Iceberg and Parquet formats without first copying it into the database. Applications continue to use existing PostgreSQL interfaces. The change follows AWS’s recent acquisition of the company behind DuckDB. During queries Aurora reads only the necessary data and columns from S3 and can cache frequently used data. Developers can see metrics such as rows scanned, data read from S3, and cache hits. AWS illustrated the capability with a financial example that joined seven days of recent transactions in Aurora with five years of history in a Parquet file on S3. External Iceberg catalogs can be linked via AWS Glue. There is no separate feature fee, though compute and S3 read costs may rise.