This article evaluates AWS S3 Table Buckets, a feature integrating Apache Iceberg into Amazon S3. It covers performance gains from data compaction, trade-offs between AWS-managed and self-managed catalogs, and integration limits with Athena, Redshift, Spark, and other query engines. The post highlights use cases for smaller teams or AWS-focused shops, discussing whether S3 Table Buckets are worth adopting for larger, multi-engine setups. Readers gain insights on key capabilities, constraints, and the future trajectory of this AWS service.
On that Reddit discussion they never mention S3 Table Buckets, but just tables on S3 buckets.
Also, my understanding is that dbt on Athena is not going to work very well, because dbt likes to create/drop tables when you do full refreshes and this is something that Athena cannot do with S3 Table Buckets
As a former maintainer of dbt-athena, I can tell you for sure that s3 buckets tables are not yet supported, no related PR in the last week's about that. Also, AWS made possible after the announcement to insert, update, delete via Athena (first weeks was yet not possible) , but table creation and metadata initialization it's still not possible natively via Athena, therefore some workaround is required.
There seem (stress on this word) to be folks writing to S3 on reddit FWIW they seem to use Glue/Athena:
https://www.reddit.com/r/dataengineering/comments/1hw9c0o/comment/m622opz/?context=3
Thanks for putting this article together.
Thanks for reading it.
On that Reddit discussion they never mention S3 Table Buckets, but just tables on S3 buckets.
Also, my understanding is that dbt on Athena is not going to work very well, because dbt likes to create/drop tables when you do full refreshes and this is something that Athena cannot do with S3 Table Buckets
Ah yes, that will be it
As a former maintainer of dbt-athena, I can tell you for sure that s3 buckets tables are not yet supported, no related PR in the last week's about that. Also, AWS made possible after the announcement to insert, update, delete via Athena (first weeks was yet not possible) , but table creation and metadata initialization it's still not possible natively via Athena, therefore some workaround is required.