AWS has added support for Iceberg materialized views to its managed cloud data warehouse service, Amazon Redshift, in an effort to help enterprises lower analytics costs.
Materialized views are precomputed, stored results of queries that data warehouses such as Redshift can use to avoid repeatedly running compute-intensive queries when the same analytical workloads are run repeatedly, reducing compute consumption and costs.
Redshift’s added support for Iceberg materialized views, which are query results stored in Iceberg tables, means enterprises with data spread across multiple warehouses and analytics engines such as Spark and Athena can reuse the same precomputed results from Redshift across those environments without having to replicate data or build separate pipelines, AWS wrote in a blog post.
The support also lets Redshift use Iceberg materialized views generated by other analytics engines, giving enterprises greater interoperability across data platforms, it added.
Added support could reduce integration complexity
That interoperability should reduce the integration burden and complexity on data engineering teams as they don’t have to “waste time” copying data and building ETL pipelines, said Pareekh Jain, principal analyst at Pareekh Consulting.
The reduced duplication of pipelines could also improve the efficiency and productivity of data teams across an enterprise, according to Amit Chandak, chief analytics officer at IT consulting firm Kanerika.
“Enterprise teams often calculate the same business metrics separately for dashboards and data science workloads, with each copy requiring its own pipeline. With support for Iceberg materialized views, the need for duplicate pipelines disappears, along with the confusion over which version of a metric is correct,” Chandak said.
That reduced confusion is also likely to help enterprises scale agentic applications.
With Iceberg materialized views, enterprises can now store the results of frequently used queries, such as business metrics, in an open table, reducing the risk of agents working with different versions of the same metric and improving consistency in their responses, Chandak said.
As enterprises deploy more agents across functions and use cases, that consistency can help them scale agentic applications with greater confidence, he added.
Reduction in analytics-related compute costs
For CIOs, however, the bigger benefit of the interoperability is the reduction in analytics-related compute costs, according to Jain.
“It is helpful particularly in multi-engine environments and agentic AI. If Redshift, Spark, Athena, and AI applications currently perform the same calculations separately, enterprises are paying for that compute multiple times. Iceberg materialized views allow Redshift to calculate once and let multiple systems reuse the result,” Jain said.
Those same savings could even extend to agentic applications or workloads, said Manoj Chandra Jha, principal analyst at Nord-IQ Research.
Interoperable materialized views can help enterprises avoid repeatedly computing the same metrics for different agentic applications, reducing the compute costs associated with agents repeatedly invoking the same underlying data queries as they reason through tasks, Jha said.
The added support also unlocks architectural flexibility for CIOs, according to Chandak.
Previously, optimization work was largely tied to the data engine that performed it, limiting its use across a heterogeneous data environment, Chandak said.
Iceberg materialized views, in contrast, will allow CIOs to reuse performance optimizations across analytics engines, making it easier for enterprises to add, switch, or combine data platforms as workloads evolve, he added. That should also reduce their dependence on any single analytics engine, he further said.
Read the original article: