What’s new for Data Analytics in the AI driven, autonomous and agentic era

Today’s fast-paced environment demands more than just data access; it requires a real-time data activation flywheel. A new reality is emerging where AI, infused directly into the data landscape, works hand-in-hand with intelligent agents. These agents act as catalysts, unlocking insights for everyone, and enabling the autonomous, real-time action that’s critical for success. Google’s Data & AI Cloud is built to power this flywheel, bringing AI to data for continuous, real-time data activation — a focus that’s attracting 5x more organizations to BigQuery than the two leading cloud companies that exclusively offer data warehouse and data science platforms.

Examples of leading companies include:

• Radisson Hotel Group boosted campaign productivity by 50% and revenue by over 20% using Gemini model fine-tuning with BigQuery. 

• Gordon Food Service used BigQuery to unify 170+ data sources, creating a scalable modern data architecture and an AI-ready foundation. This improved real-time response to critical business needs, enabled comprehensive analytics, driving significant customer adoption increase for their ordering applications, providing timely insights to their employees, while reducing costs and gaining market share. 

• J.B. Hunt is transforming logistics for shippers and carriers by consolidating fragmented systems, including Databricks, onto a unified BigQuery platform. 

• General Mills saves over $100 million using BigQuery and Vertex AI by giving employees secure access to LLMs to answer questions based on structured and unstructured data.

“We didn’t just need a place to store or consume data, we wanted a collaborator that could help us scale the most advanced data management in the industry.” – Jaime Montemayor, Chief Digital & Technology Officer, General Mills

Today, we’re announcing several new innovations with our autonomous data to AI platform powered by BigQuery, alongside our unified, trusted, and conversational BI platform with Looker:

1. Specialized agents for every user: New assistive and agentic experiences, grounded in your trusted data and available in BigQuery and Looker, are set to simplify and accelerate the work of data engineers, data scientists, analysts and business users.

2. Accelerating data science and advanced analytics: We are enhancing data science workflows in BigQuery with new AI-assisted notebooks and unlocking new insights with our BigQuery AI Query Engine, alongside seamless integration with real-time and open-source technologies.

3. Autonomous data foundation: New autonomous capabilities in BigQuery capture, manage, and orchestrate all data types, including native support for unstructured data handling and open data formats like Iceberg.

Let’s take a deeper look at each of these developments.

1. Specialized agents for every user

We believe AI should be accessible to everyone. We have made AI driven assistive experiences broadly available in BigQuery and Looker, and now we have expanded to specialized agents that best meet the needs for all data roles, including:

• Data engineering agent capabilities, embedded in BigQuery pipelines (GA), deliver support to build data pipelines, perform data preparation (GA) like transformation and enrichment of data, maintain data quality with anomaly detection (preview), and automate metadata generation. Traditionally data engineers spend countless hours cleaning, transforming, and validating data — these agents replace tedious and time consuming tasks and enable trusted data, boosting productivity of your data teams.

• Data science agent (GA), embedded within Google’s Colab notebook, enables every stage of model development. It automates feature engineering, provides intelligent model selection, enables scalable training, and faster iteration. This agent allows data science teams to focus on building advanced data science workflows, instead of wrestling with data and infrastructure.

• Looker conversational analytics (preview) empowers every user to interact with data using natural language. Expanded capabilities developed in partnership with DeepMind, not only conducts advanced analysis but also explains its thinking transparently, empowering all users to understand the agent’s behavior and seamlessly resolve ambiguities. In addition, Looker’s semantic layer improves accuracy by as much as two thirds. As users reference business terms like ‘revenue’ or ‘segments,’ the agent knows exactly what you mean and can calculate metrics in real-time, ensuring it delivers accurate, relevant, and trusted results. Additionally, we are launching a conversational analytics API (preview) for developers to build and embed conversational analytics into applications and workflow.

To power intelligence across assistive and agentic experiences in the BigQuery autonomous data to AI platform, we are also launching BigQuery knowledge engine (preview). It leverages the power of Gemini to analyze schema relationships, table descriptions, and query histories to generate metadata on the fly, model data relationships, and recommend business glossary terms. This knowledge engine is the foundation of AI powered experiences, including AI-powered data insights and semantic search (GA) across BigQuery, grounding AI and agents in business context.

Today all of our Gemini powered assistive and agentic experiences in BigQuery and Looker are available to all customers within the existing tiers of our pricing models — no add-ons!

2. Accelerating data science and advanced analytics

Our BigQuery autonomous data to AI platform is fundamentally changing how data scientists and analysts work, enabling new AI driven data science experiences and new engines to process complex data and support advanced analyses in real-time.

First, we’re supercharging the BigQuery notebook experience with AI. We are introducing intelligent SQL cells that understand your data’s context and provide smart suggestions as you write code and enable you to join data sources directly within your notebook. We are also adding native exploratory analysis and visualization capabilities, making it easy to explore data, as well as adding features to enable easier collaboration with colleagues. Data scientists can also schedule analyses to run and refresh insights periodically. And, to share insights more broadly across the organization, we are introducing the ability to build interactive data apps – dynamic, user-friendly interfaces powered by your notebook.

Building on this enhanced notebook environment, we are also announcing BigQuery AI query engine to support advanced, AI-driven analytics. This engine enables data scientists to move beyond simply retrieving structured data to seamlessly processing both structured and unstructured data together with added real-world context. The BigQuery AI query engine co-processes traditional SQL alongside Gemini to inject runtime access to real-world knowledge, linguistic understanding, and reasoning abilities. A data scientist can now ask questions like: “Which products in our inventory are primarily manufactured in countries with emerging economies?” The foundation model inherently knows which countries are considered emerging economies. Analysts can ask: “Which products are included in these social media images?” and our new engine processes the unstructured images and matches them to your product catalog. This engine supports a broad range of use cases, including building richer features for models, performing nuanced segmentation, and uncovering insights previously out of reach.

Furthermore, we empower users with the best of the open-source ecosystem, enhanced for the cloud. Google Cloud for Apache Kafka (GA) facilitates real-time data pipelines for event sourcing, model scoring, messaging and real-time analytics, powering serverless execution of Apache Spark workloads within BigQuery (preview). Customer use of our serverless Spark capability has nearly doubled in the past year, and we have enhanced this engine to provide 2.7x faster processing than the prior year.

BigQuery allows data scientists to leverage the tools they need on Google’s serverless and scalable architecture, whether it’s SQL, Spark, or the semantic power of foundation models, enabling faster innovation without traditional infrastructure challenges.

“We see SQL and Spark as two complementary ways of accessing and transforming data. Spark is especially useful to us in use cases that require complex business logic, which although niche, are extremely business-critical. Having an unified platform for SQL, Spark and AI, with the development experience in notebooks will considerably simplify these critical use cases.” – Andrés Sopeña Pérez, Head of Data and AI, Trivago 

3. An autonomous data foundation across the entire data lifecycle 

Underpinning our specialized agents and advanced analytics engines is an autonomous data foundation designed for the complexity of modern data. We are fundamentally changing the landscape by making unstructured data a first-class citizen within BigQuery. This is achieved with new capabilities in the platform including autonomous and invisible governance, orchestration for diverse data workloads, and a commitment to flexibility via open formats, ensuring your data is always ready for any data science or AI challenge. And we do all of this while minimizing operational overhead and delivering optimal price performance.

The biggest untapped opportunity for many organizations lies in the potential of their unstructured data. While structured data has pathways to analysis, unique insights embedded in images, audio, video, and text are often hard to extract and underutilized, and typically reside in siloed systems. BigQuery directly tackles this challenge by making unstructured data a first-class citizen with multimodal tables (preview), allowing you to bring rich, complex data types alongside structured data for unified storage and querying. To effectively manage this comprehensive data estate, our enhanced BigQuery governance (preview) provides a single, unified view for data stewards and professionals to handle discovery, classification, curation, quality, usage, and sharing, including automated cataloging (GA) and metadata generation (experimental). Moreover, to ensure timely insights from all your data streams, BigQuery continuous queries (GA) enable instant analysis and action on streaming data using SQL, regardless of its original format.

Our advanced support of multimodal data, both structured and unstructured, is driving adoption – customer use of Google’s AI models in BigQuery for multimodal analysis has grown by 16x year over year. Our integrated approach to data and AI is also cost effective, together BigQuery and Vertex AI are between 8-16x more cost efficient when compared to other independent data warehouse and AI platforms.

Our commitment to an open ecosystem remains paramount. BigQuery tables for Apache Iceberg (preview), delivers the flexibility of an open data lakehouse alongside the performance and integrated tooling of BigQuery, allowing you to connect your Iceberg data to SQL, Spark, AI and third party engines in an open and interoperable manner. This offering provides adaptive and autonomous table management, delivers high-performance streaming, auto-AI generated insights, near infinite serverless scale, and advanced governance. Through integration with Cloud storage, our managed service provides centralized fine-grained access control management and fail-safe capabilities.

Finally, the autonomous data to AI platform is self-optimizing. It scales resources, manages workloads, and helps ensure their cost-effectiveness with advanced workload management capabilities (GA). Furthermore, we’ve simplified purchasing with the new BigQuery spend commit (GA), unifying spend across our BigQuery platform, providing flexibility to move spend across data processing engines, streaming, governance, and more.

Get started on your data and AI journey by taking advantage of our BigQuery data migration offer. We can’t wait to learn about the ways you are innovating using data.