Cloud Storage provides simple, scalable, secure, and cost-effective object storage for customers to store, access, and manage their data. This year at Google Cloud Next ‘23, we presented a number of new Cloud Storage capabilities designed to help you serve your storage needs for mission-critical applications of all kinds, including AI/ML and data analytics workloads. Here’s a concise recap of all the new features and capabilities that we announced.
Optimized storage for AI/ML and data analytics
Our customers increasingly rely on object storage when running their data-intensive workloads (AI/ML, batch analytics, streaming analytics, and high performance computing). To help you with this critical and fast-growing area, we announced multiple capabilities in the areas of programmability, performance, and manageability.
Programmability – APIs for easy integration into your business
- Cloud Storage Fuse (generally available): Mount and access a cloud storage bucket as a local filesystem, optimized for AI/ML and GKE workloads (learn more)
- Pub/Sub to Cloud Storage subscriptions (GA): Simplify streaming data ingestion to your object storage buckets with just a few clicks (learn more)
- Transfer for HDFS (preview in Q4 ‘23): Use Storage Transfer Service to easily transfer petabytes of Hadoop/Spark workloads to Google Cloud
Performance – Optimal price/performance for your data-intensive workloads
- Cloud Storage client library transfer manager (preview): Improved read/write performance in client libraries by parallelizing uploads and downloads (learn more)
- Anywhere Cache (preview in Q4 ‘23): Elastically scalable zonal SSD read cache to minimize your egress bandwidth costs at predictable low latencies
- gRPC API (preview in Q4 ‘23): New Cloud Storage API option that provides more efficient routing for analytics workloads, reducing overall execution time
- Improved Hadoop connector (GA in Q4 ‘23): Improved write performance for Hadoop/Spark workloads on Cloud Storage via parallelization and disk buffering
Manageability – End-to-end data lifecycle tailored for ease of use and security
- Event-driven transfer (GA): Listen to event notifications to trigger Storage Transfer Service (learn more)
- Managed folders (preview in Q4 ‘23): Prefix/folder level fine-grained access control to enable simplicity and security for data-intensive workloads
Built for mission-critical workloads
No matter what workload you leverage on object storage, areas such as security, governance, availability, scalability, and management are important. Here are the capabilities we announced for mission-critical workloads of all kinds.
Intelligent storage – Easily understand and manage your data at scale
- Storage Insights datasets (preview in Q4 ‘23): Visualize and analyze your storage usage and trends within BigQuery. This supplements currently available Storage Insights inventory reports (learn more)
- Autoclassfor existing buckets (preview in Q4 ‘23): Automatically transition objects to different storage classes based on last access time, without needing to create a new bucket
Enterprise Security and Governance: Safeguard assets while meeting security, governance, regulation, and compliance requirements
- Custom audit information (GA): Additional headers to give user, application, and job context to audit logs (learn more)
- Object retention lock (in preview): WORM-compliant immutable object storage by setting retain-until policies on individual objects
- Assured Workloads(GA): Integrated and packaged security product controls to help meet compliance requirements in United States public sector with added support for ITAR & IL5 compliance regimes. Includes controls for data residency, data access and data sovereignty for data at rest, in use and in transit. (learn more)
- Custom organization policy (preview in Q4 ‘23): Custom guardrails authored and enforced by customers on GCP resources (bucket) to meet their organization governance requirements
Resilience and Scalability: Enable geo-redundancy while maintaining strict data residency controls, disaster recovery, and real-time monitoring
- Soft delete (preview in Q4 ‘23): Quickly recover your data from accidental deletion during a specified retention period
- Replication monitoring (preview in Q4 ‘23): Near real-time replication status is now provided for default and turbo replication in dual-region and multi-region buckets
- Additional dual-regions (GA): Flexible and performant geo-redundancy with new support for Canada and Australia regions (learn more)
To learn more about how to leverage Cloud Storage in your application, try out one of the new Google Cloud Jump Start Solution tutorials. For a list of all of Google Cloud Next ‘23 announcements, check out the Next ‘23 Wrap-Up.