At Beam, we’re building the next-generation digital social safety net. Our platform is used by state and local governments, community based organizations, and universities to administer public benefits and cash assistance programs. Our partners are able to drive low-friction application processes, tackle time-consuming eligibility determination processes, and disburse critical payments to vulnerable populations.
Recently, we migrated our entire infrastructure from AWS to Google Cloud with no downtime or impact to our users. Additionally, we transformed aspects of our application and engineering processes along the way to benefit from the newest cloud technologies. We are more effective and greener — all while cutting our cloud bill in half. In the following post, we’ll review Beam’s migration strategy, what we expected to get out of our move to Google Cloud, and some of our key results and takeaways.
Architecture decisions
When migrating to Google Cloud, Beam’s overall architectural strategy was to choose the services that would most effectively accelerate the integration of product-differentiating technologies, provide a scalable yet easy-to-understand cloud infrastructure, and introduce minimal administrative burden on the engineering team.
We want our engineers to focus on building a user-centered product with the features that our government partners need instead of supporting the outdated and inefficient infrastructure they used previously. This priority drove our decision to adopt Google Cloud services.
Cloud Run
Beam’s platform consists of many containerized apps that need to interact and independently scale to deliver services to users. The platform used to run in self-hosted AWS Kubernetes clusters, but the engineering team found that the administrative burden of the clusters was holding them back. Using AWS to implement Kubernetes didn’t provide the ease of management and developer-friendly UI that we needed to scale.
This is where Google Cloud’s managed container offerings really shine. Cloud Run was the perfect fit for our small team, which wanted to deploy many highly scalable, interrelated container services with the least administrative burden possible. If we needed more flexibility down the road, we could easily redeploy our services to GKE. And since Cloud Run now supports sidecars and multi-container services, there was no reason not to adopt this fully managed container service.
AlloyDB for PostgreSQL
Beam uses PostgreSQL for our core databases, and AlloyDB for PostgreSQL, Google Cloud’s fully managed PostgreSQL-compatible database, seemed like the perfect fit for our needs. Even though AlloyDB was a relatively new service when we began our migration, we wanted a highly scalable database with the ability to replicate across zones and regions for disaster recovery and high-volume reporting traffic — and AlloyDB fit that bill. Additionally, AlloyDB’s Query Insights dashboard seemed like it would close the database administration (DBA) gap and help us tune the performance of our application. Finally, we wanted a database product that didn’t require hand-holding and dedicated DBAs to manage, which would allow our teams to focus on innovation.
ML and AI services
The accelerated pace of improvement around machine learning and AI technologies has generated both intense excitement and trepidation. If implemented responsibly, AI stands to reduce the time tax and revolutionize the government’s ability to serve its people. Done carelessly, however, AI could amplify harm and bias and leave vulnerable populations further behind.
In this challenging environment, Beam’s responsible AI strategy is enabled by cutting-edge Google Cloud AI, such as Vertex AI. Our AI offerings aim to radically reduce administrative burden on applicants and overhead on program administration by streamlining the eligibility determination process from documents and case histories. Google Cloud enables us to spend more time on responsibility and equity considerations than we could otherwise.
The migration
I’ve done around half a dozen major infrastructure migrations in my career, and Beam’s migration from AWS to Google Cloud was the smoothest. In part, this was thanks to Beam’s awesome engineering team, but also due to how easy Google Cloud makes a cloud-to-cloud migration.
Here’s a summary of how we did it:
-
We stood up all our infrastructure with Terraform so everything would be documented, repeatable, and compliant going forward.
-
Using Database Migration Service, we created continuous replication from our PostgreSQL databases in AWS to AlloyDB instances.
-
Meanwhile, our developers focused on modernizing our service architecture with things like Pub/Sub for asynchronous job queueing and Cloud Storage for file uploads.
-
While we were migrating to Google Cloud, we rebuilt our DevOps processes around GitHub.
-
We used GitHub Actions to deploy our container services to Cloud Run as part of our new CI/CD pipelines.
-
We ran load tests on our Cloud Run services to ensure they would scale while keeping latency low for our end users.
-
On cutover day, we briefly paused writes to our databases, and we switched our DNS from the AWS endpoints to the Cloud Run URLs.
-
Due to the continuous data sync between our previous databases and AlloyDB, our users experienced no outages or data loss during the switch, and for us at Beam, it was a huge relief to be running our application in a more lightweight yet modern cloud platform.
Key results and takeaways
Six months later, we are thrilled with the performance of our platform rebuilt around Cloud Run and AlloyDB. With these improvements, we are on track to distribute hundreds of millions of dollars across program areas, including workforce development, child care, disaster recovery, guaranteed income, and housing.
Cloud Run just works — services scale up and down to match traffic patterns, and our deployments are easy to manage with GitHub Actions and container service definitions stored in yaml. When manual intervention is needed, for example, to route traffic between different application revisions, the Cloud Run UI is vastly superior to the container orchestration tools we previously used.
AlloyDB feels like having a team of DBAs working to keep our database healthy. Latency is shockingly low, even as our business has grown. With a few clicks in the console, we created a separate read-only pool for our data analytics and reporting needs, and we restored a point-in-time backup to a new cluster to retrieve a previous version of user data. And by purchasing committed use discounts, we’ve cut our database costs in half for the next three years.
Some more benefits we’ve seen include the following:
-
Security: Google prioritizes security in the areas important to Beam and its government partners. For example, default encryption at rest across all Google Cloud services, easy keyless service accounts with Workload Identity, and Security Command Center are some of the features that empower a small but security-focused engineering team.
-
Developer-friendly: Our developers find Google Cloud documentation easier to understand, and SDK integrations are straight-forward and quick compared to other cloud providers.
-
Innovation-enabling: Our cross-functional team was able to build effective AI models for pilot partners on top of Cloud Run and other services.
-
Socially-conscious and environmentally responsible: We’re proud to have Google Cloud as our strategic cloud partner for non-technical reasons too. Google Cloud’s sustainability strategy is the most ambitious of all the key cloud providers, and their philanthropic initiatives align with Beam’s mission and with the values of our partners and investors.
Most importantly, we’ve seen all these benefits for us and our partners while reducing our total monthly cloud costs by half!
Learn more about how Google Cloud can help your startup, and unlock resources at the recently launched Startup Learning Center.