Celebrating 10 Years of Google Cloud’s BigQuery: A Look Back

Congratulations BigQuery on celebrating 10 years! As an early entrant to the Google Cloud Partner ecosystem, we at Maven Wave have enjoyed seeing the platform grow and evolve from beta to the industry leader it is today. Let’s take a look back at the evolution of BigQuery over the past 10 years.

During the beta stage, though the product had a few rough edges, it was still quite ahead of its time by providing a robust and low-cost Platform as a Service (PaaS) cloud database. And we would know how early it was to the game – we remember having to explain to many companies what PaaS was! We have since seen it satiate the modern business appetite for collecting large amounts of data and highly performant querying.

In the early days of BigQuery, many companies became able to collect and curate large sets of data with then-emerging technologies in the Hadoop ecosystems but found out the business required performant interactive reporting against those larger datasets as well. Perhaps because Google is at the root of that Hadoop “family tree,” BigQuery addressed both needs early on. Early adopters in cloud databases leveraged BigQuery’s separation of data storage and computing power to make big data accessible to the business for reporting.

BigQuery has had a proven roadmap that has not only been innovative over the last 10 years but has also fueled the needs of modern analytics by providing analysts, engineers, and data scientists with the newest toolsets to exploit the data. While cloud-native BigQuery is the centerpiece of the always growing and highly integrated Google Cloud Platform, the new wave set of tools can quickly be incorporated into data solutions.

Specifically, over the last few months, we’ve been able to collaborate with many healthcare clients by wrangling COVID-19 data from various sources. We’ve discovered advantages for our clients by leveraging many of the BigQuery integrated GCP family of products such as:

• Healthcare API: We’ve been working with research and healthcare clients to process patient data from the various Electronic Medical Record systems ( i.e. Epic and Cerner). The data can be easily processed with the Healthcare API as long as the data adheres to the HL7/FHIR guidelines
• Dataflow: Dataflow provides us powerful BigQuery ingestion and transformation for both batch and streaming workloads in one framework.
• BigQuery ML: BigQuery ML enables users to create and execute machine learning models in BigQuery by using an easy to learn SQL interface. Inside an intuitive and interactive web interface, a predictive model can be generated against any data living in the BigQuery ecosystem. This often serves as kickoff point to more detailed predictive modeling in GCP AI Platform, or in some cases stands on its own as useful business insight.
• Column Level Security: Patient Health and Personally Identifiable Information are easily controlled by BigQuery policies and group management We have found it easy to deploy robust data security models quickly and efficiently. The features typically surpass the minimum requirements of those teams in charge data sensitivity and loss prevention.
• Data Catalog: Data catalog provides automated metadata management out-of-the-box. Our teams are using Data Catalog as a best practice on all our projects. The fully managed service creates a foundation for data governance, compliance, and proving data lineage to the business for a nominal cost.

Many of our client’s data solutions are largely successful because the BigQuery product continues to provide accessible, relevant, affordable, integrated, and future-forward technologies. We are excited to see what happens as the BigQuery journey continues. Congratulations again on 10 exciting years!

If you’re interested in discovering how BigQuery can help transform your company’s analytics capabilities, Maven Wave can help. We hold the Google Cloud Data Analytics Partner Specialization, proving our expertise and success in building customer solutions in the data analytics field. Contact us to learn more.