Modern Cloud Governance

It’s difficult to overstate the importance of cloud governance. Poorly-governed cloud instances not only expose businesses to undue security risks and untethered costs but also significantly limit the innovation potential provided by modern cloud infrastructure.

Digital platforms and online customer experiences have gone from being competitive differentiators to table stakes in every industry. For that reason, it’s essential to find a governance methodology that can provide the necessary cost and security controls while enabling agility and innovation to keep up in the marketplace.

IT leaders have the opportunity to deliver a secure, performant, and cost-effective cloud while simultaneously enhancing developer productivity if they are willing to rethink how the traditional configuration management database (CMDB) works and invest in tools that enable automated governance. 

How We Got Here

Traditional approaches to IT governance limit performance and security risk by imposing methodical processes for the allocation of finite IT resources. Amazon Web Services’ (AWS) foundational innovation was simply to increase developers’ productivity by eliminating time wasted securing infrastructure resources. AWS made resources available to developers with a simple API call. While this was a boon to developers, the approach was antithetical to most enterprise IT departments who jealously guarded limited infrastructure resources and sought to maintain strict security and performance standards. 

The early adopters of public cloud for the enterprise quickly found it to be a form of shadow IT, falling outside the scope of traditional governance structures and data center tools. Developers experienced greater productivity, but costs ran out of control and basic security was often overlooked. In an attempt to rein in the cost and risk of shadow IT, vendors began creating centralized cloud management platforms that created a choke point, which IT departments could use to control developers’ access to public cloud resources. 

The Problem with CMPs

The difficulty with using cloud management processes (CMPs) is that they create a “lowest common denominator” effect: If a CMP has to speak to multiple public cloud platforms, then it must provide a consistent set of capabilities across them all. This restricts the consumption of only those services that are common between cloud platforms. This was less of an issue when public cloud was largely defined by the use of virtual machines. However, PaaS and SaaS offerings are more prevalent today, and the differences between cloud providers has become greater while CMPs’ capabilities have lagged behind. This rise of CMPs and the lowest common denominator effect has removed the greatest benefit of the public cloud for developers: agility and innovation.

Infrastructure As Code: A New Solution — With a Caveat

When forward-thinking organizations began realizing that CMPs were holding them back, many turned to DevOps tools and techniques like Infrastructure as Code (IaC), which created standard deployment templates and configurations that could be source controlled with standard software development tools like Git. Organizations could now build new infrastructure with the confidence that it would be secure and compliant with governance policies — and developers could use well-understood tools and processes to build it.

However, using IaC still has a limiting effect on agility and innovation. Organizations must make a large investment up front to design and code the needed IaC before it can be used. Leveraging a new cloud service means investing in new IaC code or manually provisioning the service, which bypasses the governance provided by IaC. Infrastructure as code is an important evolution that enables developers to deploy cloud resources with consistency, but it is ineffective as a governance tool.

The Next Step

Moving forward, IT organizations must evolve from gatekeepers to innovation enablers. Instead of locking down the consumption of public cloud by means of a CMP or approved IaC tools, developers should be consuming cloud resources in the myriad of ways that cloud providers make possible. Allowing a wide variety of consumption options means developers can work with their preferred tools and use their preferred processes — regardless of their level of infrastructure sophistication. With some creative thinking and the proper tools, it’s possible to balance the required control for effective governance with the agility needed for innovation.

Rethinking the CMDB

The first critical change is to flip the role of the CMDB on its head. In an ideal form, CMDBs should be a perfect representation of reality (i.e., every IT resource methodically cataloged and perfectly represented as an entry in the database). However, the reality is most CMDBs are out of date and are missing significant amounts of critical information. They require constant manual upkeep to avoid drifting further from reality each day. The problem compounds when not all servers have the right agents, or the agents don’t gather all of the needed information, thus necessitating human intervention. 

On the other hand, public cloud provides all of the needed configuration information of any resource simply and consistently via its API. Rather than the CMDB telling the infrastructure how it should be configured, we should instead let the infrastructure tell the CMDB what is actually configured. By taking this “outside-in” approach, the CMDB is always (eventually) accurate, reflecting what services have been deployed and how they are configured — regardless of the tools used to provide those resources.

Eventually, the CMDB shifts to a consistent model because it can only know about a new instance after the fact. There are methods for also providing governance that limits out-of-policy instance creation before it happens, but that’s a subject worthy of its own blog post. With this reimagined CMDB in place, developers can consume anything they need in the cloud and the IT organization will know about it nearly instantaneously via the CMDB. 

Retooling Governance

The second critical change is to create a rules engine that can constantly monitor the CMDB for adherence to governance policies established by the organization. A universal rules engine can apply the same governance standards regardless of the cloud provider, enabling multi- and hybrid cloud operations. The scope and structure of the governance policies are unique to each organization, but three key pillars to think about when crafting a governance framework are security, cost-effectiveness, and availability/reliability. 

With a rules engine constantly scanning the CMDB for adherence to governance policies, traditional IT methods of control (e.g., change review boards and architectural committees) will become governance committees setting policy and designing controls. This methodology will also free IT teams from the Sisyphean task of creating the deployment or IaC templates needed to account for every developer platform and use case. Developers, by nature, will always try new services and new ways of leveraging existing services; if they have to wait for that service or pattern to be encoded by IT in a template or IaC code, they could face significant delays — or worse, the company may invest thousands or tens of thousands of dollars to create this new template only to find that it is never consumed again (i.e., another form of shadow IT).

Consider the case of a well-meaning developer who might manually create a new VM instance with a public IP address. In a well-designed organization that has automated governance, this will likely breach a governance policy and will be quickly identified in the CMDB by the rules engine. At this juncture, the VM could simply be deprovisioned and an alert sent to the user informing them of their error. 

Automated responses are not limited to security and compliance alone. Aspects like tagging can be enforced to ensure that costs can be appropriately allocated, and CMDB data can be combined with monitoring data to recommend instance rightsizing and identifying disused instances for termination. This approach to cloud governance can help create and maintain a cost-optimized cloud. 

Additionally, a clear and complete set of governance policies will cause audits to become trivial affairs. Compliance to requirements is codified in the rules engine and constantly evaluated against the configuration information in the CMDB. 

Getting the maximum value out of cloud requires eliminating sources of friction caused by legacy IT governance practices. Adopting a modern cloud governance mindset and tools will allow IT leaders to enable innovation while simultaneously delivering a secure, compliant, performant, and cost-effective cloud. At Maven Wave, our Managed Cloud Operations service provides cutting-edge thinking and tools that enable our clients to adopt a modern, automated approach to cloud governance. Contact one of our experts to learn more.

This article first appeared here as a post on the author’s personal LinkedIn page.