Structured and auditable changes to infrastructure

Note: I’m going to use AWS services as most of my examples for this post, but that’s just because I’m most familiar with them, the patterns found below are not limited to just AWS and can be applied to any cloud provider or self-hosted where similar patterns exist.

Introduction

Every service has some amount of supporting infrastructure required to support it. This includes any virtual servers (EC2 or other), storage (ex. S3, DynamoDB), load balancing, etc. basically any resources that your service uses that is not your direct business logic could be considered infrastructure. If you use continuous integration and change control on your business logic, then why would you not apply the same rules to your infrastructure?

Allowing and requiring developers to make changes using the UI introduces risk that one might make a mistake and bring down your production service. Continuing from my last post about infrastructure names, you could also make a mistake in any of regional clones.

CloudFormation

CloudFormation is service offered for all AWS accounts that enables you to represent any resource as a simple YAML or JSON file. Instead of making changes by clicking through the UI, you configure AWS resources in a YAML or JSON file and submit it to the CloudFormation console.

An example template might look like this:

1
2
3
4
5
Resources:
  WebsiteFileS3Bucket:
    Type: AWS::S3::Bucket
    Properties:
      AccessControl: Private

You then check this into your source control and apply it on the UI. This creates a new S3 bucket with an auto generated name. Now, you might want to enable static website hosting so you update the template with this:

1
2
3
4
5
6
WebsiteFileS3Bucket:
  Type: AWS::S3::Bucket
  Properties:
    AccessControl: Private
    WebsiteConfiguration:
      IndexDocument: index.html

This can be code-reviewed along with any other changes any other developers can discover any mistakes. Infrastructure as code reduces a set of clicks on the console to an easier to read configuration.

1
2
3
4
5
6
 WebsiteFileS3Bucket:
   Type: AWS::S3::Bucket
   Properties:
     AccessControl: Private
+    WebsiteConfiguration:
+      IndexDocument: index.html

Any issues detected will cause CloudFormation to automatically roll-back to the previously known good template version.

In summary, there’s a number of key benefits of describing your infrastructure as code:

  • Code reviewed - Spot and prevent mistakes from causing outages
  • Audited - Know exactly which engineer made a change to an AWS resource
  • Durable - Some mistakes are detected and rolled-back automatically by CloudFormation
  • Revision controlled - Can follow the history of your infrastructure and git blame as needed
  • Concise - Infrastructure as code is simpler to read and understand than a series of UI interactions or shell commands
Copyright - All Rights Reserved

Comments

Comments are currently unavailable while I move to this new blog platform. To give feedback, send an email to adam [at] this website url.