Infrastructure as Code: Everything you ever wanted to know but were afraid to ask

When (not) to consider automated infrastructure

As our colleague likes to say Lukas Klášterskýif you are new to the cloud, "don't think big" and get to know it thoroughly first. On the other hand, if the cloud is already a standard for you, you're running several larger applications and everything is "running as it should", then maybe it's time to think about Infrastructure as Code (IaC).

So what is the main goal of IaC? As the name suggests, it is, have the infrastructure defined in a similar way to the application source code. The result is therefore IaC blueprint (or artifact) that describes "what this part of the infrastructure should look like". In simple terms, we can say that each part of the infrastructure has a clear blueprint that it defines: "This is what my database is supposed to look like." whether: "This is what my Redis Cluster."

Of course, you can argue that each component of each of your applications is different, individual. In that case, IaC is probably not for you. Highlights synergy effect across the whole environment if you have a clear defined architectural standards (and at the same time follow them), which you then implement using IaC.

However, if you have no standard or it is difficult to define one, there may be specific use-cases where IaC will make sense.

I'll be a bit ahead of you: this example (simplified) shows how such a IaC blueprint for database (in an AWS environment, blueprint defined using AWS CloudFormation):

Now we won't go into detail about what each part means, but I wanted to show you "what it really looks like" right at the beginning.

Why automate infrastructure or use cases

Okay, so we know roughly what IaC is, and we have a rough idea of what it might be good for. Let's take a look at specific use of IaC.

For those of you who prefer brevity to detailed examples, the summary is roughly as follows:

When to consider Infrastructure as a Code

• I have clearly defined infrastructure standards.
• I need to dynamically create individual application environments.
• There are frequent changes in the environment that I want to manage centrally.

When for me Infrastructure as a Code is rather not

• I am just getting acquainted with the cloud, I have rather basic experience with running a cloud environment.
• My infrastructure is primarily "static", without many changes.
• Application infrastructure is highly individual and it is not possible to define a general binding standard.

In general, we can say that Infrastructure as a Code is an integral part of cloud automation and automation itself is one of the main motivations for using the cloud.

Use case 1: Dynamic environment

This scenario typically involves development or testing environments. Why should I run a development environment permanently (and pay for it) when development needs it "only when it is deploying and testing something"? Or maybe I have multiple development teams working on the same application. To avoid "clashes" between them, I want to be able to give each team prepare an identical environment (and then delete it again when it's no longer needed).

If you define such an environment using IaC blueprints, you can dynamically deploy the entire environment and delete it when you no longer need it. And all this is completely automated within your CI/CD pipeline.

Use case 2: Infrastructure as an integral part of the application

This scenario supports one of the important aspects of cloud-native application development, which is parity between environments. Ideally, I want my the test environment was the same as the production (maybe smaller in terms of performance), but architecturally identical.

This approach greatly simplifies my search for potential bugs and the overall troubleshooting environment. The description of the infrastructure is therefore an integral part of the application delivery and I am able to deploy such an application into the target environment immediately.

In a moment we will say that there are different types of blueprints and one of them is blueprints for configuring the landing zone (cloud environment) itself. In this use case, the goal is to have all environments (from a general perspective, not in the context of the application itself) set up exactly identically - to enforce the standard.

For example, in all environments, a specific tagging policy, an audit trail is configured in all environments, and all environments "by default" have a backbone configured in some (identical) way.

Use case 3: Automated infrastructure lifecycle

Over time, standards are changed, expanded or otherwise modified. Thanks to IaC, I can make modifications in a simple way or other changes above all infrastructureand thus manage the entire infrastructure lifecycle. From its creation through all its changes to its final closure.

For example, I defined at the beginning that I want to have 14 daily backups for all databases and that each database can be accessed by administrators from a specific network range. Over time, the decision was made that it was necessary from a business perspective to maintain 30 backups, and at the same time the organization expanded to include another branch office whose employees also needed access to all databases.

I can in a simple way modify my original blueprint for the database, these requests to it Embed and then automated editing all databases created from this blueprint. Similarly, I can, for example, remove all components created based on this blueprint in bulk at any time.

Blueprints everywhere you look

As I have already hinted a moment ago, there are basically three basic categories of IaC blueprints:

• Infrastructure
• Configuration environment (landing zone)
• Security

Blueprints for application infrastructure

These blueprints are then further divided into:

• Generic
• Application-specific

Under generic blueprints application we can imagine for example the already mentioned database. It is represented by one single IaC blueprint and all applications use only it. Another example could be the deployment of a containerization infrastructure or a virtual server.

Application-specific blueprints are part of one specific application and are not used by another application. These can be, for example, application parameters stored in AWS System Manager Parameter Store or Azure AppConfig) or specific application secrets in AWS Secret Manager or Azure Vault.

Blueprints for environment configuration (landing zone)

I mentioned these blueprints a while ago. They are designed to configure the entire cloud environment and are always deployedregardless of what application is running within the environment.

These include, for example, components of the type Budgets (to have some mechanism for cost control and reporting deployed in each environment), backbone network configuration (e.g. standard routing tables to my Hub network, as described by Jakub Procházka in his article Untangling cloud networks) or it can be for example some serverless function responsible for forwarding security logs to the central SIEM environment.

Safety blueprints

This is a specific category of blueprints that are also deployed for every environmentbut it's within his purview security department. The purpose of security blueprints is to define security policies that control or directly enforce that my security requirements are met (typically using components such as AWS Config or Azure Policy).

Examples include compliance with tagging policywhether all components are really encrypted, if specific parameters of individual components are set in a specific way, etc.

You could argue that security blueprints are unnecessary because all parameters are set "as they should be" thanks to application blueprints. This is true, but only when such a blueprint is deployed. Over time, individual cloud resources may be modified (human error, unconscious change, etc.). The purpose of security blueprints is therefore to continuous control of the entire environment. They can therefore be seen as "control blueprints".

How it all plays together

So if you decide (and you should) to implement all three types of IaC blueprints, the result will look something like this:

First, the individual blueprints for landing zone configurations are deployed to the environment, then the security blueprints, and finally the individual infrastructure blueprints.

This will achieve a state where:

• each environment (in this case, the AWS environment) is deployed according to the global standard,
• my colleagues in the security department know that all the requirements of the compliance environment are implemented,
• the application itself is deployed in a clearly defined standard.

Blueprint lifecycle

We should definitely not forget that individual blueprints change and evolve over time as your internal architectural or security requirements change. It is therefore important to have a defined process for how blueprints change over time.

We treat them similarly to other program code.

Usually the resulting blueprints are stored in central repository (for example AWS S3 or Azure Storage Account), from which they are deployed. Under no circumstances should this repository allow direct modification of these blueprints!

Typical controlled and audited blueprint modification scenario is such that its individual versions are stored in some central source-code repository (for example GIT). When a new version of the blueprint is changed (commuted), the CI/CD pipeline is started, the first step of which is to validate the changes made (review process).

If the review is successful, the new version of the blueprint is deployed to the test environment, where it is tested that "everything is as it should be", and then the new version of the blueprint is finally made available in the central blueprint repository.

Desired state and why it is so important

Let's describe what it is desired state and why you should care. Desired state approach describes "target state of the object", i.e. that my (for example) database should look like "so and so".

It therefore uses declarative approach. I declare "as it should be". Which is a different approach than you may be used to - imperative approachwhere I define "what to configure".

An example of a declarative approach:

• My database should be called MyDb.
• The size of the database is to be 20 GB.
• The database performance is 2 CPUs and 4 GB RAM.
• The database disk is to be encrypted.

An example of the imperative approach:

• Create a database with a name MyDb.
• Configure the database MyDb so that it is 20 GB in size.
• Configure the database MyDb so that it has 2 CPUs and 4 GB of RAM.
• Encrypt the database disk MyDb.

The great advantage of declarative writing is that I have that desired state - that is, information on how it should be. If I have information about the target state, I am able to easily check the current status against the target at any time.

For example, at the beginning the database was encrypted, but over time someone (due to human error, for example) decrypted it. If I know the current and target state, I am able to detect this situation, including detailed "what is different than it should be" information.

Example of using desired state

Sample functionality AWS CloudFormation Drift, a technology that allows us to detect deviations from the target state:

Here at first glance I see there's been a modification Resources ManagementSecurityGroupwhich does not currently correspond to its target state. On the basis of this information, I have the possibility to perform, if necessary corrective actions.

AWS Cloud Formation Drifts is one of the tools falling under the category of continuous cloud compliance, which is a slightly broader area dedicated to the continuous evaluation of defined rules. This includes a plethora of tools - both native, for example Azure Policy, AWS Config or the aforementioned Cloud Formation Drifts, as well as external, for example, the tool CheckPoint CloudGuard.

Topic continuous cloud compliance is rather for a separate article - don't worry, it will be soon!

So what tools can I use?

If you've read this far, congratulations! So the concept of Infrastructure as a Code has probably caught your attention and now you're probably wondering how to actually implement it.

First the decision you have to make is whether you want to focus on just one specific cloud environment, or whether you want to manage everything from on-premise across multiple cloud environments with one tool.

If your goal is to be truly Hybrid, our recommendation is to use the tool Terraform from the company HashiCorp. This tool is extremely powerful, but from my perspective, its complex deployment can be a relatively complicated task. On the other hand, it will offer you a truly hybrid approach to IaC across any environment.

If, on the other hand, you only want to focus on one specific cloud environment, I would recommend using the tools of your chosen cloud provider.

In the environment Amazon Web Services it's a tool CloudFormation. The astute reader will have noticed that this article uses samples primarily from this tool, which I personally prefer.

What can be interesting is the possibility to use the associated tool Cloud Development Kit (CDK) for creating CloudFormation blueprints. The CDK will probably be used by those of you whose cloud environment is mainly dedicated to the development department. The CDK allows you to create CloudFormation blueprints using individual programming languages such as JavaScript, TypeScript, Python, Java, C#, and Go.

If your cloud environment is Azure, you will definitely take advantage Azure Resource Manager (ARM) templates.

ARM is a native and integral part of Azure. If you're observant, you'll have noticed that Azure translates all your operations (performed in the Azure Graphical Console) related to the creation of any resource into an ARM template, which it then applies.

If you want to get some inspiration on how these "internal" ARM templates look like, you can try creating any source and download this generated ARM template.

Conclusion on Infrastructure as Code

If you're serious about cloud automation, Infrastructure as Code is definitely the way to go in terms of any tool. And honestly - if you're thinking about how to use the cloud to its fullest in your environment, then you should automatically think about automating.

At first glance, Infrastructure as Code may look complicated, but believe me, once you try deploying a few components into your environment and get familiar with the whole concept of IaC, you probably won't want to build your environment "by hand" anymore. We'll come back to this topic in a future article in our Cloud Encyclopedia.