Preface

Been working on multi region deployments a bit in 2026. As a result of this I’ve been working with replicating AWS Secrets Manager secrets across regions.

You can find more on how multi-region keys work here. Essentially in backup and recovery architecture, they allow you to process encrypted data without interruption even during a region outage.

In basic terms, data maintained in the backup region can be decrypted in the backup region and data newly encrypted in the backup region can be descrypted in the primary region once that region is restored.

So Region A fails, you flip traffic to Region B and once Region A is restored the data encrypted during downtime in the backup region can still be descrypted by the original region.

Now, having worked with AWS for years now the times when something doesn’t work as expected are few and far between. It can be easy to think something isn’t working as expected for you and quickly think; Ah this must be a bug. However, you think a little bit more and also of the number of users and scale AWS has and usually it’s not a bug and you realise you may have been doing something not as it was intended to be done.

Having said that, I believe I’ve discovered some unexpected behaviour. At the time of writing, I most likely haven’t been the first to discover it but I’ll explain below.

Secret Replication

Secret replication was launched in 2021 it allows you to create regional read replicas for secrets, managed by AWS Secrets Manager. Allowing you to access secrets in multiple regions, similar to how I described above with AWS.

I must preface here, that from my experience with AWS outages, the main services that cause revenue impact are usually CDN or compute based, think EC2, ECS, EKS & Lambda. A scenario is where ECS is impacted in us-east-1 and new tasks cannot be launched due to API errors, it’s likely that the Secrets Manager APIs in the same region would be unaffected at that time. Although if you’re going to implement multi-region and disaster recovery you will want to cover all bases. As the last thing you want is your region switchover to fail due to Secrets Manager API errors across regions during an outage. Replicating secrets allows this issue to be combatted.

How to actually do it

For simplicity, I’ve used my beloved CloudFormation to create a simple Secret in us-east-1 using the following template (note there’s no region replication):

Resources:
  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: MultiRegionTest
      Description: Test MultiRegionSecretWithMrk
      SecretString: TestString

After that goes create complete, I’ve done an update to add ReplicaRegions. The only issue is that I don’t have any multi region KMS keys in my Account. I’ve taken mrk-1234abcd12ab34cd56ef12345678990ab as the multi-region key from the AWS docs here

Resources:
  Secret:
    Type: AWS::SecretsManager::Secret
    Properties:
      Name: MultiRegionTest
      Description: Test MultiRegionSecretWithMrk
      SecretString: TestString
      ReplicaRegions:
        - Region: us-east-2
          KmsKeyId: mrk-1234abcd12ab34cd56ef12345678990ab

The surprising result is that the CloudFormation stack Update completes without any issues:

And upon checking the Secrets Manager Console and using the CLI you’ll see InProgress set on the Replication:

$ aws secretsmanager describe-secret --secret-id MultiRegionTest
[...]
    "PrimaryRegion": "us-east-1",
    "ReplicationStatus": [
        {
            "Region": "us-east-2",
            "KmsKeyId": "mrk-1234abcd12ab34cd56ef12345678990ab",
            "Status": "InProgress"
        }
    ]
}

Then after AWS finalises making the ReplicateSecretToRegions API calls you’ll see a failure:

    "ReplicationStatus": [
        {
            "Region": "us-east-2",
            "KmsKeyId": "mrk-1234abcd12ab34cd56ef12345678990ab",
            "Status": "Failed",
            "StatusMessage": "Replication failed: Secrets Manager can't encrypt the secret value: Invalid keyId 'mrk-1234abcd12ab34cd56ef12345678990ab' (Service: AWSKMS; Status Code: 400; Error Code: NotFoundException; Request ID: cb9c4973-1626-4c99-8e57-42f7f886d859; Proxy: null)"
        }
    ]

This is not only native to CloudFormation but the same behaviour occurs via any CreateSecret API, for example via CLI:

$ aws secretsmanager create-secret --name TestSecretMrkFailure --add-replica-regions Region=us-west-2,KmsKeyId=mrk-1234abcd12ab34cd56ef12345678990aa
{
    "ARN": "arn:aws:secretsmanager:us-east-1:777864510236:secret:TestSecretMrkFailure-4BS2cc",
    "Name": "TestSecretMrkFailure",
    "ReplicationStatus": [
        {
            "Region": "us-west-2",
            "KmsKeyId": "mrk-1234abcd12ab34cd56ef12345678990aa",
            "Status": "InProgress"
        }
    ]
}

What’s even more strange is that when providing a made up MRK for this secret I get a response that Replication succeeded:

$ aws secretsmanager describe-secret --secret-id TestSecretMrkFailure
{
    "ARN": "arn:aws:secretsmanager:us-east-1:777864510236:secret:TestSecretMrkFailure-4BS2cc",
    "Name": "TestSecretMrkFailure",
    "LastChangedDate": "2026-04-22T22:13:45.699000+01:00",
    "CreatedDate": "2026-04-22T22:13:45.687000+01:00",
    "PrimaryRegion": "us-east-1",
    "ReplicationStatus": [
        {
            "Region": "us-west-2",
            "KmsKeyId": "mrk-1234abcd12ab34cd56ef12345678990aa",
            "Status": "InSync",
            "StatusMessage": "Replication succeeded"
        }
    ]
}

Upon further testing, upon initial creation the Replication succeeds however the above secret was created with no initial value, when you attempt to update the secret with a Value you will get an error on Replication. I guess that tells us that when ReplicateSecretToRegions is called with a SecretId additional actions or validation is done on AWS’ side.

As this is an API side issue it would affect all IaC providers too, Terraform, Pulumi, CDK etc.

Conclusion

I would expect the Create Operation to fail on validation if an invalid mrk is provided but that doesn’t seem to be the case. In fact there looks to be zero validation on the KmsKeyId value at all:

$ aws secretsmanager create-secret --name TestSecretMrkFailure4 --add-replica-regions Region=us-west-2,KmsKeyId=MyNameIsDylan$$$$
{
    "ARN": "arn:aws:secretsmanager:us-east-1:777864510236:secret:TestSecretMrkFailure4-6IyW1q",
    "Name": "TestSecretMrkFailure4",
    "ReplicationStatus": [
        {
            "Region": "us-west-2",
            "KmsKeyId": "MyNameIsDylan1610916109",
            "Status": "InProgress"
        }
    ]
}

I’ll look at ways to combat against this in future posts…

PS: remember to delete these secrets after use, they’re $0.40 per secret per month. A replica secret is considered a distinct secret and will also be billed at $0.40 per replica per month.

Table of Contents

1. Preface
2. Platform Engineering at Scale
3. AI in DevOps: The Slop and the Promise
4. FinOps: Cost vs. Business Outcomes
5. Cloudflare’s Growing Market Share
6. Getting Back to Basics with Books
7. Conclusion

Preface

As 2025 comes to an end I thought I’d look ahead to 2026 and take some stock on the current state of DevOps/Platform Engineering and some general ramblings on what I think is to come this year and in the coming years.

In some sense a lot has changed since I began working in DevOps in 2018 but at the same time some things stay the same. I think 2025 was the first year that we couldn’t say Kubernetes didn’t exist a decade ago and since then I have not managed to avoid manually scaling clusters and control planes as ETCD ran OOM due to growing pods. I also still haven’t avoided updating clusters in the old fashioned way (maybe EKS Auto Mode will solve that issue in 2026 though).

In this post, I’ll share my thoughts on what changed in the past year and what I expect to see in 2026. I feel with my experience working at different scales, from smaller teams to large enterprises with 000’s of developers I’ve seen a lot of change over the past few years.

Platform Engineering at Scale

Platform engineering has become a significant focus recently, and for good reason. From what I’m seeing almost all enterprises have adopted or are currently adopting internal develop platforms. Backstage from Spotify is still the defacto as it’s open source however I would not knock the traditional set up of running Jenkins. A backstage IDP does have the feel of a frontend for triggering GitHub Actions and GitLab runners much like was done in the past with frontends for Jenkins. I do think we’ll see growth in the IDP space especially with plugins to come with LLMs and chat bots to provision infrastructure via runners.

That said in smaller companies where a full blown IDP isn’t always necessary. A well-crafted set of CLI tools, scripts, and documentation can achieve similar results without the overhead and it can also scale. A CLI tool with a specific set of instructions to allow devs to provision infrastructure can often have perfect results and be a crowd pleaser among devs without the bloat of dealing with an IDP with maintainance and downtime.

AI in DevOps: The Slop and the Promise

2025 was very much the year AI become mainstream in developers lives and in DevOps. Tools for provisioning infrastructure from natural language are beginning to pop up and some of them seem promising [1].

However, it’s too early to say if these tools can reliably and safely provision production grade infrastructure that is SOX and PCI compliant amongst other things. MCPs for DataDog, Splunk and LLMs in general are at the stage where they can quickly point out a point of failure or issue when troubleshooting an issue. With the increase in lines of code outputted by these LLMs and approved by developers in to the codebase it will be interesting to see if troubleshooting and supporting production incidents is shifted left from SRE/DevOps Engineers and directly in to the hands of development teams. I would see that happening as a major pro for both but I don’t think we’re there yet.

Much like well written sofrware, without appropriate context and well-crafted instructions, AI tools can generate outputs that fail in production environments. I’ve spent considerable time debugging Terraform configurations that seemed sound until they encountered edge cases or security requirements. Like a Postgres instance provisioned with auto major upgrades enabled - LLMs can sometimes favour breaking things over being conservative.

Looking to 2026, I expect this technology to mature significantly. Companies will start more widespread use of gateway LLMs [2]. Rather than having multiple places to use AI tools as is today, we’ll see consolidation to preferred choices that integrate effectively with existing workflows. LLM gateways are a tool for this as they open up wider access to models, track usage and cost.

FinOps: Cost vs. Business Outcomes

Cost optimization is a passion of mine and with the spend in AI really should be of more importance than ever. But in practice, it can often be a hinderence to delivery at early stages and be treated as an afterthought. There’s an old adage of optimising for business outcomes vs. optimising for cost and usually the business outcomes come first and optimisation after.

2026 might not be the year that FinOps goes full mainstream but it’s quite a year for Anthropic, OpenAI and the chasing field in AI. Not to mention it’s the second fiscal year of AI on public companies books. One of two things will happen from AI and it’s increased efficiency. There’s no doubt there’s an increase in productivty using the tools but if this doesn’t in turn equate to an increase in revenue (ie. more money being spent by customers) then the other optimisation could be in the form of cost-reduction. Hopefully that is on the Cloud Bill rather than in headcount…

For 2026 though, I expect FinOps to continue to grow and become more integrated into development processes. InfraCost [3] is making this easy with direct code scanning and change management from within the PR. We’ll also see growth from CloudZero and other cost management tools. Rather than separate teams conducting postmortems on expensive deployments, we’ll see cost considerations built into CI/CD pipelines and development workflows. Tools will provide real time cost feedback, allowing developers to make informed decisions without sacrificing speed.

The key will be finding the right balance. With enough cost awareness to prevent waste, but not so much that it hinders innovation.

Cloudflare’s Growing Market Share

Cloudflare has been around for a while now and first came to my attention in 2020 when I saw some smaller customers in AWS leaving CloudFront for Cloudflare but back then they didn’t have the product offering for compute that AWS had. This was followed by their R2 offering which offers no egress and cheaper storage costs to S3. I believe they’ll continue to grow and start taking enterprise business from AWS. Their edge network and security offerings have matured significantly, providing compelling alternatives to traditional cloud providers. And in 2026 while I haven’t used their Workers service I think it might be the year I dip my toes - and from an AWS man that is telling.

In 2026, expect to see more enterprises adopting Cloudflare for specific use cases, potentially leading to hybrid architectures where AWS handles core compute while Cloudflare manages edge services and security. So even if their compute runs on ECS or EKS it might be fronted by CloudFlare with some workers handling event driven processing with R2. This is dependent on the cost side of things I mentioned above coming in to play.

Getting Back to Basics with Books

Following tech and DevOps online provides a lot of information, but much of it feels generic and not focused on enterprise scale challenges. I also find my time being spent on Twitter (X) and Reddit is increasing to the point that I don’t like. While any time spent on those sites for me is not disirable they are often the place to get updates on new releases and follow people who publish content. Although in 2026 I’ll be moving this to LinkedIn and Slack communities - mainly the AWS Builder Community Slack and the FinOps Slack. I’ve also recently purchased three O’Reilly books to get back to fundamentals and gain deeper knowledge. I’m looking forward to getting through these:

I’m hoping these books will boost my solid foundation of software engineering and architecture that online content often lacks. In a field moving as fast as cloud computing, it’s easy to get caught up in the latest trends without understanding the underlying principles. These books should help bridge that gap.

Conclusion

So, in my opinion 2026 will continue to be a year of maturation in DevOps and Platform Engineering. IDPs will continue to grow in popularity for enteprises and AI will become more integrated to workflows but at a more refined scale. Likely with a reduction in AI providers being used (Cursor or Claude Code - not both), and cost optimization might not be baked into development processes yet but will continue to pop up. We’ll continue to see Cloudflare rise and I’m looking forward to following any product announcements they have this year. These might challenge traditional cloud providers, and a renewed focus on fundamentals will ensure sustainable growth.

From my experience, the most successful organizations will be those that balance innovation with pragmatism, adopting new technologies while maintaining a strong foundation. Some that go ‘all-in’ on AI will bear fruit, while others just won’t see a payoff and will have to pivot back to focussing on what their products where before AI and see where they can continue to grow there. As we enter 2026, I’m excited to see how these trends play out in the real world and we’ll see if I’m on the money or completely off the mark…

[1] https://spacelift.io/intent [2] https://llmgateway.io [3] https://www.infracost.io/

Table of Contents

1. Overview of the Workflow
2. Prerequisites
3. Step-by-Step Implementation
   1. S3 Bucket Setup
   2. Create EventBridge Rule
   3. Define the Input Transformer
   4. Create Batch Job Definition
   5. Finalize and Deploy
4. Conclusion

In AWS, connecting different services seamlessly is a key to building robust, automated workflows. A common scenario is passing properties from an S3 event to an AWS Batch job using EventBridge. This workflow allows you to trigger a Batch job when a specific event happens in an S3 bucket, passing along important metadata or configurations as part of the job’s environment variables.

In this blog post, we’ll explore how to implement this architecture using Infrastructure as Code (IaC) with Terraform. This approach ensures your infrastructure is version-controlled, repeatable, and scalable.

Overview of the Workflow

The basic workflow involves the following steps:

1. S3 Event: An object is created or modified in an S3 bucket.
2. EventBridge Rule: The event triggers an EventBridge rule.
3. Input Transformer: The EventBridge rule uses an Input Transformer to extract and transform specific properties from the S3 event.
4. AWS Batch: The transformed properties are passed as environment variables to the AWS Batch job.

Prerequisites

Before we dive into the setup, ensure you have the following:

• An S3 bucket set up to store your files.
• An AWS Batch compute environment and job queue configured.
• Basic understanding of Terraform and AWS services like S3, EventBridge, and Batch.

Step-by-Step Implementation

1. S3 Bucket Setup

First, you need an S3 bucket where your files will be uploaded. This bucket will emit events that will trigger the AWS Batch jobs.

resource "aws_s3_bucket" "example_bucket" {
  bucket = "my-event-trigger-bucket"
}

2. Create EventBridge Rule

Next, create an EventBridge rule that listens for specific events from the S3 bucket. This rule will filter for events like `s3:ObjectCreated:*` and trigger the Batch job.

resource "aws_cloudwatch_event_rule" "s3_event_rule" {
  name        = "s3-event-rule"
  event_pattern = <<EOF
{
  "source": ["aws.s3"],
  "detail-type": ["AWS API Call via CloudTrail"],
  "detail": {
    "eventName": ["PutObject", "CompleteMultipartUpload"],
    "requestParameters": {
      "bucketName": ["${aws_s3_bucket.example_bucket.bucket}"]
    }
  }
}
EOF
}

3. Define the Input Transformer

The Input Transformer in EventBridge allows you to manipulate the event data before passing it to the Batch job. For example, you can extract the S3 bucket name and object key from the event and pass them as environment variables.

resource "aws_cloudwatch_event_target" "batch_target" {
  rule = aws_cloudwatch_event_rule.s3_event_rule.name
  arn  = aws_batch_job_queue.my_job_queue.arn

  input_transformer {
    input_paths = {
      "bucket" = "$.detail.requestParameters.bucketName"
      "key"    = "$.detail.requestParameters.key"
    }

    input_template = <<EOF
{
  "jobName": "my-batch-job",
  "jobQueue": "${aws_batch_job_queue.my_job_queue.arn}",
  "jobDefinition": "${aws_batch_job_definition.my_job_definition.arn}",
  "containerOverrides": {
    "environment": [
      {"name": "S3_BUCKET", "value": <bucket>},
      {"name": "S3_KEY", "value": <key>}
    ]
  }
}
EOF
  }
}

4. Create Batch Job Definition

Finally, define the Batch job that will be triggered. The job definition should include the environment variables that are passed from the EventBridge Input Transformer.

resource "aws_batch_job_definition" "my_job_definition" {
  name = "my-batch-job-definition"

  container_properties = jsonencode({
    image      = "my-docker-image"
    vcpus      = 2
    memory     = 2048
    environment = [
      {
        name  = "S3_BUCKET"
        value = "placeholder" // This will be overwritten by EventBridge
      },
      {
        name  = "S3_KEY"
        value = "placeholder" // This will be overwritten by EventBridge
      }
    ]
  })

  type = "container"
}

5. Finalize and Deploy

With the S3 bucket, EventBridge rule, Input Transformer, and Batch job definition in place, you can deploy your infrastructure using Terraform. Make sure you apply the configuration and monitor the process to ensure everything is working as expected.

Conclusion

By following this setup, you can efficiently pass properties from S3 events to AWS Batch jobs using EventBridge. This method is powerful for building event-driven architectures that respond dynamically to changes in your S3 buckets. With Terraform, you ensure that your infrastructure is easily manageable and reproducible.

For more detailed information on the specific configurations and additional features, check out the official AWS documentation on EventBridge and Batch integration.

Happy automating!

Table of Contents

1. Preface
2. Ecosystem
3. Compatibility/Support
4. OpenSource vs. Licencing vs. Native
5. Speed
6. Associated Toolsets & Accessories
7. Programming and Dev Experience
8. Future of Both Tools
9. Conclusion

Preface

My journey with IaC began in 2018, focusing exclusively on CloudFormation. Over the years, I honed my skills to become a Subject Matter Expert in CloudFormation while working at AWS. This involved a lot of things, mainly ,providing complex work evidence of solving CFN issues for customers and also passing a board with the CFN dev team in Seattle. However, since 2022, Terraform has beeen the go-to tool for all things IaC where I work.

This was quite a change, and while I still use CFN for a lot of my personal projects I’ve been using Terraform almost entirely in work apart from a few SAM stacks deployed via Terraform modules.

This transition offered me a unique perspective on both tools, and in this post, I’ll share my insights, comparing CloudFormation and Terraform across various dimensions when deploying infra to AWS. I’ll also show why I think CloudFormation is improving and closing the gap on Terraform and what I think is to come from both over the next few years.

Ecosystem

CloudFormation: As a native AWS service, CloudFormation is tightly integrated with AWS, providing out-of-the-box support for almost every AWS resource. Its ecosystem is robust, with a wealth of templates and a supportive community. However, it can be somewhat insular, primarily catering to AWS resources.

Terraform: Terraform’s ecosystem is expansive, supporting multiple providers beyond just AWS. Its modular approach, with the use of Terraform Registry, offers a vast collection of modules contributed by the community, enhancing its adaptability and extensibility across various cloud platforms and services.

Compatibility/Support

CloudFormation: CloudFormation launched back in 2011 and whether you view this as a pro or con any template written on Day 1 of CloudFormation can still be deployed today without issues or errors. For example, when working older, more mature code bases this can be a benefit during updates. AWS will always support that JSON or YAML template.

Whereas with Terraform, when you go to revisit your long running RDS DB that’s reaching EOL support, you’ll be faced with major upgrade changes for the terraform-provider-aws. Similarly, your template_files may no longer be supported and you need to spend time updating code to use the newer templatefile function. There’ll always be that management overhead when using Terraform.

Some other issues I’ve noticed creeping up recently are with resource AWS support and updates from Terraform.

At re:invent 2023 we got periodic recording for Config on November 26th, this was supposed to come with OOTB support from CloudFormation via the RecordingMode parameter, we didn’t get it that night but had to wait until December 11th via 1861 not too long considering it’s AWS’ busiest week of the year and a lot of staff are in Vegas:

Type: AWS::Config::ConfigurationRecorder
Properties:
[...]
  RecordingGroup: 
    RecordingGroup
  RecordingMode: 
    RecordingMode
[...]

But it took a whopping 3 months to reach Terraform support on February 23rd 2024 when it was delivered via 34577!

Similarly, with CodePipeline we got the V2 Pipeline Type for re:Invent week on November 22, 2023. This allows you to use a couple of new features but most importantly allowed you to avail of cheaper Pipeline costs via the new type. This came with same day support via CloudFormation PipelineType parameter.

This was brought to Terraform in January 2024 via 34122.

I’m not complaining about having to wait here and understand that the aws terraform provider is maintained by a small team, I just suspected these changes to be implemented at a quicker rate with the maturitity of the AWS provider. The speed of feature delivery is food for though when using these tools at scale.

Another example of gaps in support I was surprised to see were with data sources. To this day there’s no support for HTTP API Gateway custom domain name data blocks in Terraform, I suspect shortly after publishing this though it will be provided via 36027:

data "aws_apigatewayv2_domain_name" "example" {
  domain_name = "api.example.com"
}

Some things Terraform should be commended on though, is its support for multiple providers and tooling. For example, if using CloudFormation and mTLS with API Gateway your only real native option is to use ACM Private CA to issue certificates with a checks AWS Bill cost of $400/month per certificate. If you’re using Terraform you can use the TLS Provider and self sign certs for your API gateways truststore:

###############################################
# S3 bucket and TLS certificate for truststore
###############################################

resource "aws_s3_bucket" "truststore" {
  bucket = "${random_pet.this.id}-truststore"
  #  acl    = "private"
}

resource "aws_s3_object" "truststore" {
  bucket                 = aws_s3_bucket.truststore.bucket
  key                    = "truststore.pem"
  server_side_encryption = "AES256"
  content                = tls_self_signed_cert.example.cert_pem
}

resource "tls_private_key" "private_key" {
  algorithm = "RSA"
}

resource "tls_self_signed_cert" "example" {
  is_ca_certificate = true
  private_key_pem   = tls_private_key.private_key.private_key_pem

  subject {
    common_name  = "example.com"
    organization = "ACME Examples, Inc"
  }

  validity_period_hours = 12

  allowed_uses = [
    "cert_signing",
    "server_auth",
  ]
}

The equivalent to the above is a lambda-backed custom resource in CFN, which would not be graceful at all. Using a mix of resource providers is where Terraform can outshine CloudFormation when deploying to AWS on occasions.

OpenSource vs. Licencing vs. Native

This is one the hottest topics recently in regards to CloudFormation vs. Terraform. In the past, a very strong argument to use Terraform was that it’s “open source”. However in 2023, HashiCorp threw a curve ball by adopting a Business Source Licence. I won’t go in to what’s involved in all of that in this post but you can think of services that used to be free and aren’t anymore.

Further, this sparked the creation of OpenTofu a truly open-source alternative. So with a lack of transparency from HashiCorp you’re faced with continuing to use Terraform and hope it remains free or seriously consider OpenTofu - not exactly a trivial task and something you want your dev team discussing. Anyone using CloudFormation won’t be having this conversation.

On the CloudFormation side, back in the day CloudFormation was a completely closed shop and owned by a dev team who were responsible for implementing all of the AWS’ resources in to CFN. As the number of services AWS created grew this became unsustainable, especially with 2 pizza teams - it was difficult to stay consistent.

Today, there’s still a CFN service team in AWS but they’ve built a platform where on launch other Service Teams in AWS must write their own provider code for CFN. This makes things consistent across all of the teams.

While I don’t see the CloudFormation engine ever becoming fully open-source a lot of the resource providers have been made available on GitHub here which is a welcome change to having to contact AWS Support directly for all resource issues and having no transparency in the past.

There’s also the cloudformation-coverage-roadmap which provides an amicable amount of transparency in to what is and isn’t coming to CloudFormation.

Speed

Speed of deployment has been talked about a lot in CFN vs. TF but I haven’t seen any legitimate benchmark data.

From personal use though, I do feel that Terraform is quicker to deploy. Especially for specific AWS resources such as CloudFront, R53 and IAM updates. However, some things do just take a while. For example, with ECS Services I know that CloudFormation won’t mark the resource as CREATE_COMPLETE until a task has reached the steady state it pings DescribeTask calls to achieve this which is known as Resource Stabilization.

Terraform feels quicker to re-deploy failed deployments too as it doesn’t rollback each resource similar to the way CloudFormation does. Also with Terraform you’d traditionally fail quicker if a resource type or variable did not match an expected value. With CloudFormation you would have only hit this at Stack Creation.

However, just this week CloudFormation announced the following - Experience up to 40% faster stack creation with AWS CloudFormation, I’m yet to experiment with this new CONFIGURATION_COMPLETE state but it looks like Stabilization is being performed in parallel now.

So while Terraform can be faster to deploy resources, you mightn’t get that baked in stabilisation that comes with CloudFormation. Terraform does have better parallelism capabilities though.

Associated Toolsets & Accessories

cdk migrate, tf lint, cfn lint, checkov, tfsec, aqua

Both tools here perform similarly in this regard with CloudFormation and Terraform both having tools to lint and enforce security best practice. cfn lint and tf lint both offer similar rule sets. cfn-guard also feels a lot like tfsec although the latter has faced a rename recently from tfsec to trivy - again something we don’t see with CloudFormation are renames.

When using Open Policy Agent CloudFormation allows you to use hooks which can be finicky with OPA as hooks only support Python or Java. Because of this, you need the OPA CFN Hook which does work. If you really want to use OPA with Terraform you’ll need to pay for Terraform Cloud:

checkov is also useful for both CFN and TF and is free for now. The CDK also supports features for validation during synthesis time via CfnGuardValidator and supports OPA and Chekov.

Programming and Dev Experience

The capabilities of HCL is where Terraform excels in comparison to CloudFormation. When using both these tools at scale you begin to face teething issues. While CloudFormation has a decent set of Pseudo parameters that can be used to enable multi-account, multi-region deploys such as AWS::AccountId & AWS::Region the Intrinsic Functions also work but aren’t a breeze to play with. They have expanded on these with recent additions via the AWS::LanguageExtensions transforms but again it’s a little convoluted.

When using CFN at scale with Nested Stacks you’ll also face issues with Imports/Exports and Cross-Stack references and dependencies. I’m sure anyone reading this will wince at the following error:

StackA Export StackA:ExportsOutputRefMyTableCD79AAA0A1504A18 cannot be deleted as it is in use by StackB

I would recommend avoiding these completely and using Parameter Store and dynamic references for non-sensitive values.

Terraforms development experience trumps CloudFormation here considerably. From my time working with tf there’s something useful there when you need to reference an existing resource or something else. For example, data sources for references, modules for re-usabilility, outputs for debugging and referencing values and the list goes on.

dynamic Blocks can also be extremely useful within modules when aiming to create slightly different variations of resources but re-use the same code. I haven’t even mentioned for_each or count. The list of built-in functions is endless too. You’ve also got the random provider to take care of dynamic placement of resources in subnets:

data "aws_availability_zones" "available" {
state = "available"
}

resource "random_shuffle" "az" {
input = data.aws_availability_zones.available.names
result_count = 3
}

CloudFormation definitely comes out second in this contest as the more primitive tool. For more “dev” DevOps Engineers and Software Engineers working at scale, Terraform will naturally be the preffered choice here.

Future of Both Tools

CloudFormation: As AWS continues to evolve, CloudFormation is expected to remain a key player, potentially expanding its capabilities and integration with other AWS services. Its future likely includes enhancements in usability and template management. With the additional abstraction of the CDK, I wonder will we ever see a rebuilt CFN engine that doesn’t translate go, typescript or python code back in to JSON or YAML templates like the CDK does - but today this is where we’re at.

Terraform: I’m not sure exactly where Terraform’s trajectory is going to go. With the introduction of Terraforms licencing, the recent creation of OpenTofu and the delays in introducing new resources I’m beginning to think that Terraform are going to push on Terraform Cloud and begin to cash in on what a great tool it has been over the years. I would not be surprised to see them enforce billing on Enterprise customers or be acquired this year. Having said that, I think they will look to develop towards broader ecosystem support and implement more providers.

Conclusion

Both CloudFormation and Terraform have their strengths and areas for improvement. My personal experience has shown me the nuances of each, and I believe the choice between them often depends mainly on the projects needs and organizational context. For example, anything multi-cloud you’re not choosing CloudFormation.

I like to think that within an enterprise you can use both. Either or. As long as there’s the correct tooling in place to use Terraform well and in a consistent manner. This is where the toolsets and accessories become more important.

I also wouldn’t turn anyone away from using the aws_cloudformation_stack terraform resource, while it may appear to be an antipattern it can combine the best of worlds at times.

Having said that, with the recent improvements CloudFormation has made and the changes in licencing and uncertainty from Terraform. I would have to crown the long term IaC winner in my opinion as … CloudFormation.

One of the first things I noticed about Quito was the altitude. Standing at an elevation of about 2,850 meters (9,350 ft), it’s the second-highest official capital city in the world.

The altitude was definitely something to get used to, and I even with my level of fitness and countless Altitude Classes @ AltiPeak on the long mile road, I still found myself a small bit short of breathe when walking up the steep hills. But within a day or so, my body started adjusting to the height.

We went on the Hostels local walking tour the next morning. This was a great way to see some of the city’s most iconic sights, and get a feel for the culture. Our guide gave us the low-down on why half of the buildings were never fully complete (you only pay taxes on the building if it’s complete) and he also brought us around to some of the best food spots in the city. ADD MORE HERE

We stayed at the Secret Garden Hostel, full of backpackers’ with great views of the city skyline. Staff were friendly, the food was good and they give a 50% discount on food and drink if you leave a review - this was availed of in full.

Quito was a great place to start and the walking tour from the Hostel was a must do. TBC…..

Next stop was to catch a night bus to Cuyabeno which is at the foot of the Amazon Rainforest in Ecuador.

This is the first post on my new webpage. This will be the place where I’ll be posting some updates publicly to track my web dev skills over time.

This post may be changed periodically or even removed over time.