A Case for a Test Landing Zone

Background

We notice that often organizations deploy a single landing zone solution once and further changes are made on the same landing zone as the organization matures. Code Pipeline Errors are fixed on the fly with a sense of urgency and dedicated focus leading to unplanned efforts. Landing zone solution, being complicated, may not be amenable to a rollback to an earlier working configuration. This blog is proposing a test landing zone as a test-bed.

Quick introduction on AWS Landing Zone

The AWS Landing zone is a customizable solution provided by AWS for managing a multi account AWS environment. As against Control Tower, some organizations prefer to use a landing zone solution due to the high level of customizations provided by modifying the code. And because we are dealing with code, we need to treat it the way one would treat any code…. i.e Take it through the different development phases.

“Quality is not an act; it is a habit” –Aristotle.

World over, as a quality best practice, all software code needs to be tested. And an important code like the landing zone solution is no exception.

AWS Landing Zone solution is offered through AWS Solution Architect, Professional services or through APN partners like Infosys that provides a configurable setup of a landing zone with rich customization options.

Please check https://aws.amazon.com/solutions/implementations/aws-landing-zone/ for latest updates.

AWS Landing Zone Solution Overview

  • AWS Landing Zone solution help customers quickly setup secure, multi-account AWS environment based on best practices.
  • Implement initial security baseline and automate the setup of environments for running secure and scalable workloads.
  • Deploys Account Vending Machine (AVM) for provisioning and automatically configuring new accounts.
Landing Zone Solution
Landing Zone Solution Overview (source AWS)

About code pipeline…

The AWS code pipeline is the primary tool used for deployment of landing zone solution and any updates. Any errors reported in the pipeline usually do not disturb the existing workloads running under the landing zone. But all engineers want to always have a clean code pipeline. Troubleshooting code pipeline errors are time consuming and here is where a test landing zone comes handy, where the code is tested and then deployed on the production landing zone.

AWS Code Pipeline
AWS Code Pipeline for Landing Zone solution (source AWS)

The Dev workloads can be deployed on the Test Landing zones while the Prod and QA workloads can be deployed on the Prod landing zone. VPN connectivity as a cost effective solution for private connectivity with the on-premise can be utilized for connectivity to the workloads from on-premises. In the below diagram, other than the backhaul connectivity, the test and the prod landing zone configuration are near exact replica.

Prod and Test Landing Zones
Production and Test Landing zones.

A standard folder structure is defined for the AWS Landing zone solution configuration. This folder structure is shown below. This is customizable for any specific additional requirement.

Landing Zone Folder Structure
Default Landing zone folder structure

Landing Zone Deployment integration with GitHub

The configuration data of the landing zone solution can be stored in a Git repository (preferred and recommended) or uploaded in a pre-defined S3 bucket. Steps to be taken for GitHub integration is as shown below.

  • Download and extract AWS Landing Zone configuration zip file.
  • Connect to new Git repository, extract the file contents to the base of Git repository and not in aws-landing-zone-configuration folder
  • Navigate to AWS CodePipeline console in AWS Organizations account and select AWS-Landing-Zone-CodePipeline pipeline.
  • In the first stage, edit the source and change the source provider to GitHub.
  • Configure repository name and branch for GitHub.
  • Configure Output artifacat #1 = SourceApp and Select Update.
  • Once integrated, changes committed to Git repository will trigger the update pipeline to push changes to AWS LZ configuration.
Git Web Hook
Integrating LZ pipeline with Git Web Hooks

Some of the possible Landing zone customizations

  • AWS Transit Gateway for inter VPC and On-premise communication.
  • Azure AD integration using SAML2.0 for authentication.
  • Customize AVM to include Transit Gateway connection and automated AAD integration.
  • Customize config rules and other guard rails to adapt to organization requirement.
  • Creation of OUs

A Possible Git branch structure…

The whole purpose of the test environment for the landing zone is to test how structurally identical Landing Zone reacts to changes to the LZ configuration upon deployment. To maintain this identical configuration, we should have the Test LZ configuration files in the same Git repository as the Production.

Git Branches Branch name Description
1 Master Production LZ branch – baseline with clean LZ pipeline.
2 Testbr Test LZ branch – all feature branch changes are deployed here first. Some configs (like the manifest.yml) specific to the Test LZ should be retained.
3 Feature Holds new changes to LZ configuration: new stacks, or changes to old ones.

Scenario walkthrough…

As an example, let’s say, we would like to deploy a new service control policy. Example of SCP is extracted from this link. Below json statement needs to be inserted into landing zone policy_file: policies/aws-landing-zone-non-core-mandatory-preventive-guardrails.json. This SCP prevents any deployment outside the two US regions.

{
    "Sid": "DenyAllOutsideUS",
    "Effect": "Deny",
    "NotAction": [
        "ec2:DescribeRegions",
        "ec2:DescribeTransitGateways",
        "ec2:DescribeVpnGateways",
        "sts:*",
        "waf:*",
        "wafv2:*",
    ],
    "Resource": "*",
    "Condition": {
        "StringNotEquals": {
            "aws:RequestedRegion": [
                "us-east-1",
                "us-west-1"
            ]
        },
        "ArnNotLike": {
            "aws:PrincipalARN": [
                "arn:aws:iam::*:role/BillingRole",
                "arn:aws:iam::*:role/Role2AllowedToBypassThisSCP"
            ]
        }
    }
}
Git Flow for Landing zone configuration

Git Steps

  1. <Create branch feature1 from main branch.>
  2. git checkout feature1
  3. <Edit aws-landing-zone-core-mandatory-preventive-guardrails.json>
  4. git commit -m “Edited json file”
  5. git reflog <note the commit id for the edited json file>
  6. git checkout testbr
  7. git cherry-pick CommitId
  8. git push <Check for Test LZ pipeline errors. Correct configurations to address errors>
  9. git checkout feature1
  10. git push <if Test LZ pipeline has no errors>
  11. <Create a pull request to merge from feature1 to main branch>
  12. <Merge pull request after review>

To Summarize…

  • All codes need to be tested and therefore landing zone solutions need to be tested in a test environment.
  • Normally an error in the landing zone code pipeline does not impact existing workloads, but a need exists to ensure that the pipeline is error-free, so that deployment of new workloads will be easy.
  • The test landing zone solution configuration needs to be as close as possible to the production landing zone. To make effective use of the test landing zone, the development workload may be deployed on the test landing zone.

Author Details

James Mani Vathalloor

Cloud and IaaC Enthusiast. Experience in architecting cloud migration, design and build Landing Zone.

Leave a Comment

Your email address will not be published.