Showing posts with label cloudformation. Show all posts
Showing posts with label cloudformation. Show all posts

Thursday, June 10, 2021

Jamstack Cloudformation

Problem and Audience

A simple clouformation template makes it easy to stamp out AWS infrastructure for a jamstack web site. A jamstack is a web site composed of static presentation (non-API) resources (html, css, javascript) assembled at build time - as opposed to a site that requires some kind of dynamic server side rendering of resources at request time. A nice feature of this architecture is that it allows a site to be served inexpensively from a serverless object storage system like S3. We manage https://apps.frickjack.com by copying presentation resources to an S3 bucket that acts as an origin for a cloudfront distribution.

We originally setup the infrastructure for https://apps.frickjack.com by clicking around the AWS web console (like this), but we want to tear down that infrastructure, and move to cloudformation managed infrastructure to realize the benefits of infrastructure as code including:

  • it is less work to manipulate infrastructure by editing json files than clicking through the web console
  • a cloudformation template allows us to deploy multiple copies of our architecture (for different products, test environments, etc) in a consistent way
  • cloudformation templates capture best practices and institutional conventions, and allow infrastructure to evolve over time
  • tracking the cloudformation template and parameters in git gives an audit trail
  • cloudformation makes automation easy

Jamstack Requirements

We have a handful of requirements for our jamstack infrastructure. First we want the S3 bucket to remain private and encrypted. Even though the content of the bucket is publicly accessible via the cloudfront CDN, making the origin bucket conform to standard S3 best practices simplifies compliance, since we do not need to note exceptions to organization policies that expect a bucket to be private and encrypted. The cloudfront CDN is granted access to the private bucket via a bucket policy that gives read permission to an origin access identity associated with the cloudfront distribution.

Most of our other requirements are addressed by adjusting knobs on our cloudfront configuration. For example, we want all http traffic to be redirected to https. We want to use an input parameter to our cloudformation stack to associate an alias domain with the distribution - we manage the DNS setup for the alias in another Route53 stack. We use another input parameter to feed the ARN of ourACM-managed TLS certificate. We configure cloudfront to require clients to use TLS 1.2 or better.

Finally, our little stack tools make it easy to follow the tagging conventions that we want to enforce across our infrastructure.

the little stack

We setup the following template (also in github) to start managing our jamstack infrastructure with cloudformation. The template takes advantage of the nunjucks extensions to cloudformation templates supported by our little stack automation.

One "gotcha" that we ran into was that we originally intended to setup a new stack with the apps.frickjack.com domain alias already in use by our live CDN, copy our web content to the new bucket (modify our codebuild CI pipeline configuration to do that for us), then update DNS to point the apps.frickjack.com domain at the new CDN. However, it turns out that cloudfront does not allow two distributions to have the same alias, so we setup our new CDN with a temporary alias, then took a few minutes of downtime while we removed the apps.frickjack.com alias from the old CDN, then updated our new stack.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Metadata": {
    "License": "Apache-2.0"
  },
  "Description":
    "Generalize AWS s3-cdn sample template from - https://github.com/awslabs/aws-cloudformation-templates/blob/master/aws/services/S3/S3_Website_With_CloudFront_Distribution.yaml",
  "Parameters": {
    "CertificateArn": {
      "Type": "String",
      "Description": "ACM Certificate ARN"
    },
    "DomainName": {
      "Type": "String",
      "Description": "The DNS name of the new cloudfront distro",
      "AllowedPattern": "(?!-)[a-zA-Z0-9-.]{1,63}(?<!-)",
      "ConstraintDescription": "must be a valid DNS zone name."
    },
    "BucketSuffix": {
      "Type": "String",
      "Description": "The suffix of the bucket name - prefix is account number",
      "AllowedPattern": "[a-zA-Z0-9-]{1,63}",
      "ConstraintDescription": "must be a valid S3-DNS name"
    }
  },
  "Resources": {
    "S3Bucket": {
      "Type": "AWS::S3::Bucket",
      "Properties": {
        "AccessControl": "Private",
        "BucketName": { "Fn::Join": [ "-", [ { "Ref" : "AWS::AccountId" }, { "Ref": "BucketSuffix" } ] ] },
        "BucketEncryption": {
          "ServerSideEncryptionConfiguration" : [ 
            {
              "BucketKeyEnabled" : "true",
              "ServerSideEncryptionByDefault" : {
                "SSEAlgorithm": "AES256"
              }
            }
          ]
        },        
        "WebsiteConfiguration": {
          "IndexDocument": "index.html",
          "ErrorDocument": "error.html"
        },
        "Tags": [
          {{ stackTagsStr }}
        ]
      }
    },
    "CloudFrontOriginIdentity": {
      "Type": "AWS::CloudFront::CloudFrontOriginAccessIdentity",
      "Properties": {
        "CloudFrontOriginAccessIdentityConfig": {
          "Comment": "origin identity"
        }
      }
    },
    "BucketPolicy": {
      "Type": "AWS::S3::BucketPolicy",
      "Properties": {
        "Bucket": { "Ref": "S3Bucket" },
        "PolicyDocument": {
          "Version": "2012-10-17",
          "Statement": [
            { 
              "Effect": "Allow",
              "Principal": {
                "AWS": { "Fn::Sub": "arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ${CloudFrontOriginIdentity}" }
              },
              "Action": "s3:GetObject",
              "Resource": { "Fn::Sub": "arn:aws:s3:::${S3Bucket}/*" }
            }
          ]
        }
      }
    },
    "CdnDistribution": {
      "Type": "AWS::CloudFront::Distribution",
      "Properties": {
        "DistributionConfig": {
          "Aliases": [
            { "Ref": "DomainName" }
          ],
          "Origins": [
            { 
              "DomainName": { "Fn::Sub": "${S3Bucket}.s3.${AWS::Region}.amazonaws.com" },
              "Id": "S3-private-bucket",
              "S3OriginConfig": {
                "OriginAccessIdentity": { "Fn::Sub": "origin-access-identity/cloudfront/${CloudFrontOriginIdentity}" }
              }
            }
          ],
          "DefaultRootObject": "index.html",
          "Enabled": "true",
          "Comment": { "Ref": "DomainName" },
          "DefaultCacheBehavior": {
            "AllowedMethods": [ "GET", "HEAD", "OPTIONS" ],
            "CachedMethods": [ "GET", "HEAD", "OPTIONS" ],
            "TargetOriginId": "S3-private-bucket",
            "ForwardedValues": {
              "QueryString": "false",
              "Cookies": {
                "Forward": "none"
              }
            },
            "ViewerProtocolPolicy": "redirect-to-https"
          },
          "ViewerCertificate": {
            "AcmCertificateArn": { "Ref": "CertificateArn" },
            "SslSupportMethod": "sni-only",
            "MinimumProtocolVersion": "TLSv1.2_2019"
          }
        },
        "Tags": [
          { "Key": "Name", "Value": { "Ref": "DomainName" } },
          {{ stackTagsStr }}
        ]
      }
    }
  },
  "Outputs": {
    "CdnAliasDomain": {
      "Value": { "Fn::GetAtt": [ "CdnDistribution", "DomainName" ] },
      "Description": "The URL of the newly created website"
    },
    "BucketName": {
      "Value": { "Ref": "S3Bucket" },
      "Description": "Name of S3 bucket to hold website content"
    }
  }
}

Summary

A simple clouformation template makes it easy to stamp out AWS infrastructure for a jamstack web site.

Monday, June 07, 2021

route53 and cloudformation

Problem and Audience

Managing route53 records with cloudformation is a good idea for the same reasons that tracking other resources with cloudformation (or terraform or whatever) is better than clicking around in the web console - namely:

  • it is less work to manipulate route53 records by editing json files than clicking through the web console
  • tracking the cloudformation template and parameters in git (or whatever code repository) gives an audit trail
  • cloudformation makes automation easy

We setup the following cloudformation template to start managing our simple route53 zones with cloudformation. The template takes advantage of the nunjucks extensions to cloudformation templates supported by our little stack automation.

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Parameters": {
    "DomainName": {
      "Type": "String",
      "Description": "the domain name"
    }
  },
  "Resources": {
    "HostedZone": {
      "Type" : "AWS::Route53::HostedZone",
      "Properties" : {
          "HostedZoneTags" : [ 
            {{ stackTagsStr }}
          ],
          "Name" : { "Ref": "DomainName" }
        }
    }

    {% if stackVariables.aliasList.length %}
    ,
      {% for item in stackVariables.aliasList %}
      "AliasA{{ item.resourceName }}": {
        "Type" : "AWS::Route53::RecordSet",
        "Properties" : {
            "AliasTarget" : {
              "DNSName" : "{{ item.target }}",
              "HostedZoneId": "{{ item.hostedZoneId }}"
            },
            "Comment" : "{{ item.comment }}",
            "HostedZoneId" : { "Ref": "HostedZone" },
            "Name" : "{{ item.domainName }}",
            "Type" : "A"
          }
      },
      "AliasAaaa{{ item.resourceName }}": {
        "Type" : "AWS::Route53::RecordSet",
        "Properties" : {
            "AliasTarget" : {
              "DNSName" : "{{ item.target }}",
              "HostedZoneId": "{{ item.hostedZoneId }}"
            },
            "Comment" : "{{ item.comment }}",
            "HostedZoneId" : { "Ref": "HostedZone" },
            "Name" : "{{ item.domainName }}",
            "Type" : "AAAA"
          }
      }

      {% if not loop.last %} , {% endif %}
      {% endfor %}
    {% endif %}

    {% if stackVariables.mxConfig %}
    ,
    "MX": {
      "Type" : "AWS::Route53::RecordSet",
      "Properties" : {
          "Comment" : "mx mail config",
          "HostedZoneId" : { "Ref": "HostedZone" },
          "Name" : { "Ref": "DomainName" },
          "ResourceRecords" : {{ stackVariables.mxConfig.resourceRecords | dump }},
          "TTL" : "900",
          "Type" : "MX"
        }
    }
    {% endif %}

    {% if stackVariables.cnameList.length %}
    ,
    {% for item in stackVariables.cnameList %}
    "Cname{{item.resourceName}}": {
      "Type" : "AWS::Route53::RecordSet",
      "Properties" : {
          "Comment" : "{{ item.comment }}",
          "HostedZoneId" : { "Ref": "HostedZone" },
          "Name" : "{{ item.domainName }}",
          "ResourceRecords" : [ "{{ item.target }}" ],
          "TTL" : "900",
          "Type" : "CNAME"
        }
    }
    {% if not loop.last %} , {% endif %}
    {% endfor %}

    {% endif %}

    {% if stackVariables.txtList.length %}
    ,
    {% for item in stackVariables.txtList %}
    "Txt{{item.resourceName}}": {
      "Type" : "AWS::Route53::RecordSet",
      "Properties" : {
          "Comment" : "{{ item.comment }}",
          "HostedZoneId" : { "Ref": "HostedZone" },
          "Name" : { "Ref": "DomainName" },
          "ResourceRecords" : [ {{ item.txtValue | dump }} ],
          "TTL" : "900",
          "Type" : "TXT"
        }
    }
    {% if not loop.last %} , {% endif %}
    {% endfor %}

    {% endif %}

  },

  "Outputs": {
    "NameServers": {
      "Description": "hosted zone nameservers",
      "Value": { "Fn::Join": [",", { "Fn::GetAtt": [ "HostedZone", "NameServers" ] }] }
    }
  }
}

Summary

Managing route53 zones with cloudformation is the right thing to do.

Monday, May 17, 2021

Supporting Cloudformation Patterns with Nunjucks

Problem and Audience

Since cloudformation templates do not natively support the dynamic resource provisioning patterns required for many cloud architectures, various extensions and template generators have emerged (like CDK and SAM). The little stack tools allow the use of the nunjucks template language in cloudformation templates to support various infrastructure patterns.

Overview of little tools

The little tools include the little stack helpers for deploying infrastructure defined by a declarative template file. The little tools include their own library of cloudformation templates. Ideally a template is defined in a generic way, but accepts input parameters that allow different stacks (like prod and dev) to be deployed, so a (template.json, parameters.json) pair defines each infrastructure stack. The end user follows this workflow.

  • select a cloudformation template from the library
  • create a parameters json file defining the input variables that the template requires - the parameters file format extends the cli skeleton (from aws cloudformation update-stack --generate-cli-skeleton) with a littleware block - for example:
{
    "StackName": "name of the stack",
    "Capabilities": [
        ... cloudformation capabilities if any 
        "CAPABILITY_NAMED_IAM"
    ],
    "TimeoutInMinutes": 5,
    "EnableTerminationProtection": true,
    "Parameters" : [
        ... cloudformation input parameters
    ],
    "Tags": [
        ... tags for the stack
            {
                "Key": "org",
                "Value": "applications"
            },
            {
                "Key": "project",
                "Value": "api.frickjack.com"
            },
            {
                "Key": "stack",
                "Value": "reuben"
            },
            {
                "Key": "stage",
                "Value": "dev"
            },
            {
              "Key": "role",
              "Value": "api"
            }
    ],
    "Littleware": {
        "TemplatePath": "lib/cloudformation/cloud/api/authclient/root.json ... path to the template",
        "Variables": { ... supplemental nunjucks input variables
            "authnapi": {
                "lambdaVersions": [
                    {
                        "resourceName": "lambdaVer20200523r0",
                        "description": "initial prod version"
                    },
                    {
                        "resourceName": "lambdaD001000003D20200618r0",
                        "description": "little-authn 1.0.3"
                    },
                    {
                        "resourceName": "lambda20201205r0",
                        "description": "little-authn 1.0.4"
                    },
                    {
                        "resourceName": "lambda20201216r0",
                        "description": "little-authn 1.0.5"
                    }
                ],
                "prodLambdaVersion": "lambda20201216r0",
                "gatewayDeployments": [
                    {
                        "resourceName": "deploy20200523r0",
                        "description": "initial deployment"
                    }
                ],
                "prodDeployment": "deploy20200523r0",
                "betaDeployment": "deploy20200523r0"
            },
            "sessmgr": {
                "kmsKeys": [
                    "sessmgr20210416"
                ],
                "kmsSigningKey": "sessmgr20210416",
                "kmsOldKey": "sessmgr20210416",
                "kmsNewKey": "sessmgr20210416",
                "jwksUrl": "https://cognito-idp.us-east-2.amazonaws.com/us-east-2_860PcgyKN/.well-known/jwks.json",
                "cloudDomain": "dev.aws-us-east-2.frickjack.com",
                "cookieDomain": ".frickjack.com",
                "lambdaImage": "027326493842.dkr.ecr.us-east-2.amazonaws.com/little/session_mgr:3.0.0",
                "lambdaVersions": [
                    {
                        "resourceName": "sessmgr20210416v2m6p1",
                        "description": "initial prod version"
                    },
                    {
                        "resourceName": "sessmgr20210515v3m0p0",
                        "description": "v3.0.0"
                    }
                ],
                "prodLambdaVersion": "sessmgr20210515v3m0p0",
                "gatewayDeployments": [
                    {
                        "resourceName": "deploy20210416r0",
                        "description": "initial deployment"
                    },
                    {
                        "resourceName": "deploy20210514",
                        "description": "add /versions"
                    }
                ],
                "prodDeployment": "deploy20210514",
                "betaDeployment": "deploy20210514"
            }
        }
    }
}
  • use the various little stack commands to create, update, and monitor the cloudformation stack - for example:
    little stack filter ./stackParams.json
    little stack validate ./stackParams.json
    little stack create ./stackParams.json
    little stack events ./stackParams.json
    little stack update ./stackParams.json
    ...

Cloudformation Patterns

Here are a couple examples to illustrate how little stack cloudformation templates use nunjucks to implement patterns that would be difficult with cloudformation alone.

Template decomposition

Splitting a large template between multiple files makes it easier to work with, and nunjucks' import directive provides the functionality to do that. For example, the root.json file of this api gateway template imports separate files to define resources for each API accessed via the gateway.

{% import "./authnApiStage.js.njk" as authnApi with context %}
{% import "./sessionMgrApiStage.js.njk" as sessmgr with context %}

The same import functionality allows the api resource to import its openapi definition from an external file:

    "apiGateway": {
      "Type" : "AWS::ApiGateway::RestApi",
      "Properties" : {
          "Description" : "simple call-through to lambda api",
          "EndpointConfiguration" : {
            "Types": ["EDGE"]
          },
          "MinimumCompressionSize" : 128,
          "Name" : "{{ "authn_api-" + stackParameters.DomainName }}",
          "Body": {% include "./authnOpenApi.json" %},
          "Tags": [
            {{ stackTagsStr }}
          ]
        }
    },

Resource Versioning

Resource versioning is a pattern that a few AWS API's (lambda, kms, and API gateway deployments anyway) rely on, but is not supported well by cloudformation. For example, this little template deploys infrastructure for littleware's session manager API. The "beta" stage of the API is backed by a lambda function, and the "prod" stage of the API is backed by a lambda alias that references a lambda version (snapshot) of the same lambda function. When a user wants to test new lambda code, she updates a variable in the parameters file to point at the Docker image with the new code (a sample parameters file is here)

"Littleware": {
  "TemplatePath": "lib/cloudformation/cloud/api/authclient/apiGateway.json",
  "Variables": {
        ...
    "sessmgr": {
      ...
      "lambdaImage": "027326493842.dkr.ecr.us-east-2.amazonaws.com/little/session_mgr:2.6.1",
      ...

When the new code is ready to be promoted to production, then the developer publishes a new version of the lambda, and points the production alias at that version. The parameters file defines variables like these:

    "lambdaImage": "027326493842.dkr.ecr.us-east-2.amazonaws.com/little/session_mgr:2.6.1",
    "lambdaVersions": [
        {
            "resourceName": "sessmgrVer20210416r0",
            "description": "initial prod version"
        }
    ],
    "prodLambdaVersion": "sessmgrVer20210416r0",

The nunjucks-enhanced cloudformation template looks like this:

    {% for item in stackVariables.sessmgr.lambdaVersions %}
      "{{ item.resourceName }}": {
        "Type" : "AWS::Lambda::Version",
        "Properties" : {
            "FunctionName" : { "Ref": "sessMgrLambda" },
            "Description": "{{ item.description }}"
          }
      },
    {% endfor %}

    "sessMgrLambdaAlias": {
      "Type" : "AWS::Lambda::Alias",
      "Properties" : {
          "Description" : "prod stage lambda alias",
          "FunctionName" : { "Ref": "sessMgrLambda" },
          "FunctionVersion" : { "Fn::GetAtt": ["{{ stackVariables.sessmgr.prodLambdaVersion }}", "Version"] },
          "Name" : "gateway_prod"
        }
    },

The kms API supports a similar mechanism for rotating keys where user code accesses an encryption key through an alias that can be moved to point at a new (rotated) key. Littleware's session manager infrastructure defines 3 alias to asymmetric KMS keys used to sign and verify JWT's: kmsSigningAlias, kmsNewAlias, kmsOldAlias. The signing alias points at the key for signing tokens (tokens expire after an hour). The "old" alias points at the key that was used for signing JWT's in the past, so token verification code can load the old public key after a key rotation. The "new" alias points at the key that will become the signing key after the next key rotation. We define the "new" key, so that verification code can just load all 3 keys at startup time, and continue to work after a key rotation (assuming keys rotate less frequently than we restart our services).

The little stack parameters file defines a name for each kms key managed by a stack, and a target for each kms alias.

            "sessmgr": {
                "kmsKeys": [
                    "sessmgr20210416"
                ],
                "kmsSigningKey": "sessmgr20210416",
                "kmsOldKey": "sessmgr20210416",
                "kmsNewKey": "sessmgr20210416",

The template consumes those variables.

    {#
       Support KMS key rotation.
       Add a new key when it's time to rotate, and
       move the key alias there. 
    #}
    {% for keyName in stackVariables.sessmgr.kmsKeys %}
    "{{ keyName }}": {
      "Type" : "AWS::KMS::Key",
      "Properties" : {
          "Description" : "asymmetric kms key for session mgr jwt signing and validation",
          "KeyPolicy" : {
              "Id": "key-consolepolicy-3",
              "Version": "2012-10-17",
              "Statement": [
                  {
                    "Sid": "Enable IAM User Permissions",
                    "Effect": "Allow",
                    "Principal": {
                      "AWS": {"Fn::Join": ["", 
                        ["arn:aws:iam::", {"Ref": "AWS::AccountId"}, ":root"]
                        ]}
                    },
                    "Action": "kms:*",
                    "Resource": "*"
                  }
              ]
          },
          "KeySpec" : "ECC_NIST_P256",
          "KeyUsage" : "SIGN_VERIFY",
          "PendingWindowInDays" : 7,
          "Tags": [
            { "Key": "Name", "Value": "{{ keyName }}" },
            {{ stackTagsStr }}
          ]
        }
    },
    {% endfor %}

    {% set kmsSigningAlias %}{{ "alias/littleware/api/" + (stackParameters.DomainName | replace(".", "-")) + "/sessMgrSigningKey" }}{% endset %}

    {# old signing key - rotated out #}
    {% set kmsOldAlias %}{{ "alias/littleware/api/" + (stackParameters.DomainName | replace(".", "-")) + "/sessMgrOldKey" }}{% endset %}

    {# new signing key - not yet used for signing #}
    {% set kmsNewAlias %}{{ "alias/littleware/api/" + (stackParameters.DomainName | replace(".", "-")) + "/sessMgrNewKey" }}{% endset %}

    "kmsSigningKey": {
      "Type" : "AWS::KMS::Alias",
      "Properties" : {
          "AliasName" : "{{ kmsSigningAlias }}",
          "TargetKeyId" : { "Ref": "{{ stackVariables.sessmgr.kmsSigningKey }}" }
        }
    },
    "kmsOldKey": {
      "Type" : "AWS::KMS::Alias",
      "Properties" : {
          "AliasName" : "{{ kmsOldAlias }}",
          "TargetKeyId" : { "Ref": "{{ stackVariables.sessmgr.kmsOldKey }}" }
        }
    },
    "kmsNewKey": {
      "Type" : "AWS::KMS::Alias",
      "Properties" : {
          "AliasName" : "{{ kmsNewAlias }}",
          "TargetKeyId" : { "Ref": "{{ stackVariables.sessmgr.kmsNewKey }}" }
        }
    },

The kms alias names are passed as part of the json configuration to the session manager lambda function. Nunjucks' dump filter makes it easy to generate and stringify json:

    "sessMgrLambda": {
      "Type" : "AWS::Lambda::Function",
      "Properties" : {
        "PackageType": "Image",
        "Code" : {
          "ImageUri": "{{ stackVariables.sessmgr.lambdaImage }}"
        },
        "Description" : "session manager API lambda",
        "Environment" : {
          "Variables": {
            "JAVA_TOOL_OPTIONS": "-Djava.util.logging.manager=org.apache.logging.log4j.jul.LogManager",
            "LITTLE_CLOUDMGR": {{
              {
                "little.cloudmgr.domain" : stackVariables.sessmgr.cloudDomain,
                "little.cloudmgr.sessionmgr.type": "aws",
                "little.cloudmgr.sessionmgr.localconfig": {}, 
                "little.cloudmgr.sessionmgr.awsconfig": {
                    "kmsPublicKeys": [
                      kmsSigningAlias, kmsOldAlias, kmsNewAlias
                    ],
                    "kmsSigningKey": kmsSigningAlias,
                    "oidcJwksUrl": stackVariables.sessmgr.jwksUrl
                },
                "little.cloudmgr.sessionmgr.lambdaconfig": {
                    "corsDomainWhiteList": [ stackVariables.sessmgr.cookieDomain ],
                    "cookieDomain": stackVariables.sessmgr.cookieDomain
                }
              } | dump | dump
            }}
          }
        },
        "FunctionName" : { "Fn::Join": [ "-", ["sessmgr", "{{ stackParameters.DomainName | replace(".", "-") }}",  { "Ref": "StackName" }, { "Ref": "StageName" }, "prod"]] },
        "MemorySize" : 768,
        "Role" : { "Fn::GetAtt": ["sessMgrRole", "Arn"] },
        "Tags": [
          {{ stackTagsStr }}
        ],
        "Timeout" : 5,
        "TracingConfig" : {
          "Mode": "Active"
        }
      }
    },

Summary

The little stack tools allow the use of the nunjucks template language in cloudformation templates to support various infrastructure patterns.