Production Ready Kumologica: Github Action

Deploy Kumologica flow to the AWS account using Kumologica CLI and Github Action

Photo by Richy Great on Unsplash

This is a continuation of previous article:

Production Ready Kumologica: AWS Deploy

This article puts together all commands discussed in previous article into github action workflow that fully automates deployment of Kumologica workflow into AWS Account.

AWS Setup

Deployment of Kumologica flow to the AWS Account requires correct AWS account setup:

1. Create S3 Bucket used by cloud formation ‘prepare’ command. Replace ‘CHANGE_IT_TO_REAL_BUCKET_NAME’ placeholder in code examples below with the real bucket name.
2. Create AWS IAM user with access keys and sufficient permissions. Save values of access key id and access key secret for Github actions secrets setup.

For instructions how to setup AWS IAM user and role with correct access policy to deploy Kumologica flows to AWS Account refer section Configuring AWS in our Getting Started Guide or run kumologica-designer.yaml script.

Github Action Workflow

Below is probably the most simplistic github action workflow that does build and deployment of Kumologica flow into aws account. The file will be split here into logical sections with comments.

Configuration

The initial section contains name of the workflow, trigger definition and environment variables of the workflow (env).

For simplicity, this section declares KL_ENVIRONMENT variable to define all environment variables required by Kumologica flow and KL_TRIGGERS variable to define all triggers required by Kumologica flow.

name: |
Example github action workflow demonstrating deployment
of Kumologica flow to the aws account
on:
  push:
    branches:
      - main
env:
  AWS_REGION: CHANGE_IT_TO_YOUR_REGION
  LAMBDA_NAME: cli-demo-lambda
  BUCKET_NAME: CHANGE_IT_TO_REAL_BUCKET_NAME
  KL_ENVIRONMENT: '{"Variables": {"key": "value"}}'
  KL_TRIGGERS: '[{"api": {"apiId": "CHANGE_IT_TO_API_GATEWAY_ID", "parentId": "CHANGE_IT_TO_PARENT_ID", "stage": "test", "resource": "accounts"}},  {"event": {"expression": "cron(0 1 * * ? *)", "reference": "1am", "name": "CliBuildDemoEvent1am"}}, {"event": {"expression": "rate(1 minute)", "reference": "5min", "name": "CliBuildDemoEvent5min"}}]'

Setting up job

This section sets up all components required by workflow:

• node js
• kumologica cli
• checkout source code from github

Kumologica sdk is installed with--ignore-scripts flag. This bypasses installation of Kumologica Designer GUI. It also speeds up installation time.

It also requires definition of two github action secrets in repository:

• AWS_ACCESS_KEY_ID
• AWS_SECRET_ACCESS_KEY

with the values received during creation of AWS IAM User (Setup Section)

jobs:
  deploy:
    name: Deploy kumologica flow
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [14.x]

    steps:
    # checkout repository
    - name: checkout sources
      uses: actions/checkout@v2

    # setup aws credentials
    - uses: actions/checkout@v2
    - name: Configure AWS credentials
      uses: aws-actions/configure-aws-credentials@v1
      with:        
        aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
        aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}    
        aws-region:  ${{ env.AWS_REGION }}

    # setup right node version
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}

    # install kumologica cli
    - name: Install kumologica cli
      run: |
        npm install -g @kumologica/sdk --ignore-scripts

The following steps are almost a copy of the commands: kl build, kl export, aws cloud formation package and aws cloudformation deploy from the previous article:

KL Build

Not much to add here, for details about building aws lambda see previous article.

    # build kumologica flow
    - name: build kumologica flow lambda
      run: kl build aws

Kl export

Cloud formation script will be named ‘template.json’ and created in ‘build’ subdirectory.

    # generate cloudformation script
    # note we pass environment variables so iam role will
    # contain valid resources
    - name: generate cloudformation script
      run: |
        kl export cloudformation --project-directory . \
           --environment '${{ env.KL_ENVIRONMENT }}' \
           --zip-file-name lambda.zip \
           --lambda-name ${{ env.LAMBDA_NAME }} \
           --bucket-name ${{ env.BUCKET_NAME }} \
           --triggers '${{ env.KL_TRIGGERS }}'

Aws cloudformation package

There is no github action for cloud formation package command. Fortunately aws cli is pre installed in the github action container:

    # cloud formation package
    - name: cf package
      run: |
        aws cloudformation package \
          --template-file ./build/template.json \
          --s3-bucket ${{ env.BUCKET_NAME }} \
          --s3-prefix ${{ env.LAMBDA_NAME }} \
          --output-template-file template.yml

AWS cloudformation deploy

There is dedicated action for aws cloud formation deploy command. In this specific case cloud formation stack will not fail when deployment has no material changes.

Capability: CAPABILITY_NAMED_IAM is used since kl export command sets IAM role names.

    # cloudformation deploy
    - name: Deploy CloudFormation Stack
      uses: aws-actions/aws-cloudformation-github-deploy@v1.0.3        
      with:
        name: ${{ env.LAMBDA_NAME }}
        template: template.yml
        capabilities: CAPABILITY_NAMED_IAM, CAPABILITY_AUTO_EXPAND
        no-fail-on-empty-changeset: "1"

Deployment Check

The github workflow will execute every time changes are pushed to the main branch.

Once deploment succeeds the Kumologica workflow is testable using following command (remember to replace CHANGE_IT_TO_API_GATEWAY_ID and CHANGE_IT_TO_YOUR_REGION with values from env. section):

curl https://{CHANGE_IT_TO_API_GATEWAY_ID}.execute-api.{CHANGE_IT_TO_YOUR_REGION}.amazonaws.com/test/customers/accounts

[{"account":"11112222","name":"John Smith"}]

Conclusion

With the Kumologica CLI commands and its configurability it is very easy to orchestrate deployments into cloud provider using your preferred CI/CD provider. The example here covers Github Actions, however it should not be a problem to make a changes to support for example bitbucket actions.

Remember Kumologica is free to download and use. Go ahead and give it a try, we would love to hear your feedback.

More information

• The complete github action workflow yml file has been added to the source code of previous article in our kumologica-demos github repository.
• For Kumologica CLI Reference Guide see: Kumologica CLI Reference
• For more information about Kumologica SDK Installation see: Kumologica SDK Installation
• Join our community group on discord
• Visit kumologica.com for information about sdk, designer, documentation, tutorials, support and professional services.