Production Ready Kumologica: Github Action

Deploy Kumologica flow to the AWS account using Kumologica CLI and Github Action

Kumologica
4 min readMar 20, 2023
Kumologica Github Actions
Photo by Richy Great on Unsplash

This is a continuation of previous article:

This article puts together all commands discussed in previous article into github action workflow that fully automates deployment of Kumologica workflow into AWS Account.

AWS Setup

Deployment of Kumologica flow to the AWS Account requires correct AWS account setup:

  1. Create S3 Bucket used by cloud formation ‘prepare’ command. Replace ‘CHANGE_IT_TO_REAL_BUCKET_NAME’ placeholder in code examples below with the real bucket name.
  2. Create AWS IAM user with access keys and sufficient permissions. Save values of access key id and access key secret for Github actions secrets setup.

For instructions how to setup AWS IAM user and role with correct access policy to deploy Kumologica flows to AWS Account refer section Configuring AWS in our Getting Started Guide or run kumologica-designer.yaml script.

Github Action Workflow

Below is probably the most simplistic github action workflow that does build and deployment of Kumologica flow into aws account. The file will be split here into logical sections with comments.

Configuration

The initial section contains name of the workflow, trigger definition and environment variables of the workflow (env).

For simplicity, this section declares KL_ENVIRONMENT variable to define all environment variables required by Kumologica flow and KL_TRIGGERS variable to define all triggers required by Kumologica flow.

name: |
Example github action workflow demonstrating deployment
of Kumologica flow to the aws account
on:
push:
branches:
- main
env:
AWS_REGION: CHANGE_IT_TO_YOUR_REGION
LAMBDA_NAME: cli-demo-lambda
BUCKET_NAME: CHANGE_IT_TO_REAL_BUCKET_NAME
KL_ENVIRONMENT: '{"Variables": {"key": "value"}}'
KL_TRIGGERS: '[{"api": {"apiId": "CHANGE_IT_TO_API_GATEWAY_ID", "parentId": "CHANGE_IT_TO_PARENT_ID", "stage": "test", "resource": "accounts"}}, {"event": {"expression": "cron(0 1 * * ? *)", "reference": "1am", "name": "CliBuildDemoEvent1am"}}, {"event": {"expression": "rate(1 minute)", "reference": "5min", "name": "CliBuildDemoEvent5min"}}]'

Setting up job

This section sets up all components required by workflow:

  • node js
  • kumologica cli
  • checkout source code from github

Kumologica sdk is installed with--ignore-scripts flag. This bypasses installation of Kumologica Designer GUI. It also speeds up installation time.

It also requires definition of two github action secrets in repository:

  • AWS_ACCESS_KEY_ID
  • AWS_SECRET_ACCESS_KEY

with the values received during creation of AWS IAM User (Setup Section)

jobs:
deploy:
name: Deploy kumologica flow
runs-on: ubuntu-latest
strategy:
matrix:
node-version: [14.x]

steps:
# checkout repository
- name: checkout sources
uses: actions/checkout@v2

# setup aws credentials
- uses: actions/checkout@v2
- name: Configure AWS credentials
uses: aws-actions/configure-aws-credentials@v1
with:
aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
aws-region: ${{ env.AWS_REGION }}

# setup right node version
- name: Use Node.js ${{ matrix.node-version }}
uses: actions/setup-node@v1
with:
node-version: ${{ matrix.node-version }}

# install kumologica cli
- name: Install kumologica cli
run: |
npm install -g @kumologica/sdk --ignore-scripts

The following steps are almost a copy of the commands: kl build, kl export, aws cloud formation package and aws cloudformation deploy from the previous article:

KL Build

Not much to add here, for details about building aws lambda see previous article.


# build kumologica flow
- name: build kumologica flow lambda
run: kl build aws

Kl export

Cloud formation script will be named ‘template.json’ and created in ‘build’ subdirectory.

    # generate cloudformation script
# note we pass environment variables so iam role will
# contain valid resources
- name: generate cloudformation script
run: |
kl export cloudformation --project-directory . \
--environment '${{ env.KL_ENVIRONMENT }}' \
--zip-file-name lambda.zip \
--lambda-name ${{ env.LAMBDA_NAME }} \
--bucket-name ${{ env.BUCKET_NAME }} \
--triggers '${{ env.KL_TRIGGERS }}'

Aws cloudformation package

There is no github action for cloud formation package command. Fortunately aws cli is pre installed in the github action container:

    # cloud formation package
- name: cf package
run: |
aws cloudformation package \
--template-file ./build/template.json \
--s3-bucket ${{ env.BUCKET_NAME }} \
--s3-prefix ${{ env.LAMBDA_NAME }} \
--output-template-file template.yml

AWS cloudformation deploy

There is dedicated action for aws cloud formation deploy command. In this specific case cloud formation stack will not fail when deployment has no material changes.

Capability: CAPABILITY_NAMED_IAM is used since kl export command sets IAM role names.

    # cloudformation deploy
- name: Deploy CloudFormation Stack
uses: aws-actions/aws-cloudformation-github-deploy@v1.0.3
with:
name: ${{ env.LAMBDA_NAME }}
template: template.yml
capabilities: CAPABILITY_NAMED_IAM, CAPABILITY_AUTO_EXPAND
no-fail-on-empty-changeset: "1"

Deployment Check

The github workflow will execute every time changes are pushed to the main branch.

Once deploment succeeds the Kumologica workflow is testable using following command (remember to replace CHANGE_IT_TO_API_GATEWAY_ID and CHANGE_IT_TO_YOUR_REGION with values from env. section):

curl https://{CHANGE_IT_TO_API_GATEWAY_ID}.execute-api.{CHANGE_IT_TO_YOUR_REGION}.amazonaws.com/test/customers/accounts

[{"account":"11112222","name":"John Smith"}]

Conclusion

With the Kumologica CLI commands and its configurability it is very easy to orchestrate deployments into cloud provider using your preferred CI/CD provider. The example here covers Github Actions, however it should not be a problem to make a changes to support for example bitbucket actions.

Remember Kumologica is free to download and use. Go ahead and give it a try, we would love to hear your feedback.

More information

  • The complete github action workflow yml file has been added to the source code of previous article in our kumologica-demos github repository.
  • For Kumologica CLI Reference Guide see: Kumologica CLI Reference
  • For more information about Kumologica SDK Installation see: Kumologica SDK Installation
  • Join our community group on discord
  • Visit kumologica.com for information about sdk, designer, documentation, tutorials, support and professional services.

--

--

Kumologica

Kumologica is the first low-code development solution that makes your integration services run on serverless compute regardless the cloud provider.