TzookB

My Recent Circle Project Deep Dive

Thu Jan 16 2020

As I'm quite proud with what our current CircleCI flow is doing. Especially as I managed to cut the run time in 70% from the original build, and got all of the "tasks" run separataly.

Here is the document I wrote down to explain to the team what is going on inside Circle:

Circle CI

We use Circle version 2.1 for our builds. That means we can use their latest features like: workflows, commands and orbs.

Workflows

We now have three different workflows for specific set of branches:

  • deploy_staging_prod
  • test_braches
  • deploy_integration

Each of them has its own set of jobs to run that are specific to our needs.

deploy_staging_prod

branches: prod, prod_preview, staging, staging_preview

this workflow is for our production deploys and it consists of the full stack of jobs we have.

Jobs:

  • checkout_code
  • unit_test
  • lint_code
  • prep_env_vars
  • build_code
  • deploy
  • publish_sentry

test_braches

branches: all branches, except: master, master_preview, prod, prod_preview, staging, staging_preview

this workflow is for our in development branches. It aims to be very fast and only give developers quick feedback on their code.

Jobs:

  • checkout_code
  • unit_test
  • lint_code

deploy_integration

branches: master, master_preview

This build our integration preview where the team can preview the work before taking it to the next phase of staging and production. Here we skip the code checks as we relied on that on the above feature branches checks. But this should be fast so the rest of the team could see the output of dev team.

Jobs:

  • checkout_code
  • prep_env_vars
  • build_code
  • deploy

Jobs

Jobs are a part of worfklows and the contains a list of rules to perform for a specific goal. For example checking out the code, installing dependecies and run tests.

We have a list of different jobs in the system and we will describe each one.

  • checkout_code
  • unit_test
  • lint_code
  • prep_env_vars
  • build_code
  • deploy
  • publish_sentry

checkout_code

This job is in charge of fetching the code, installing npm dependecies and caching both of them for future jobs to use. It takes advantage of Circle caching in cache of same git commits or unchange lock files.

unit_test

This job is pulling the code from our cached data. Then it runs our unit tests to verify the code works as expected.

lint_code

This job is pulling the code from our cached data. Then it runs linting on the code.

prep_env_vars

This job is pulling the code from our cached data. Then it performs bash commands to generate the need .env.production file. Attaching "sentry release" version if we are in a deployment branch. In the end it persists the created file to workflow workspace.

build_code

This job is pulling the code from our cached data, and pulling the created env file from the workspace data. Then it will bundle the app into the release version files. In the end it will attach the bundled files in to the workflow workspace.

deploy

This job is pulling the code from our cached data, and pulling the created env file from the workspace data. Then it will run the deployment bash script that will upload the code to S3 and invalidate our cdn.

publish_sentry

This job is pulling the code from our cached data. Gets the current git commit id and creates and releases to sentry.

Commands

Commands are a great way to "DRY" (don't repeat yourself). You define a set of steps you need and you can use one liner commands in your main jobs that take advantage of those commands.

Our different commands:

  • get_app_code
  • get_project
  • slack_on_deployment
  • skip_if_cache_exists
  • save_cache_flag

get_app_code

This command is simply fetching our cached codebase.

get_project

This command is fetching our cached codebase and the cached npm dependencies.

slack_on_deployment

This is how we notify slack on jobs progress. So we want to send a slack messsage in case a job failed or succeeded. It has severl parameters that can be used for different behaviours.

skip_if_cache_exists

There are certain jobs we would like to skip if nothing was changed. For example we dont need to run unit tests if there was no code change. So we are caching a flag and in case the flag exists we just skip the rest of the job.

save_cache_flag

This command create a file as flag that will be cached for future jobs to check and skip on with the commands "skip_if_cache_exists"

executors

These are another way to define the containers our jobs will run on. Currently we have only one, but if there are jobs that require PHP, Golan etc. You can create an executer an reuse it in your jobs.

version: 2.1

executors:
  my-executor:
    docker:
      - image: circleci/python:3.7.1-node-browsers

orbs:
  slack: circleci/slack@3.4.1

jobs:
  checkout_code:
    executor: my-executor
    steps:
      - checkout
      - save_cache:
          key: code-{{ .Environment.CIRCLE_SHA1 }}
          paths:
            - ./
      - restore_cache:
          key: deps-{{ checksum "yarn.lock" }}
      - run:
          name: yarn-install
          command: yarn install
      - save_cache:
          key: deps-{{ checksum "yarn.lock" }}
          paths:
            - ./node_modules
      - slack_on_deployment:
          success_message: "deployment started for [$CIRCLE_BRANCH]"
          failure_message: "deployment failed to start for [$CIRCLE_BRANCH]"

  prep_env_vars:
    executor: my-executor
    steps:
      - get_project
      - run:
          name: Prep Env Vars
          command: |
            while IFS='' read -r var || [[ -n "$var" ]]; do
              ./scripts/prep-var.sh $CIRCLE_BRANCH $var
            done < BUILD_VARS
      - run:
          name: Attach Git Version To Sentry env
          command: |
            if [[ "$CIRCLE_BRANCH" =~ ^(staging|staging_preview|prod|prod_preview)$ ]]; then
              SENTRY_VERSION=$(git rev-parse --verify HEAD)
              echo "SENTRY_RELEASE_VERSION=$SENTRY_VERSION" >> .env.production
            else
              echo "$CIRCLE_BRANCH is not in the list"
            fi
      - persist_to_workspace:
          root: ./
          paths:
            - .env.production
      - slack_on_deployment:
          fail_only: true
          failure_message: "deployment failed for [$CIRCLE_BRANCH] because of env_prep"

  unit_test:
    executor: my-executor
    steps:
      - skip_if_cache_exists:
          skiptype: "unittests"
      - get_project
      - run: 
          name: Unit Test
          command: yarn test --maxWorkers=4
      - save_cache_flag:
          skiptype: "unittests"
      - slack_on_deployment:
          fail_only: true
          failure_message: "deployment failed for [$CIRCLE_BRANCH] because of unit tests"
  
  lint_code:
    executor: my-executor
    steps:
      - skip_if_cache_exists:
          skiptype: "linting"
      - get_project
      - run: 
          name: Lint Code
          command: yarn lint
      - save_cache_flag:
          skiptype: "linting"
      - slack_on_deployment:
          fail_only: true
          failure_message: "deployment failed for [$CIRCLE_BRANCH] because of linting"

  build_code:
    executor: my-executor
    steps:
      - get_project
      - attach_workspace:
          at: ~/project
      - run:
          name: build_code
          command: yarn build && yarn build-storybook -o public/storybook
      - persist_to_workspace:
          root: ./
          paths:
            - ./public
      - slack_on_deployment:
          fail_only: true
          failure_message: "deployment failed for [$CIRCLE_BRANCH] because of code build"

  deploy:
    executor: my-executor
    steps:
      - get_app_code
      - attach_workspace:
          at: ~/project
      - run:
          name: Deploy to S3
          command: ./scripts/deploy.sh
      - slack_on_deployment:
          success_message: "deployment successful for => [$CIRCLE_BRANCH]"
          failure_message: "deployment failed for [$CIRCLE_BRANCH]"

  publish_sentry:
    executor: my-executor
    steps:
      - get_app_code
      - run:
          name: install sentry-cli
          command: curl -sL https://sentry.io/get-cli/ | bash
      - run:
          name: set Sentry release version
          command: echo $(git rev-parse --verify HEAD) > SENTRY_RELEASE_VERSION
      - run:
          name: create sentry release version
          command: sentry-cli releases --org $SENTRY_ORG --project $SENTRY_PROJECT new $(cat SENTRY_RELEASE_VERSION)
      - run:
          name: finalizing sentry release
          command: sentry-cli releases --org $SENTRY_ORG finalize $(cat SENTRY_RELEASE_VERSION)
      - run:
          name: finalizing sentry release
          command: sentry-cli releases --org $SENTRY_ORG deploys $(cat SENTRY_RELEASE_VERSION) new --env $CIRCLE_BRANCH

commands:
  get_app_code:
    steps:
      - restore_cache:
          name: restore app code
          key: code-{{ .Environment.CIRCLE_SHA1 }}
  get_project:
    steps:
      - restore_cache:
          name: restore app code
          key: code-{{ .Environment.CIRCLE_SHA1 }}
      - restore_cache:
          name: restore dependencies
          key: deps-{{ checksum "yarn.lock" }}
  slack_on_deployment:
    description: |
      notify deployments slack channel
    parameters:
      only_for_branches:
        default: "master,master_preview,prod,prod_preview,staging,staging_preview"
        description: comma-separated list of branches for which to send notifications. No spaces.
        type: string
      fail_only:
        default: false
        description: should notify on fail only or success as well
        type: boolean
      success_message:
        default: ""
        description: msg on success
        type: string
      failure_message:
        default: ""
        description: msg on failure
        type: string
    steps:
      - slack/status:
          channel: "deployments"
          success_message: "<<parameters.success_message>> - $CIRCLE_WORKFLOW_ID"
          failure_message: "<<parameters.failure_message>> - $CIRCLE_WORKFLOW_ID"
          only_for_branches: <<parameters.only_for_branches>>
          fail_only: <<parameters.fail_only>>

  skip_if_cache_exists:
    description: |
      a command to exit the job for selected branch
    parameters:
      skiptype:
        description: type of job to skip
        type: string
    steps:
      - restore_cache:
          key: skipcheck-<<parameters.skiptype>>-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
      - run: 
          name: if cache exists exit
          command: |
            FILE=~/cachedflags/job.<<parameters.skiptype>>.flag
            if test -f "$FILE"; then
                echo "$FILE exist"
                circleci step halt
            else
                echo "$FILE doesnt exist"
            fi
  save_cache_flag:
    description: |
      a command that will create the cache
    parameters:
      skiptype:
        description: type of job to skip
        type: string
    steps:
      - run:
          name: create job flag file
          command: mkdir -p ~/cachedflags/ && touch ~/cachedflags/job.<<parameters.skiptype>>.flag
      - save_cache:
          key: skipcheck-<<parameters.skiptype>>-{{ .Environment.CIRCLE_BRANCH }}-{{ .Environment.CIRCLE_SHA1 }}
          paths:
            - ~/cachedflags/job.<<parameters.skiptype>>.flag

workflows:
  deploy_staging_prod:
    jobs:
      - checkout_code:
          filters:
            branches:
              only:
                - prod
                - prod_preview
                - staging
                - staging_preview
      - unit_test:
          requires:
              - checkout_code
      - lint_code:
          requires:
              - checkout_code
      - prep_env_vars:
          requires:
              - checkout_code
      - build_code:
          requires:
              - unit_test
              - lint_code
              - prep_env_vars
      - deploy:
          requires:
              - build_code
      - publish_sentry:
          requires:
              - deploy

  test_braches:
    jobs:
      - checkout_code:
          filters:
            branches:
              ignore:
                - master
                - master_preview
                - prod
                - prod_preview
                - staging
                - staging_preview
      - unit_test:
          requires:
              - checkout_code
      - lint_code:
          requires:
              - checkout_code

  deploy_integration:
    jobs:
      - checkout_code:
          filters:
              branches:
                  only:
                    - master
                    - master_preview
      - prep_env_vars:
          requires:
              - checkout_code
      - build_code:
          requires:
              - prep_env_vars
      - deploy:
          requires:
              - build_code