Github actions

Github actions is cloud service which is offered by Github. It is the process for continuous integration and continuous delivery.

Continuous Integration

Continuous Integration is a process of building, testing and merging the new code to existing code

Continuous Delivery/Deployment

Continuous deployment is a process of releasing the new version of the application/package which the integrated changes

Workflow Automation by Github

• CI/CD - testing, building and deploying the code /application

• Repository Management - Automate code reviews, issue management etc.

Git Quick Guide

To configure git locally with username and user email

git config --global user.name "your username"
git config --global user.email "your email"

To initialize project

git init

This will add an invisible .git folder to track all the file

Create Commits

We have two parts in creating commits

1. To stage the changes

2. To commit the stages changes

# To stage
git add <filename> <filename>
#or
git add . # for all the changed files

#To commit
git commit -m <"message for commit">

To know the status of the commit

git status

To know the entire commit log/history

git log

Move between commits

when we do git log HEAD=>main which means the head points to the latest commit in main branch

We might have multiple commits; we can use checkout command to move between commits or branches

when we move to different old commit the the HEAD will point to that commit and we can move to latest by

git checkout main

git checkout <commit_id> | <branch?>

Undo Commits

To undo a commit we can use

git revert <commit_id>

This will add a new commit by reverting that commit and we can see that in git log also

To reset / delete the commit we can use

git reset --hard <commit_id>

but this will entirely delete that commit content and log history also.

Git Ignore

There might be some local files/ env files which we do not want to commit for these files we will create a separate file called .gitignore where in this file we will add all the file/directory names which needs to be ignored

Branches

Git branches are a way to separate the code and make fixes to new version of code and merge them to the main branch which leads to clean practices and by not fidgeting the production code

To create a branch

git branch <branch name>

To change default branch in local

git checkout <branch_name>

To delete branch

git branch -D <branch_name>

To create and change default branch

git checkout -b <branch_name>

Merge branches

We need to go base branch first and then run the below command from which we need to get the code to be merged

(base->main)
git merge <branch_name>

To set url of remote github repository

git remote set-url origin <url>
git remot add origin <url>

To get url of remote github

git remote get-url origin

Github Actions

The Github actions has three key elements in it.

• Workflows

• Jobs

• Steps

Workflow

• Attached to github repository

• Contain one or more jobs

• Triggered upon events

Jobs

• Define a Runner (execution environment)

• Contain one or more steps

• Run in parallel (default) or sequential

• Can be conditioned

Steps

• Execute a shell script or an action

• Can use custom or third party action

• Steps are executed in order

• can be conditioned

Simple Hello world runner

# name of the workflow
name: first action
# triggger , here it is manual for workflow_dispatch
on: workflow_dispatch
jobs:# reserved key
#user defined job name
  first_job:
# defining the runner
    runs-on: ubuntu-latest
    steps:
#name for steps
      - name: Print greeting
# run the shell script command
        run: echo "Hello world"
      - name: Print Good bye
        run: echo "done -bye"

To run multiple commands in same run

run: |
    echo "First output"
    echo "Second output"

To run multiple jobs

name: Test code and deploy
on: workflow_dispatch
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Get Code
        uses: actions/checkout@v3
      - name: Install node js v18
        uses: actions/setup-node@v3
        with:
          node-version: 18
      - name: Install mocha
        run: npm install --global mocha
      - name: Run test
        run: mocha test/add.test.js
  deploy: 
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Get Code
        uses: actions/checkout@v3
      - name: Install node js v18
        uses: actions/setup-node@v3
        with:
          node-version: 18
      - name: Deploy simulation
        run: echo "code deployed"

In the above example we have 2 jobs

Jobs

• The jobs are can be multiple in number

• The jobs by default will run in parallel but if we want to run one job after the other we can use needs:<job-name>

• For every job we need to assign a new runner machine

Actions

• Actions are predefined/thirdparty ymls which makes our ci/cd flow easy.

• We can create our own actions and use that

• to use actions we will use uses key and with action name and for options in action we will use with

Event Triggers

Please refer this link to see all the triggers for the workflow

we can add multiple event triggers using array

Context:

name: To display the github context variable
on: workflow_dispatch
jobs:
  info:
    runs-on: ubuntu-latest
    steps:
      - name: Display github context
        # Here the github is the context variable where will get all the details about the context
        run: echo "${{toJSON(github)}}"

Activity Types and Filters

• We have different events in GitHub actions like pull_request, push, issue, for these events we will have activity types and for other we will have filters to execute the github action

name: Events Demo 1
on:
  pull_request: #this is the event
    types: # this is the activity type for opened we will trigger wf
      - opened
    branches: # this is the filter to traget branches only we will trigger wf
      - main # main
      - 'dev-*' # dev-new dev-this-is-new
      - 'feat/**' # feat/new feat/new/button
  workflow_dispatch:
  push: # this is also event
    branches: # this is the filter to target branches only we will trigger wf
      - main # main
      - 'dev-*' # dev-new dev-this-is-new
      - 'feat/**' # feat/new feat/new/button
      # developer-1
    paths-ignore: # This is to ignore( when changes are in this dir ignore)
      - '.github/workflows/*'
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Output event data
        run: echo "${{ toJSON(github.event) }}"
      - name: Get code
        uses: actions/checkout@v3
      - name: Install dependencies
        run: npm ci
      - name: Test code
        run: npm run test
      - name: Build code
        run: npm run build
      - name: Deploy project
        run: echo "Deploying..."

1. There are many events, some are repository related (pull request, push) and some are general (scheduled)

2. Initial approval is required for the pull_request which is from forked repositories to avoid spamming

3. Workflows gets cancelled when the jobs fail or else we can manually cancel if we need it.

Job Artifacts, outputs and Caching dependency

Job Artifacts

• when we build code or run tests we will get binaries or code reports these are job artifacts which we can use it to pass to other job or download it through the github UI

Job Outputs

• The job ouputs are just variable data which is generated in one job and can be used in other job

Caching dependency

• The dependency caching is when we need some dependencies like (node_modules or pycache) to run the application or build it we can cache it to lower the resource usage and time

name: Deploy website
on:
  push:
    branches:
      - main
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
    # here we are caching dependencies using cache and it takes the depency path which is 
# described in docs and key is a unique hash generated when the package.lock.json is chnaged
# if the package the lock doesnt change this means it can take the cache or else it will re rerun
# the dependencies
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        run: npm ci
      - name: Lint code
        run: npm run lint
      - name: Test code
        run: npm run test
  build:
    needs: test
    runs-on: ubuntu-latest
# for using the output variable we will use this outputs and with outputs variable names and map 
# it to the variables which we give before ={} with format steps.<id>.outputs.<file_name_before{}>

    outputs:
      script-file: ${{ steps.publish.outputs.script-file }}
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        run: npm ci
      - name: Build website
        run: npm run build
# Here we are taking a filename and assigning to varaible to use it in another job
      - name: Publish JS filename
# this id is given to access the output variable
        id: publish
# we are running the linux command to get filenamae and output that to $GITHUB_OUTPUT 
        run: find dist/assets/*.js -type f -execdir echo 'script-file={}' >> $GITHUB_OUTPUT ';'
    # The upload artifacts uses an action to get the artifacts(build files)
      - name: Upload artifacts
        uses: actions/upload-artifact@v3
       # This with takes a name for artifact and path with which we can upload/download pass it to other job
        with:
          name: dist-files
          path: dist
          # path: |
          #   dist
          #   package.json
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
    # Here we are using download-artifact to get artifact with above same name
      - name: Get build artifacts
        uses: actions/download-artifact@v3
        with:
          name: dist-files
    # we are outputting the contents of the folder
      - name: Output contents
        run: ls
      - name: Output filename
        run: echo "${{ needs.build.outputs.script-file }}"
      - name: Deploy
        run: echo "Deploying..."

Secrets & Environment variables

Environment Variables

• Dynamic values can be used in code

• Can be defined at WF /job/step level

• Accessible via interpolation $ or env context

Secrets

• Some dynamic values should not be exposed

• Can be stored at Repository level or environment level

• Secrets can be referred by secrets context

Github Action Environment

• Jobs can refer different environemnts

• We can store secrets at environment level

• It Allows extra protection

name: Deployment
on:
  push:
    branches:
      - main
      - dev
env:
  MONGODB_DB_NAME: gha-demo
jobs:
  test:
    environment: testing # This is where we tell wf to take env testing secrets 
    runs-on: ubuntu-latest
    env: # Reserved key to declare envs
      MONGODB_CLUSTER_ADDRESS: cluster0.ntrwp.mongodb.net # This is hard coded envs
      MONGODB_USERNAME: ${{ secrets.MONGODB_USERNAME }} # This is stored in secrets at repo/env level
      MONGODB_PASSWORD: ${{ secrets.MONGODB_PASSWORD }}
      PORT: 8080
    steps:
      - name: Get Code
        uses: actions/checkout@v3
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: npm-deps-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        run: npm ci
      - name: Run server
        run: npm start & npx wait-on http://127.0.0.1:$PORT #This is the interpolation way to access envs
      - name: Run tests
        run: npm test
      - name: Output information
        run: |
          echo "MONGODB_USERNAME: $MONGODB_USERNAME"
  deploy:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Output information
        env:
          PORT: 3000
        run: |        
          echo "MONGODB_DB_NAME: $MONGODB_DB_NAME"
          echo "MONGODB_USERNAME: $MONGODB_USERNAME"
          echo "${{ env.PORT }}"

Conditional Workflow

• In workflows we can use if condition to move forward or stop with the flow of the jobs

• we have multiple functions to deal with it

always() # always the job or step should move forward
failure() # on failure it shows
success() # for success jobs /steps
cancelled() # for cancelled one's
continue-on-error # This will continue even if there are errors (ignore step failures)

name: Website Deployment
on:
  push:
    branches:
      - main
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        # here we are using if condition whether to check cache is hit if not we will run npm ci
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Lint code
        run: npm run lint
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Test code
        id: run-tests
        run: npm run test
      - name: Upload test report
        # here we are checking for failure in step and check outcome is failure or not
        if: failure() && steps.run-tests.outcome == 'failure'
        uses: actions/upload-artifact@v3
        with:
          name: test-report
          path: test.json
  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Build website
        id: build-website
        run: npm run build
      - name: Upload artifacts
        uses: actions/upload-artifact@v3
        with:
          name: dist-files
          path: dist
  deploy:
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Get build artifacts
        uses: actions/download-artifact@v3
        with:
          name: dist-files
      - name: Output contents
        run: ls
      - name: Deploy
        run: echo "Deploying..."
  report:
    needs: [lint, deploy]
    # here we are checking failure for job level any job failure this will show up
    if: failure()
    runs-on: ubuntu-latest
    steps:
      - name: Output information
        run: | 
          echo "Something went wrong"
          echo "${{ toJSON(github) }}"

Matrix strategy

This strategy will make our workflows to run in different environments

We can add or remove these configurations

name: Matrix Demo
on: push
jobs:
  build:
    # this will continue even if there are errors
    continue-on-error: true
    strategy:
      matrix:
        # we can have multiple combinations like this
        node-version: [12, 14, 16]
        operating-system: [ubuntu-latest, windows-latest]
        include:
            # we can include extra combination here
          - node-version: 18
            operating-system: ubuntu-latest
        exclude:
            # we can exclude existing combination
          - node-version: 12
            operating-system: windows-latest
    runs-on: ${{ matrix.operating-system }}
    steps:
      - name: Get Code
        uses: actions/checkout@v3
      - name: Install NodeJS
        uses: actions/setup-node@v3
        with:
          node-version: ${{ matrix.node-version }}
      - name: Install Dependencies
        run: npm ci
      - name: Build project
        run: npm run build

Reusable workflow

• Workflows can be used via workflow_call

• work with any job or steps and we uses all inputs,outputs and secrets in this context

#reusable workflow
name: Reusable Deploy
on: 
  workflow_call: # callable in other workflow
    inputs: # takes input
      artifact-name: # input name
        description: The name of the deployable artifact files
        required: false
        default: dist 
        type: string
    outputs: # gives output
      result:
        description: The result of the deployment operation
        value: ${{ jobs.deploy.outputs.outcome }}
    # secrets:
      # some-secret:
        # required: false
jobs:
  deploy:
    outputs:
      outcome: ${{ steps.set-result.outputs.step-result }}
    runs-on: ubuntu-latest
    steps:
      - name: Get Code
        uses: actions/download-artifact@v3
        with:
          name: ${{ inputs.artifact-name }}
      - name: List files
        run: ls
      - name: Output information
        run: echo "Deploying & uploading..."
      - name: Set result output
        id: set-result
        run: echo "step-result=success" >> $GITHUB_OUTPUT

workflow using above worflow

name: Using Reusable Workflow
on:
  push:
    branches:
      - main
jobs:
  lint:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Lint code
        run: npm run lint
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Test code
        id: run-tests
        run: npm run test
      - name: Upload test report
        if: failure() && steps.run-tests.outcome == 'failure'
        uses: actions/upload-artifact@v3
        with:
          name: test-report
          path: test.json
  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Get code
        uses: actions/checkout@v3
      - name: Cache dependencies
        id: cache
        uses: actions/cache@v3
        with:
          path: node_modules
          key: deps-node-modules-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        if: steps.cache.outputs.cache-hit != 'true'
        run: npm ci
      - name: Build website
        id: build-website
        run: npm run build
      - name: Upload artifacts
        uses: actions/upload-artifact@v3
        with:
          name: dist-files
          path: dist
  deploy:
    needs: build
    uses: ./.github/workflows/reusable.yml # here we are using above workflow
    with:
      artifact-name: dist-files # sending input
    # secrets:
      # some-secret: ${{ secrets.some-secret }}
  print-deploy-result:
    needs: deploy
    runs-on: ubuntu-latest
    steps:
      - name: Print deploy output
        run: echo "${{ needs.deploy.outputs.result }}" # getting the output
  report:
    needs: [lint, deploy]
    if: failure()
    runs-on: ubuntu-latest
    steps:
      - name: Output information
        run: | 
          echo "Something went wrong"
          echo "${{ toJSON(github) }}"

Containers

• We can run the application inside a custom container in the jobs where it is handier to run the application in custom environment

• Build your own one or use community images

Service containers

• Extra services can be using by steps in jobs

• where the application can have a connection with the container db which we can setup (like docker compose)

• This can also be custom image/ community images

name: Deployment (Container)
on:
  push:
    branches:
      - main
      - dev
env:
  CACHE_KEY: node-deps
  MONGODB_DB_NAME: gha-demo
jobs:
  test:
    environment: testing
    runs-on: ubuntu-latest
    # This the container on job level to setup env to run our job
    # container:
    #   image: node:16
    env:
      MONGODB_CONNECTION_PROTOCOL: mongodb
      MONGODB_CLUSTER_ADDRESS: 127.0.0.1:27017 # here if we are running on containers then we can use label as IP and no port is required
      MONGODB_USERNAME: root
      MONGODB_PASSWORD: example
      PORT: 8080
    services:
      mongodb:
        image: mongo
        ports: # if running on container, no need to use ports
          - 27017:27017
        env:
          MONGO_INITDB_ROOT_USERNAME: root
          MONGO_INITDB_ROOT_PASSWORD: example
    steps:
      - name: Get Code
        uses: actions/checkout@v3
      - name: Cache dependencies
        uses: actions/cache@v3
        with:
          path: ~/.npm
          key: ${{ env.CACHE_KEY }}-${{ hashFiles('**/package-lock.json') }}
      - name: Install dependencies
        run: npm ci
      - name: Run server
        run: npm start & npx wait-on http://127.0.0.1:$PORT # requires MongoDB Atlas to accept requests from anywhere!
      - name: Run tests
        run: npm test
      - name: Output information
        run: |
          echo "MONGODB_USERNAME: $MONGODB_USERNAME"
  deploy:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - name: Output information
        env:
          PORT: 3000
        run: |        
          echo "MONGODB_DB_NAME: $MONGODB_DB_NAME"
          echo "MONGODB_USERNAME: $MONGODB_USERNAME"
          echo "${{ env.PORT }}"