This post go over the installation, basic resource, and a use cases of Tekton Pipelines.

More information can be found on there website tekton.dev.

Tekton Installation

Pipeline

To install Tekton Pipeline on your cluster just run the following kubectl command.

kubectl apply --filename https://storage.googleapis.com/tekton-releases/pipeline/latest/release.yaml

If you want to deploy a specific release replace latest with previous/vX.Y.Z. Releases can be found on there GitHub here.

Operator

It can also be installed using the Tekton Operator which can be found here

kubectl apply -f https://storage.googleapis.com/tekton-releases/operator/latest/release.yaml

Dashboard

Tekton also offers a web dashboard to view Tekton resources. Documentation and installation can be found here.

Tekton Concepts

I recommend you checkout the official documentation here since its better than my explanation.

Steps

At the lowest level, Tekton implements a Step resource. A Tekton Step defines an atomic operation in a pipeline, such as cloning down a Git repository, kicking off a build, and so forth.

Tasks

Defines a series of Steps that run sequentially to perform logic Every Task runs as a pod on the Kubernetes cluster, with each step running in its own container.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: hello
spec:
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        echo "Hello World"

TaskRuns

TaskRun execute the steps in a task.

apiVersion: tekton.dev/v1
kind: TaskRun
metadata:
  name: hello-task-run
spec:
  taskRef:
    name: hello

View logs of output kubectl logs --selector=tekton.dev/taskRun=hello-task-run

Pipelines

A Pipeline defines an ordered series of Tasks arranged in a specific execution order as part of the CI/CD workflow

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: hello-goodbye
spec:
  tasks:
    - name: hello
      taskRef:
        name: hello
    - name: goodbye
      runAfter:
        - hello
      taskRef:
        name: goodbye

PipelineRuns

A PipelineRun, represented in the API as an object, sets the value for the parameters and executes a Pipeline

apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: hello-goodbye-run
spec:
  pipelineRef:
    name: hello-goodbye

Params

Unless you want the task to run with the same input every time parameters can be passed in from the Pipeline, TaskRuns or PipelineRuns.

Documentation on parameters in tasks.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: task-with-parameters
spec:
  params:
    - name: foo
      description: description for the parameter
      default: test
      type: string
  steps:
    - name: echo
      image: alpine
      script: |
        #!/bin/sh
        echo "$(params.foo)"

Docs on parameters in Pipelines can be found here.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: pipeline-with-parameters
spec:
  params:
    - name: input
      type: string
      description: Value for task
      default: default
  tasks:
    - name: task-with-parameters
      taskRef:
        name: task-with-parameters
      params:
        - name: foo
          value: $(params.input)

Calling the pipeline with parameters documentation.

Workspaces

Workspaces allow you to pass in storage that will be used by your tasks. General documentation can be found here.

In this example the task will be passed a workspace that will be used to store files.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: read-from-directory
spec:
  workspaces:
    - name: source
      description: location to share files between tasks
  steps:
    - name: read-dir
      image: docker.io/library/bash
      script: |
        #!/bin/sh
        ls "$(workspaces.source.path)"

When a PipelineRun is triggered a workspace will be passed in which will be passed to the Pipeline and then the tasks associated if they request it.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: read-from-directory-pipeline
spec:
  workspaces:
    - name: shared-data
      description: location to share files between tasks
  tasks:
    - name: read-from-directory
      taskRef:
        name: read-from-directory
      workspaces:
        - name: source
          workspace: shared-data

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: read-from-directory-pipeline-run-1
spec:
  pipelineRef:
    name: read-from-directory-pipeline
  workspaces:
    - name: shared-data
      emptyDir:
        sizeLimit: 2Gi

Tekton also supports workspaces from secrets, configmaps, PVC, volume claims, and emptyDir.

Triggering Jobs

Pipelines can be triggered manually with PipelineRuns but that does not scale well.

Tekton Triggers

Documentation can be found here.

kubectl apply --filename https://storage.googleapis.com/tekton-releases/triggers/latest/release.yaml
kubectl apply --filename https://storage.googleapis.com/tekton-releases/triggers/latest/interceptors.yaml

Release can be found here on Github.

Useful Resources

Use Cases

Automated Image Building

I want to highlight the combination of pulling from GitHub and building images using Kaniko.

The task is build on the following tektoncd catalogs git-clone and kaniko

This task could be separated into two tasks but then you would need a persistent storage between them.

apiVersion: tekton.dev/v1
kind: Task
metadata:
  name: clone-build-push
spec:
  description: |
    This task will Clone your Git repository and then build the image with kaniko before pushing it to your artifact repository.
    The steps are a combination of the Tekton catalogs git-clone & kaniko.    
  workspaces:
    - name: ssh-directory
      optional: true
      description: |
        A .ssh directory with private key, known_hosts, config, etc. Copied to
        the user's home before git commands are executed. Used to authenticate
        with the git remote when performing the clone. Binding a Secret to this
        Workspace is strongly recommended over other volume types.        
    - name: basic-auth
      optional: true
      description: |
        A Workspace containing a .gitconfig and .git-credentials file. These
        will be copied to the user's home before any git commands are run. Any
        other files in this Workspace are ignored. It is strongly recommended
        to use ssh-directory over basic-auth whenever possible and to bind a
        Secret to this Workspace over other volume types.        
    - name: ssl-ca-directory
      optional: true
      description: |
        A workspace containing CA certificates, this will be used by Git to
        verify the peer with when fetching or pushing over HTTPS.        
    - name: source
      description: The git repo will be cloned onto the volume backing this Workspace.
    - name: dockerconfig
      description: Includes a docker `config.json`
      optional: true
      mountPath: /kaniko/.docker
  params:
    - name: IMAGE
      description: Name (reference) of the image to build.
    - name: DOCKERFILE
      description: Path to the Dockerfile to build.
      default: ./Dockerfile
    - name: CONTEXT
      description: The build context used by Kaniko.
      default: ./
    - name: EXTRA_ARGS
      type: array
      default: []
    - name: BUILDER_IMAGE
      description: The image on which builds will run (default is v1.5.1)
      default: gcr.io/kaniko-project/executor:v1.5.1@sha256:c6166717f7fe0b7da44908c986137ecfeab21f31ec3992f6e128fff8a94be8a5
    - name: url
      description: Repository URL to clone from.
      type: string
    - name: revision
      description: Revision to checkout. (branch, tag, sha, ref, etc...)
      type: string
      default: ""
    - name: refspec
      description: Refspec to fetch before checking out revision.
      default: ""
    - name: submodules
      description: Initialize and fetch git submodules.
      type: string
      default: "true"
    - name: depth
      description: Perform a shallow clone, fetching only the most recent N commits.
      type: string
      default: "1"
    - name: sslVerify
      description: Set the `http.sslVerify` global git config. Setting this to `false` is not advised unless you are sure that you trust your git remote.
      type: string
      default: "true"
    - name: crtFileName
      description: file name of mounted crt using ssl-ca-directory workspace. default value is ca-bundle.crt.
      type: string
      default: "ca-bundle.crt"
    - name: subdirectory
      description: Subdirectory inside the `output` Workspace to clone the repo into.
      type: string
      default: ""
    - name: sparseCheckoutDirectories
      description: Define the directory patterns to match or exclude when performing a sparse checkout.
      type: string
      default: ""
    - name: deleteExisting
      description: Clean out the contents of the destination directory if it already exists before cloning.
      type: string
      default: "true"
    - name: httpProxy
      description: HTTP proxy server for non-SSL requests.
      type: string
      default: ""
    - name: httpsProxy
      description: HTTPS proxy server for SSL requests.
      type: string
      default: ""
    - name: noProxy
      description: Opt out of proxying HTTP/HTTPS requests.
      type: string
      default: ""
    - name: verbose
      description: Log the commands that are executed during `git-clone`'s operation.
      type: string
      default: "true"
    - name: gitInitImage
      description: The image providing the git-init binary that this Task runs.
      type: string
      default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.40.2"
    - name: userHome
      description: |
        Absolute path to the user's home directory.        
      type: string
      default: "/home/git"
  results:
    - name: commit
      description: The precise commit SHA that was fetched by this Task.
    - name: url
      description: The precise URL that was fetched by this Task.
    - name: committer-date
      description: The epoch timestamp of the commit that was fetched by this Task.
    - name: IMAGE_DIGEST
      description: Digest of the image just built.
    - name: IMAGE_URL
      description: URL of the image just built.
  steps:
    - name: clone
      image: "$(params.gitInitImage)"
      env:
      - name: HOME
        value: "$(params.userHome)"
      - name: PARAM_URL
        value: $(params.url)
      - name: PARAM_REVISION
        value: $(params.revision)
      - name: PARAM_REFSPEC
        value: $(params.refspec)
      - name: PARAM_SUBMODULES
        value: $(params.submodules)
      - name: PARAM_DEPTH
        value: $(params.depth)
      - name: PARAM_SSL_VERIFY
        value: $(params.sslVerify)
      - name: PARAM_CRT_FILENAME
        value: $(params.crtFileName)
      - name: PARAM_SUBDIRECTORY
        value: $(params.subdirectory)
      - name: PARAM_DELETE_EXISTING
        value: $(params.deleteExisting)
      - name: PARAM_HTTP_PROXY
        value: $(params.httpProxy)
      - name: PARAM_HTTPS_PROXY
        value: $(params.httpsProxy)
      - name: PARAM_NO_PROXY
        value: $(params.noProxy)
      - name: PARAM_VERBOSE
        value: $(params.verbose)
      - name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
        value: $(params.sparseCheckoutDirectories)
      - name: PARAM_USER_HOME
        value: $(params.userHome)
      - name: WORKSPACE_OUTPUT_PATH
        value: $(workspaces.source.path)
      - name: WORKSPACE_SSH_DIRECTORY_BOUND
        value: $(workspaces.ssh-directory.bound)
      - name: WORKSPACE_SSH_DIRECTORY_PATH
        value: $(workspaces.ssh-directory.path)
      - name: WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND
        value: $(workspaces.basic-auth.bound)
      - name: WORKSPACE_BASIC_AUTH_DIRECTORY_PATH
        value: $(workspaces.basic-auth.path)
      - name: WORKSPACE_SSL_CA_DIRECTORY_BOUND
        value: $(workspaces.ssl-ca-directory.bound)
      - name: WORKSPACE_SSL_CA_DIRECTORY_PATH
        value: $(workspaces.ssl-ca-directory.path)
      securityContext:
        runAsNonRoot: true
        runAsUser: 65532
      resources:
        limits:
          cpu: 200m
          memory: 1Gi
        requests:
          cpu: 100m
          memory: 500Mi
      script: |
        #!/usr/bin/env sh
        set -eu

        if [ "${PARAM_VERBOSE}" = "true" ] ; then
          set -x
        fi

        if [ "${WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND}" = "true" ] ; then
          cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.git-credentials" "${PARAM_USER_HOME}/.git-credentials"
          cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.gitconfig" "${PARAM_USER_HOME}/.gitconfig"
          chmod 400 "${PARAM_USER_HOME}/.git-credentials"
          chmod 400 "${PARAM_USER_HOME}/.gitconfig"
        fi

        if [ "${WORKSPACE_SSH_DIRECTORY_BOUND}" = "true" ] ; then
          cp -R "${WORKSPACE_SSH_DIRECTORY_PATH}" "${PARAM_USER_HOME}"/.ssh
          chmod 700 "${PARAM_USER_HOME}"/.ssh
          chmod -R 400 "${PARAM_USER_HOME}"/.ssh/*
        fi

        if [ "${WORKSPACE_SSL_CA_DIRECTORY_BOUND}" = "true" ] ; then
           export GIT_SSL_CAPATH="${WORKSPACE_SSL_CA_DIRECTORY_PATH}"
           if [ "${PARAM_CRT_FILENAME}" != "" ] ; then
              export GIT_SSL_CAINFO="${WORKSPACE_SSL_CA_DIRECTORY_PATH}/${PARAM_CRT_FILENAME}"
           fi
        fi
        CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}"

        cleandir() {
          # Delete any existing contents of the repo directory if it exists.
          #
          # We don't just "rm -rf ${CHECKOUT_DIR}" because ${CHECKOUT_DIR} might be "/"
          # or the root of a mounted volume.
          if [ -d "${CHECKOUT_DIR}" ] ; then
            # Delete non-hidden files and directories
            rm -rf "${CHECKOUT_DIR:?}"/*
            # Delete files and directories starting with . but excluding ..
            rm -rf "${CHECKOUT_DIR}"/.[!.]*
            # Delete files and directories starting with .. plus any other character
            rm -rf "${CHECKOUT_DIR}"/..?*
          fi
        }

        if [ "${PARAM_DELETE_EXISTING}" = "true" ] ; then
          cleandir || true
        fi

        test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}"
        test -z "${PARAM_HTTPS_PROXY}" || export HTTPS_PROXY="${PARAM_HTTPS_PROXY}"
        test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}"

        ls -al "${PARAM_USER_HOME}"/.ssh

        git config --global --add safe.directory "${WORKSPACE_OUTPUT_PATH}"
        /ko-app/git-init \
          -url="${PARAM_URL}" \
          -revision="${PARAM_REVISION}" \
          -refspec="${PARAM_REFSPEC}" \
          -path="${CHECKOUT_DIR}" \
          -sslVerify="${PARAM_SSL_VERIFY}" \
          -submodules="${PARAM_SUBMODULES}" \
          -depth="${PARAM_DEPTH}" \
          -sparseCheckoutDirectories="${PARAM_SPARSE_CHECKOUT_DIRECTORIES}"
        cd "${CHECKOUT_DIR}"
        RESULT_SHA="$(git rev-parse HEAD)"
        EXIT_CODE="$?"
        if [ "${EXIT_CODE}" != 0 ] ; then
          exit "${EXIT_CODE}"
        fi
        RESULT_COMMITTER_DATE="$(git log -1 --pretty=%ct)"
        printf "%s" "${RESULT_COMMITTER_DATE}" > "$(results.committer-date.path)"
        printf "%s" "${RESULT_SHA}" > "$(results.commit.path)"
        printf "%s" "${PARAM_URL}" > "$(results.url.path)"        
    - name: build-and-push
      workingDir: $(workspaces.source.path)
      image: $(params.BUILDER_IMAGE)
      args:
        - $(params.EXTRA_ARGS)
        - --dockerfile=$(params.DOCKERFILE)
        - --context=$(workspaces.source.path)/$(params.CONTEXT)
        - --destination=$(params.IMAGE)
        - --digest-file=$(results.IMAGE_DIGEST.path)
      securityContext:
        runAsUser: 0
      resources:
        limits:
          cpu: '3'
          memory: 12Gi
        requests:
          cpu: 500m
          memory: 1Gi
    - name: write-url
      image: docker.io/library/bash:5.1.4@sha256:c523c636b722339f41b6a431b44588ab2f762c5de5ec3bd7964420ff982fb1d9
      resources:
        limits:
          cpu: 50m
          memory: 100Mi
        requests:
          cpu: 25m
          memory: 50Mi
      script: |
        set -e
        image="$(params.IMAGE)"
        echo -n "${image}" | tee "$(results.IMAGE_URL.path)"

This Pipeline only calls the clone-build-push task, it could also include a task that calls a webhook to notify that a new image has been created.

apiVersion: tekton.dev/v1
kind: Pipeline
metadata:
  name: git-build-pipeline
spec:
  description: |
    git-build-pipeline takes a git repository and a commit SHA and validates that cloning the revision succeeds.
    It then will build a image and push it to a artifact repository.    
  params:
    - name: base
      type: string
      description: the service name
    - name: branch
      type: string
      description: The git branch name
    - name: commit
      type: string
      description: The git commit to fetch.
    - name: image
      type: string
      description: the image to build
    - name: url
      type: string
      description: The git repository URL to clone from
  workspaces:
    - name: shared-data
      description: |
        This workspace will receive the cloned git repo and be passed
        to the next Task for the commit to be checked.        
    - name: git-credentials
      description: github ssh credentials
  tasks:
    - name: clone-build-push
      taskRef:
        name: clone-build-push
      workspaces:
        - name: source
          workspace: shared-data
        - name: ssh-directory
          workspace: git-credentials
      params:
        - name: url
          value: $(params.url)
        - name: revision
          value: $(params.commit)
        - name: depth
          value: '0'
        - name: IMAGE
          value: $(params.image)
        - name: CONTEXT
          value: $(params.base)/app

This is an example PipelineRun that could be used. I would recommend that you use Tekton Triggers or some other form of automated creation of the run.

apiVersion: tekton.dev/v1
kind: PipelineRun
metadata:
  name: build-manual-pipelineRun
spec:
  pipelineRef:
    name: git-build-pipeline
  serviceAccountName: tekton-service-account
  podTemplate:
    securityContext:
      fsGroup: 65532
  workspaces:
    - name: shared-data
      emptyDir:
        sizeLimit: 2Gi
    - name: git-credentials
      secret:
        secretName: github-secrets
  params:
    - name: base
      value: SERVICE
    - name: branch
      value: BRANCH
    - name: commit
      value: COMMIT_HASH
    - name: image
      value: IMAGE_REPOSITORY:TAG
    - name: url
      value: [email protected]:ORG/REPO.git

What the example is looking for file structure in the Git repository

git-root
  - SERVICE
    - app
      - Dockerfile
  - SERVICE-2