Concourse and Secret Management

Concourse and Secret Management

One aspect of creating good CI/CD pipelines is the management of passwords and other credentials required for deployment.

A typical concourse pipeline will poll for updates in a git repo, do a build and then push the results to a PaaS such as Kubernetes or Cloud Foundry.

A Working Example.

For this example, we will use bucc. bucc is an all in one deployment of Bosh, UAA, Credhub, and Concourse. If you don’t have access to a working Concourse/Credhub environment, this is an excellent place to start.

  1. Install bucc per the documentation in here.

  2. Install Credhub cli from here

Adding a secret to Concourse.

Concourse will retrieve credentials from Credhub by looking them up based on their path.

Global credentials for a team can be placed directly under the team name. Credentials for a specific pipeline can be organized under the team name/pipeline name.

Properties can be referenced in a pipeline using standard property replacement in Concourse

Running this job should reveal the password.

Using other Secret types.

Credhub can store other types of secrets besides just key-value pairs. Examples include SSH keys, JSON structures, and Certificates. A complete list of types are available here.

An SSH Key contains two parts. public_key and private_key. It can be imported as follows.

You can now access the ssh key in your script.

  • You must flatten the key to a single line before importing it.

alternatively you can pass a file containing the key to import.

Using SSH keys

A typical use case in concourse is polling git for updated commits.

Generating Secrets

In addition to storing secrets, Credhub can be used to generate them.

  • Generate a new SSH key pair

  • retrieve the public key