In this blog post, you’ll learn how to use Google Cloud Build to build a Docker image from a private GitHub or Bitbucket repository.
This post is part of the Dockerized Django Back-end API with Angular Front-end Tutorial. Check out all the parts of the tutorial there.
In the last part of the tutorial, we’ve learned how to build a Docker image in Google Cloud Build.
What happens if we host our code and Dockerfile in a private GitHub or Bitbucket account?
No worries.
In this blog post, we’ll learn how to build the Docker image from a private GitHub or Bitbucket repository in Cloud Build.
To get the code to where we left off in the last blog post, use:
$ git checkout v1.17
Just follow the steps below and don’t forget to have fun!
1. Login to GCP.
2. Enable the Cloud KMS API for your project.
Use this link to enable Cloud KMS. We’ll use Cloud KMS for managing the SSH key we’ll use to access our repository.
3. Open the command-line in GCP (i.e. Cloud Shell).
4. Create a new directory and an SSH key for GitHub/Bitbucket.
Next, we’ll create an SSH key for accessing the repository in a new directory.
$ mkdir buildversion1
$ cd buildversion1
$ ssh-keygen -t rsa -b 4096 -C "you@yourdomain.com"
Save the key as version1key and don’t give it a passphrase.
5. View the PUBLIC key and add it to GitHub/Bitbucket.
Use the cat command to show the public key:
$ cat version1key.pub
Notice the .pub at the end of the filename.
Next, add this key to either GitHub or Bitbucket, depending on where you’re hosting your private repository.
5.1 Adding the key to GitHub
For GitHub, the key you’ve created is called a deploy key.
Add it to your GitHub private repo as follows.
From your repository page, click Settings, then go to Deploy Keys in the sidebar, then click Add deploy key.
Give it the title gcp and paste in the public key you got using the cat command above.
For your information, one deploy key can access one repo, so you need to create a machine user to access multiple repos. See the deploy keys GitHub documentation for more details.
5.2 Adding the key to Bitbucket
For Bitbucket, the key you’ve created is called an access key.
Add it to the Bitbucket repo as follows.
From your repository page, click Settings, then go to Access keys in the sidebar, then click Add key.
Give it the label gcp and paste in the public key you got using the cat command above.
In contrast to a deploy key on GitHub, you can add an access key to multiple repositories in Bitbucket. See the access keys Bitbucket documentation for more details.
6. Create a Cloud KMS KeyRing and CryptoKey.
A Cloud KMS CryptoKey is used to encrypt and decrypt the SSH key. CryptoKeys are stored in KeyRing objects.
$ gcloud kms keyrings create version1-keyring --location=global
$ gcloud kms keys create github-key \
--location=global --keyring=version1-keyring \
--purpose=encryption
If you’re using Bitbucket, then it would be more helpful to name the key bitbucket-key instead of github-key.
Of course, should you decide to use a different keyring or key name, use those same names in the next commands.
7. Encrypt the PRIVATE key before using it in a build.
$ gcloud kms encrypt --plaintext-file=version1key \
--ciphertext-file=version1key.enc \
--location=global --keyring=version1-keyring --key=github-key
This will create a file called version1key.enc containing the encrypted private key.
8. Delete the clear PRIVATE key so it can’t be used by others.
$ rm version1key
9. Grant the Cloud Build service account decrypt permission.
$ gcloud kms keys add-iam-policy-binding \
github-key --location=global --keyring=version1-keyring \
--member=serviceAccount:<your-service-account>@cloudbuild.gserviceaccount.com \
--role=roles/cloudkms.cryptoKeyEncrypterDecrypter
Replace <your-service-account> with the number associated with your Cloud Build service account.
To get your service account email address, just go to the Google Cloud Platform Console IAM menu. The service account email address contains @cloudbuild.gserviceaccount.com.
To authenticate the GitHub or Bitbucket server, we need to provide a known_hosts file containing the rsa key for github.com or bitbucket.org.
From within the buildversion1 directory you’ve created in step 4, run:
$ ssh-keyscan -t rsa github.com > known_hosts
or if you’re using Bitbucket:
$ ssh-keyscan -t rsa bitbucket.org > known_hosts
11. Prepare the build config YAML file.
Create and copy over the build_gcp.yaml file from the repo to:
$ nano buildversion1_private.yaml
In your buildversion1 directory you should now have the following files:
version1key.enc – the encrypted private key.version1key.pub – the public key.known_hosts – containing the SSH fingerprint for GitHub or Bitbucket.buildversion1_private.yaml – the build config file for accessing the private repo.
We will now modify buildversion1_private.yaml to clone the repository from a private GitHub/Bitbucket account.
The build file below decrypts the SSH key, configures git to use the SSH key, and clones the repository at git@github.com:dnstanciu/drf-angular-docker-tutorial.git.
You can of course change dnstanciu to your own GitHub username and drf-angular-docker-tutorial.git to your own private GitHub repository.
If you’re using Bitbucket, the repo link should look something like git@bitbucket.org:dnstanciu/drf-angular-docker-tutorial.git, so replace the link appropriately.
Additionally, ensure the you’re using your own names for the KMS keyring (e.g. mine is version1-keyring) and for the KMS key (e.g. mine is github-key).
Lastly, change yourproject to your own GCP project ID.
12. Submit the build.
You’re now ready to submit the build to Cloud Build. Just run:
$ gcloud builds submit --config buildversion1_private.yaml .
You can find the build config file for the private repo as build_gcp_private.yaml in the repo root by viewing my git commit or using:
$ git checkout v1.18
Summary
In this part of the tutorial, we’ve learned how to build a Docker image from a private GitHub or Bitbucket repository in Google Cloud Build. We’ve covered creating an SSH deploy key if you’re using GitHub or an access key if you’re using Bitbucket.
In the next part of the tutorial, we’ll see how to connect Django to a production-grade PostgreSQL database hosted on Cloud SQL.
Credit: For this tutorial, I’ve used the following resources:
