Today I Learned, the Google Cloud Platform / Terraform Edition

Written by / Original link on Sep. 13, 2019


1f30d.png This week I’ve been goofing around with Google Cloud Platform and Terraform to manage it. I’ve learned quite a few things, that might be of help to others, so here goes …

There’s a difference between IAM policy for service account (google_service_account_iam) and IAM policy for projects (google_project_iam_member).

The former is used to allow other users to impersonate/control a certain Service Account, the second is used to define what a Service Account can access and do.

Should’ve RTFM‘d


To apply IAM Policies onto a Service Account the invoker needs the “Project IAM Admin” (roles/resourcemanager.projectIamAdmin) role. If not you’ll get back a 403:

returned error: Error applying IAM policy for project "my-project": Error setting IAM policy for project "my-project":
googleapi: Error 403: The caller does not have permission, forbidden


There’s a bug in Terraform which prevents terraform apply from setting IAM Roles when you have a Service Account or User with an uppercase character in their e-mail address (e.g. Firstname.Lastname@domain.tld).

Terraform will automatically convert all e-mail address to all-lowercase variants (e.g. firstname.lastname@domain.tld). Therefore GCP’s Resource Manager won’t be able to apply the policies as the Rresource does not exist. Accounts that inherit access to the project are not affected.

The error you get back is quite obscure:

returned error: Error applying IAM policy for project "my-project": Error setting IAM policy for project "my-project":
googleapi: Error 400: Request contains an invalid argument., badRequest

To see whether you have such a conflicting user, check the output of gcloud projects get-iam-policy PROJECT-ID


You can partially apply a Terraform State by using Resource Targets. You need this, for example, when storing secrets onto Google Cloud Platform: as you rely on a keyring+keycode to store the secrets, and you need encrypted secrets for your services you can’t apply the whole thing at once, as you need the keyring/keycode to be available before you can generate encrypted secrets from plaintext strings.

To solve this:

  1. Run Terraform to only target the google_kms_key_ring and google_kms_crypto_key resources:

    terraform apply -target=google_kms_key_ring.my_key_ring -target=google_kms_crypto_key.my_crypto_key
  2. Generate your secrets and store them in the keyring using helper scripts such as gcloud-kms-scripts which I created just for that.
  3. Use the encrypted secrets in your .tf files
  4. Run terraform apply as you’d normally do


Applying IAM Policies to a newly created Service Account won’t always work, as it takes some time before the SA is available for use. There’s a workaround in which you trigger a sleep command using local-exec, yet I’m hoping that this will be solved in a future release of Terraform (perhaps with a delay Terraform Resource?).

resource "null_resource" "before" {

resource "null_resource" "delay" {
  provisioner "local-exec" {
    command = "sleep 10"
  triggers = {
    "before" = "${}"

resource "null_resource" "after" {
  depends_on = ["null_resource.delay"]


It’s possible to impersonate a Service Account from within your Terraform code. First you connect using your main account, and then generate a short lived token for the SA. From the Terraform docs I got this snippet:

provider "google" {
    scopes = [

data "google_service_account_access_token" "default" {
 provider = "google"
 target_service_account = ""
 scopes = ["userinfo-email", "cloud-platform"]
 lifetime = "300s"

data "google_client_openid_userinfo" "me" { }

output "source-email" {
  value = "${}"

provider "google" {
   alias  = "impersonated"
   access_token = "${data.google_service_account_access_token.default.access_token}"

data "google_project" "project" {
  provider = "google.impersonated"
  project_id = "target-project"

Due to the provider = "google.impersonated" part, the google_project will run as


That’s it! I hope these might have been of help to you, saving you some lookup work …

Did this help you out? Like what you see?
Consider donating.

I don’t run ads on my blog nor do I do this for profit. A donation however would always put a smile on my face though. Thanks!

2615.png Buy me a Coffee ($3)

gcloud iam gabrielaio original-content terraform bram

« ESNext: Proposals to look forward to (Fronteers Jam Session) - text-decoration-* CSS Properties Coming to Firefox »