The Terraform Gap Nobody Talks About: Managing AWS Resources the Provider Doesn't Support

At some point every team running Terraform against AWS hits the same wall: a resource, or a specific option on a resource, that AWS shipped and the Terraform AWS provider hasn’t caught up to yet. It’s not rare — provider coverage lags the AWS API by design, and for anything niche enough, it can lag for a long time. The problem isn’t that the gap exists. It’s that most of the advice for working around it quietly breaks the two things Terraform is actually for: state and idempotency.

The problem, precisely

I needed to manage an AWS resource setting that had no argument in the provider’s resource schema. aws_the_thing existed, but the specific option I needed wasn’t a supported field. The AWS API supported it fine — this was a provider gap, not an AWS limitation.

Why the obvious fix isn’t enough

The standard first move is a null_resource with a local-exec provisioner that shells out to the AWS CLI. It works, once. The problem shows up on the second terraform apply:

No state. Terraform has no record of what the script actually did, so it can’t tell you if reality has drifted from what you asked for.
No idempotency by default. Re-running the same aws CLI command on every apply either errors on “already exists” or silently creates a duplicate, depending on the API.
Nothing to reference downstream. If another resource needs the ARN of the thing you just created out-of-band, you’re back to hardcoding it or parsing shell output into a data source, which is its own mess.

That’s three of Terraform’s core guarantees gone for the sake of one workaround.

The pattern: `terraform_data` as the state anchor

terraform_data (the replacement for the old bare null_resource) gives you a real state slot to attach outputs to, plus a triggers_replace map that controls exactly when the provisioner reruns — not “every apply,” only when the inputs you care about actually change.

resource "terraform_data" "unsupported_thing" {
  triggers_replace = {
    config_hash = md5(jsonencode(var.thing_config))
  }

  provisioner "local-exec" {
    command = "python3 ${path.module}/scripts/apply_thing.py"
  }

  provisioner "local-exec" {
    when    = destroy
    command = "python3 ${path.module}/scripts/destroy_thing.py"
  }
}

The boto3 script on the other end is where idempotency actually gets enforced — Terraform can’t do that part for you, so the script has to behave like a well-written Terraform provider would: check the current state of the world before mutating it.

def apply_thing(client, config: dict) -> str:
    existing = client.describe_things(Filters=[{"Name": "tag:managed-by", "Values": ["terraform"]}])
    if existing["Things"]:
        return existing["Things"][0]["ThingArn"]
    created = client.create_thing(**config)
    return created["ThingArn"]

Check-then-act, tagged so the check is unambiguous, return the identifier either way. Run this script twice with the same input and you get the same ARN back both times — which is the entire bar for calling something idempotent.

Getting the ARN back into Terraform

A local-exec provisioner alone can’t hand a value back to Terraform state — it only reports success or failure. To reference the created resource from other resources, write the ARN to a file the provisioner produces and read it with an external data source, or have the script’s stdout captured via a wrapping external program block instead of local-exec. I use the file-plus-local_file/external-source combination when other resources need the value; for a pattern that’s genuinely fire-and-forget, terraform_data’s own id is enough to prove the step ran.

Handling destroy

The when = destroy provisioner is the part people skip, and it’s the part that turns this from a hack into something you can trust in a real environment. Without it, terraform destroy removes the state entry and leaves the actual AWS resource running — an orphan that shows up as a mystery line item or a mystery security exposure weeks later. The destroy script needs the same check-then-act discipline: look up the resource before trying to delete it, and don’t fail the whole destroy run if it’s already gone.

When this pattern is worth it

Not every gap is worth bridging. Before reaching for this, I check three things:

Is this a permanent gap or a “coming soon” one? If there’s an open PR against the provider close to merging, sometimes waiting a release cycle is genuinely less work.
Does this resource change often? A bridge you write once for a resource that’s created once and never touched again barely needs the idempotency machinery. It’s the resources under active iteration where drift and duplicate creation actually bite.
Can I contribute the fix upstream instead? For anything not tied to a private/internal API, opening a PR against the provider fixes it for the next person hitting the same gap, and removes my own script from the maintenance burden a release or two later.