Using Terraform HTTP Provider to Make HTTP Calls for Provisioning and Decommissioning

Using Terraform HTTP Provider to Make HTTP Calls

Terraform’s HTTP provider allows you to make HTTP requests as part of your infrastructure-as-code workflow. This is particularly useful when you need to interact with external APIs during provisioning or decommissioning of resources.

Why Use the HTTP Provider?

The HTTP provider is helpful when:

1. You need to fetch configuration data from an external API
2. You want to trigger webhooks during resource creation or destruction
3. You need to register or deregister resources with external systems
4. You want to validate external service availability before provisioning

Installing the HTTP Provider

Add the HTTP provider to your Terraform configuration:

terraform {
  required_providers {
    http = {
      source  = "hashicorp/http"
      version = "~> 3.4"
    }
  }
}

Basic HTTP Data Source

The http data source makes a GET request and exposes the response for use in your configuration:

data "http" "example" {
  url = "https://api.example.com/config"

  request_headers = {
    Accept        = "application/json"
    Authorization = "Bearer ${var.api_token}"
  }
}

output "response_body" {
  value = data.http.example.response_body
}

Making HTTP Calls for Provisioning

When provisioning resources, you might need to register them with an external service. Here’s an example of using null_resource with local-exec provisioner to make HTTP calls:

variable "api_endpoint" {
  description = "API endpoint for resource registration"
  type        = string
}

variable "api_token" {
  description = "API authentication token"
  type        = string
  sensitive   = true
}

resource "aws_instance" "web_server" {
  # Replace with a valid AMI ID for your region
  # You can use data sources to fetch the latest AMI dynamically
  ami           = "ami-xxxxxxxxxxxxxxxxx"
  instance_type = "t2.micro"

  tags = {
    Name = "WebServer"
  }
}

resource "null_resource" "register_server" {
  depends_on = [aws_instance.web_server]

  triggers = {
    instance_id = aws_instance.web_server.id
  }

  provisioner "local-exec" {
    command = <<-EOT
      curl -X POST "${var.api_endpoint}/register" \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer ${var.api_token}" \
        -d '{"instance_id": "${aws_instance.web_server.id}", "private_ip": "${aws_instance.web_server.private_ip}"}'
    EOT
  }
}

Making HTTP Calls for Decommissioning

When destroying resources, you may need to deregister them from external systems. Use the when = destroy provisioner:

resource "null_resource" "manage_server_lifecycle" {
  depends_on = [aws_instance.web_server]

  triggers = {
    instance_id  = aws_instance.web_server.id
    api_endpoint = var.api_endpoint
    api_token    = var.api_token
  }

  # Register on creation
  provisioner "local-exec" {
    command = <<-EOT
      curl -X POST "${self.triggers.api_endpoint}/register" \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer ${self.triggers.api_token}" \
        -d '{"instance_id": "${self.triggers.instance_id}"}'
    EOT
  }

  # Deregister on destruction
  provisioner "local-exec" {
    when    = destroy
    command = <<-EOT
      curl -X DELETE "${self.triggers.api_endpoint}/deregister/${self.triggers.instance_id}" \
        -H "Authorization: Bearer ${self.triggers.api_token}"
    EOT
  }
}

Using the HTTP Provider with Terraform Cloud

When working with Terraform Cloud or Enterprise, you can use the HTTP provider to fetch dynamic configuration:

data "http" "config" {
  url = "https://config-service.example.com/terraform/settings"

  request_headers = {
    Accept = "application/json"
  }
}

locals {
  config = jsondecode(data.http.config.response_body)
}

resource "aws_instance" "app" {
  ami           = local.config.ami_id
  instance_type = local.config.instance_type
  
  tags = local.config.tags
}

Complete Example: Service Registration and Deregistration

Here’s a complete example that demonstrates both provisioning and decommissioning HTTP calls:

terraform {
  required_providers {
    http = {
      source  = "hashicorp/http"
      version = "~> 3.4"
    }
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
    null = {
      source  = "hashicorp/null"
      version = "~> 3.2"
    }
  }
}

variable "service_registry_url" {
  description = "URL of the service registry API"
  type        = string
}

variable "auth_token" {
  description = "Authentication token for the service registry"
  type        = string
  sensitive   = true
}

# Check if the service registry is available before provisioning
data "http" "health_check" {
  url = "${var.service_registry_url}/health"

  request_headers = {
    Accept = "application/json"
  }
}

resource "aws_instance" "application" {
  count = data.http.health_check.status_code == 200 ? 1 : 0

  # Replace with a valid AMI ID for your region
  ami           = "ami-xxxxxxxxxxxxxxxxx"
  instance_type = "t3.micro"

  tags = {
    Name        = "ApplicationServer"
    Environment = "production"
  }
}

resource "null_resource" "service_lifecycle" {
  count = length(aws_instance.application)

  triggers = {
    instance_id     = aws_instance.application[count.index].id
    private_ip      = aws_instance.application[count.index].private_ip
    registry_url    = var.service_registry_url
    auth_token      = var.auth_token
  }

  # Register the service on creation
  provisioner "local-exec" {
    command = <<-EOT
      curl -X POST "${self.triggers.registry_url}/services" \
        -H "Content-Type: application/json" \
        -H "Authorization: Bearer ${self.triggers.auth_token}" \
        -d '{
          "id": "${self.triggers.instance_id}",
          "name": "application-server",
          "address": "${self.triggers.private_ip}",
          "port": 8080,
          "tags": ["production", "web"]
        }' \
        --fail --silent --show-error
    EOT
  }

  # Deregister the service on destruction
  provisioner "local-exec" {
    when    = destroy
    command = <<-EOT
      curl -X DELETE "${self.triggers.registry_url}/services/${self.triggers.instance_id}" \
        -H "Authorization: Bearer ${self.triggers.auth_token}" \
        --fail --silent --show-error || true
    EOT
  }
}

output "instance_id" {
  value = length(aws_instance.application) > 0 ? aws_instance.application[0].id : null
}

output "service_registry_status" {
  value = data.http.health_check.status_code == 200 ? "available" : "unavailable"
}

Best Practices

1. Use triggers wisely: Store values you’ll need during destruction in the triggers block
2. Handle failures gracefully: Add || true to destruction commands to prevent failed deregistration from blocking destroy
3. Use sensitive variables: Mark API tokens as sensitive to prevent them from appearing in logs
4. Validate responses: Check HTTP status codes before proceeding with dependent resources
5. Implement retry logic: Consider adding retry logic for transient failures

Limitations

• The HTTP data source only supports GET requests
• For POST, PUT, DELETE operations, use local-exec provisioners with curl
• Provisioners are a last resort - prefer native Terraform providers when available

Conclusion

The Terraform HTTP provider is a powerful tool for integrating external APIs into your infrastructure workflow. By combining HTTP data sources with provisioners, you can automate the complete lifecycle of your resources, including registration and deregistration with external services.