» Terraform Cloud Agents

Terraform Cloud Agents allow Terraform Cloud to communicate with isolated, private, or on-premises infrastructure. By deploying lightweight agents within a specific network segment, you can establish a simple connection between your environment and Terraform Cloud which allows for provisioning operations and management. This is useful for on-premises infrastructure types such as vSphere, Nutanix, OpenStack, enterprise networking providers, and anything you might have in a protected enclave.

The agent architecture is pull-based, so no inbound connectivity is required. Any agent you provision will poll Terraform Cloud for work and carry out execution of that work locally.

» Before Install

» Supported Operating Systems

Agents currently only support 64 bit Linux operating systems. You can also run the agent within Docker using our official Terraform Agent Docker container.

» Supported Terraform Versions

Agents support Terraform versions 0.12 and above. Workspaces configured to use Terraform versions below 0.12 will not be able to select the agent-based execution mode.

» Networking Requirements

In order for an agent to function properly, it must be able to make outbound requests over HTTPS (TCP port 443) to the Terraform Cloud application APIs. This may require perimeter networking as well as container host networking changes, depending on your environment. The IP ranges are documented in the Terraform Cloud IP Ranges documentation.

Additionally, the agent must also be able to communicate with any services required by the Terraform code it is executing. This includes the Terraform releases distribution service, releases.hashicorp.com (supported by Fastly), as well as any provider APIs. The services which run on these IP ranges are described in the table below.

Hostname Port/Protocol Directionality Purpose
app.terraform.io tcp/443, HTTPS Outbound Polling for new workloads, providing status updates, and downloading private modules from Terraform Cloud's Private Module Registry
registry.terraform.io tcp/443, HTTPS Outbound Downloading public modules from the Terraform Registry
releases.hashicorp.com tcp/443, HTTPS Outbound Updating agent components and downloading Terraform binaries
archivist.terraform.io tcp/443, HTTPS Outbound Blob Storage

» Operational Considerations

The agent is distributed as a standalone binary which can be run on any supported system. By default, the agent will run in the foreground as a long-running process that will continuously poll for workloads from Terraform Cloud. We strongly recommend pairing the agent with a process supervisor to ensure that it is automatically restarted in case of an error.

Agents do not guarantee a clean working environment per Terraform execution. Each execution is performed in its own temporary directory with a clean environment, but references to absolute file paths or other machine state may cause interference between Terraform executions. We strongly recommend that you write your Terraform code to be stateless and idempotent. You may also want to consider using single-execution mode to ensure your agent only runs a single workload.

» Updating

The agent will automatically receive application code updates for the agent only. Administrators are required to update the host operating system and all other installed software.

» Security Considerations

Agents should be considered a global resource within an organization. Once configured, any workspace owner may configure their workspace to target the organization's agents. This may allow a malicious workspace to access state files, variables, or code from other workspaces targeting the same agent, or access sensitive data on the host running the agent. For this reason, we recommend carefully considering the implications of enabling agents within an organization, and restricting access to your organization to only trusted parties.

» Limitations

Agents are designed to allow you to run Terraform operations from a Terraform Cloud workspace on your private infrastructure. The following use cases are not supported by agents:

  • Connecting to private infrastructure from Sentinel policies using the http import
  • Connecting Terraform Cloud workspaces to private VCS repositories

For these use cases, we recommend you leverage the information provided by the IP Ranges documentation to permit direct communication from the appropriate Terraform Cloud service to your internal infrastructure.

Organizations are limited to 20 pools each.

» Managing Agent Pools

Agents are organized into pools, which can be managed in Terraform Cloud's UI. Each workspace can specify which agent pool should run its workloads.

» Create a new Agent Pool

  1. Navigate to Organization Settings > Agents and click "New agent pool".

    Screenshot: The Agents page with no pools

  2. Give your pool a name, then click "Continue". This name will be used to distinguish your pools when changing the settings of a workspace.

    Screenshot: Step 1 of pool creation, naming the agent pool

  3. Give your token a description and click "Create Token".

    Screenshot: Step 2 of pool creation, token creation

  4. Click "Finish".

» Delete an Agent Pool

  1. Navigate to Organization Settings > Agents and click on the name of the pool you would like to delete.

  2. Click "Delete agent pool".

    Screenshot: Deleting an agent pool

  3. Confirm the deletion of the pool by clicking "Yes, delete agent pool".

    Screenshot: Confirmation to delete an agent pool

    Screenshot: Unable to delete an agent pool with attached workspace

» Revoke an Agent Token

You may revoke an issued token from your agents at any time.

Revoking a token will cause the agents using it to exit. For agents to continue servicing workspace jobs, they must be reinitialized with a new token. Under normal circumstances, it may be desirable to generate a new token first, initialize the agents using it, then revoke the old token once no agents are using it. Agent tokens display information about the last time they were used to help you decide whether they are safe to revoke.

  1. Navigate to Organization Settings > Agents, then click on the agent pool you would like to manage.

  2. Click "Revoke Token" for the token you would like to revoke.

    Screenshot: The Agent tokens list with the "Revoke Token" link highlighted

  3. Confirm the deletion of the token by clicking "Yes, delete token".

    Screenshot: The "Revoke agent token" confirmation modal

» Managing Agents

The agent software runs on your own infrastructure. Agent pool membership is determined by which token you provide when starting the agent.

» Download and Install the Agent

  1. Download the latest agent release, the associated checksum file (.SHA256sums), and the checksum signature (.sig).
  2. Verify the integrity of the downloaded archive, as well as the signature of the SHA256SUMS file using the instructions available on HashiCorp's security page.
  3. Extract the release archive. The unzip utility is available on most Linux distributions and may be invoked as unzip <archive file>. Two individual binaries will be extracted (tfc-agent and tfc-agent-core). These binaries must reside in the same directory for the agent to function properly.

» Start the Agent

Using the Agent token you copied earlier, set the TFC_AGENT_TOKEN and TFC_AGENT_NAME environment variables.

export TFC_AGENT_TOKEN=your-token
export TFC_AGENT_NAME=your-agent-name
./tfc-agent

Once complete, your agent will appear on the Agents page and display its current status.

Screenshot: The Agents page with one connected agent in the idle state

» Optional Configuration: Running an Agent using Docker

Alternatively, you can use our official agent Docker container to run the Agent.

docker pull hashicorp/tfc-agent:latest
docker run -e TFC_AGENT_TOKEN=your-token -e
TFC_AGENT_NAME=your-agent-name hashicorp/tfc-agent

» Optional Configuration: Single-execution mode

The Agent can also be configured to run in single-execution mode, which will ensure that the Agent only runs a single workload, then terminates. This can be used in combination with Docker and a process supervisor to ensure a clean working environment for every Terraform run.

To use single-execution mode, start the agent with the -single command line argument.

» Configuring Workspaces to use the Agent

To configure a workspace to execute runs using an agent:

  1. Open the workspace from the main "Workspaces" view, then navigate to "Settings > General" from the dropdown menu.
  2. Select Agent as the execution mode, as well as the agent pool this workspace should use.
  3. Click "Save Settings" at the bottom of the page.

Screenshot: The Workspace General settings page, with agent selected for execution mode

» Run Details

Runs which are processed by an agent will have additional information about that agent in the details section of the run:

Screenshot: Run details with agent information

» Running Multiple Agents

You may choose to run multiple agents within your network for improved resiliency. If there are multiple agents available within an organization, Terraform Cloud will select the first available agent.

» Troubleshooting

» Viewing Agent Statuses

Screenshot: Possible agent statuses

Agent status appear on the Organization Settings > Agents page and will contain one of these values:

  • Idle: The agent is running normally and waiting for jobs to be available.
  • Busy: The agent is running normally and currently executing a job.
  • Unknown: The agent has not reported any status for an unexpected period of time. The agent may yet recover if the agent's situation is temporary, such as a short-lived network partition.
  • Errored: The agent encountered an unrecoverable error or has been in an Unknown state for long enough that Terraform Cloud considers it errored. This may indicate that the agent process was interrupted, has crashed, a permanent network partition exists, etc. If the agent was in the process of running an operation (such as a plan or apply), that operation has been marked as errored. If the current agent process does manage to recover, it will be instructed to exit immediately.
  • Exited: The agent exited normally, and has successfully informed Terraform of it doing so.

» Agent Capacity Usage

Agents are considered active and count towards the usage of the organization's total capacity if they are in the Idle, Busy, or Unknown state. Agents which are in the Errored or Exited state do not count towards the usage of the total capacity.

The number of active agents you are eligible to deploy is determined by the number of Concurrent Runs your organization is entitled to. Learn more about Terraform Cloud pricing here.

Agents are considered active and count towards the organization's total agent allowance if they are in the Idle, Busy, or Unknown state. Agents which are in the Errored or Exited state do not count towards the organizations total agent capacity.

Agents in the Unknown state continue to be counted against the organization's total agent allowance, as this status is typically an indicator of a temporary communication issue between the agent and Terraform Cloud. Unknown agents which do not respond after a period of 2 hours will automatically transition to an Errored state, at which point they will not count against the agent allowance.

» Viewing Agent Logs

Output from the Terraform execution will be visible on the run’s page within Terraform Cloud. For more in-depth debugging, you may wish to view the agent’s logs, which are sent to stdout and configurable via the -log-level command line argument. By default, these logs are not persisted in any way. It is the responsibility of the operator to collect and store these logs if they are needed.