June 20-22 Announcing HashiConf Europe full schedule: keynotes, sessions, labs & more Register Now
  • Overview
    • Enforce Policy as Code
    • Infrastructure as Code
    • Inject Secrets into Terraform
    • Integrate with Existing Workflows
    • Manage Kubernetes
    • Manage Network Infrastructure
    • Manage Virtual Machine Images
    • Multi-Cloud Deployment
    • Terraform CLI
    • Terraform Cloud
    • Terraform Enterprise
  • Registry
  • Tutorials
    • About the Docs
    • Intro to Terraform
    • Configuration Language
    • Terraform CLI
    • Terraform Cloud
    • Terraform Enterprise
    • Provider Use
    • Plugin Development
    • Registry Publishing
    • Integration Program
    • Terraform Tools
    • CDK for Terraform
    • Glossary
  • Community
GitHubTerraform Cloud
Download

    Terraform Enterprise Admin

  • Overview
    • Pre-Install Checklist
      • Overview
      • AWS Reference Architecture
      • Azure Reference Architecture
      • GCP Reference Architecture
      • VMware Reference Architecture
      • PostgreSQL Requirements
      • Mounted Disk Requirements
      • Minio Setup Guide
      • Externally Managed Vault
    • Network Requirements
      • RedHat Linux Requirements
      • CentOS Linux Requirements
    • Overview
      • 1. Run Installer
      • 2. Configure in Browser
      • Automated Installation
      • Active/Active
      • Initial User Automation
      • Encryption Password
    • Uninstall
      • Overview
      • Configuration
      • Team Membership
      • Attributes
      • Login
        • Overview
        • ADFS
        • Azure Active Directory
        • Okta
        • OneLogin
      • Troubleshooting
    • Overview
      • Automated Recovery
      • Upgrades
      • Log Forwarding
      • Monitoring
      • Backups and Restores
      • Admin CLI Commands
      • Terraform Cloud Agents on TFE
      • Demo to Mounted Disk Migration
    • Terraform Cloud Agents on TFE
      • Accessing the Admin Interface
      • General Settings
      • Customization
      • Integration Settings
      • Managing Accounts & Resources
      • Module Sharing
      • Admin API
      • Updating Terraform Enterprise License
    • Terraform Enterprise Logs
    • Overview
    • Architecture Summary
    • Reliability & Availability
    • Capacity & Performance
    • Security Model
    • Overview
      • Overview
      • v202205-1
      • v202204-2
      • v202204-1
      • v202203-1
      • v202202-1
      • v202201-2
      • v202201-1
      • Overview
      • v202112-2
      • v202112-1
      • v202111-1
      • v202110-1
      • v202109-2
      • v202109-1
      • v202108-1
      • v202107-1
      • v202106-1
      • v202105-1
      • v202104-1
      • v202103-3
      • v202103-2
      • v202103-1
      • v202102-2
      • v202102-1
      • v202101-1
      • Overview
      • Overview
      • Overview
  • Support
  • Application Usage

  • Overview
  • Plans and Features
  • Getting Started
    • API Docs template
    • Overview
    • Account
    • Agent Pools
    • Agent Tokens
    • Applies
    • Audit Trails
    • Comments
    • Configuration Versions
    • Cost Estimates
    • Feature Sets
    • Invoices
    • IP Ranges
    • Notification Configurations
    • OAuth Clients
    • OAuth Tokens
    • Organizations
    • Organization Memberships
    • Organization Tags
    • Organization Tokens
    • Plan Exports
    • Plans
    • Policies
    • Policy Checks
    • Policy Sets
    • Policy Set Parameters
      • Modules
      • Providers
      • Private Provider Versions and Platforms
      • GPG Keys
    • Runs
    • Run Tasks
    • Run Tasks Integration
    • Run Triggers
    • SSH Keys
    • State Versions
    • State Version Outputs
    • Subscriptions
    • Team Access
    • Team Membership
    • Team Tokens
    • Teams
    • User Tokens
    • Users
    • Variables
    • VCS Events
    • Workspaces
    • Workspace-Specific Variables
    • Workspace Resources
    • Variable Sets
      • Overview
      • Module Sharing
      • Organizations
      • Runs
      • Settings
      • Terraform Versions
      • Users
      • Workspaces
    • Changelog
    • Stability Policy
    • Overview
    • Creating Workspaces
    • Naming
    • Terraform Configurations
      • Overview
      • Managing Variables
      • Overview
      • VCS Connections
      • Access
      • Notifications
      • SSH Keys for Modules
      • Run Triggers
      • Run Tasks
    • Terraform State
    • JSON Filtering
    • Remote Operations
    • Viewing and Managing Runs
    • Run States and Stages
    • Run Modes and Options
    • UI/VCS-driven Runs
    • API-driven Runs
    • CLI-driven Runs
    • The Run Environment
    • Installing Software
    • Users
    • Teams
    • Organizations
    • Permissions
    • Two-factor Authentication
    • API Tokens
      • Overview
      • Microsoft Azure AD
      • Okta
      • SAML
      • Linking a User Account
      • Testing
    • Overview
    • GitHub.com
    • GitHub.com (OAuth)
    • GitHub Enterprise
    • GitLab.com
    • GitLab EE and CE
    • Bitbucket Cloud
    • Bitbucket Server and Data Center
    • Azure DevOps Services
    • Azure DevOps Server
    • Troubleshooting
    • Overview
    • Adding Public Providers and Modules
    • Publishing Private Providers
    • Publishing Private Modules
    • Using Providers and Modules
    • Configuration Designer
  • Migrating to Terraform Cloud
    • Overview
    • Using Sentinel with Terraform 0.12
    • Manage Policies
    • Enforce and Override Policies
    • Mocking Terraform Sentinel Data
    • Working With JSON Result Data
      • Overview
      • tfconfig
      • tfconfig/v2
      • tfplan
      • tfplan/v2
      • tfstate
      • tfstate/v2
      • tfrun
    • Example Policies
    • Overview
    • AWS
    • GCP
    • Azure
      • Overview
      • Service Catalog
      • Admin Guide
      • Developer Reference
      • Example Customizations
      • V1 Setup Instructions
    • Splunk Integration
    • Kubernetes Integration
    • Run Tasks Integration
    • Overview
    • IP Ranges
    • Data Security
    • Security Model
    • Overview
    • Part 1: Overview of Our Recommended Workflow
    • Part 2: Evaluating Your Current Provisioning Practices
    • Part 3: How to Evolve Your Provisioning Practices
    • Part 3.1: From Manual Changes to Semi-Automation
    • Part 3.2: From Semi-Automation to Infrastructure as Code
    • Part 3.3: From Infrastructure as Code to Collaborative Infrastructure as Code
    • Part 3.4: Advanced Workflow Improvements

  • Terraform Cloud Agents

  • Other Docs

  • Intro to Terraform
  • Configuration Language
  • Terraform CLI
  • Terraform Cloud
  • Terraform Enterprise
  • Provider Use
  • Plugin Development
  • Registry Publishing
  • Integration Program
  • Terraform Tools
  • CDK for Terraform
  • Glossary
Type '/' to Search

»Capacity and Performance

The maximum capacity and performance of Terraform Enterprise is dependent entirely on the resources provided by the Linux instance it is installed on. There are a few settings that allow Terraform Enterprise's capacity to be adjusted to suit the instance.

»Memory + Concurrency

The amount of memory to allocate to a Terraform run and the number of concurrent runs are the primary elements in understanding capacity above the base services.

By default, Terraform Enterprise allocates 512 MB of memory to each Terraform run, with a default concurrency of 10 parallel runs. Therefore, by default Terraform Enterprise requires 5.2 GB of memory reserved for runs.

After factoring in the memory needed to run the base services that make up the application, the default memory footprint of Terraform Enterprise is approximately 4 GB.

»Settings

The settings for per-run memory and concurrency are available in the dashboard on port 8800, on the Settings page, under the Capacity section. They can also be set via the application settings JSON file when using the automated install procedure.

»Increasing Concurrency

To increase the number of concurrent runs, adjust the capacity_concurrency setting. This setting is not limited by system checks; it depends on the operator to provide enough memory to the system to accommodate the requested concurrent capacity. For example, if capacity_concurrency is set to 100 and the worker memory is set to 512, the instance would require a minimum of 52 GB of memory just for Terraform runs. The rest of the TFE application requires a minimum of 4 GB of memory in addition to the Operating System requirements.

»Adjusting Memory

The default memory limit of 512 MB per Terraform run is also configurable. Note that this setting is not limited by system checks; it depends on the operator to provide enough memory to the system to accommodate the requested limits. If the memory limit is adjusted to 1024 MB with the default capacity of 10, the instance would require, at a minimum, 10 GB of memory reserved for Terraform runs.

»Downward Adjustment

We do not recommend adjusting the memory limit below 512 MB. Memory is Terraform's primary resource and it becomes easy for it to go above smaller limits and be terminated mid-run by the Linux kernel.

»CPU

The required CPU resources for an individual Terraform run vary considerably, but in general they are a much more minor factor than memory due to Terraform mostly waiting on IO from APIs to return.

Our rule of thumb is 10 Terraform runs per CPU core, with 2 CPU cores allocated for the base Terraform Enterprise services. So a 4-core instance with 16 GB of memory could comfortably run 20 Terraform runs, if the runs are allocated the default 512 MB each.

As of the v202109-1 TFE release, you can use the capacity_cpus Replicated configuration option to set the maximum number of CPU cores that can be allocated to a Terraform run. When capacity_cpus is set, the configuration places a hard quota on the number of cores that a Terraform operation and underlying provider plugin logic can consume. This can be an effective tool to prevent one expensive workspace from monopolizing the CPU resources of the host.

»Disk

The amount of disk storage available to a system plays a small role in the capacity of an instance. A root volume with 200 GB of storage can sustain a capacity well over 100 concurrent runs.

»Disk I/O

Because of the amount of churn caused by container creation as well as Terraform state management, highly concurrent setups will begin pushing hard on disk I/O. In cloud environments like AWS that limit disk I/O to IOPS that are credited per disk, it's important to provision a minimum number to prevent I/O related stalls. Low disk I/O can create significant performance issues.

This resource is harder to predict than memory or CPU usage because it varies per Terraform module, but we generally recommend a minimum of 50 IOPS per concurrent Terraform run. So if an instance is configured for 10 concurrent runs, the disk should have 500 IOPS allocated. For reference, on AWS, an EBS volume with an allocated size of 250 GB comes with a steady state of 750 IOPS.

We recommend using a disk with a minimum of 500 IOPS, but high load systems should consider increasing this significantly. For example, a production instance with a consistently high level of utilization and a concurrency of 10 should ideally have a disk with about 3,000 IOPS. Internal testing has shown performance increases with additional IOPs up to 8,000. Scaling the disk beyond 8,000 IOPs does not significantly improve performance.

github logoEdit this page
  • Overview
  • Docs
  • Extend
  • Privacy
  • Security
  • Press Kit
  • Consent Manager