holdenssplendiddigests.zenbloomer.com
←Back

How Do AWS T Instances CPU Credits Actually Work?

When managing cloud infrastructure, especially on AWS, understanding the nuances of burstable instance types like the AWS T family is crucial to optimizing both performance and cost. These instances introduce the concept of CPU credit balance, which can be confusing at first glance but offers real advantages when used appropriately. This post dives deep into how AWS T instances CPU credits work, compares shared CPU definitions across cloud providers, highlights the dangers of always-on small services hiding cloud waste, and suggests best practices for measuring real peak demand using the right observation windows and percentiles, not averages.

Overview of AWS T Family and Burstable Performance

The AWS T family (T2, T3, T3a, and T4g) is designed around burstable performance, a concept meaning these instances provide baseline CPU capacity with the ability to burst above that baseline for short periods. This is enabled through a system of CPU credits:

  • CPU credits are earned during periods of low CPU usage.
  • Each credit grants the instance the ability to use 100% of one vCPU for one minute.
  • Credits accumulate in a CPU credit balance.
  • When the balance runs out, the instance can no longer burst above the baseline CPU performance, leading to throttling.

The result is a cost-effective instance ideal for workloads with variable CPU demand—low most of the time but occasional spikes.

But How Do CPU Credits Really Work?

Let’s break down the mechanics:

  1. Baseline Performance: Each T instance size specifies an average baseline CPU utilization as a percentage of one full vCPU. For example, a t3.medium has a baseline of 20% CPU utilization.
  2. Accumulation: When the instance uses less than the baseline, it accumulates CPU credits at a fixed rate.
  3. Consumption: When the instance bursts above baseline, it consumes credits at a rate equivalent to the CPU usage above baseline.
  4. Balance Cap: Credits are capped to a maximum balance, preventing an unbounded accrual.
  5. Unlimited Mode: For T3 and T4g, you can enable “Unlimited Mode” where throttling does not occur, but you pay for excess CPU usage beyond accrued credits.

This credit system enables efficient resource utilization but requires engineers to understand their workload’s CPU demand patterns deeply.

Why Always-On Small Services Hide Cloud Waste

One common mistake is placing always-on small services on burstable instances assuming their low average CPU usage means low cost and good performance. However, these services usually:

  • Have repetitive but low CPU usage—sometimes under baseline, sometimes barely hitting baseline.
  • Rarely burst significantly.
  • Accumulate CPU credits slowly and consume them steadily, never getting meaningful bursts.

This leads to two forms of waste:

  1. Hidden Inefficiency: The instance is effectively running at a small fixed CPU amount but paying for an instance capable of bursting. Using a small fixed instance or micro instance might be better.
  2. Credit Starvation Risks: Even a slight transient spike can cause throttling if credits are depleted, impairing availability and performance.

It’s better to match the CPU capacity to consistent demand rather than relying on burst credits for always-on services that never meaningfully burst.

Shared CPU Definitions Differ by Provider

While AWS uses CPU credits to define burstable instances, other cloud providers approach the shared CPU model differently.

Provider Shared CPU Model Measurement Mechanics Typical Use Cases AWS CPU credits on burstable instances (T family) CPU credit balance based on baseline and burst, throttled after credits run out Variable workloads with occasional CPU bursts Azure B-series VMs (CPU credits) Similar CPU credit accumulation and consumption but differs in earned rates and costs Similar burstable workloads, also supports "paid bursting" Google Cloud Shared vCPU, no explicit credit system vCPUs are fractional; performance depends on host load and cgroup limits Small workloads, no guaranteed burst mechanisms

It’s critical to note that the definition of shared CPU is not standardized. AWS’s CPU credits provide explicit bursting mechanics and throttling policies, while Google Cloud simply shares physical cores with less explicit guarantees, and Azure’s CPU credits have different thresholds and methods.

When migrating or right-sizing across clouds, do not treat vCPU counts as direct performance guarantees, especially for burstable or shared CPU instances.

Measuring Peaks With the Right Observation Window

Cost and performance decisions based on average https://computingforgeeks.com/shared-cpu-cloud-waste-migration-guide/ CPU usage are dangerously insufficient when evaluating burstable performance instances.

  • Average CPU utilization smooths spikes and hides critical peak behavior.
  • CPU credits are earned and consumed minute-by-minute. Observations should align with these timeframes.
  • Measuring only at coarse intervals like hourly or daily aggregates loses vital detail.

To accurately size burstable instances and understand credit balance dynamics, use monitoring tools and telemetry that provide:

  • Minute-level granularity (or better).
  • CPU utilization percentiles like P95 and P99, capturing the high spike behavior, not just the mean.
  • Spike duration metrics showing how long the instance sustains CPU over baseline.

For example, if a service spikes at 90% CPU but only for 10 seconds every 10 minutes, it might not need a bigger instance. But if high CPU lasts for multiple minutes, the T instance might exhaust CPU credits and throttle, degrading performance.

Percentiles and Spike Duration vs. Averages

Focusing on percentiles like P95 and P99 instead of averages prevents underestimating peak load which often causes throttling. Engineers reading this should always ask:

  • What does the P95 CPU usage look like?
  • What are the spike durations at these percentiles?
  • What happens to the CPU credit balance during these intervals?

This data should precede decisions on changing instance sizes or performance tiers.

Using AWS Compute Optimizer and Azure Advisor in Burstable Instance Sizing

AWS Compute Optimizer and Azure Advisor are two excellent tools that support cloud cost and performance efficiency, and they both consider burstable instances in their recommendations, but with differences.

AWS Compute Optimizer

AWS Compute Optimizer provides:

  • Instance rightsizing recommendations including T family instances.
  • Detailed insights on CPU utilization patterns, including burstable instances' CPU credit balances.
  • Actionable advice based on multi-dimensional metrics with recommendations to upgrade or downgrade.

Importantly, Compute Optimizer evaluates not just average CPU, but the duration and percentile utilization to avoid suggesting a downgrade that would cripple burstable performance.

Azure Advisor

Azure Advisor provides:

  • Similar recommendation capabilities, including for B-series burstable VMs.
  • Optimization advice considers CPU credit accrual and consumption for B-series VMs.
  • Cost-saving ideas coupled with workload performance impacts.

However, Azure’s metrics and thresholds differ slightly, so it's essential to understand the specific CPU credit mechanics of your cloud provider before blindly applying recommendations.

Practical Recommendations

  • Don’t treat vCPU counts as performance guarantees: The presence of a vCPU does not mean consistent or full utilization availability on shared or burstable instances.
  • Always analyze P95 and P99 CPU usage and spike durations before resizing: This prevents performance regressions due to throttling.
  • Use AWS Compute Optimizer or Azure Advisor as aids, not absolutes: They provide valuable signals but should be combined with manual observation aligned to the workload's patterns.
  • Avoid placing always-on small services on burstable instances: Those that never actually burst are better suited for fixed baseline instance types.
  • Enable unlimited mode cautiously: While it prevents throttling, it may lead to unexpected cost spikes if the workload is regularly bursting beyond the accrued credits.
  • Observe CPU credit balance trends: Running out of credits is your critical signal of persistent load beyond baseline.

Summary

Understanding how AWS T instances CPU credits actually work is vital for engineers aiming to optimize cloud cost and performance. The burstable performance model offers flexibility and efficiency but demands a precise comprehension of the CPU credit system and your workload’s true CPU usage patterns.

Always consider percentile-based CPU metrics over averages, recognize the differing shared CPU definitions among cloud providers, and use available optimization tools thoughtfully. Finally, avoid the trap of running always-on services on burstable instances without meaningful bursting behavior, as that typically introduces hidden cloud waste rather than savings.

By applying these principles, you can confidently decide when and how to leverage AWS T family instances to balance cost optimization with predictable performance.