s3-bucket-size

S3 bucket size, by CloudWatch

Returns the size of an S3 bucket, every storage class added up, its number of objects, and how old these figures are, read from the daily storage metrics Amazon S3 sends to CloudWatch, with an access key that may only read. S3 is never asked: the bucket is not listed, and no object is read. Three watches are offered, each one set on the probe.

Versions

Generic: the daily storage metrics are sent by S3 for every general purpose bucket, free of charge, without any setting. The collector asks for the version 2010-08-01 of the CloudWatch API in the Query protocol, which AWS says it keeps supporting beside the JSON protocol its newer SDKs use.

How it works

1. GET https://monitoring.<aws:region>.amazonaws.com/?Action=ListMetrics&Dimensions.member.1.Name=BucketName&Dimensions.member.1.Value=<aws:bucket>&MetricName=BucketSizeBytes&Namespace=AWS%2FS3&Version=2010-08-01, page after page by NextToken: the storage classes the bucket holds, each under the dimension StorageType (StandardStorage, StandardIAStorage, GlacierStorage…).

<ListMetricsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
  <ListMetricsResult>
    <Metrics>
      <member>
        <Dimensions>
          <member><Name>BucketName</Name><Value>backups-example</Value></member>
          <member><Name>StorageType</Name><Value>StandardStorage</Value></member>
        </Dimensions>
        <MetricName>BucketSizeBytes</MetricName>
        <Namespace>AWS/S3</Namespace>
      </member>
    </Metrics>
    <NextToken>…</NextToken>
  </ListMetricsResult>
  <ResponseMetadata><RequestId>…</RequestId></ResponseMetadata>
</ListMetricsResponse>

2. GET https://monitoring.<aws:region>.amazonaws.com/?Action=GetMetricStatistics&…&MetricName=NumberOfObjects&…StorageType=AllStorageTypes&Period=86400&Statistics.member.1=Average&StartTime=<now − 3 days>&EndTime=<now>, then the same for BucketSizeBytes once per storage class found: the daily points of the last three days.

<GetMetricStatisticsResponse xmlns="http://monitoring.amazonaws.com/doc/2010-08-01/">
  <GetMetricStatisticsResult>
    <Label>BucketSizeBytes</Label>
    <Datapoints>
      <member><Timestamp>2026-10-27T00:00:00Z</Timestamp><Average>53687091200</Average><Unit>Bytes</Unit></member>
    </Datapoints>
  </GetMetricStatisticsResult>
  <ResponseMetadata><RequestId>…</RequestId></ResponseMetadata>
</GetMetricStatisticsResponse>

Every request is signed by aws.sign (Signature Version 4, service monitoring).

• size is the sum, over the storage classes, of the newest daily value of BucketSizeBytes, in GiB (1 073 741 824 bytes), to the thousandth: S3 counts every object, current and noncurrent, their metadata, and the parts of the multipart uploads not completed.
• objects is the newest daily value of NumberOfObjects, which counts the objects of every class, current and noncurrent, the delete markers and the parts of the multipart uploads not completed.
• age is the time from that point to now, in hours: S3 sends one point a day, and a figure a day or two old is the common case.
• A bucket for which CloudWatch lists no size stops the collector misconfigured: a name or a region that is not the bucket's. No number of objects in three days stops it unreachable.
• A region with other characters than lower-case letters, digits and hyphens stops the collector misconfigured before anything leaves. A code 400, 401 or 403 stops it misconfigured, with the error code AWS writes (InvalidParameterValue, AccessDenied, SignatureDoesNotMatch…); a throttled request (Throttling), a code 500 or above, or any other code but 200 stops it unreachable.

The keys come from protocol:aws: the secret key never leaves the poller, only the signature it makes. The answers are not secret.

Prerequisites

• An IAM user with an access key, or a role whose temporary keys are set with their session token, allowed these actions and nothing else:

{
  "Version": "2012-10-17",
  "Statement": [
    { "Effect": "Allow", "Action": ["cloudwatch:ListMetrics", "cloudwatch:GetMetricStatistics"], "Resource": "*" }
  ]
}

No permission on S3 is needed. The access key ID goes in protocol:aws:access-key-id, the secret key in protocol:aws:secret-access-key, the session token, for temporary keys only, in protocol:aws:session-token.

• The poller reaches monitoring.<region>.amazonaws.com in HTTPS, and its clock is right within a few minutes: AWS refuses a request signed more than fifteen minutes away from its own time. The certificates of AWS are issued by public authorities: the profile "Standard" trusts them.

Variables

Outputs

Watches

Limits

• One bucket per probe. The directory buckets (S3 Express One Zone) are not tried.
• The figures are those of the day before, at best: S3 sends them once a day.
• A storage class the bucket no longer holds for two weeks drops out of ListMetrics, which lists the metrics that received data in the last two weeks.
• Not checked against an AWS account: the metrics, their dimensions and their storage types are those of the Amazon S3 User Guide ("Metrics and dimensions"); the actions, their parameters and their fields those of the Amazon CloudWatch API Reference ("ListMetrics", "GetMetricStatistics") and of the Query protocol of AWS (Smithy, "AWS query protocol"); the reference no longer gives a sample of an XML answer.