Notifications

65 views

Description

  • When downloading AWS billing report from S3 bucket in "us-east-1" region, S3 returned the following error,
Unable to establish connection to https://s3.amazonaws.com: com.amazonaws.services.s3.model.AmazonS3Exception: 
The specified key does not exist. (Service: Amazon S3; Status Code: 404; Error Code: NoSuchKey; Request ID: xxxxxx; S3 Extended Request ID: xxxxxx), S3 Extended Request ID: xxxxxx

Release or Environment

  • Instance with CMP plugin enabled.

Cause

  • The billing file in S3 bucket could not be found when generic S3 endpoint is used.

Resolution

Logic:

  • In general when downloading the billing report, we do the following

1. Makes a call to S3 to find the region the targeted bucket resides in

2. Append the region information to the endpoint URL

  • However, if S3 bucket region is in "us-east-1", S3 will return region as "US" in step #1 above. The end result is the following generic S3 endpoint built: https://s3.amazonaws.com.
  • This is in-line with S3 API documentation,

https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingBucket.html

https://docs.aws.amazon.com/general/latest/gr/rande.html#ct_region

Probable Cause:

  • The probable issue here might be due to different kind of setting the bucket has, which in turn would lead S3 API not being able to fetch the files in the bucket.
  • This has to be verified and validated with internal S3 Admin and possibly AWS Support to see why billing file in S3 bucket is not found when generic S3 endpoint is used.

Workaround:

  • In order to check and make this work (i.e. by Changing Endpoint to use as "https://s3.us-east-1.amazonaws.com") can be achieved by adding following code block in "AWSBillingProbe" MID Server Script Include at lines 73 - 75:
=================
if (region == "US")
region = "us-east-1";
=================

Note:

  • Once this is fixed internally revert changes which were made to the "AWSBillingProbe" MID Server Script Include and re-run the billing to check the behavior.
  • If issue still persists please contact ServiceNow Technical Support for further assistance.

Additional Information

Before proceeding with above steps kindly review and validate the steps mentioned in the below KB's.

  1. KB0723524 - AWS billing Download error either with Status Code: 403 or Status Code: 404
  2. KB0647606 - Cloud Management Troubleshooting - Billing

Article Information

Last Updated:2020-01-06 07:36:05
Published:2020-01-06
Pasted image.png