Issue
Contents
1. Overview
While downloading the billing information from AWS the following error is shown:
Downloading dataUnable to establish connection to https://s3.amazonaws.com: com.amazonaws.services.s3.model.AmazonS3Exception:
The AWS Access Key Id you provided does not exist in our records.
(Service: Amazon S3; Status Code: 403; Error Code: InvalidAccessKeyId; Request ID: 00A2720DB45B6E4E), S3 Extended Request ID: hU5c2E/UwRUGQ+ucSbRJercaLeY7eHW3fO4XjVu6gnDibi4Icouo3Qr++K0Kw4oWeMjjyV/LTZ8=
2. Steps to Reproduce
- Log in to the instance
- Impersonate with the user having Billing configuration at Cloud Admin Portal.
- Navigate to Cloud Admin Portal > Analyze > Billing > Billing Schedules
- Verify the AWS billing schedule and click to open (Define the schedule for downloading AWS billing data)
- Verify the provided Billing Bucket name (Review: Billing and Usage Reporting for S3 Buckets) >> The S3 bucket name already configured.
- Execute "Execute Now" from the top right of the Billing schedule page to initiate a new billing.
- Once the Schedule starts, it will error out as follows:
3. Investigations
- Verify if there are any recent modifications made to the bucket name on the billing schedule
- Verify that the files of the bucket are of the following format
{accountNumber}-aws-billing-detailed-line-items-with-resources-and-tags-{year}-{month}-.csv.zip (without the brackets)
- Make sure the not trying to download billing data future dates
- Make sure the requested billing dates while executing a new download and verify at Amazon S3 Bucket has that information
- Make sure the AccountID and the Secret key are correctly configured for the Service Account of AWS provided in the Billing Schedule configuration
- Log in to the AWS console and verify the S3 bucket provided to the schedule has correct configuration and verified successfully
4. Cause
- AWS has multiple configuration methods for S3 bucket billing, but, ServiceNow currently supports only "Detailed Billing Report with Resources and Tags", the newer method is "Cost and Usage Report". The customer might have configured "Cost and Usage Report". See:
- Once S3 bucket configured with "Detailed Billing Report with Resources and Tags" will be associated with the Account ID, the account must need match with the Service Account configured in the Cloud Billing Schedule, in the customer environment, the Service Account was different and the S3 Bucket configured do not contain the same
- The customer might be using the Primary Account for credentials usage and the child accounts might be using the same credentials for Discovery, but for Billing it needs every Service Account should have their own credentials configured
5. Solution
- Make sure the S3 Billing bucket configured with "Detailed Billing Report with Resources and Tags" and the files in the bucket have below format.
{accountNumber}-aws-billing-detailed-line-items-with-resources-and-tags-{year}-{month}-.csv.zip (without the brackets)
- Make sure the Service Account ID in billing schedule and the Account ID in the above file format {accountNumber} is the same
- Make sure the Service Account have the correct credentials (AccessID and Secret key)
- Verify the Service Account credentials in the AWS console Command line Configuring the AWS CLI (Refer: AWS configure)
- Below is the error occurred while testing and make sure the credentials pass
