Some of the customers with big billing data are complaining that the current Azure billing process is too slow. It turns out that the size of data is too big for the transformation process.
- The Billing Job or the Billing Schedule is very slow to complete.
- The Billing job seems to be stuck or hung at some point and no progress in the background.
- The Billing job unable to complete or no data available in transform history.
- The schedule shows that 2019/02 import succeeded but 2019/01 import is stuck.
- Workflow shows that the process is stuck during transformation. which means parsing is done and all billing data is in import table - sn_cmp_imp_azure_cost
- # of records in sn_cmp_imp_azure_cost for 2019/01 : 1,271,202
- According to transform history, two transforms have been stuck since 02-19-2019. These two transforms are causing the issue as per the above screenshot.
Steps to Reproduce
- Login to the instance
- Cloud Management > Cloud Admin Portal > Analyze > Billing
- Billing Schedules > Verify the billing schedule in the hung state > Open the record
- Observe either the Billing job is either hung or running very slow than expected
There are multiple updates with different issues on billing with Azure and AWS, and this is a part of the parent issue, hence consolidated all the fixes and released the Workaround along with updates.
Please refer "Mismatch in AWS / Azure and CMP Dashboard billing data" for the updates to implement as a workaround
The fix for this issue contains:
- Added new indexes for fast retrieval
- Created AzureBillingCSVRollupProcessor. This script will be exected to populate monthly rolled-up data.
- Enabled the new processor by replacing the value (sn_cmp.AzureBillingCSVRollupProcessor()) for provider: Azure and key: billing_processor_constructor
- Created a new property : sn_cmp.billing.use_tag_values_for_pivot for tracking tag_values change. The default is "false", Modified CmpBillingCSVProcessor & AzureBillingCSVProcessor
IMP: When applying the attached updateset, also make sure that this property should be set to 0 - sn_cmp.billing.update_threshold = 0
Related Problem: PRB1331941