Notifications

947 views
How to get a rough estimate of the clone progress

Description


It is challenging to track the clone progress to see if it is still moving, due to lack of monitors or information. The process explained in this article shows you how to get a rough estimate of clone progress when copying over the data.

Procedure


  1. When you clone an instance, you get a change context in the clone change, for example, <datacenter>/u_cid_clone_engine_context.do?sys_id=663feebcdbb972043353f22ebf96190a.
  2. When you open this, you see details from the Source and Target Instance Info Tab: 

    Source: instancesource 

    DB: db123456.<db_source_server>.service-now.com 

    Source Socket: /tmp/mysqld_instancesource_3400.sock

    Target: instancetarget

    Target DB: MySQL Server@db654321.<db_target_server>.service-now.com:3401

  3. With this information, you can get a rough estimate of the size of Source. To do this, constantly check the Target DB for its size and compare for a percentage:
    1. SSH to db.
    2. Run script to get size

(Result in KB) sudo du -s /glide/mysqld/<Instance_Port_Size>

OR

(Result in MB,GB,TB) sudo du -hs /glide/mysqld/<Instance_Port_Size>

For example:

********************************************* 

Source: instancesource 

********************************************* 

DB: db123456.<db_source_server>.service-now.com 

PROCESS: 

ssh db123456.<db_source_server>.service-now.com 

sudo du -s /glide/mysqld/instancesource_3400_giga/ 

RESULT: 

3652849276 /glide/mysqld/instancesource_3400_giga/ 

OR

sudo du -hs /glide/mysqld/instancesource_3400_giga/ 

RESULT: 

3.5TB /glide/mysqld/instancesource_3400_giga/ 

 

********************************************* 

TARGET: instancetarget 

********************************************* 

DB: db654321.<db_target_server>.service-now.com 

PROCESS: 

ssh db654321.<db_target_server>.service-now.com

sudo du -s /glide/mysqld/instancetarget_3401_xlarge/ 

RESULT: 

184684540 /glide/mysqld/instancetarget_3401_xlarge/ 

OR

sudo du -hs /glide/mysqld/instancetarget_3401_xlarge/ 

RESULT: 

176GB /glide/mysqld/instancetarget_3401_xlarge/ 

4. From this result, you can provide a percentage of data copied across:

Current state: 

184684540 / 3652849276

176 / 3483 GB 

5.1% 

 

Noting the DB size for production at the start of the change and using that as an estimate of the DB size, you can compare the growing size of the Target DB and determine a percentage.

While it is a rough estimate, it helps when customers ask for an update and it may assist them in estimating how much has been copied across.

 ADDITIONS:

Thanks for this addtion Thijs, updating now with Jos's enhancement:

 

the following command enhances the suggestion from Thijs to run the command every minute. It adds the timestamp and the percentage currently copied (based on the hardcoded value of the size on the source):
while sleep 60; do curdatum=`date`; curval=`sudo du -s /glide/mysqld/<instance>_<port>_<size> |awk 'NR==1{print $1}'`;echo -n $curdatum; echo -n " - ";echo -n $curval;echo -n " - ";awk "BEGIN {printf \"%.3f%\n\", $curval/<size on host in bytes>*100}";done

e.g. 
while sleep 60; do curdatum=`date`; curval=`sudo du -s /glide/mysqld/surfmirror_3401_ultra |awk 'NR==1{print $1}'`;echo -n $curdatum; echo -n " - ";echo -n $curval;echo -n " - ";awk "BEGIN {printf \"%.3f%\n\", $curval/7630779816*100}";done

 

Applicable Versions


[Enter text here]

Additional Information


[Optional - remove this entire section if you do not need to list additional helpful resources]

 

Process

  1. When you clone an instance, you get a change context in the clone change, for example, <datacenter>/u_cid_clone_engine_context.do?sys_id=663feebcdbb972043353f22ebf96190a.
  2. When you open this, you see details from the Source and Target Instance Info Tab: 

    Source: instancesource 

    DB: db123456.<db_source_server>.service-now.com 

    Source Socket: /tmp/mysqld_instancesource_3400.sock

    Target: instancetarget

    Target DB: MySQL Server@db654321.<db_target_server>.service-now.com:3401

  3. With this information, you can get a rough estimate of the size of Source. To do this, constantly check the Target DB for its size and compare for a percentage:
    1. SSH to db.
    2. Run script to get size

(Result in KB) sudo du -s /glide/mysqld/<Instance_Port_Size>

OR

(Result in MB,GB,TB) sudo du -hs /glide/mysqld/<Instance_Port_Size>

For example:

********************************************* 

Source: instancesource 

********************************************* 

DB: db123456.<db_source_server>.service-now.com 

PROCESS: 

ssh db123456.<db_source_server>.service-now.com 

sudo du -s /glide/mysqld/instancesource_3400_giga/ 

RESULT: 

3652849276 /glide/mysqld/instancesource_3400_giga/ 

OR

sudo du -hs /glide/mysqld/instancesource_3400_giga/ 

RESULT: 

3.5TB /glide/mysqld/instancesource_3400_giga/ 

 

********************************************* 

TARGET: instancetarget 

********************************************* 

DB: db654321.<db_target_server>.service-now.com 

PROCESS: 

ssh db654321.<db_target_server>.service-now.com

sudo du -s /glide/mysqld/instancetarget_3401_xlarge/ 

RESULT: 

184684540 /glide/mysqld/instancetarget_3401_xlarge/ 

OR

sudo du -hs /glide/mysqld/instancetarget_3401_xlarge/ 

RESULT: 

176GB /glide/mysqld/instancetarget_3401_xlarge/ 

4. From this result, you can provide a percentage of data copied across:

Current state: 

184684540 / 3652849276

176 / 3483 GB 

5.1% 

 

Noting the DB size for production at the start of the change and using that as an estimate of the DB size, you can compare the growing size of the Target DB and determine a percentage.

While it is a rough estimate, it helps when customers ask for an update and it may assist them in estimating how much has been copied across.

 ADDITIONS:

Thanks for this addtion Thijs, updating now with Jos's enhancement:

 

the following command enhances the suggestion from Thijs to run the command every minute. It adds the timestamp and the percentage currently copied (based on the hardcoded value of the size on the source):
while sleep 60; do curdatum=`date`; curval=`sudo du -s /glide/mysqld/<instance>_<port>_<size> |awk 'NR==1{print $1}'`;echo -n $curdatum; echo -n " - ";echo -n $curval;echo -n " - ";awk "BEGIN {printf \"%.3f%\n\", $curval/<size on host in bytes>*100}";done

e.g. 
while sleep 60; do curdatum=`date`; curval=`sudo du -s /glide/mysqld/surfmirror_3401_ultra |awk 'NR==1{print $1}'`;echo -n $curdatum; echo -n " - ";echo -n $curval;echo -n " - ";awk "BEGIN {printf \"%.3f%\n\", $curval/7630779816*100}";done

 

 

Article Information

Last Updated:2018-10-16 10:34:58
Published:2018-07-25