Notifications

45989 views

Description

NOTE: If you would like to provide feedback or have any questions please leave comments on this blog: Clone - Frequently Asked Questions - F.A.Q


The goal of this article is to answer generic frequent requests/questions ServiceNow Technical Support receives in relation to clones. I if you have follow-up questions, please contact Technical Support.

NOTE: If you are experiencing issues with attachments missing after clones this is because of a known issue that was fixed in Dec 2018 and is working as designed 

NOTE: I have included an XML file (cloneFAQ.xml) if you import this XML file into your instance it will create a module link in your instance which will show under clone and it will redirect to this FAQ page for up to date information about clones.

NOTE: We have seen issues with customer's not able to cancel their clone using the link on the clone form, this is due to a field not appearing on the form and is due to the form being customised. More details in the Can I rollback my clone? Section on how to fix and make this work.

FAQ

How does the clone work?Can I rollback my clone?If I check the audit and log data option, why is my other data excluded?
What do all the options do for cloning?Can I clone from a specific date?Can I request a backup to be taken before my clone? How can I determine the available backup for my instance?
Node in charge of the clone is offlineThe clone pre-flight check for valid capacity mapping has failed" on clone to demo or pov instance as targetUnable to login to instance after clone?
How do Excluders work?How do Preservers work?How do clone cleanup scripts work?
Can I have cleanup scripts per instance?After a clone, comments/updates all show as a different user and time?What is a draft? why can I not delete these?
Why is clone 'on hold'?How can I preserve my ATF jobs?I have renamed my instance and cannot create a new clone target?
How to do a full clone?If I clone to another instance will the release change?Node in charge of the clone is Offline?
What is the difference between Standby Database and Source InstanceHow can I exclude and preserve Users, Roles and groupsWhy is my text search not running after clone
Why are my attachment missing after a cloneCan I preserve plugins during a cloneIf preserving records do i need to preserve the attachments also?
Can I clone over my production instance and how?What tables should I not exclude?I excluded a specific table, after the clone the data is still there
If I change preservers and excluders will it work straight away in my next clone?Why have my email accounts been cloned to my target even when excluding data?I unchecked the exclude options but still after a clone the excluders still ran?
I can't add an existing table to the preserversThe clone failed with error "Cannot clone since target instance is a demo


How does the clone work?

In response to a clone request, the ServiceNow platform performs the following tasks:

    1. Generates a file to preserve operational data on the target server.

      This file contains the data preserved by data preservers.

    2. Creates a new database for the target instance
    3. Restores the database schema and data from the latest backup to the newly created database
    4. After all data is copied across it starts the exclude/preserver functions
    5. If Exclude options are ticked
      • Tables in the exclude module are truncated (All data is cleared out)
    6. File created in Step 1 for preservers is loaded into the target database (overwriting any data from the restore)
    7. Email functionality is disabled in the target database
    8. Once all completed the target instance repoints to the newly created database
    9. All Clone cleanup scripts are executed in target instance.
      • Regenerate Text indexes
      • Clear scheduled job node associations
    10. The clone is complete and the instance is accessible for the customer.
    11. After clone completes a backup is taken of the old Target database (Can be rolled back for up to 7 days)
    12. Once backup completes the old target database is retired.

Can I roll back my clone?

As we take a backup of the database after the cloning process we can rollback the clone up to 7 days after the clone was run.

Pre-Madrid:

To request a rollback please raise an case in HI to request a rollback of a clone.

Madrid and above:

After a clone is complete there is a UI action 'Rollback clone' if you click this button it will roll back the clone using the same automation that Technical Support will use to roll back the clone.

Any issues faced or clarification about the rollback of a clone please contact Technical Support and we will be happy to help. 

NOTE:

If the Cancel clone link is not working or responding, it has been related to a field "Cloud details" not being on the form. The reason this field has not been added to your form is because it would have been skipped as you have customised the clone form. 

To resolve the issue , please navigate to the clone Form and revert it to the base version. This will enable the "Cloud details" field on the form and then the "Cancel clone" link will work.
Another option is to configure the clone Form and add the "Cloud details" field on the form. 


If I check the audit and log data option, why is my other data excluded?

  • In releases before Kingston, the exclude audit and log data feature is not just for excluding audit and log data, if this is checked then it will run all the excludes in the exclude module.
  • In Kingston and newer releases, the exclude audit and log data is only for excluding audit and log data, there is another option "Exclude tables specified in Exclusion List" which will run all the excludes from the exclude module.

What do all the options do for cloning?

London and newer

In London we added the feature: Amount of data copied from the Task table, see Request a clone [London documentation]

FieldDescription
Exclude tables specified in Exclusion ListPrevent cloning records from the source instance specified in the System Clone > Exclude Tables module. Use this option to create empty but usable tables on the target instance. By default, the system excludes tables for auditing, license usage, logging, and notifications. This option is selected by default.
Note: This option is not supported by the legacy clone engine.
Exclude audit and log dataPrevents cloning audit and log records from the source instance. Use this option to create an empty but usable audit and log tables on the target instance. This option is selected by default.
Exclude large attachment dataPrevents the cloning of large attachments such as video files, image files, and other typically large binary file types. Excludes all common binary file types, regardless of file size. When selected, the clone also excludes attachments from the Attachments [sys_attachment] and Attachment Documents [sys_attachment_doc] tables that meet all these criteria.
  • The attachment table_name value does not indicate it is a small file. Small attachments have table name values that start with ZZ_.
  • The attachment data type value indicates it is a large file such as application or video.
  • The attachment table_name is not one of these system tables: sys_certificate, ecc_agent_jar, ecc_agent_mib, sys_store_app, or invisible.sys_store_app.

This option is selected by default.

Amount of data copied from the Task tableSelect the amount of data to clone from the source instance Task table. By default, the target instance receives the last 90 days of Task table records from the source instance.
Preserve themeEnables data preservers for the target instance theme and CSS elements. This option is selected by default.

Can I clone from a specific date?

Clone requests always use the latest backup for the source instance,

  • If you wish to use a date older then the latest backup you will need to raise a backup restore request in HI 
  • If you wish to take a fresh backup before taking the clone then you can raise a case for Technical Support to add you to the backup list so a fresh backup will be taken before the clone processes (Know that this can increase clone time as it needs to take a fresh backup which can take some time)

Can I request a backup to be taken before my clone?

 If you wish to have a fresh backup to be taken just before the clone is kicked off you can do this by raising a case in HI so that your instance to be added to on Demand Backup list to take a backup just before the clone

  • This can increase clone times, if your backup takes 6 hours, it will need to wait 6 hours for the backup to complete before it runs the clone engine.

How can I determine the available backup for my instance?

Get a "List of Available Backups of an Instance" from Now Support (HI)

  • Log in to Now Support (HI)
  • Click "Service Catalog" on the left nav
  • Select "Instance Management" in the Catalog Menu
  • Click the "List of backups for the instance" tile

My clone failed with message "Node in charge of the clone is offline"

This error message happens when the instance is being repointed to the new database and a health check is done on the node and it is offline causing this message. This can be safely ignored as the clone is working as designed it is just a timing issue with the health check.

This is a known problem, see KB0689695: Node in charge of the clone generates an offline error message on the instance clone history.

My clone failed with message "The clone pre-flight check for valid capacity mapping has failed" on clone to demo or pov instance as target

This error message happens when you are trying to clone over an instance which is too small and is not set up to grow in size.

Demo and POV instances are built to only grow to a specific size, so they can be cloned too if the source instance is also small, but once it grows to a specific size you cannot use demo or POV instances for cloning. You must then clone over a sub prod instance (which has the capability to grow in size) 

Also to note we cannot increase the size of a demo or POV instance, we will always request to clone over a sub prod instance.

Why am I unable to log in after a clone?

This happens more often than you would think.

When cloning over an instance it brings the settings from production this can include SSO and login functionality that can cause you not to be able to login into the sub production instance after the upgrade.

See KB0635905: The advantage of having a separate local login admin account when using SSO/LDAP.

This illustrates the benefits of always having a local account you can always log into, not only just for cloning.

To avoid this issue from happening for cloning you can do the following.

Method 1: Create a local admin user on the source instance.

  1. Create a local admin user with local password in your source instance
  2. Clone over sub prod instance
  3. Log in to instance using side_door.do with the username and password from step 1

Method 2: Create local admin user on target instance and include preserver.

  1. Create a local admin user with local password in your target instance
  2. Create clone preserver for user (sys_user), user roles [sys_user_has_roles], and user groups [sys_user_grmember]
  3. Clone over sub prod instance
  4. Log in to instance using side_door.do with the username and password from step 1

How do Excluders work? 

  • All Exclude settings must be set up on your SOURCE instance
  • Created  per table, you cannot use condition-based Excluders
  • They are only actioned for the SOURCE data
  • When excluding a table it truncates the table so no data will be kept from the source instance
  • By default the exclude options are selected for cloning
  • If you remove the options for excluding before cloning you will not exclude any data
  • You can Exclude and Preserve the same Table

How do Preservers work? 

  • All Preservers settings must be set up on your SOURCE instance
  • You can only preserve data not tables or columns
  • Created per table, you can use condition to only preserve specific data
  • They are only actioned for the TARGET data
  • Data will be imported after the Exclusions are actioned.
  • These will always run on every clone.
  • You can Exclude and Preserve the same Table
  • You can only preserve a table from the target instance if it also exists in the source instance (This is because table structure cannot be preserved only data)
  • Also, preservers are for preserving key data and not a mass amount of data, if you do preserve mass amounts of data the clone can break or take a very long time to run. Would advise not to preserve mass amounts of data.

More information can be found in KB0717012 - Clone results based on Exclusion and Preserver configuration

How do clone cleanup scripts work? 

  • After the clone completes all cleanup scripts are created as scheduled jobs and run till completion
  • There is no such ordering in cleanup scripts they will run in different orders
  • Must be defined on the SOURCE instance.

Can I have cleanup scripts per instance? 

  • You cannot have cleanup scripts specifically for certain instances
  • But you can define in the cleanup script to check the instance name system property and then depending on the result do specific jobs
  • EG:
var instanceName = gs.getProperty('instance_name');
if (instanceName == 'testdev')
{
// Do specific jobs for dev instance
}
else if (instanceName == 'testsandbox')
{
// Do specific jobs for sandbox instance
}

After a clone, comments/updates all show as a different user and time?

This is as designed, if you clone with excluders you are excluding the sys_audit table, the sys_audit table is used to keep the user, date and time for each comment and update so without the sys_audit it will use the sys_created_by and sys_created_at fields for all comments on the record.

If you view the history of the record all the updates will show as update 0

More information can be found in KB0690828: Cloning - Activity formatter and Audit not showing correct user and time.

What is a draft? why can I not delete these?

When you click on the UI to submit a clone, a record is created in the clone table, if you do not proceed with the clone or click out of the pop-up window the record will stay in your system as a 'draft' but this can be safely ignored. The reason these cannot be deleted is because they have a link to an internal instance at ServiceNow which manages the Clone requests.

Why is clone 'on hold'

When a clone State is set to 'Hold', this means the server rejected the clone request. Here are some of the most common reasons:
  • The clone was not ready to proceed by the scheduled time.
  • In pre-London releases, additional clone requests were submitted before the first one completed. The source clone for these requests are the same. We only allow 1 clone per database server source.
  • Target instance is not downgraded.
  • Problem with the backup used for the clone.

More information can be found in KB0694499 - Several clones in the state of 'Hold'

How can I preserve my ATF jobs?

We have had many requests on how to preserve ATF jobs during a clone. Preserving is not the best option as ATF was not designed to work with preservers but it was designed to be moved via update sets.

The best option would be to move your ATF tests to Production via update sets, then when you clone prod over a sub prod the ATF tests are there for you to use. If you commit the ATF tests in prod it doesn't mean you need to run them in prod it just makes it much simpler to clone over as you don't need to worry about preserving anything in the clone.

Note: If you are currently testing an ATF job in a sub prod and it's not ready to be pushed to prod but you need to clone, then export the ATF test via an update set, clone over your target instance and then recommit your update sets containing that ATF Test

I have renamed my instance and cannot create a new clone target

After you have renamed your instance you have tried to log in to the instance and tried to create a new clone target for the instance, this will not work as the instance is already in your instance under the old name.

To fix this you can follow the steps detailed in KB0550896: Unable to add clone target after instance rename.

How to do a full clone?

When someone asks to do a full clone this means that when you clone you should untick all the exclude options. If you are in London you will also have the option Amount of data copied from the task tables you should set this to all.

This means no excluders will run in the clone and it will be as close a replica of your source instance as possible. 

If I clone to another instance will the release change.

Yes, as when you are cloning you are making a replica of your source instance, so it will match in table structure and version.

So if you clone a Kingston over London, the target instance will be a Kingston release.

and vice-versa:

If you clone a London over a Kingston, the target instance will become a London release.

"Node in charge of the clone is Offline"

This error appears in your instance clone history and is a known error that is fixed in Madrid, this is tracked in PRB1262541/KB0689695: Node in charge of the clone generates an offline error message on the instance clone history.

This is a cosmetic issue only and is not an indicator that the clone has failed, if you check and see that the change request in HI is completed successfully then the clone has completed successfully and you can safely ignore this message. 

What is the difference between Standby Database and Source Instance 

When cloning you may get an option to select either Standby database and Source instance.

This is an old option and is no longer needed when requesting clones, it is removed in the newer releases and you shouldn't see it appear, but if you still see it appear you can safely leave and ignore this option. 

How Can I exclude and Preserve Users, Roles, and Groups 

In some situations, you would like to not bring over the users from the source instance and only preserve the target instances users. This will step you through the best way to exclude and preserve Users. 

To do this you need to create the following Excluders:

  • sys_user
  • sys_user_grmember
  • sys_user_has_role

This will not bring over any users from the source instance or any of the references to groups and roles, but now you need to preserve the sys_user in the target instance, but something that many people forget is you also need to preserve roles, groups and also the links from users to roles and groups, this is a common mistake and after cloning you are missing the admin role and cannot log in, this is because the links to the roles are missing.

Now you should create these preservers: (Add any conditions depending on your requirements)

  • sys_user
  • sys_user_role
  • sys_user_group
  • sys_user_grmember
  • sys_user_has_role

This will preserve all the users, roles and groups in the target instance, but it will also preserve the permissions of those users to those groups and roles.

If you place these in your instance, after a clone you will have all the same users from your target instance and will still have all the access from before the clone.

Why is my text search not running after my clone

When cloning over an instance you may experience the issue that your text searches are not working.

This is because one of the clone cleanup scripts that run after a clone completes is to reindex this instance for search, while this is running your search will not work.

If you look at jobs running you will see text index events process, if you look on your stats.do you will see something like this:

 

You can monitor this to completion, once done your search functionality will start to work. 

Why are my attachments missing after a clone?

You may have realized that lately after your clones have completed you are missing attachments in the target instance and has been causing issues. 

This is actually working as designed but this exclusion started to work correctly after December 2018 after an internal problem was fixed with the clone automation. This is outside the ServiceNow releases so this will affect all releases and all clones from then. The fix was for the clone option: Exclude large attachment data. 

As documented in the previous question: What do all the options do for cloning?

It states it will remove sys_attachment records that don't meet that criteria, it is not based on the size as the name suggests.

So this is working now as designed and documented through our documentation if you wish to keep your attachments you will need to uncheck the Exclude large attachment data checkbox upon cloning.   

Can I preserve plugins from the target instance?

In some cases when testing plugins in your target instance you wish to preserve these plugins so you can continue to test out these plugins and work on them. 

Unfortunately, the way we do clones we do a restore of the source instance over a new database, after the clone you will need to activate the plugins again or request them via HI if required.

If Preserving records do I need to preserve the attachments also?

When cloning you may want to preserve certain records on your target instance like for example some incidents, you also want to make sure after the clone that the attachments linked to these incidents are preserved also.

Currently, in our automation, if you preserve a record that has attachments we automatically preserve those attachments without having to create another preserver specifically for the attachment table. 

Can I clone over my production instance and how?

This is risky and would advise against it but in some cases, you may want to perform a clone over your production instance, this may be for a go-live, if your instance is already live this is not a good idea and would advise you to contact Technical Support for assistance.

To add your production instance as a clone target you can follow the below steps:

  1. Log in to your production instance
  2. Go to the system properties list
  3. Find system property: glide.db.clone.allow_clone_target
  4. Set this to 'true'

This will now allow this instance to be created as a clone target

Cloning over this instance is the same as any other request for a clone. After cloning is complete this property should be then set back to 'false' so you don't accidentally clone over your prod instance later.

What tables should I not exclude?

Upon looking at issues raised through support we have seen a number of tables that have been excluded and causes issues after the clone, we recommend that if this table needs to be excluded that you also preserve it, or if you just wish to delete some of the data from that table to use a clone cleanup script.

Tables to not exclude

  • sys_properties
  • sys_trigger
  • sys_plugins (this can potentially cause the target instance to go down)
  • sys_user (as stated before but if preserved with the other tables it should be fine)

Excluding these can put your target instance into a bad state, and you will need to rollback the clone, adjust and clone again.

I excluded a specific table, but after the clone, the data is still there?

This is a very common question that comes up to support. You enter in an excluder but after the clone, the data is still there.

This can be a number of different causes:

  1. You unchecked the checkbox to exclude tables for the clone
  2. You tried to exclude a child table that is using the TPH extension model EG: incident. 

The system cannot exclude tables that extend the Task table and are also flattened into it as part of the table per hierarchy extension model. Since these extended tables are actually part of the same physical database table, the system clones the data when it clones the Task table. You can exclude tables that extend the Task table under two conditions. Either the system stores the tables in their own physical tables as part of the table per class extension model, or you exclude the Task table itself.

If I change preservers and excluders will it work straight away in my next clone?

A common question, you have just adjusted your excluders or preservers and want to know if you have to wait for some time before cloning. The answer is NO, once you change the excluders and preservers, those changes will be enforced in the next requested clone.

Why have my email accounts been cloned to my target even when excluding data?

On occasion after a clone before turning on email you see that your production email account has been copied into your sub production instance. You then check that you did have the exclude tables option checked but still seem to have come across.

When this issue has happened it is because normally on one occasion you have cloned to a sub prod and not excluded the table, therefore the sys_email_account has been copied across to your target instance, then you have a preserver for sys_email_account so it would have been preserved every clone after that.

Workaround: Clone Cleanup Script

  1. Log in on the production instance
  2. Check all the sys_id's of the email accounts that you would not ever want in your sub prods
  3. Create a clone cleanup script in your production that will delete all the email accounts with those sys_id's

Example Script:


// Add sys_ids to the encoded query (queryString)
// Like queryString = "sys_idIN206594e8db9eb3002e14c6fc34961998,6ab25213db003300eaf8e64305961927"

var
queryString = "sys_idIN"; var emailclean = new GlideRecord('sys_email_account'); emailclean.addEncodedQuery(queryString); emailclean.deleteMultiple();

This will mean after all clones it will delete these sys_email_accounts even if they were cloned over or not. If you add new email accounts you will need to add to this script, but this will stop the email accounts ever coming from your source instance.

I unchecked the exclude options but still after a clone the excluders still ran?

Recently we have seen an increase in this issue. After requesting a clone and unticking the exclusion options after the clones it seems like excluders still run why??

This has been linked to a UI change in the request form and some customers had customized the form therefore not getting the changes.

A change was made to the UI to hide the options for excluding in an option tab that you need to click to open and view. Since this happened some customers have customized the form or may have already customized the form to show those values instead of seeing the options in the hidden section, therefore having the field twice on the form.

The problem here is that the values will always come from the last value it finds so if you unchecked the options on the main form it will take the values from the options as that is the last value for that field it finds.

To fix this you will need to update the form so the fields only appear once on the form over all sections. Once this is fixed it will not happen again.

I can't add an existing table to the preservers?

Sometimes you are trying to add a data preserver for a specific table and you cannot find it in the list and wondering why?? This is most probably due to the scope of the table.

If you want to preserve a table that is part of a specific scope you need to do the following:

  1. Switch to the scope of the table by clicking the cogwheel (gear icon) on the top right > navigate to 'Developer' > pick the scope of the table
  2. Then go to the Data preserver table.
  3. Now creating the data Preserver you will be able to select that specific table and save.

The clone failed with error "Cannot clone since target instance is a demo"

A change was made recently (Feb 2020) to avoid the issue described earlier in Clone Mapping.

The issue is when a you request a clone over a demo instance, it fails with storage capacity checks as the source instance is too large to store all the information on the target instance. This is because Demo instances are not configured to expand their storage capacity past a set limit.

Due to certain circumstances, this change to not allow clones over demo instances has now been rolled back and cloning will run again as normal for cloning over demo instances, with the original issue of if the source is too large the clone will not work due to capacity reasons.



Article Information

Last Updated:2020-08-30 17:07:31
Published:2020-08-31