293 views

Description

When copies are done in parallel, the ExecutionTracker used by DBOnlineAlter will be manipulated from multiple threads. It's thread-unsafe for multiple reasons, one of which is that it holds (via ProgressWorker) a GlideRecord.

This leads to Update Set commits failing with errors such as:

2018-02-01 02:28:13 (993) worker.7 worker.7 [TableChunkCopier] Starting [online_alter] operations
2018-02-01 02:28:14 (401) glide.db.online_alter.0.change_request SYSTEM [0:00:00.235] Copied chunk 1 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:14 (407) glide.db.online_alter.1.change_request SYSTEM [0:00:00.240] Copied chunk 2 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:14 (408) glide.db.online_alter.2.change_request SYSTEM [0:00:00.240] Copied chunk 3 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:14 (412) glide.db.online_alter.3.change_request SYSTEM [0:00:00.244] Copied chunk 4 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:14 (413) glide.db.online_alter.4.change_request SYSTEM [0:00:00.245] Copied chunk 5 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:14 (414) glide.db.online_alter.0.change_request SYSTEM Time: 0:00:00.235 id: jtiqas_1[glide.6] for: INSERT IGNORE INTO tmp_c1075823340t (...)
2018-02-01 02:28:14 (419) glide.db.online_alter.4.change_request SYSTEM Time: 0:00:00.245 id: jtiqas_1[glide.10] for: INSERT IGNORE INTO tmp_c1075823340t (...)
2018-02-01 02:28:14 (420) glide.db.online_alter.1.change_request SYSTEM Time: 0:00:00.240 id: jtiqas_1[glide.7] for: INSERT IGNORE INTO tmp_c1075823340t (...)
2018-02-01 02:28:14 (439) glide.db.online_alter.3.change_request SYSTEM Time: 0:00:00.244 id: jtiqas_1[glide.9] for: INSERT IGNORE INTO tmp_c1075823340t (...)
2018-02-01 02:28:14 (441) glide.db.online_alter.2.change_request SYSTEM Time: 0:00:00.240 id: jtiqas_1[glide.8] for: INSERT IGNORE INTO tmp_c1075823340t (...)
...
2018-02-01 02:28:22 (854) glide.db.online_alter.1.change_request SYSTEM [0:00:00.198] Copied chunk 126 of 127 to tmp_c1075823340t[change_request]
2018-02-01 02:28:22 (860) glide.db.online_alter.1.change_request SYSTEM Time: 0:00:00.198 id: jtiqas_1[glide.7] for: INSERT IGNORE INTO tmp_c1075823340t (...)
2018-02-01 02:28:22 (863) worker.7 worker.7 SEVERE *** ERROR *** Something went wrong during online alter, cleaning up temporary table/triggers
java.sql.SQLException: java.lang.NullPointerException
        at com.glide.db.bulk_copy.TableChunkCopier.copyData(TableChunkCopier.java:367)
        at com.glide.db.bulk_copy.TableChunkCopier.copy(TableChunkCopier.java:219)
        at com.glide.db.bulk_copy.TableChunkCopier.copy(TableChunkCopier.java:279)
       at com.glide.db.ddl.DBOnlineAlter.copyData(DBOnlineAlter.java:175)
        at com.glide.db.ddl.DBOnlineAlter.doOnlineCopy(DBOnlineAlter.java:137)
        at com.glide.db.ddl.DBOnlineAlter.alterTable0(DBOnlineAlter.java:367)
        at com.glide.db.ddl.DBOnlineAlter.alterTable(DBOnlineAlter.java:341)
        at com.glide.db.ddl.TableAlterer.alter(TableAlterer.java:34)
        at com.glide.db.ddl.TableAlterer.alter(TableAlterer.java:20)
        at com.glide.db.bootstrap.xml.StorageTableChangeCommitter.commit(StorageTableChangeCommitter.java:79)
        at com.glide.db.bootstrap.xml.BootstrapBatcher.commit(BootstrapBatcher.java:393)
        at com.glide.update.UpdateSetCommit.commit(UpdateSetCommit.java:350)
        at com.glide.update.UpdateSetCommit.handleAlterBatching(UpdateSetCommit.java:343)
        at com.glide.update.UpdateSetCommit.handleLoads(UpdateSetCommit.java:236)
        at com.glide.update.UpdateSetCommit.loadSetItems(UpdateSetCommit.java:184)
        at com.glide.update.UpdateSetCommit.commitUpdateSet0(UpdateSetCommit.java:162)
        at com.glide.update.UpdateSetCommit.commitUpdateSet(UpdateSetCommit.java:141)
        at com.glide.update.UpdateSetWorker.commitUpdateSet(UpdateSetWorker.java:1288)
        at com.glide.update.UpdateSetWorker.startWork(UpdateSetWorker.java:190)
        at com.glide.worker.AbstractProgressWorker.startAndWait(AbstractProgressWorker.java:123)
        at com.glide.worker.HierarchicalProgressWorker.startAndWait(HierarchicalProgressWorker.java:34)
        at com.glide.worker.BackgroundProgressJob.execute(BackgroundProgressJob.java:54)
        at com.glide.schedule.JobExecutor.executeJob(JobExecutor.java:103)
        at com.glide.schedule.JobExecutor.execute(JobExecutor.java:89)
        at com.glide.schedule.GlideScheduleWorker.executeJob(GlideScheduleWorker.java:219)
        at com.glide.schedule.GlideScheduleWorker.lambda$process$66(GlideScheduleWorker.java:161)
        at com.glide.worker.TransactionalWorkerThread.executeInTransaction(TransactionalWorkerThread.java:35)
        at com.glide.schedule.GlideScheduleWorker.process(GlideScheduleWorker.java:161)
        at com.glide.schedule.GlideScheduleWorker.run(GlideScheduleWorker.java:72)

Steps to Reproduce

 

  1. Commit an update set that adds a column to an existing table populated with data.

    For more information, see the product documentation topic Commit an update set.

    The alter may fail under certain circumstances due to heavy use of threads involved in the transaction.

Workaround

Change the value of the glide.db.online_alter_threads system property to 1. If the property does not exist, create it.

  1. Enter 'sys_properties.list' in the navigation filter.
  2. Click New and create the system property with the following values:
    • Name: glide.db.online_alter_threads
    • Type: integer
    • Value: 1
  3. Click Submit.

 

 


Related Problem: PRB1234411

Seen In

There is no data to report.

Intended Fix Version

London

Fixed In

Jakarta Patch 8
Kingston Patch 3

Safe Harbor Statement

This "Intended Fix Version" information is meant to outline ServiceNow's general product direction and should not be relied upon in making a purchasing decision. The information provided here is for information purposes only and may not be incorporated into any contract. It is not a commitment, promise, or legal obligation to deliver any material, code, or functionality. The development, release, and timing of any features or functionality described for our products remains at ServiceNow's sole discretion.

Associated Community Threads

There is no data to report.

Article Information

Last Updated:2018-06-15 14:17:08
Published:2018-02-07