Important tips to be successful and avoid issues in the production environment - 

- Work on a copy of the production environment (with data). If not, time/effort will be spent on issues that the non-prod environments have, but the production environment does not have. 
- When taking a backup of the environment, please make sure all the following are backed up while the system is stopped. If using VMs, highly recommend taking snapshots - 
  + Filesystem - TIBCO_HOME  - from all servers including ActiveMatrix (AMX) Administrator / EMS. Preserve timestamps, please. 
  + Filesystem - CONFIG_HOME - from all servers including AMX Administrator / EMS. Preserve timestamps, please. 
  + Filesystem - SHARED_MOUNT used by AMX / BPM / EMS (as applicable). Preserve timestamps, please. 
  + Database - AMX / BPM / EMS (as applicable)

The resolution section of this article covers steps that are needed to be performed before/after the upgrade process is complete. 

This article summarizes steps that are needed to avoid issues during the upgrade process and/or post-upgrade.

ALL

Below upgrade steps serve as a supplement to the product documentation and should not be treated as a replacement. 

==============  
A. Prior to the Upgrade - (can be done weeks before the scheduled upgrade - as long as there are no app upgrades/deployments)
==============  

1. Review the documentation - 
   - 4.2.0 readme
   - 4.2.0 system requirements
   - Release Notes, most importantly - Closed Issues for the versions you will be covering in the upgrade. i.e. if upgrading from 3.1.0 to 4.2.0, then go thru 4.0.0, 4.1.0 & 4.2.0 release notes
   - 4.2.0 - Upgrade documentation - 
     + Upgrade Introduction & Impact
     + Upgrade Overview & Steps
   - Latest hotfix readme for both AMX v3.3.0 & BPM v4.2.0. There maybe new enhancements (to existing features) which are enabled via a configuration change. 

2. Install and apply the latest AMX platform hotfix on each server (this can also be done pre-upgrade - before installing BPM 4.2.0 in TIBCO_HOME)

3. Housekeeping (should be done regularly) -  All of the below practices will help with the mandatory post-upgrade task of event collector (EC) data migration  
   - List how many older versions of any app exists. Validate with business if any of the older process instances (PI) are still needed. If not, then cancel them. 
   - Remove any older DAAs/templates from Admin > Software Management -- before this check the version has been removed from BPM database and node runtime
   - Check if there pending messages on EMS queues. Queue's with pattern *_ErrorQueue do not have any receivers. Messages can get accumulated over a period of time, due to TX failures. If there are no ongoing runtime issues, then purge such queues.
   - Purge EC data - both instance & non-instance. 

4. Make sure the system requirements are still met. System configuration may get changed from what it should be -
   - Unix user limits - open file / max user processes etc
   - Oracle - open cursors

Refer pre-install tasks - https://docs.tibco.com/pub/amx-bpm/4.2.0/doc/html/bpmhelp/GUID-05753A7B-3CC5-4423-AF50-F3C3E23C7B5A.html

5. Perform below validation to check if any older application versions are not in Prepare For Undeploy (PFU) state - 

https://support.tibco.com/s/article/TIBCO-ActiveMatrix-BPM-product-upgrade-may-fail-while-deploying-BPM-applications

6. Make sure hosts/nodes have been updated to use JRE 8. 

7. Validate whether Stats Collector (SC) data is used or not. If not, SC should be disabled. Refer - https://support.tibco.com/s/article/How-to-disable-Statistics-Collector

==============  
B. Pre-upgrade - (prior to running TIBCO Configuration Tool [TCT] to upgrade)
==============  

1. Perform all the pre-upgrade steps as listed in the documentation - 

https://docs.tibco.com/pub/amx-bpm/4.2.0/doc/html/bpmhelp/GUID-AFA89426-8AF5-4AFB-B7CA-264DC5EB6241.html

2. Prior to installing BPM 4.2.0 in TIBCO_HOME, take a backup of - 

$TIBCO_HOME/administrator/3.3/samples/admin-scripts-base.xml
$TIBCO_HOME/administrator/3.3/scripts/basic_cli.xml

This is due to the issue documented here - 
https://support.tibco.com/s/article/TIBCO-ActiveMatrix-BPM-upgrade-may-require-an-additional-step-to-restore-ActiveMatrix-Administrator-command-line-interface

After installing BPM 4.2, restore these 2 files. 

3. Install the latest 4.2.0 hotfix (HF) as well (only on machines where admin exists). 

Note - When running TCT to upgrade, HFs are also applied

4. Increase SystemNode & TCT heap size - For example - change Xmx to 4G & MaxPermSize to 2G -- as feasible

5. Take a full backup of the environment before running the upgrade. 

==============  
C. Upgrade - 
==============  

1. In TCT - only choose the last option. i.e. only to upgrade AMX BPM application (upgrade-bpm)

==============  
D. Post Upgrade (prior to releasing the system for business) - 
==============  

1. Login into Openspace / Workspace right after the upgrade. -- Only to validate the login is successful post-upgrade

2. Perform all the post-upgrade and post-installation tasks (as applicable) - 

https://docs.tibco.com/pub/amx-bpm/4.2.0/doc/html/bpmhelp/GUID-BAF0FACA-CD9C-4B9E-B65E-87E9CA6EF3CF.html

3. Check for the below issues - 

https://support.tibco.com/s/article/Process-Manager-background-thread-gets-an-error-after-upgrade-to-TIBCO-ActiveMatrix-BPM-v4-2
https://support.tibco.com/s/article/Process-Instance-creation-can-fail-with-a-NullPointerException-after-applying-TIBCO-ActiveMatrix-BPM-4-2-0-HF-2-or-4-1-0-HF-8-or-4-0-0-HF-4

4. Change the below index - 

https://support.tibco.com/s/article/Modifying-the-Indexes-Used-on-the-ec-managed-obj-lifecycle-Table-Avoids-Deadlocks-on-SQL-Server-and-Improves-Performance

5. If the system has been upgraded from AMX BPM v3.1.0 or lower, ensure DataSourceDirect connection pool has been increased as per the post-upgrade steps -- possibly doubled (50 -> 100)

6. During the upgrade, the bindings corresponding to 4.2.0 version are installed with SOAP v1.1. If SOAP v1.2 was being used before the upgrade, then this configuration change needs to be repeated post-upgrade. Please refer - https://docs.tibco.com/pub/amx-bpm/4.2.0/doc/html/bpmhelp/GUID-A955485B-5FEE-48FE-918B-EFC1C8D3C439.html

7. Ensure the old SOAP API has been replaced with the new 4.2.0 version. This will apply to any client (e.g. - BW / Mediation / BPM app etc) which invoke amx.bpm.app APIs.