Background

The upgrade is not reversible. If we do have to rollback, anything manually entered into the new version is likely to be lost.

See the upstream documentation for the upgrade process: https://wiki.civicrm.org/confluence/display/CRMDOC46/Upgrading+CiviCRM+for+Drupal+7

Preparation

TODO: Document MFT process for creating any point release. DRAFT: https://etherpad.wikimedia.org/p/Civi_release_process
Time the staging upgrade.
Test extensively on staging.

Scheduling

This upgrade will take 2 hours of preparation, 11 hours of database migration, then some follow-up sanity checking. It will disrupt mailing list updates and donation analytics.

Warn everyone

Tell all Civi users, online campaign leads, and third parties we interface with. Spell out all the impacts on people's work. Give enough notice that the ruckus settles down and we either have buy-in or a new date, well in advance of the planned upgrade.

Potential impacts:

Campaigns can stay up (see caveats below)
Donation numbers will not be available for campaigns. All tech will be able to say is, "processing in France is still up". Clickthrough numbers will be available.
Silverpop export will stop, so it won't be possible to exclude recent donations from newly cut mailing lists.
When Civi is back up, all these holes will be backfilled by the regular automated jobs.

Do it

Take the Site Offline

drush vset maintenance_mode 1

Disable Jenkins entirely.

Disable watchdogs and timeouts for MySQL

(make appropriate log entry in #wiki_tech with !log logentry or Tim Starling will be angry)

Kill the slow db query process killer.

root@db-master:~# killall -9 kill-slow-queries

Ensure that CiviCRM is not your Drupal homepage

In Administer » Site Configuration » Site Information, if the default home page value is "civicrm", change it. Probably to "node".

Disable all CiviCRM integration / extension modules (NOT CiviCRM itself)

Find an up to date list of modules in mediawiki/vagrant. For this upgrade, it will be:

MODULES="contribution_audit \
contribution_tracking \
environment_indicator \
exchange_rates \
ganglia_reporter \
globalcollect_audit \
large_donation \
log_audit \
offline2civicrm \
oneoffs \
paypal_audit \
queue2civicrm \
recurring \
recurring_globalcollect \
syslog \
thank_you \
twigext_l10n_tests \
wmf_audit \
wmf_campaigns \
wmf_civicrm \
wmf_common \
wmf_communication \
wmf_communication_tests \
wmf_contribution_search \
wmf_fredge_qc \
wmf_logging \
wmf_refund_qc \
wmf_reports \
wmf_test_settings \
wmf_unsubscribe \
wmf_unsubscribe_qc \
wmf_zendesk_reports \
worldpay_audit"

drush pm-disable -y $MODULES

Clear caches

Clear the database cache:

drush cc all

Delete all files in [yoursite]/sites/default/files/civicrm/templates_c/

Backup your CiviCRM database

Include triggers. Back up the civicrm and fredge databases, and drupal as well just to be a good citizen.

TESTING ONLY: At this point, you might want to restore a dump of the previous version:

mysql -u $CIVIADMIN -p -e "drop database dev_civicrm;"
mysql -u $CIVIADMIN -p -e "create database dev_civicrm;"
time gunzip -c civicrm-20150923.sql.gz | mysql dev_civicrm -u $CIVIADMIN -p

(real 172m50.430s user 10m53.093s sys 0m25.318s), plus 1 CPU pinned by mysqld for the 3 hours.

Do any pending OS maintenance

This is a great time to install the latest stable Linux kernel, packages, and reboot.

Run the Upgrade script

Run from the commandline, using:

(time drush -v -v civicrm-upgrade-db |& tee /tmp/civi-migration.log) ;\
(echo "Completed with exit status [$?], at" `date` | mail -s "Civi upgrade complete" tech@FOOBAR.org)

If something goes wrong and the upgrade can't complete, you will have to drop the entire DB and start over. Beware of new tables, simply restoring old tables from a backup will cause table already exists errors.

Re-enable compatible CiviCRM integration / extension modules

drush pm-enable -y $MODULES

Restore CiviCRM as the Drupal homepage if appropriate

If you had to change your homepage from "civicrm" to "node" In Administer » Site Configuration » Site Information, now would be an excellent time to change it back.

Unkill the slow db query killer

root@db1008:~# nohup ./kill-slow-queries &

Notes

In the event that you are going for a test install before running it on the real thing, and your cloned instance is presenting you with the drupal install screen, make sure the web server has the appropriate permissions on the cloned files. (Why it deduces you simply haven't installed the site yet, I have no idea...)