Drupal 7 and upgrading Media module to 2.x

For a few years now I've been hosting and maintaining a Drupal 7 installation for a non-profit club. I chose Drupal because I needed to provide a website that a non-technical person could easily publish content on.

I've written this post in the hope that it might just help someone and also to vent some frustration over the house-of-cards that is Drupal.

Why Drupal?

I did not want to run Wordpress because at the time, Wordpress had a sorry security and upgrade reputation. I'll note now that Wordpress has slowly rehabilitated that reputation, but still not to the point that I'd be willing to look after a Wordpress site myself.

Drupal is no walk in the park. In fact, I've been constantly annoyed by just how difficult maintaining a Drupal installation is. Layers upon layers of modules need to be installed before you have a dynamic and user-friendly website.

Drupal Drawbacks

Upgrading modules is generally easy, but still a chore in particular because I do not personally like the automated upgrade methods. For anyone maintaining many Drupal installations or a popular site where quick adoption of security patches is mandatory I am sure automated updates are a lifesaver.

I'll skip mentioning the scores of transient errors (Drupal love your system memory long time) and tedious frustrations (I cannot for the life of me work out how to get clean URLs working) that one encounters with Drupal in general, because I want to cover off some notes about the Media module and the version change to 2.x.

There was a major security problem discovered in the Media module version 1.x, which required an upgrade to 2.x. The upgrade instructions were so convoluted and the user problems post-upgrade were so frightful that I elected to wait it out and let other people suffer the bugs and pain. Meanwhile, I didn't need to take any action to implement the workaround to the security issue, I already had restricted access for untrusted users:

Prevent anonymous or untrusted users from accessing the media browser through permissions configuration -- 7.x-2.8 release notes

The Drupal cron job pestered me for months about needing to upgrade and it didn't feel good to ignore a necessary security update. I finally found the time to go back and review the upgrade process again and review the potential challenges. This exercise revealed an exemplar of how Drupal support and documentation can go woefully wrong.

Upgrade Documentation

The users and developers bravely struggled through the issues, attempting to document things within the framework of the Drupal website. Meaning that in order to upgrade from 1.x to 2.x and understand all the potential issues before upgrade, I needed to review these pages and hope that I'd discovered all the relevant documentation.

1. Media
2. media 7.x-2.8
3. upgrading from 1.x to 2.x support
4. Upgrading Media 7.x-1.x to 7.x-2.x
5. Document Upgrade Path from Media 1.x to Media 2.x/3.x
6. Comparison between Media 1.x and 2.x
7. Critical database error after updating to this version
8. Media + CKEditor + Media CKEditor recipe for setup
9. File Entity (fieldable files) 

I was lucky in that I didn't have to mess around with the Views or Features modules or rework any fields (see link 4 above). This wasn't entirely luck, in that I've taken the approach with Drupal modules (and pages) to make customisations only when absolutely necessary; stemming from my abject fear of the nightmare scenario the Media upgrade represented.

How I Upgraded from 1.6 to 2.8

Well I hope this helps someone.

1. Download the module tarball and extract it into the usual modules directory, overwriting the original module files.
2. Download the File Entity module and extract it into the usual modules directory. File Entity is a new module to install because the functionality used to be a part of Media but has now been removed.
3. Delete the file_entity directory in the Media module's directory. This is a directory resident from the older 1.x Media install. See link 6 above for more info.

At this point the site was generally working, but I was not able to run the database update mechanism (update.php) because of an issue I had not seen coming. My Drupal runs on an Ubuntu 14.04.5 LTS trusty install and the Database Update mechanism would not run unless the System version was 7.33. The Drupal version I had was 7.26.

Okay, they got me. Insert slow-clap here.

Since my Drupal install appeared to be operating okay, I considered waiting it out until a newer version of the drupal7 package was available and then running the database update. However, while I was doing some other maintenance on the Drupal install, the site came back with an error similar to the critical database error described in link 7 above and it was immediately obvious that the database update needed to be run in order to restore the site.

I considered doing a restore from backup. Eventually, I elected to add another dpkg source and install a newer Drupal version. I selected a Debian source for this although on reflection, taking the Ubuntu xenial source surely would have been the smarter option.

Before you do this yourself, read the rest of this post because I do not recommend mixing Ubuntu and Debian sources.

/etc/apt/sources.list:

deb http://ftp.debian.org/debian jessie-backports main

Then I ran a package update and installed drupal7 from this source. I got a couple of questions from the package installer about the database to use (as if it was a first install), but that was okay.

# apt-get -t jessie-backports install drupal7

I commented out the jessie-backports source and put a hold on the package, because at this stage I'm not clear what I'll do next.

# dpkg --set-selections drupal7 hold

The xenial source is Ubuntu 16.04 and so that is the next upgrade path for me, except that the xenial Drupal version is currently a lower version than the jessie-backports version I now have installed. To be clear, I believe that I should have used the xenial source. Quite a mess I made here, but I can figure this one out later.

Meanwhile, because I'd hosed Drupal earlier, I couldn't access the database update page. I had to put the site into "update_free_access" mode so that I could run the update engine without actually being logged in.

/etc/drupal/7/sites/moff.tech/settings.php:

$update_free_access = TRUE;

Then I could access the update.php page and run the database update, which completed successfully but with some vague messages but the site was up and running. If you do this, remember to set the "free access" back to FALSE after you're done.

Final Notes

The Media upgrade from 1.x to 2.x is a monster and it's likely that everyone who undertakes it will strike their own unique set of issues.

The Media module has been folded into Drupal 8 and so upgrading to 8 might skip this headache.

The File Entity module has not been folded into Drupal 8 so if you need that functionality then you need to install it as a module. Under Drupal 7, you must install File Entity with the upgrade of Media.

Check your Drupal version before you run the Media upgrade. You need System version 7.33 before you can run the database update.