Monday, June 26, 2017

Switching Your Drupal to HTTPS

This post describes the steps I undertook to take to switch my Drupal site over to HTTPS. I'd made several attempts at this after getting a certificate via Let's Encrypt but ran into problems with mixed content, meaning that Firefox would not render the page properly while there was a mixture of encrypted and non-encrypted objects being loaded from the website.

I finally found the time to sit down and work it out. There is a nice explanation on the Drupal website and to a large extent I followed the 'best possible' solution there. My post adds some extra points you should know.

Let's Encrypt

I've covered the topic of Let's Encrypt in some detail in other places on this blog, so there's no need for me to cover this off. Get your certificate from Let's Encrypt and make sure that your webserver (in my case apache2) is using the certificate and responds to the https:// form of your Drupal website.

Go Full Encryption

As I mentioned above, the explanation on the Drupal website is a good place to start. I fiddled around with the Secure Login module and also with using $conf['https'] = TRUE; in the settings.php file.  In the end the "best possible security" option was the simplest and strongest solution.

Redirect Everything To HTTPS

Take the information from the Drupal website and implement the VirtualHost configuration to redirect all HTTP to HTTPS. Invoking the "Redirect" example on the VirtualHost, rather than the "Rewrite" example is easier and more elegant.

Mixed Content Problems

It's at this point that you may encounter mixed content problems, meaning that the Drupal site will not render correctly and have a yellow icon where the green padlock should be in the URL bar.

There are two things to do. The first thing is to check for objects that are actually loading over HTTP. If you're using Firefox, press F12 to load firebug and you can audit/search the source of the page for "http" objects. Do not confuse an http link to an external site (such as in an <a href..> tag) with content that is actually loading.

Mixed content means that your page has actually caused the browser to download an object from over HTTP instead of HTTPS. In my case I discovered a logo image present on every page had a hard-coded http:// address. Once I fixed that, I still had mixed content errors.

The next thing to change is the $base_url of your Drupal site, to indicate a default to https on all urls. This should have the effect of changing all the relative links that Drupal generates on the fly when rendering pages.

/etc/drupal/7/sites/moff.tech/settings.php:
$base_url = 'https://moff.tech';
In my case, these two steps got me the green padlock. It's possible that your install could have hard-coded HTTP object sources inside pages. All of that would need to be tidied up and depending on the size of your site a tedious exercise.

Good luck and please do share here any tips (war stories) from your experience!

Sunday, June 18, 2017

Drupal 7 and upgrading Media module to 2.x

For a few years now I've been hosting and maintaining a Drupal 7 installation for a non-profit club. I chose Drupal because I needed to provide a website that a non-technical person could easily publish content on.

I've written this post in the hope that it might just help someone and also to vent some frustration over the house-of-cards that is Drupal.

Why Drupal?

I did not want to run Wordpress because at the time, Wordpress had a sorry security and upgrade reputation. I'll note now that Wordpress has slowly rehabilitated that reputation, but still not to the point that I'd be willing to look after a Wordpress site myself.

Drupal is no walk in the park. In fact, I've been constantly annoyed by just how difficult maintaining a Drupal installation is. Layers upon layers of modules need to be installed before you have a dynamic and user-friendly website.

Drupal Drawbacks

Upgrading modules is generally easy, but still a chore in particular because I do not personally like the automated upgrade methods. For anyone maintaining many Drupal installations or a popular site where quick adoption of security patches is mandatory I am sure automated updates are a lifesaver.

I'll skip mentioning the scores of transient errors (Drupal love your system memory long time) and tedious frustrations (I cannot for the life of me work out how to get clean URLs working) that one encounters with Drupal in general, because I want to cover off some notes about the Media module and the version change to 2.x.

There was a major security problem discovered in the Media module version 1.x, which required an upgrade to 2.x. The upgrade instructions were so convoluted and the user problems post-upgrade were so frightful that I elected to wait it out and let other people suffer the bugs and pain. Meanwhile, I didn't need to take any action to implement the workaround to the security issue, I already had restricted access for untrusted users:
Prevent anonymous or untrusted users from accessing the media browser through permissions configuration -- 7.x-2.8 release notes
The Drupal cron job pestered me for months about needing to upgrade and it didn't feel good to ignore a necessary security update. I finally found the time to go back and review the upgrade process again and review the potential challenges. This exercise revealed an exemplar of how Drupal support and documentation can go woefully wrong.

Upgrade Documentation

The users and developers bravely struggled through the issues, attempting to document things within the framework of the Drupal website. Meaning that in order to upgrade from 1.x to 2.x and understand all the potential issues before upgrade, I needed to review these pages and hope that I'd discovered all the relevant documentation.
  1. Media
  2. media 7.x-2.8
  3. upgrading from 1.x to 2.x support
  4. Upgrading Media 7.x-1.x to 7.x-2.x
  5. Document Upgrade Path from Media 1.x to Media 2.x/3.x
  6. Comparison between Media 1.x and 2.x
  7. Critical database error after updating to this version
  8. Media + CKEditor + Media CKEditor recipe for setup
  9. File Entity (fieldable files) 
I was lucky in that I didn't have to mess around with the Views or Features modules or rework any fields (see link 4 above). This wasn't entirely luck, in that I've taken the approach with Drupal modules (and pages) to make customisations only when absolutely necessary; stemming from my abject fear of the nightmare scenario the Media upgrade represented.

How I Upgraded from 1.6 to 2.8

Well I hope this helps someone.
  1. Download the module tarball and extract it into the usual modules directory, overwriting the original module files.
  2. Download the File Entity module and extract it into the usual modules directory. File Entity is a new module to install because the functionality used to be a part of Media but has now been removed.
  3. Delete the file_entity directory in the Media module's directory. This is a directory resident from the older 1.x Media install. See link 6 above for more info.
At this point the site was generally working, but I was not able to run the database update mechanism (update.php) because of an issue I had not seen coming. My Drupal runs on an Ubuntu 14.04.5 LTS trusty install and the Database Update mechanism would not run unless the System version was 7.33. The Drupal version I had was 7.26.

Okay, they got me. Insert slow-clap here.

Since my Drupal install appeared to be operating okay, I considered waiting it out until a newer version of the drupal7 package was available and then running the database update. However, while I was doing some other maintenance on the Drupal install, the site came back with an error similar to the critical database error described in link 7 above and it was immediately obvious that the database update needed to be run in order to restore the site.

I considered doing a restore from backup. Eventually, I elected to add another dpkg source and install a newer Drupal version. I selected a Debian source for this although on reflection, taking the Ubuntu xenial source surely would have been the smarter option.

Before you do this yourself, read the rest of this post because I do not recommend mixing Ubuntu and Debian sources.

/etc/apt/sources.list:
deb http://ftp.debian.org/debian jessie-backports main
Then I ran a package update and installed drupal7 from this source. I got a couple of questions from the package installer about the database to use (as if it was a first install), but that was okay.
# apt-get -t jessie-backports install drupal7
I commented out the jessie-backports source and put a hold on the package, because at this stage I'm not clear what I'll do next.
# dpkg --set-selections drupal7 hold
The xenial source is Ubuntu 16.04 and so that is the next upgrade path for me, except that the xenial Drupal version is currently a lower version than the jessie-backports version I now have installed. To be clear, I believe that I should have used the xenial source. Quite a mess I made here, but I can figure this one out later.

Meanwhile, because I'd hosed Drupal earlier, I couldn't access the database update page. I had to put the site into "update_free_access" mode so that I could run the update engine without actually being logged in.

/etc/drupal/7/sites/moff.tech/settings.php:
$update_free_access = TRUE;
Then I could access the update.php page and run the database update, which completed successfully but with some vague messages but the site was up and running. If you do this, remember to set the "free access" back to FALSE after you're done.

Final Notes

The Media upgrade from 1.x to 2.x is a monster and it's likely that everyone who undertakes it will strike their own unique set of issues.

The Media module has been folded into Drupal 8 and so upgrading to 8 might skip this headache.

The File Entity module has not been folded into Drupal 8 so if you need that functionality then you need to install it as a module. Under Drupal 7, you must install File Entity with the upgrade of Media.

Check your Drupal version before you run the Media upgrade. You need System version 7.33 before you can run the database update.