The ultimate Joomla! update guide
As you are here, I assume you know very well, that using the latest Joomla! version is important for security and security of those who visit the website. And you are probably aware how time-consuming and boring the proper update process is. In this guide I will show you some tricks how to speed up the whole update process.
This guide is exclusively about updating (for example from Joomla 3.4.1 to the latest version). The upgrade topic (from Joomla 2.5 to 3.x) will be raised in the other article.
All tools & tricks we use in a single guide
Managing websites is a challenge that our team has been facing for years. During this time, our web developers have tested lots of tools and learnt many tricks to ease the pain of updating. To help you with this struggle, I gathered all know-how we have acquired over the years of managing websites in this guide.
Start preparations earlier
There are several things you can do in advance, to be ready for the zero day. Additionally, update is a great time to clean up your website or to check if everything was developed along with good practices. If not, you won’t have a better time to do this than now. Prepare yourself, plan and get a checklist to do. It makes execution easier.
List custom modifications
From our experience almost every Joomla website, has some customizations in the code. It is very common for developers to modify extensions or even Joomla core to their customers’ needs . Unfortunately, those changes are often hard coded, so they will be overridden by the Joomla update. That’s why, it is beneficial to list all these customizations as you will need to paste them into an updated website code.
How to find hard coded custom modifications?
If you use a version control system (such as Git or SVN) you can observe changes by going through your commit history in the files you haven’t created yourself. If don’t like the idea of digging in your past commits, you don’t use a version control system or you manage a website you haven’t created yourself you may have problem with listing all hard coded modifications. Here’s how we do it. Our secret sauce is to use an IDE (Integrated Development Environment), such as NetBeans or phpStorm to create diff showing all hard coded changes in the code.
Creating diff with NetBeans
First of all, download again unmodified version of Joomla you currently use. For example if your websites runs Joomla 3.4.1 and you think of updating it to version 3.4.8, download Joomla 3.4.1. Now, compare your current files to the unmodified files, you have just downloaded. The same is correct when it comes to any extensions you use.
NetBeans enables you to diff files even if you are not using versioning:
- Select the two files in the Projects, Files or Favorites view.
- Invoke the context menu and choose Tools | Diff to compare the two files. Note that this submenu appears only if you select two files.
- A diff window is opened highlighting all the differences.
Creating diff with phpStorm
First step is same as above, download again unmodified version of Joomla you currently use. For example if your websites runs Joomla 3.4.1 and you think of updating it to version 3.4.8, download Joomla 3.4.1. Now, compare your current files to the unmodified files, you have just downloaded. The same is correct when it comes to any extensions you use.
- Keeping the Ctrl key pressed, click two files in the Project tool window.
- On the context menu of the selection, choose Compare Files, or press Ctrl+D. The Differences Viewer for Files opens, with the differences being color-highlighted.
Finding alterations in Joomla database core tables
Apart from modifications in files you should also mind checking alterations in Joomla database core tables structure, because those changes may be overridden as well. In some rare cases, it may even break the update process.
Creating database structure diff with phpStorm
To compare databases’ structure using phpStorm select two data sources or tables, then select ‘Compare’ and press Ctrl+D. To save, right-click the table and choose output format and then specify desired location.
Transform hard coded changes - if you can
Most of hardcoded Joomla modifications can be rewritten along with good practices. It’s just the matter of time and resources available. If you can - do it! It will save you time in future updates. More tips on that in paragraph “Add custom modification” below.
If you can't, I advise you to - at least - mark custom modifications in the code using such comments:
/* CUSTOM CORE CHANGE - START */
/* CUSTOM CORE CHANGE - STOP */
You don’t use it = you don’t need this
Usually, during web development process you try different extensions. If you didn’t get rid of all downloaded, but unused extensions earlier - now it’s time. It reduces potential security risk (probably you don’t keep them updated, so there is a chance they have security holes) and lets you to keep your website clean. Moreover, it will be beneficial to empty Joomla trash and optymize database.
It is easy to find unused plugins or templates, but the situation with modules is totally different. There is no easy way to find modules that have no instance which is turned on. For this purpose you can use this SQL query:
SELECT T2 . *
FROM ###_extensions T2
LEFT OUTER JOIN ###_modules T1 ON T2.element = T1.module
WHERE T1.module IS NULL
AND T2.type = 'module'
where ### is database prefix of this Joomla
Browse through beta version. Uncover potential problems
We browse through Joomla beta version very often, and I truly recommend you this. If you have time don’t hesitate to compare beta code version to the actual one. It’s always worth doing, because it allows you to find the differences in the code and identify potential problems you may encounter after the update.
Things to do before the update
Be sure to take a backup of your live website. If something goes wrong while updating, you will be secure. And if you have never done the backup (hope not!), this is the time.
Firstly, in case of a large website, make sure you have enough disk space available. Then, you can do the backup manually or to save time, you can use automated tools. Akeeba Backup is the most widely used backup tool.
Bad things happen to good people. That’s why, it is extremely important to verify the backup you created before the update. It is the only way to know if the website can be restored just in case. Remember to verify the exact backup files that has been stored, as the backup file may be damage during transfer from one server to another.
Compare structure and file size
The first thing you can do to ensure you have a good backup is restoring Joomla from your backup on another server or your local computer and compare size of the uncompressed website with the one on the server. Additionally, make sure you can see all the folders and files and you are able to browse them. It’s not a 100% safe method, but at least you will be able to notice some big issues.
The better idea is to compare checksums of all the files. Again you will need to restore the backup somewhere first. Then I recommend creating a script in either PHP or Bash (if you have console access on the hosting). First, you run the script on the deployment server to get the full list of the files with checksums (XML is a great output idea). Then you run it again on the server where you have restored the website. And finally you comprate both XMLs to see if they are identical. This way you know that your backup is rock solid.
Verify database backup
Apart from verifying the files, it is also important to check the database backup. At least consider opening a .sql file in a text editor to see if the tables and data are represented. To do it the proper way I recommend SQL Schema Compare or SQL Data Compare.
Should I use development environment?
If you have resources, it is always worth to update your Joomla website on development environment. To migrate website from one location to another easily you can use Akeeba Kickstart.
Preparing test scenarios
A white screen of death is noticeable immediately after something went wrong. But what about missing meta tag? You won’t see it missing right away, if you don’t have something to compare it with. Preparing a test scenario will solve this problem for you. Here’s how we do it.
Preparing screenshots for visual comparison
To compare the visible elements of the website you can do a visual comparison. In order to do that, you need to generate screenshots before the update. Here are some tools to help you with preparing screenshots:
If you plan to update on development environment, you can just compare old and updated website displaying them side by side on two screens.
Writing test scenarios with Codeception
Two most popular solutions for websites testing are Codeception and Casper.JS. In this example we will show how to write a simple test scenario in Codeception. Here you can find the information how to set up the testing environment.
Here’s a sample test scenario that checks if I can log to Joomla admin panel.
Once you are ready with your test scenarios run them and make sure that they all pass. You can find information how to run Codeception tests in section “Running test scenarios with Codeception” below.
Having your websites covered with automated tests gives you confidence. But it is never 100%, because you are not able to predict all the possible scenarios not to mention writing test scenarios for all of them.
Although writing test scenarios takes a while, they will be very helpful to check if nothing get broken after the update. You can use them during every future update so it’s investment that pays off in time.
Make a list of all third-party extensions in use (components, modules, plugins, languages and templates too) and determine whether they are compatible with the latest version of Joomla. Unfortunately, the only way is to check extensions’ compatibility on developer's’ website. Seems like a huge waste of time? My sentiments exactly. To be honest our web developers often used to take the risk (you always have a verified backup, don’t you?) and update the website even if they lacked information on plugin compatibility.
Deactivate non-core system plugins?
It’s often beneficial to deactivate all system plugins, which are not created by Joomla! Project before the update.
To do so, navigate to Extensions >> Manage, then pick ‘Manage’ in the box on the left. Use filter ‘Plugin’ for - Select Type- and ‘system’ for -Select Folder- option lists.
I recommend to note, which plugins are deactivated before you disable them all. After the update you will know which should be enabled and which not.
The time has come. Update
The moment is now
Finally, you are well prepared, and ready to proceed to updating your Joomla. There are two ways to do this:
- one site after another
- all websites at once
One site after another
You can update your site with a single click. If update is available there will be a message indicating this and a button to press. Click the button ‘Install the Update’.
All websites at once
If you manage multiple websites, it can be time-consuming to go to every website admin panel and update Joomla. There are tools, which can help you with this issue like Watchful.li or MyJoomla, for example.
After the update
Activate system plugins
Enable system plugins you disabled before. I recommend to do this one by one and every time check if any errors occurred and what is going on with your website.
Add custom modifications
If there were any custom modifications in the code you need to add them again. If possible, do this along with good practices, and don’t hard code it again. It will help you to save time on future updates. Here are some tips how to do it “the right way”.
Usually, you can override modules layouts, but only them (other extensions’ authors don’t allow you to do this) . Here’s how to override layouts.
Everything is ready now, so if you followed the instructions above and have created test scenarios or screenshots, now it’s time to use them.
Running test scenarios with Codeception
Run your test in Codeception with the run command:
$ php codecept.phar run
Here's the output you should see:
Let's get a detailed output:
$ php codecept.phar run acceptance --steps
You should see a step-by-step report on the performed actions.
In /output folder you will also find a screen and HTML code for every test scenario that have failed.
Comparing screenshots with CSS
First of all, create screenshot of the updated website the same way you have done it before. Then, I recommend using a CSS filter to find the difference between screenshots. This simple trick is very useful and easy.
This can be achieved with a single CSS filter:
-webkit-filter: invert(100%) opacity(50%);
Here you can find more information how to do this.
Rollback when updating fails
You updated your website but the site is not working properly? Sometimes, when Joomla upgrading fails, you need to rollback to the previous version, especially if you updated a live website.
Restoring website with Akeeba Backup and Akeeba Kickstart
Download current version of Akeeba Kickstart and upload kickstart.php file and the translation INI files to the server on which you want to restore your website. Upload all parts of the backup archive to this server. Then, go to the Kickstart URL on your target server and click ‘Start’ to begin your website restoring. Detailed information you can find on Akeeba Backup site.
Is it possible to make it easier?
Dreaming of a perfect solution
Tools and tricks mentioned above were very helpful, but we had been dreaming of something that will make a significant difference. We were looking for something, what could further improve time-consuming and laborious web admins’ work. Eventually, we decided to create a solution by ourselves.
Wasting time? Never again
We were fed up with restoring every backup and verifying if no file got corrupted, so we made a tool to do this for us. It wasn’t once or twice, when we haven’t noticed a bug during manual tests after the update and it was our customer to discover it. You can imagine he wasn’t happy. To avoid such unpleasant situations we designed test engine automatically pointing out potential errors to fix. We were in need of one platform to see update status of all websites we were responsible for running different CMS? We made a TODO list that refreshes automatically every day. We have saved countless hours thanks to this solution. Recently, we decided to share it and make it available for everyone. If you want, you can try it here.
Together we can make the Internet a safer place
I really hope, you will find my post useful and helpful while updating your Joomla to the newest version. You know it is great you’re doing this? If you’re not aware you should know and remember: thanks to people like you the Internet is safer place for all of us.
In the end of this post, I would like to thank Perfect Web's developers team. Especially, I thank Aleksander Kuczek, Jan Krzywanek and Piotr Moćko for help and all the advices.