Transitions and GeoIP in Piwik 1.9

I am using Piwik for over three years now and have seen it grown from version 0.4 to todays release of Piwik 1.9.

Update to 1.9

Piwik will tell you to upgrade to the latest version on the top right corner. Once you click on the link to upgrade you will be guided through a very easy upgrade process.

On this page, select to "Update Automatically". After the process has finished you will see the confirmation that everything worked perfectly.

Piwik 1.9 comes with a major database upgrade which, depending on your database size, might take a long time to execute. No matter what you do, make sure you have an backup of your database just in case anything goes wrong.

Transitions

At first sight, not much has changed in the Piwik UI. Go to Actions -> Page Titles and hover one of the entries. You will see two icons, the right one being the evolution of this entry and the left icon will show the transition interface.

The transition interface will show you how your users got to that page and how the proceeded to the next page.

You can read more about Piwik Transitions and how to work with them in the Piwik docs.

GeoIP

Piwik always offered a way to detect a users country based on the language they were using.

The default location provider guesses a visitor's country based on the language they use. This is not very accurate, so we recommend installing and using GeoIP.

As this is not a reliable and satisfactory way to determine where your users are from, Piwik now comes bundled with the GeoIP plugin, which was separately available for the last five years.

When properly installed, Piwik 1.9 will report the city of the visitors in the Live! Widget when you hover the country flag.

There also is a new widget available that you can add to the dashboard to view the cities you website is most popular in.

(City Widget)

Install GeoIP PHP

Installing the GeoIP database is easy. Download the database from Maxmind to your misc/ directory and unpack it.

cd /var/www/piwik/misc
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
gunzip GeoLiteCity.dat.gz

In the Piwik UI go to Settings -> Geolocation and select "GeoIP (Php)" as geolocation provider. As the documentation states, this provider is great if you don't have many visits, but will become a performance bottleneck if you have a lot of page visits. The problem is, that the PHP provider has to load the database for each request.

Install GeoIP PECL with PHP-FPM

A faster solution is a PECL (PHP Extension Community Library) extension. Combined with PHP-FPM this solution is very fast and therefore the recommended solution by Piwik. To install the extension (I am assuming a self-compiled PHP on a *NIX here), download the sources from the PECL website:

cd /opt
wget -O - http://pecl.php.net/get/geoip > geoip-latest.tgz
tar -xvf geoip-latest.tgz
cd geoip-1.0.8
phpize .
./configure
make && make install

This will install the geoip extension on your system. Now you will need to tell PHP to load it. Find your php.ini and add the following line to the [PHP] section:

extension=geoip.so

Additionally you need to tell the extension where to look for the GeoIP database(s). Find the [gd] section and prepend the following snippet:

[geoip]
geoip.custom_directory = /usr/share/GeoIP/

Adjust the directory to where you store your GeoIP database.

Update all previous visits with GeoIP information

Visits that have not been recorded with Piwik 1.9 do not have have any geolocation data. The developers were aware of the situation and provide a simple script that re-evaluates all previously tracked visits and fills in the missing geolocation data (see link.

# cd /var/www/piwik/misc/others
# php geoipUpdateRows.php 
54893 rows to process in piwik_log_visit and piwik_log_conversion...
0% done...
2% done...
4% done...
[..]
done!

Note: Providers reallocate IP address ranges very often, so the further back you go with your data (e.g. visits from over 2 years ago), the more errors you will get.

Automatically update GeoIP database

To avoid wrong geolocation information, you should update your GeoIP database regularly. Create a bash script in the misc/ directory of your piwik installation.

DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
cd $DIR
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
rm GeoLiteCity.dat
gunzip GeoLiteCity.dat.gz

The Maxmind database is updated on the first tuesday of every month, so lets schedule our cronjob to the first wednesday of every month. The crontab file has the following format:

• 1. Minute (0-59)
• 2. Hour (0-23)
• 3. Day of the Month (1-31)
• 4. Month of the Year (1-12)
• 5. Day of the Week (0-6) 0 being Sunday.
• 6. the executable string

Now sadly the "day of week" does not work as one would expect, as cron will execute a line if "day of the month" or "day of the week" matches, making it impossible to configure it to only run a command on the first Wednesday. With a little help of bash we can make it work: [ "$(date '+%u')" == 3]. This effectively gives us the following entry for the cronjob (type crontab -e to edit your cronjobs):

0    0    1-7    *    *    [ "$(date '+%u')" == 3] && /path/to/update_geodb.sh

Conclusions

Piwik 1.9 comes with several new features. In this article we have seen the Transitions and GeoIP plugin that will help you gain a better understanding on how visitors use your page and where they are from.

SiteSearch is a plugin not covered in this article. It enables you to track how visitors use the search functionality on your page and which results they pick.

All in all, Piwik has grown to a very mature and feature-rich analytics solution with a lot of great features still to come in Piwik 2.0.