Drupal 8 RC2 is out and the official release of Drupal 8 is not too far away. As a hosting company that is officially supporting Drupal and offering Drupal optimised hosting, we thought it would be a must to get ready for the Drupal 8 launch in advance. Having enough time to do good fine tuning + independent performance testing of our Drupal 8 VPS offering, along with getting some numbers to estimate the performance and to share it with the world.
Preface
The Internet is fulfilled with a lot of posts about Drupal 8 performance benchmarks against Drupal 7, and they don’t sound optimistic. Apparently, a few basic “ab” testing reveal Drupal 8 to be up to 3x times slower than Drupal 7, for example, this article outlines it:
- http://alexrayu.com/blog/drupal-739-vs-drupal-8-rc1-performance
This result made us do our own investigation, and a few other articles (published on whimleers.com and buytaert.net) revealed cases where Drupal 8 could be much faster compared to Drupal 7.
- Building really fast websites with Drupal 8
- Drupal 8 now has page cache enabled by default
- Making Drupal 8 fly
In our own experience of preparing, fine tuning and performance testing of customers’ Drupal websites, we know that the ab testing is not the best tool to assess the real performance of a production website, whether it is Drupal or some other CMS. The ab testing is a good tool to measure raw, unoptimised resource to get an idea about its performance, outlined in numbers.
Fast VPS hosting does not always mean a fast CMS or underlaying hardware. Modern software and the know how, this is what makes Drupal 8 fly.
The three posts above outline the use of Varnish cache with Drupal cache tags, allowing to cache most of the Drupal page contents even for logged in users. This (once again) stresses the importance of correct software selection, configuration and fine tuning in order to achieve the real high performance for production websites. In our own experience, the common perception of the “fast hosting idea” as the hosting using an expensive, latest & fastest hardware with latest CPU, GPU and SSD is not 100% correct, and is only good for marketing of such hosting for geeks or people with lack or server optimisation skills, seeking to get more juice for the buck.
The real performance however, comes with the software and it’s configuration for the specifics of the application.
The real performance comes with the caching implemented in the server’s memory, with minimal disk IOPS (ZFS on Unix) and efficient compression algorithms (MariaDB 10.1, ZFS). Such configuration outperforms the fastest hardware alone, though it is still better to seek for good performing hardware for website hosting. Ideally, you would be looking for the fast hardware, combined with the know-how to get the best performance out of it. It all comes to your budget in the end.
In the past years, we deployed a lot of customers’ Drupal websites running behind the Varnish reverse proxy cache. We loved Varnish cache very, very much, it was the key to the well-performing websites. At some stage, though, we faced performance issues with busy websites running via SSL. Sadly, Varnish cannot (and will not in the observable future) handle SSL pages.
At this point we explored alternatives and our preferences went with Nginx, which proved to be very close in performance numbers to Varnish cache. However, Nginx support for SSL and more importantly SPDY protocol (recently, with the release of Nginx 1.9.5, HTTPS2 module was released) was hard to beat. In fact, Nginx could be used to completely replace Apache + Varnish synergy with similar performance and additional benefits.
Where the real bottleneck is
In modern data centres, where hardware is very performance capable, the real page delivery delay is introduced by the network/internet connection between the server and the client. The key importance of SPDY (or HTTP2) cannot be underestimated. Modern CMSs deliver pages with 50 – 200 (and sometimes even more) connections on an average. The connection latency becomes the real bottleneck that Varnish cache cannot resolve. The HTTP2 comes very handy, allowing to deliver all of the page components (such as images, JS and CSS files) in a SINGLE multiplexed connection, allowing to avoid browsers going back and forward to the server in order to load all of those images and other static files. HTTP2 saves heaps of time and bandwidth in page delivery to the end user. Testing websites via www.webpagetest.org reveals (check website loading waterfall) that even slow servers process the HTML page (PHP page rendering) in just tens (or hundreds) milliseconds. If you get hardware capable of reducing this 2x times, well, you saved few milliseconds. However, loading of other resources is what really takes the time to load. This time could be measured in thousands of milliseconds and this is where most optimisation helps.
Consider this: with SPDY (or HTTP2) in place, use of CDN only makes things worse. The client waiting times for full page load increases with the use of CDN, but decreases with the use of SPDY.
This is why we actually prefer to use Nginx vs Varnish cache.
How we configured our server
As part of this exercise we used the following tools and components to benchmark Drupal 8 website:
Hardware
- XenServer 6.5 virtualisation platform
- Cloud VPS with 8GB RAM, 8 CPUs
- 10GB storage link to ZFS on UNIX Storage
Software
- CentOS 7
- MariaDB 10.1 with InnoDB and large innodb_memory_pool_size
- Apache 2.4 w/mod_pagespeed running on port 82 behind SSL proxy
- Nginx 1.9.5 w/HTTP2 module running as SSL terminator and reverse proxy cache for static files only.
- HHVM 3.9
How we added content to our test Drupal 8 website
We used Drush devel-generate command to create 50 nodes of Page and Article content with 10 comments each.
What tools we used to perform benchmarking
We used http://loader.io and ‘ab’ tests to assess if the Drupal 8 performs to our performance standards. Our performance acceptance criteria for a Cloud VPS is to handle at least 100 users per second during 1 minute for every 1GB of RAM and 1CPU of the VPS with no timeouts. We set the timeout to be 10 seconds and loader.io simulates full page loads by a unique user (unlike the ‘ab’ test, which only hits the server, waits for the response and moves ahead).
Tests were taken for SSL pages.
What does 500 (or 1200) clients per second means
500 (or N clients) per second during our test means that the test server (www.loader.io) is simulating N number of unique client connections to the server and every client is loading full page requested (home page in our test). N clients per second is being sent by the test system without waiting for the response, in parallel. Ever page load is requested by a simulated unique client, with no cookies set.
Test Methodology
During the test, we began with little customisations of the original CentOS 7 template that we had for Managed VPS customers. We were increasing the number of visitors per second slowly, step by step until we exhausted server resources and began receiving timeouts in the responses. This was the indication of the server limit and this is where we began fine tuning the server, researching for options and comparing loader.io results. A good indicator was the outcome of the loader.io indicating the absence of response timeouts, at which point we were increasing the number of visitors further until response timeouts reappeared again.
Test Results
ab testing is included for us to compare Drupal 8 RC2 with Drupal 7, and not to indicate real world performance.
These ab tests run from the VPS itself, hitting the Drupal front page.
ab test to non-SSL port
ab -n1000 -c1 http://drupal8.redyhost.com/ 55.26 requests per second
ab -n1000 -c1 http://drupal7.redyhost.com/ 67.13 requests per second
Same ab test to SSL port
ab -n1000 -c1 https://drupal8.redyhost.com/ 36.30 requests per second
ab -n1000 -c1 https://drupal7.redyhost.com/ 41.49 requests per second
ab test to non SSL port (concurrency 20)
ab -c20 -n 1000 http://drupal8.redyhost.com/ 323.60 requests per second
ab -c20 -n 1000 http://drupal7.redyhost.com/ 408.17 requests per second
ab test to SSL port (concurrency 20)
ab -c20 -n 1000 https://drupal8.redyhost.com/ 190.51 requests per second
ab -c20 -n 1000 https://drupal8.redyhost.com/ 202.75 requests per second
Drupal 8: begin with 500 clients per second (SSL Port)
This was our initial test with 500 clients per second. As you can see, this test has no timeouts, network errors or 400/500 responses. Going with 600 clients in this configuration caused the test to be interrupted due to hitting our 50% error threshold set on the test.
Drupal 8: finish with 1200 Clients per second (SSL Port)
This is the final test after we did additional performance tuning. You can see 17 of clients received 500 response code, which was Nginx throwing an error about the inability to get a timely response from Apache. Going beyond 12000 clients per second began increasing the number of 500 errors and brought timeouts count to 100 or more.
Drupal 7: 1200 clients per second (SSL Port)
We setup Drupal 7 in the same VPS as an additional vhost. We used drush devel-generate to create 50 nodes of Page and Article content types, with 10 comments each and URL alias.
As expected, Drupal 7 responded excellently and we see 500ms less average response time compared to Drupal 8. Additionally, Drupal 7 returned 47% more successful responses during the 1-minute test.
Conclusion
As we explained at the top of this article, the real performance comes with the right software and the know how. After we fine tuned our base template, you could see that the server was able to successfully serve 2.4x times more clients for Drupal 8. Is your 8GB, 8CPU virtual server capable of serving 1200 clients per second? Share your numbers from loader.io in comments below.
When comparing the results for home page loading in Drupal 7, it worth noting that Drupal 7 delivered 19,989 responses vs 13,542 for Drupal 8, which is about 47% times more. However you can see that Drupal 8 served more content – 1.09GB vs 0.826GB in Drupal 7 (both sites had the same number of articles published, with images), which is 30% more. We do not have the precise formula to combine these metrics and come up with a single number for Drupal 8 vs Drupal 7 performance, I feel the real performance of Drupal 7.41 is around 15%-25% better comparing to Drupal 8 RC2.
We hope that Drupal 8 will perform better in near future, as it approaches first release, with either PHP 7 or future versions of HHVM.
Interested in getting high-performance Drupal 8 hosting on our platform?
ORDER PRECONFIGURED HIGH PERFORMANCE VPS NOW!
OR
CONTACT US TO DISCUSS YOUR MANAGED SECURE HOSTING REQUIREMENTS