Hostelworld offers an online booking service to thousands of hostels worldwide. Hostelworld has a vision to be the fastest-growing online provider of great value accommodation, using innovative technology to inspire independently minded travellers everywhere.
Since March 2010, eCircle has been working with Hostelworld to develop an effective email marketing strategy which focuses on customer loyalty and to increase conversions and revenue through optimised creative and automated programmes. eCircle’s partnership with 8Seconds offers multivariate testing to improve the response rate and ROI of email campaigns. Through a fully integrated solution, eCircle’s clients are now able to go further than the traditional A:B split testing, and dynamically analyse the most successful emails in real time.
Multivariate testing is a process by which more than one component of an email may be tested in a live environment. Through a fully integrated solution, multivariate testing is more than traditional A:B split testing. A:B tests are usually performed to determine the better of two content variations, whereas multivariate testing can test the effectiveness of limitless combinations. The only limit on the number of combinations and the number of variables in a multivariate test are the amount of time it will take to get a statistically valid sample.
This case study looks at how Hostelworld optimised the main header image of their June 2010 newsletter by using multivariate testing to improve response rates and ROI. Different images were shown to the recipients who opened the email (see four different header images tested). The effectiveness of the combination of images based on open rates and clicks was measured in real time once the statistically significant or pre-determined measure was reached. The optimised email was then shown automatically to the remaining recipients.
Using real time optimisation Hostelworld was able to optimise their email campaigns. Four separate image headers were tested including “Free city guides?, “Get ready to rock!?, “Dublin Hostels? and “Three to watch this month?. In addition, Hostelworld performed Segment Optimisation, which meant not only were they testing four separate images, but that the test ran over four different segments: namely four different currencies (USD, GBP, EUR, AUD).
When the email campaign series had been sent out, Hostelworld were able to see how each header image had performed and measure which image had worked the best. An average of 4,000 readers per segment was needed to determine the winning image with a 100% statistical validity. All the remaining recipients subsequently received only the best converting header image.
Following the testing phase the “Free City Guide? proved to be the best performing image. It converted on average 62% more successfully than the “Get ready to rock!? image, which was worst performing. Unsurprisingly, the “Dublin Hostels? image worked well in Europe and the UK, but the Americans and Australians did not respond as well to this header. However, the Americans and Australians did respond well to Miami, Shanghai and Buenos Aires “Three to watch this month?.
The four different images were shown 1,435 times each. 5,739 unique recipients were needed and 142 unique clicks to determine a winning header with 100% validity. The remaining 53,486 recipients (or 38,084 unique recipients) were only sent the email with the best performing image header resulting in an impressive 1,214 clicks or a click through rate of 3.19%.
When only the best performing image is shown to the remaining recipients of each segment, the average click through rate was 3.54% which is 48% more than during the testing phase!
Running a campaign that enabled recipients to decide what they like best – all in real time was the success of this campaign. The result was a boost in open rates and campaign ROI for Hostelworld.
Debs Kennedy, group strategic marketing manager, Hostelworld said, “The multivariate testing project has been a very productive exercise. Any future newsletters that we send now will be highly targeted with content that we know our subscribers are interested in receiving.”
Check out 12ahead, our brand new platform
covering the latest in cutting-edge digital marketing and creative technology from around the globe.
12ahead identifies emerging trends and helps
you to understand how they can apply to modern-day companies.
We believe 12ahead can put you and your
business 12 months ahead of the competition. Sign up for a free trial today.