I finally took the plunge and tried out optimization. I am (literally) surrounded by optimization at ZAAZ, since the optimization team sits near me and I’m hearing about it all the time. I really like the approach that ZAAZ takes, (see here for the ZAAZ process) with viewing optimization as part of a holistic process for website design/maintenance. Once you analyze the data, you know what parts to improve upon, and optimization prevents blindly implementing things based on what the developer/designer/manager likes best.
All I knew about optimization was that its the process of designing different versions of a page and then testing them to see which performs better. A/B testing is testing 2 or more different versions of a page (according to Google), while multi-variate testing is testing various sections within a page. Since I’m a noob, I went with A/B testing in Google Website Optimizer because it is simpler to set up the experiment.
I will preface this with the fact that yes, its a fake experiment that I didn’t put much thought into, and no I don’t actually care about the conversions. Here are the basic steps to setting it up with some notes on my process:
- Decide what you want to test and how you want to test it. This is the hardest part I think, though if it were a more important site I would preferably base this on analytics data for underperforming pages, campaigns, or the key conversion funnels. I didn’t care, so I made 2 versions of my usability portfolio page and a super-obvious conversion CTA. I opted to have 50% of visitors see the original and 50% see the test page.
- Enter in the URLs for the original page, test page, and conversion page. Easy enough except I wanted to set up 2 different conversion pages, and that did not appear to be allowed.
- Place the tracking code in each page. Fortunately this plays nicely with Google Analytics, but you have to be careful about the placement of the tracking code–just after the head tag and just before the closing body tag, and after the Google Analytics code. Then click to validate the tags.
- Begin the experiment…and wait for the results! It says the results come in within 24 hours, and I’m not sure how accurate that is because as of now I have more conversions than I have visitors. Not sure how that is possible.
Overall fairly painless process, though I’m skeptical about the accuracy of the data I’m getting so far. I think next time I will try out the multi-variate testing though for something more interesting. My colleague Rachel Elkington wrote a great article on combining UX principles with optimization that I also want to try out.
And I seem to not have enough hours in the day to do everything I want, which means the blog posts have been fewer and I haven’t gone running as much. My bad.
All posts
sarahd23 on Twitter
Congrats on first foray. One thing to be aware of in GWO a/b testing format is that it doesn’t split the traffic every-other-visitor to achieve an a/b split. It’s actually more random than that, so it may show that one variation is getting way more exposure at first. That will even out in the end, though.
Another “worth mention” is that you can usually do an a/b split test using GWO’s multivariate format. Don’t think that using their MV format means you have to be running many different variations all over the place. You can pretty easily set up a “univariate” test that’s the same as an A/B split.
Hi Brendan!
Those are good things to keep in mind for future. Multi-variate just sounded complicated to me. I’m still not sure how I had more conversions than visitors, but who knows why that would happen. I’ll have to try out a multi-variate test soon, but I’d like to try it out on a site where it could make a difference vs. pages that I select at random.
That is odd. From a technical standpoint, it means that the conversion script got loaded/fired more times than the pageview script due to tagging issues, OR some sort of data lag (not unheard of with a free service).