Omniture Summit | March 5 | A/B, Multivariate Testing wtih Omniture Test & Target

The second breakout session I attended was presented by Lily Chiu, Consultant, Omniture and Josh Friedman from Dell presented a case study.

Lily began by identifying one of the major problems for online marketers. This problem is that marketers have a whole bunch of different ideas that they would like to try, but the deployment of these ideas are controlled by IT. This leads to idea's not being deployed quickly enough. The solution to this problem is to use a product like Omniture Test & Target to quickly deploy and test these marketing ideas.

Test & Targeting allows you to A/B and Multivariate Testing to test multiple site layouts at the same time. This testing can help answer common problems such as:

"I know my cart page results in the most drop off but I don't know what will improve it..." and "I know my high-frequency visitors should have the best conversion rate but..."

A/B testing is where different sites are compared against each other to see how well they convert. Multivariate testing takes A/B testing to the next level by testing multiple portions of a site at the same time. With Omniture Test & Target, you can have different customers directed to different site layouts and then after a given period of time you can view the KPIs for the different layouts and see which is performing the best.

When first starting with this type of testing, Lily had 3 suggestions:

1. Start out with something simple. It should be something that is both technically easy to implement and politically easy to get approved.
2. Know your goal. You should know what you are trying to improve. For example, are you looking to increase conversion rate? or do you want to increase the time spent on the page?
3. Define your question. Make sure you know exactly what it is that you are trying to answer by doing the testing.

Lily then explained that the two most important aspects of a testing platform are:

1. Speed - The ability to test and learn from those tests quickly
2. Control - Being able to make the changes without needing to go through IT

A good testing platform, such as Omniture's Test & Target allows for both speed and control.

Next Lily presented some best practices for multivariate testing:

1. Consider your traffic
2. Define your goals and success metrics
3. Think different when designing alternatives (if all of your alternatives are too similar, then there probably won't be statistically significant results from your testing)

The presentation then continued with Josh Friedman giving a case study of Dell and how they used multivariate testing to optimize the layout for their product pages. It was interesting to see that while the pages looked somewhat similar, statistically some page layouts performed better than others. Dell was able to use this data to make the best possible layout for their pages.

In conclusion, I enjoyed this presentation as it was an excellent introduction into A/B testing and Multivariate testing. Any site that is serious about optimization would benefit greatly from Omniture's Test & Target software.