After surviving the webmaster deathmatch at pubcon Las Vegas, we were able to secure beta access to Google’s sweet new multivaritate testing system, Google website optimizer. I’ve been using it for a few weeks now and I’m liking it so far. For a free product, its awesome, but it does seem very beta still. Here’s a couple of points of contention I have with it:

Once you start an experiment, there’s no way to edit variations or any of the other experiment settings, and if you stop an experiment you can’t restart it, you have to create a new one from scratch.

In creating an experiment the only way to specify page sections to test is by putting section code in the test page and using the javascript to name it. For instance if you want to test 3 sections of the page you have to put three occurances of the code on the page and name the sections in the code. I wish there was an interface in the Website Optimizer system where I could create and name the page sections to test and add the code seperately, but this isn’t a huge deal.

When you’re creating variations for a page section to test, you have to hit save before you can preview the page, again not a huge problem, but a little confusing.

If you’ve created an experiment and not finished building it (like if you messed up) there’s no way to delete it from your list of experiments.

The navigation of the Website Optimizer seems inconsistant, sometimes I can’t find my way back to a screen I was previously on.

Sometimes I log into other google accounts (like my personal one, rather than our adwords account) and when I log back into adwords, I can access everything but the optimizer. If I click on it’s tab I’m sent momentarily back to the log in screen then back to the adwords account homepage by a redirection. I’m forced to restart my browser (this is happening on firefox).

The only success metric it seems I can measure is a direct conversion, sometimes I’d like to just check out time on site, or pageviews. Perhaps this is doable via integration with Google Analytics, but I’m not sure.

But again, these are mostly little convienance issues, and overall I love the app, especially at the price (free).

If you liked this post, don't forget to subscribe to my RSS feed or my email newsletter so you never miss the science.