Agile Marketing

Hallmark 2: Experiments Not Missions

Why Is Experimentation Important?

Google is one of the biggest practitioners of “beta” experiences—it introduced its Gmail and many other ideas just that way. Google goes even further, by testing tiny changes with small groups. If you ever notice that your Google search screen looks different from one shown the person next to you, it’s because Google is testing a new feature on a random subset of searchers.

You, too, can tweak your experience a little at a time, rather than gutting it and replacing it every few years. What happens when you don’t keep tweaking? One marketing director lamented:

“So we’re in the situation now where we built our website [five years ago] and it was a great website at that point. Well, nobody invested in keeping that foundation solid and current—and online the landscape changes so quickly. Now we need to spend a ton of money simply to rebuild the foundation. That’s a big lesson we are trying to impart to people. We are not going to do a web design project and walk away from it. We need to turn this project into a program.”

Learn from this. Every time you make a change to your website, you need to measure its results and make it better. When most people think the project’s over, you need to know it’s just starting.

If you make small changes, it’s far easier to measure their effect. First, you have less data when your change is confined to a small area on your site. Second, if you change only one thing, then any effects can be attributed to that particular change—making multiple changes muddies the waters.

The trick is to make a small change with a large impact. Start with a single page, but make sure it is an important page—one that directly leads to conversions. Make it a page with lots of traffic. Later in this chapter, we’ll show you the simplest ways to set up an experiment.

Learn how to make gradual changes, NEXT.