Experimentation has played a vital role in technology, and eCommerce is no different. Business operators are discovering how seemingly trivial changes to website elements such as button color and call-to-action copy can greatly impact conversion rates. So, as a business operator, how important is testing and how can you improve the dependency upon test results.
One strategy for evaluating the best converting combination of marketing elements is A/B testing. While A/B testing is not a new concept by any means, we argue that many people advocate for a flawed method of A/B testing—one that does not include a control group.
What is a Control Group?
A Control group is a segment of an experiment’s participants that are intentionally sheltered from viewing any of the test’s variables. The control group goes about using the product in its original state during the test. A classic example of a control group is the segment of a pharmaceutical drug study that receives doses of the placebo.
Often times, people recommend split testing two variations without mention of a control group, which is why CleverTap has coined the term A/B/C testing.
A/B/C testing, which is probably becoming clear, is the inclusion of a control group within a traditional split test.
Why a Control Group is Important
Many times, there are external variables that can have influence over the results of tests, with little indication of their existence. A control group helps validate the results and ensure they are statistically significant.
For example, a hypothetical split test was run without a control group and discovered that variation A performed 20% better than variation B, it could be difficult to see through these results. When a control group was added, however, the results revealed that the control group outperformed both variations A and B.
Look to the visual below from CleverTap for more information about A/B/C testing and why control groups are important in your testing strategy.