Why are the Test Sample Sizes needed for Multivariate Testing
so very low compared to A-B Split Testing?
The test sample size required for 95% confidence level in
A-B Split Tests is many, many times more than sample sizes
required for MultiVariate Testing…. thousands per test
cell instead of hundreds. This is because the analysis of
Phase I test results in MultiVariate Testing is not done by
comparing the click throughs and sales conversions of the
individual Phase I test versions (which A-B Split-Testing
does)
Instead, MultiVariate Testing simultaneously compares the
conversion rate of each Variation of all the Factors against
each other. Because each Variation is present multiple times
in the Phase I test versions, each Variation is being tested
multiple times during the Phase I test itself. This is why
the sample sizes for MultiVariate testing are dramatically
lower, a few hundreds instead of many thousands, than the
sample sizes required for A-B Split tests.
While A-B Split tests can never test interactions, traditional
Regression techniques do have the ability to test interactions.
But, testing interactions with the regression technique requires
a huge sample size, and therefore, a lot more time and expense.
A unique power of The Conversion Multiplier lies in the fact
that even with only 500 E-mail recipients per test cell, it
can accurately test the sales gain potential from interactions
between the Selling Factors tested.
So, how does this Advanced Multivariate Testing Technology
work?
Let's say there are 15 Selling Factors you want to test and
each of these Factors has 2 Variations you judge to be worth
testing. If you tried to test all 15 Factors and Variations
at the same time with traditional A-B Split Tests, you would
have to manually create 2 to the 15th power which would require...
32,768 different Versions of your E-mail and/or landing pages.
Needless to say, it's simply not realistic for anyone to
create and test 32,768 versions of their campaign. Not only
would trying to create so many versions of a campaign require
a phenomenal amount of money but, because you could not test
them all at the same time, it would also take 24 years to
finish tests for all the versions you want to test.
That's why no one ever does this and why The Conversion Multiplier
comes in...
... Because our advanced MultiVariate Testing technique
uses unique, highly sophisticated mathematical and statistical
calculations that enable you to produce record-setting Sales
Conversion Rates by testing only 10 to 20 Versions of your
E-mail campaign in just 2 short Test Phases that are conducted
in only a few weeks.
Let’s say you do decide to check out the potential
this new approach has for you...
Step I: We’d start your Phase I Tests by creating different
specific versions of your E-mails (and landing page if you’re
not selling directly from your E-Mail itself). Specific Versions
that are generally no fewer than 10 and no more than 20 which
are representative of all the tens of thousands of possible
permutations you would otherwise have to create and test,
were you not using this kind of advanced MultiVariate Testing.
For example, let’s take those 15 Selling Factors you
decided to test with 2 different Variations each – the
control value and a new value. If the subject line of your
E-mail is one of the Factors you want to test, the control
value of the subject line is your current subject line and
the new value is the new subject line you want to test in
order to identify which subject line produces more sales conversions.
Similarly, you have control values and new values for all
the other Factors and Variations to be tested as well.
Step II: The Conversion Multiplier then extrapolates and
identifies, from the results obtained in your Phase I tests,
the most productive Variations of each of your different Selling
Factors. Once your most productive Variations have been identified,
you are ready for your Phase 2 (and final) tests.
OK… But exactly how can this software mathematically
calculate the specific combinations of Factors and Variations
that will produce my greatest Sales & Profit gains based
on Phase 1 Test Results?
Good question. And a satisfactory answer requires making
you aware of all the following details.
A) Understand that the 10 to 20 representative
versions that were short-listed (in Step I) for your Phase
I tests were not selected at random from the tens of thousands
of combinations that could have been created for the simple
reason that selecting them at random would lead to statistically-invalid
results.
Instead, The Conversion Multiplier specifically selects those
10 to 20 versions that satisfy certain mathematical requirements
that insure the results obtained after testing these Phase
1 versions can be used for further calculations needed to
select the specifics needed for your final Phase 2 Verification
tests.
B) Consequently, once enough data about
E-Mail recipients and visitors to your landing page has been
obtained during the Phase 1 test, our software analyzes the
performance of each Variation of the first Factor that was
tested based on:
1) Performance of each Variation of the
first Factor and that Variation’s power to increase
your sales conversion rate.
2) What the impact of “the interaction"
of this Variation with all other Variations of all other
Factors is….on your conversion rate.
C) Next, all the Variations related to the
first Factor are then compared against each other to identify
those specific combinations of Variations that have the highest
likelihood of producing the greatest increase to your existing
sales conversion rate.
Only at this point do these combinations (as mentioned below,
usually only 3 to 4 combinations) then become part of the
short-list for your final Verification Test.
D) Next, the same kind of analysis is performed
for each of the other Factors, so that the software can short-list
a specific Variation for each Factor tested in Phase 1.
These short-listed Variations of all the Factors in combination
with each other (based on the software’s mathematical
analysis of Phase 1 results) produces… the Phase 2 short-list
of combinations recommended for Final Verification A-B Split
Tests.
This is what makes
The Conversion Multiplier’s kind of MultiVariate
testing and tracking so powerful…
The fact that the conversion rate improvements
produced by each of the Factors in which the new value
has beaten the control are not merely added together
but are… compounded on top of each other. |
To sum up how our software combines higher mathematics with
superior IT performance…The Phase 1 analysis is performed
using 6 different mathematical formulae that produce up to
6 unique short-listed combinations. Usually 2 or 3 of these
6 short-listed combinations turn out to be exactly the same,
leaving around 3 or 4 unique combinations. It is these 3 to
4 unique combinations that are then tested against the control
version in a classical A-B Split-Test.
Whichever specific combination in this Phase 2 Split-Test
produces the highest Sales Conversion Rate is the Winning
Combination that will be rolled-out in your E-mail campaign.
How this New Approach worked in the Real World of Websites
competing for Online Sales and Market Share
A right-on, real world example is the e-mail campaign MV
testing of a site that generates leads of dog owners.
In that test, the Winning Combination in Phase 1 was #7 of
the 18 different combinations of Factors / Variations tested,
producing a sales increase of 143.2%, the highest in the site's
web marketing history.
However, their maximum sales gain had not yet been reached!
The software completed its mathematical analysis of the Phase
1 test, (to identify sales gains possible beyond that 143.2%
gain), based on...how the interaction of each different Variation
would impact total site sales conversions.
The software then recommended 3 to 4 new combinations that
differed from the
# 18 Version that had produced the Winning 143.2% gain in
the Phase 1 tests.
Phase 2 Verification then tested these 3 to 4 mathematically-selected
finalists against the site’s original Control Version
in a Classic A-B Split test.
What happened?
The 3rd Version selected by the software analysis increased
their Sales Conversions once again… taking their Sales
Gain to 226.8%, over 50% higher than the site's previous record-setting
high of 143.2%. All in only a few weeks.
The Conversion Multiplier produced this 226.8% gain in Revenue
for the site in just a few weeks, proving that this advanced
MultiVariate Testing Technique can be many times faster than
traditional A-B Split testing.
So, If your company has Annual E-mail sales of $500,000+
and you want to learn how your company can use this Advanced
kind of MultiVariate Testing to produce record sales gains
from E-mail campaigns, Discover
the steps involved in our companies working together to
make a MultiVariate Test for your E-mail marketing campaigns
a reality.
About Us | Contact
Us | Privacy Policy | Sitemap
©2006 The Conversion Multiplier, All Rights Reserved.1-800-767-0864
|
