Using Control Groups – Help Centre

Avoid cross-contamination

It is imperative that your target and control groups do NOT receive any other campaigns with offers for the duration of this campaign. The intention is that the control group exhibits normal behavior over the duration of your campaign and that the target group’s performance is directly attributed to this campaign only.

If either group gets contaminated, you will be unable to specifically determine the cause for any change from the control group. For example, you will be unable to determine which offer caused the target group’s change in behavior.

Excluding target and control groups

To exclude an entire campaign’s target and control group members from another campaign, you will need to add a separate filter for each campaign.

To do this:

Hover over ‘Analyze’ in the top menu bar and click on ‘Group Builder’
Click on the ‘Add Filter’
Click on the ‘Marketing’ header
Select the ‘All Campaign Members’ filter
Enter the name of the campaign that had target and control members

Excluding control groups only

To exclude a campaign’s control group members only, create a separate filter for each campaign.

To do this:

Hover over ‘Analyze’ in the top menu bar and click on ‘Group Builder’
Click on the ‘Add Filter’
Click on the ‘Marketing’ header
Select the ‘Control Group Members’ filter
Enter the name of the campaign that had control group members

How many members in a control group?

This depends on how accurate you would like to measure the campaign effectiveness and Return on Investment.

In determining your desired level of accuracy, you need to also consider how many members you are comfortable to exclude from the campaign and any other campaigns during the process. The more accurate you would like the measurement to be, the bigger the control group has to be, and the more members you will exclude from receiving offers during the campaign.

Movio Cinema automatically calculates the control group size for you. The recommended size is based on two mathematical levers:

Confidence level – the confidence level tells you how sure you can be. It is expressed as a percentage and represents how often the true percentage of the target audience would behave within margin of error. The 95% confidence level means you can be 95% certain; the 99% confidence level means you can be 99% certain.
The default confidence level is 95%.
Margin of error – the margin of error is the plus-or-minus figure usually reported in newspaper or television opinion poll results.
For example, if you are using a margin of error of 4 and 47% of your control behaves in a certain way, you can be ‘sure’ that between 43% and 51% of the entire target audience would have done the same.
The default margin of error is 3%.

When you are adding a control group to your campaign, we highly recommend that you use the ‘Auto Calculate’ option, as this will return the best statistics. However, you can also use the ‘Number of Members’ or ‘Percentage of Members’ options and use the calculator to manually input your numbers.

Auto Calculate: an algorithm will calculate optimum control members
Number of Members: you can input the number of control members
Percentage of Members: you can input the percentage of control group members

Reporting

After you execute your campaign, Movio begins to track the control and target group members. You'll then be able to go into the reporting tab and measure the effectiveness of the campaign. You'll be able to see:

The difference in visitation from the control vs. target members (those who received the campaign)
The difference in various spend metrics from control vs. target members
The difference in admissions per visit from the control vs. target members
The return on investment from the campaign