AB Experiments (Firebase)
Run AB Experiments with Firebase, the LiveOps solution from Google.
Introduction
AB Experiments are set up on the Firebase Remote Config Dashboard. Ensure your Firebase instance is set up and you can access your game dashboard.
This document provides instructions for setting up the Example AB Test use case described below.
Requirements
Design your experiment - what are your hypotheses, goals, and game levers? Please see our Experiment Best Practices.
LionSDK will automatically fire the
ab_cohort
event to LionAnalytics, allowing us to analyze the experiment results.Every experiment needs a unique name, and we need a special (unique for the experiment) parameter that tracks which variant the player has been assigned for that experiment. LionSDK will automatically check for a parameter with the following naming convention and use that to fire the required
ab_cohort
event for you.“
exp_[EXPERIMENT NAME]
" (replace[EXPERIMENT NAME]
with your unique name)
Ensure you have all necessary events firing in your game that you will use for analysis. See our Analytics: 1. Planning guide for more information.
Analyze Results - Once your experiment is complete, please see the Dashboards & Analysis section here: Dashboard & Analysis
Example Experiment (AB Test) Use Case
Firebase Remote Config Experiment (AB Test) Example
The Product team wants to run an experiment using Firebase Remote Config to optimize the level at which interstitial ads start showing in the game.
Each variant/cohort of the experiment will receive a different value for the
interstitialMinLevel
parameter, allowing the team to treat users differently by assigning them differentinterstitialMinLevel
values.There will be two experiment variants:
“baseline”
and“variant A”
.Users will be equally distributed among the experiment variants.
VariantAllocationinterstitialMinLevelDescriptionVariant A
50% of players
5
Users will start seeing interstitial ads after level 5
Baseline
50% of players
3
Users will start seeing interstitial ads after level 3
Firebase Remote Config Setup
Create an experiment called
interstitialStart
in the Firebase Console.Create a parameter for your experiment named
exp_interstitialStart
in the Firebase Console. The part after “exp_
” in the remote config name MUST be the same as your experiment name. (interstitialMinLevel
→exp_interstitialStart
)The default value of the
exp_interstitialStart
parameter should bebaseline
.This parameter will pass the experiment and variant names to the Analytics
ab_cohort
event.
Now, create one or more parameters for the variables you want to test.
For example, create a parameter named
interstitialMinLevel
with a default value of3
.
Create experiment variants in the Firebase Console.
Go to
A/B Testing
→Variants
Create 2 variants and set the parameter values for each variant.
Start the experiment when you are ready.
Go to
A/B Testing
→Start Experiment
Once the experiment has concluded, you can apply the best-performing variant as the default value for all users.
Last updated