AB Experiments (Firebase)

Introduction

AB Experiments are set up on the Firebase Remote Config Dashboard. Ensure your Firebase instance is set up and you can access your game dashboard.

This document provides instructions for setting up the Example AB Test use case described below.

Requirements

1. Design your experiment - what are your hypotheses, goals, and game levers? Please see our Experiment Best Practices.

2. LionSDK will automatically fire theab_cohort event to LionAnalytics, allowing us to analyze the experiment results.

   1. Every experiment needs a unique name, and we need a special (unique for the experiment) parameter that tracks which variant the player has been assigned for that experiment. LionSDK will automatically check for a parameter with the following naming convention and use that to fire the required ab_cohort event for you.

      • “exp_[EXPERIMENT NAME]" (replace [EXPERIMENT NAME] with your unique name)

   2. Ensure you have all necessary events firing in your game that you will use for analysis. See our Analytics: 1. Planning guide for more information.

3. Analyze Results - Once your experiment is complete, please see the Dashboards & Analysis section here: Dashboard & Analysis

Example Experiment (AB Test) Use Case

Firebase Remote Config Experiment (AB Test) Example

1. The Product team wants to run an experiment using Firebase Remote Config to optimize the level at which interstitial ads start showing in the game.

2. Each variant/cohort of the experiment will receive a different value for the interstitialMinLevel parameter, allowing the team to treat users differently by assigning them different interstitialMinLevel values.

3. There will be two experiment variants: “baseline” and “variant A”.

4. Users will be equally distributed among the experiment variants.

   Variant

   Allocation

   interstitialMinLevel

   Description

   Variant A

   50% of players

   5

   Users will start seeing interstitial ads after level 5

   Baseline

   50% of players

   3

   Users will start seeing interstitial ads after level 3

Firebase Remote Config Setup

1. Create an experiment called interstitialStart in the Firebase Console.

   
2. Create a parameter for your experiment named exp_interstitialStart in the Firebase Console. The part after “exp_” in the remote config name MUST be the same as your experiment name. (interstitialMinLevel → exp_interstitialStart )

   • The default value of the exp_interstitialStart parameter should be baseline.

   • This parameter will pass the experiment and variant names to the Analytics ab_cohort event.

     
3. Now, create one or more parameters for the variables you want to test.

   • For example, create a parameter named interstitialMinLevel with a default value of 3.

     
4. Create experiment variants in the Firebase Console.

   • Go to A/B Testing → Variants

   • Create 2 variants and set the parameter values for each variant.

     
5. Start the experiment when you are ready.

   • Go to A/B Testing → Start Experiment

6. Once the experiment has concluded, you can apply the best-performing variant as the default value for all users.