AB Experiments (Firebase)

Introduction

This document provides instructions for setting up the AB Experiments exclusively for Firebase only.

Requirements

1. Lion Remote configs must be installed and setup. You can install the Lion - Remote Configs package from the package manager. More info can be found here (only looking at installation guide in the link will be enough)

2. Ensure that you have Firebase package configured and setup in your project. More info can be found here.

3. "Firebase" must be selected for the Adapter option in the "Remote Configs" settings. You can access this via the LionStudios/Settings Manger menu.

   

4. LionSDK will automatically fire the ab_cohort event to LionAnalytics, if you follow the below step,

   1. Every experiment needs a unique name, to track which variant the player has been assigned for that experiment. LionSDK will automatically check for a parameter with the following naming convention and use that to fire the required ab_cohort event for you.

      • “exp_[FULL EXPERIMENT NAME]" (replace [FULL EXPERIMENT NAME] with your unique name). Make sure to use the experiment naming convention described here for your Full EXPERIMENT NAME . Take a look at our example setup below for more details.

Example Experiment (AB Test) Use Case

Firebase Remote Config Experiment (AB Test) Example

1. Lets say the product team wants to run an experiment using Firebase Remote Config to optimize the level at which interstitial ads start showing in the game. The Product team wants to run an experiment using Firebase Remote Config on the game named: King Wing (3 letter code: kwi) to evaluate the impact of different interstitial ad timings. The platform is android (3 letter code: and). So the full experiment named becomes kwi_and_NewInterTimer, with the following variants and configurations:

   Variants

   inter_between_time

   inter_start_time

   control

   90

   90

   aggressive

   60

   60

   passive

   120

   120

2. Each variant/cohort of the experiment will receive a different value for the inter_between_time and the inter_start_time parameter, allowing the team to treat users differently by assigning them different values.

3. There will be 3 experiment variants: “control” , “aggressive”, and passive.

4. Users will be equally distributed among the experiment variants.

Firebase Remote Config Setup

1. In the Firebase dashboard go to Remote Config/Parameters and click Add parameter . Add 3 new parameters called inter_between_time , inter_start_time , and exp_kwi_and_NewInterTimer. These 3 parameters will be used in our experiment setup. A button to publish your new changes will appear and select "Publish changes".

   1. The exp_kwi_and_NewInterTimer parameter serves as an internal identifier to track which cohort variant a player belongs to. This parameter will be set to one of three values: control, aggressive, or passive. Keep default value clear.

2. Then in the Firebase dashboard go to Remote Config/AB Tests tab and click Create Remote Config experiment. Create an experiment called kwi_and_NewInterTimer in the Firebase Console. This experiment will be ran on a game called King Wing (3 letter code kwi), for android platforms. Note: You must use your own game code, platform and experiment name.

   

3. Next setup your Targeting. You can select which platform (android or iOS) you want your experiment ran on and as well as what percent of your player base will be tested. Note: You must setup the same experiment separately for both iOS and android. In our case we will be using Android.

   

4. Setup the "Goals" section.

5. Setup the variants for this experiment.

   • Looking back at our initial guideline here, we want to have 3 groups. The control, aggressive and passive.

   • Each group will have different values for the 3 new parameters (inter_between_time , inter_start_time, and exp_kwi_and_NewInterTimer) we created in Step 1.

   • Create 3 variants and set the values for each of the parameters.

   • Make sure to name the exp_kwi_and_NewInterTimer parameter to match the group it belongs to.

     

6. Click "Review" and now you can start the experiment when you are ready.

   • Go to A/B Testing → Start Experiment

7. Once the experiment has concluded, you can apply the best-performing variant as the default value for all users.