[WIP] - AB Experiments (Balancy)
Last updated
Last updated
AB Experiments are set up on the Balancy Dashboard. Ensure your Balancy instance is set up and you can access your Balancy project.
LionSDK will automatically fire theab_cohort
event to LionAnalytics, allowing us to analyze the experiment results
This document explains the requirements using an example use-case.
Install and setup Balancy in Unity project as explained here
Install LiveOps plugin in Balancy game dashboard.
At this point, you will be able to fetch remote configs from Balancy via LionSDK (Remote Config) package.
Create an experiment with an experiment name by going to your Balancy game dashboard and follow the experiment naming convention
Naming Convention → {game_code}_{platform}_{testName}
game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode
.
platform:
ios
for iOS
and
for Android
all
for tests run on both platforms simultaneously (start and end dates should be equal).
testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.
Let’s suppose there is a game with game code csc
runs an experiment to evaluate the impact of different interstitial ad timings. The experiment is named csc_and_newInterTimer
, with the following variants and configurations:
baseline
90
90
aggressive
60
60
passive
120
120
Each variant receives different values for the experiment's inter_between_time
and inter_start_time
remote configs.
Create an experiment with an experiment name following the experiment naming convention
Naming Convention → {game_code}_{platform}_{testName}
game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode
.
platform:
ios
for iOS
and
for Android
all
for tests run on both platforms simultaneously (start and end dates should be equal).
testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.
Example → csc_and_newInterTimer
Create templates/remote configs that the experiment will override for each experiment cohort
Setup the override values for each experiment cohort
For more information on the above, you can follow this link
For more information on Balancy AB test documentation. You can follow this official link