[WIP] - AB Experiments (Balancy)

Introduction

• AB Experiments are set up on the Balancy Dashboard. Ensure your Balancy instance is set up and you can access your Balancy project.

• LionSDK will automatically fire theab_cohort event to LionAnalytics, allowing us to analyze the experiment results

• This document explains the requirements using an example use-case.

Pre-requisite

• Install and setup Balancy in Unity project as explained here

• Install LiveOps plugin in Balancy game dashboard.

At this point, you will be able to fetch remote configs from Balancy via LionSDK (Remote Config) package.

Requirements

• Create an experiment with an experiment name by going to your Balancy game dashboard and follow the experiment naming convention

  • Naming Convention → {game_code}_{platform}_{testName}

    • game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode.

    • platform:

      • ios for iOS

      • and for Android

      • all for tests run on both platforms simultaneously (start and end dates should be equal).

    • testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.

Example Use-Case

• Let’s suppose there is a game with game code csc runs an experiment to evaluate the impact of different interstitial ad timings. The experiment is named csc_and_newInterTimer, with the following variants and configurations:

  Variants	inter_between_time	inter_start_time
  baseline	90	90
  aggressive	60	60
  passive	120	120

• Each variant receives different values for the experiment's inter_between_time and inter_start_time remote configs.

Requirements

• Create an experiment with an experiment name following the experiment naming convention

  • Naming Convention → {game_code}_{platform}_{testName}

    • game_code: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name in camel-case: gameThatDoesntHaveCode.

    • platform:

      • ios for iOS

      • and for Android

      • all for tests run on both platforms simultaneously (start and end dates should be equal).

    • testName: Any descriptive name for the test. Please use camel case instead of spaces for multiple words.

  • Example → csc_and_newInterTimer

    
• Create templates/remote configs that the experiment will override for each experiment cohort

  
• Setup the override values for each experiment cohort

  

For more information on the above, you can follow this link

For more information on Balancy AB test documentation. You can follow this official link