> For the complete documentation index, see [llms.txt](https://lionstudios.gitbook.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://lionstudios.gitbook.io/readme/analytics/a-b-testing.md).

# A/B Testing

## Introduction

A/B tests are a fundamental tool for optimizing game features by comparing different variants and analyzing their impact on player behavior and game performance. By running A/B tests, you can make data-driven decisions to enhance monetization, engagement, and overall player satisfaction.

## What is an A/B Test?

An A/B test involves dividing users into different groups, each exposed to a distinct variant of a feature. By comparing the performance metrics across these groups, you can determine which variant performs better and make informed decisions based on the results.

## A/B Test Analysis

We analyze A/B test results using internal analytics tools designed to provide a complete picture of performance. Feature flags allow us to place users into controlled test groups, while our systems track monetization, ad performance, engagement, and other core metrics to evaluate each experiment.

## Experiment Naming Convention

Use the following pattern for all experiment names:

`{game_code}_{platform}_{TestName}`

**Components**

* **game\_code**: Three-letter game code (e.g., FNI, SBP, FMT). If the game doesn’t have a three-letter code, use the game name: `GameName`
* **platform**:
  * `ios` for iOS
  * `and` for Android
* **TestName:** The experiment name. Use **PascalCase** for multi-word names 

**Examples**

1. `fmt_ios_NewUI`
2. `pie_and_AdventureModeStartXp`
3. `fni_and_NewWelcomePack`

## Available Functions

{% hint style="success" %}
N**ote**: If you're using our [Lion Remote Configs package](/readme/features/remote-configs.md), **AbCohort** tracking can be set up to run automatically when your app starts - no manual coding required.
{% endhint %}

#### [**`AbCohort`**](/readme/analytics/api/helper-functions.md#AbCohort)

The [**`AbCohort`**](/readme/analytics/api/helper-functions.md#AbCohort) function assigns users to a test cohort and adds the [**`AbCohort`**](/readme/analytics/api/helper-functions.md#AbCohort) parameter to all subsequent events. This function should be called every session when users log in.

### Critical Parameters

* **Required**
  * `experiment_name`: The name of the experiment (e.g., `abc_and_NewInterTimer`).
  * `experiment_cohort`: The name of the variant or cohort within the experiment (e.g., `aggressive`, `control`, `passive`).

### Implementation

* Call the [**`AbCohort`**](/readme/analytics/api/helper-functions.md#abcohort) function with the experiment name and the cohort received from your feature-flag system. Once set, [**`AbCohort`**](/readme/analytics/api/helper-functions.md#abcohort) becomes a global parameter and is automatically included in all subsequent event payloads.
* The `ab_cohort` value is automatically reset at the start of a new session.
* If your workflow requires explicitly ending an experiment during the same session, you can use [**`ClearAbCohort`**](/readme/analytics/api/helper-functions.md#ClearAbCohort) to remove the parameter immediately.

## Example

To illustrate the proper setup and naming convention for an A/B test, consider the following example:

### Experiment Setup

An anonymous game (game code `ABC`) runs an experiment to evaluate the impact of different interstitial ad timings. The experiment is named `abc_and_NewInterTimer`, with the following variants and configurations:

| Variants   | inter\_between\_time | inter\_start\_time |
| ---------- | -------------------- | ------------------ |
| control    | 90                   | 90                 |
| aggressive | 60                   | 60                 |
| passive    | 120                  | 120                |

Each variant receives different values for the experiment’s `inter_between_time` and `inter_start_time` remote config parameters.

#### ✅ Correct Function Usage

Call the `ab_cohort` function based on the user’s allocation to the variant:

* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "control")`
* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "aggressive")`
* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "passive")`

#### ❌ Incorrect Function Usage

Avoid calling the function with individual configuration values:

* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "60")`
* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "90")`
* `LionAnalytics.AbCohort("abc_and_NewInterTimer", "120")`


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://lionstudios.gitbook.io/readme/analytics/a-b-testing.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.