Lion Analytics QA Tool

The LionAnalytics QA tool lets you view the event stream from a registered device in near-real-time (less than 1 minute between firing the event and the event payload becoming available). This tool includes multiple search and filter options and export and sharing.

Introduction

Event validation is crucial to ensure that LionAnalytics events are implemented correctly. Failure to validate risks in rolling out releases that are incorrectly instrumented. In the case of wrong instrumentation, extracting insights may not be possible, and Looker dashboards may not work.

The AutoQA tool is designed to simplify and accelerate the event validation process using automation. The tool provides two views essential to the validation process:

• Event Stream:

  • Displays all events received from the device

  • Verifies that each event matches a requirement in the Event SPEC

• Event Requirements:

  • Displays all requirements from the Event SPEC

  • Verifies that each requirement is matched with at least one received event

Setup Adjust Insights

1. Install Adjust Insights application on your mobile device to learn your device ID.

   • Apple Store URL

   • Google Play Store URL

2. Open Adjust Insights app on your mobile device

3. Add your device in the Lion AutoQA tool by entering the IDFA or GAID into the Device Name section on the tool based on your testing device’s platform.

   • Apple → IDFA

   • Google → GAID

   Ensure you allow tracking of the game; otherwise, your Device ID will not be attached to the LionAnalytics events and cannot be searched for in the LionAnalytics QA Tool.

Validate analytic events with LionAnalytics QA Tool

Prerequisites

• A Google Sheet containing the Event SPEC Template 2.0, prepared by an analyst and customized for your game, must be provided. This document serves as a reference for both you and the tool, detailing how events should be implemented.

• Reinstall the app before validating events to ensure that events are captured from the beginning of the latest version. Use the Sequence ID field to order events chronologically, from first to last. Keep in mind that the Sequence ID resets to 0 with each reinstall.

Configure the Tool

Open the Lion AutoQA Tool, https://partner.lionstudios.cc/lionanalyticsrealtime/v2

1. Select your device from the Device Name dropdown that you set up earlier

   
2. Select your Game

3. Use the Event Timestamp Filter to match the time of your most recent fresh reinstall, ensuring you capture all events from the start of the user’s journey. You can set the filter so that the Sequence ID starts from 0

4. Enter the Event SPEC Template 2.0 Google Sheet URL

   1. Make sure your data analyst has delivered or reviewed the Event SPEC and confirmed that the requirements are up to date. If your game is in its early stages and does not yet have an Event SPEC, you can skip this part. In this case, you need to manually check if you have implemented events properly.

5. Hit Check Events

Event stream tab

• Expand each event with a “Failed” validation status to review the conflicts.

• Navigate to the “Parameters” tab to check for any conflicts. Resolve these conflicts by following the instructions provided in the conflict messages.

• Navigate to the “Additional Data” tab to check for any conflicts. Resolve these conflicts by following the instructions provided in the conflict messages.

Event Requirements Tab

• Expand each requirement with an “Unchecked” QA status to identify which requirements remain not validated and to gain a detailed understanding of the associated validation conditions.

  

Examples

Resolving conflicts associated with Economy object

The rewardProducts parameter expects a Product object. A Product object can include attributes such as virtual currencies, real currency, and items.

For the EconomyObjects sheet, a virtual currency object must follow these specifications:

• It should have virtual currency type as meta and virtual currency name as hammer, OR

• It should have virtual currency type as soft and virtual currency name as coin.

Additionally, for the item object:

• It may have item type as power_up and item name as hint.

Any deviation from these specified attribute values will result in conflicts.

FAQs

Which fields are required to use the QA tool?

Device ID and App Name are the only required fields.

App Version and LionAnalytics Version options are available to assist with filtering data when multiple builds are being tested.

Why do I need to register my device?

The system works by siphoning events from Lion’s existing data pipeline and saving them to a temporary database. That process happens multiple times a minute, and its speed and cost depend on the amount of data being siphoned - device ID registration is designed to minimize the amount of data being siphoned, increasing responsiveness and reducing cost.

Can I view end-user event feeds?

Only if you have their device IDs.

How are events from do-not-track (DNT) users handled?

Typical do-not-track settings result in an null (all 0’s) advertising ID (IDFA / GAID). If you have registered your device by using the advertising ID, then events will not be available in the QA tool. You can attempt to register alternative IDs (IDFV, Adjust ID)

I used the Adjust Insights application to register my IDFV; why is the QA tool not displaying my events?

The IDFV is unique to each application developer and will reset when all of a developer’s applications are removed from the device; your device will have multiple IDFVs associated with it. The Adjust Insights application displays the IDFV associated with its developer (Adjust), not the IDFV associated with the developer of the game you are playing.

Other device IDs are global and will be valid, however it is recommended that you do not restrict tracking on test devices.

Why do I see multiple conflicts under the “Conflicts” section?

A data analyst might request the use of a single event across multiple use cases. In such instances, the validation conditions for event parameters can vary depending on the specific use case. Aligning an event’s validation conditions with its use case requires contextual understanding.

To streamline this process, we identify all potential conflicts and allow you to associate the event with the correct requirements. Reviewing the events triggered immediately before and after it in the event stream can offer valuable context to resolve conflicts effectively.