Validation

Overview

Implementing events correctly is critical to ensuring that our analytics dashboards operate as intended. The AutoQA tool is designed to simplify and accelerate the event validation process using automation.

The tool provides two views essential to the validation process:

Event Stream:
- Displays all events received from the device
- Verifies that each event matches a requirement in the Event SPEC
Event Requirements:
- Displays all requirements from the Event SPEC
- Verifies that each requirement is matched with at least one received event

Prerequisites

A Google Sheet containing the Event SPEC Template 2.0, prepared by an analyst and customized for your game, must be provided. This document serves as a reference for both you and the tool, detailing how events should be implemented.

⛔

If your game is in its early stages and does not yet have an Event SPEC, you can still use the tool to QA events manually.

Enter your device ID into the tool before firing events from your device. If the device ID is not entered, the events will NOT be registered by the tool.

Reinstall the app before validating events to ensure that events are captured from the beginning of the latest version. Use the Sequence ID field to order events chronologically, from first to last. Keep in mind that the Sequence ID resets to 0 with each reinstall.

Objective

The validation process is considered complete when the following outcomes are achieved:

All events listed in the Event Stream have a “Validated” status.
All requirements listed in Event Requirements have a “Checked” status.

Steps

Configuring the Tool

Go to LionAuto QA Tool

Select the Device Name from the dropdown

Select the game
Use the Event Timestamp Filter to match the time of your most recent fresh reinstall, ensuring you capture all events from the start of the user’s journey.

💡

PRO TIP: Set the filter so that the Sequence ID starts from 0

Enter the Event SPEC Template 2.0 Google Sheet URL

👉

Make sure your data analyst has delivered or reviewed the Event SPEC and confirmed that the requirements are up to date.

⚠️

If your game is in its early stages and does not yet have an Event SPEC, you can skip this part. In this case, you need to manually check if you have implemented events properly.

Hit “Check Events”

Event Stream Tab

Expand each event with a “Failed” validation status to review the conflicts.
Navigate to the “Parameters” tab to check for any conflicts. Resolve these conflicts by following the instructions provided in the conflict messages.
Navigate to the “Additional Data” tab to check for any conflicts. Resolve these conflicts by following the instructions provided in the conflict messages.

Event Requirements Tab

Expand each requirement with an “Unchecked” QA status to identify which requirements remain not validated and to gain a detailed understanding of the associated validation conditions.

Examples

Resolving conflicts associated with Economy object

The rewardProducts parameter expects a Product object. A Product object can include attributes such as virtual currencies, real currency, and items.

For the EconomyObjects sheet, a virtual currency object must follow these specifications:

It should have virtual currency type as meta and virtual currency name as hammer, OR
It should have virtual currency type as soft and virtual currency name as coin.

Additionally, for the item object:

It may have item type as power_up and item name as hint.

Any deviation from these specified attribute values will result in conflicts.

Example Economy Objects requirement

FAQs

Why do I see multiple conflicts under the “Conflicts” section?

A data analyst might request the use of a single event across multiple use cases. In such instances, the validation conditions for event parameters can vary depending on the specific use case. Aligning an event’s validation conditions with its use case requires contextual understanding.

To streamline this process, we identify all potential conflicts and allow you to associate the event with the correct requirements. Reviewing the events triggered immediately before and after it in the event stream can offer valuable context to resolve conflicts effectively.