A/B Testing - Code

Overview

This guide includes everything needed to use the A/B Testing feature in the "Beamable SDK for Unity".

The purpose of this feature is to allow game makers to deploy new functionality to subset of players.

What is A/B Testing?

A/B Test trials are experiments that can be created in the Portal to group players into buckets either by percentage (e.g. "50% of all users") or according to criteria (e.g. Stat of PLAYER_LEVEL > 1).

Benefits of A/B Testing

Game makers often know exactly which decisions make the best experience for players. When in doubt, game makers can use an A/B Testing trial to try multiple solutions before finalizing the game design.

• Increase KPIs - Key performance indicators, including retention, can be isolated and improved through testing
• Increase Revenue - By improving KPIs, improve revenue
• Decrease Costs - Game makers can focus on a leaner feature-set which is proven to matter to players
• Decrease Risks - Roll out new gameplay features more frequently to a subset of the players. Ensure its ready before releasing to everyone

Flowchart

Here is a flowchart demonstrating an example of one A/B Testing Trial.

The game front-end loads the player manifest from the TrialDataService. The PlayerProgression data returned depends on a player-specific Stat value.

Glossary

Here are some common terms:

API

Unlike many Beamable Features, A/B Testing does not require a specific Beamable Feature Prefab to be used. The main entry point to this feature is C# programming.

Here are API highlights for beamableAPI.TrialDataService.

🚧

Gotchas

Here are hints to help explain some of the trickier concepts:

• The downloaded manifest is compressed in Gzip format. Be sure to account for that
• The url of the cloud data leaves the protocol unspecified. Be sure to prepend either "http://" or "https://"

{
   "result":"ok",
   "meta":[
      {
         "sid":1335576733310978,
         "version":1,
         "ref":"game_world_config",
         "uri":"dev-trials.beamable.com/1208935505932288/DE_1208935505932289/1335576733310978/1",
         "cohort":{
            "trial":"",
            "cohort":""
         }
      }
   ]
}

{
   "result":"ok",
   "meta":[
      {
         "sid":1335576733310978,
         "version":1,
         "ref":"game_world_config",
         "uri":"dev-trials.beamable.com/1208935505932288/DE_1208935505932289/1335576733310978/1",
         "cohort":{
            "trial":"",
            "cohort":""
         }
      }
   ]
}

Stats

The Stats feature is related to A/B Testing in a couple of ways.

1. Automatic Stats

Each player who is a member of an A/B Testing Trial automatically has these populated Stats.

These values can be accessed programmatically via set as game.private.player or set in the Beamable Portal.

2. Criteria Stats

For any A/B Testing Trial with an "Allocation" of type "Custom", the game maker specifies any player Stat and a relational operator. This determines the cohort (e.g. stat of PLAYER_LEVEL > 1).

These values can be accessed programmatically via get as game.private.player.

Adding A/B Testing

Game Maker User Experience

During development, the game maker's user experience is as follows:

Steps

📘

Related Guides

A common use case for the feature is covered in the guides.

• See Adding A/B Testing for more info

Advanced

This section contains any advanced configuration options and workflows.

Debugging Manifests

For debugging the manifests, game players can use the Admin Flow to enter commands.

Commands

📘

Best Practices

These hints make efficent use of concepts and workflows.

Unity Client

• Trial assignment occurs on session start
• Fetch the cloud data manifest again if the user changes

C# Microservice

• Fetch the manifest with every incoming request, it is cheap to do so
• Cache the cloud data in local storage or in memory