Adding A/B Testing

Overview

Here is everything needed to use the A/B Testing feature in the "Beamable SDK for Unity".

The purpose of this feature is to allow game makers to deploy new functionality to subset of players.

📘

Related Features

More details are covered in related feature page(s).

Game Maker User Experience

During development, the game maker's user experience is as follows.

Steps

Follow these steps to get started.

Step 1. Create Data

Here the data will be created. This represents the default data which all users will receive (unless overridden by the TrialTrial - An experiment to group players into buckets either by percentage or according to a player stat criteria created per Step 2).

Step

Detail

  1. Open the Portal

• See db-portal-prod.disruptorbeam.com

Note: The specific url above is required for this usage of Beamable Portal

  1. Open "Game Based Cloud Data"
  1. Upload Data

• Click "Upload YAML File"
• Populate all data fields
• Optional, choose the appropriate file from Json Data Files below
• Click "Upload"

Note: Any text/binary format is compatible with Game Base Cloud Data. Many game makers choose YAML. See Wikipedia's YAML for more info

Step 2. Create Trial

Here the TrialTrial - An experiment to group players into buckets either by percentage or according to a player stat criteria will be created. This represents the rules for if/when the default data created in Step 1 above will be overridden.

Step

Detail

  1. Open the Portal

• See db-portal-prod.disruptorbeam.com

Note: The specific url above is required for this usage of Beamable Portal

  1. Open "Trials"
  1. Create the Trial

• Choose "Allocation" of "Custom" and add 2 cohorts
• Populate all data fields
• Click "Create"

Note: See A/B Testing » Code (Glossary) for more info

  1. Assign the Data

• Assign the override data that will be loaded instead of the default data from Step 1 above
• Optional, choose the appropriate files from Json Data Files below

  1. Play the Trial

• Click "Play"
• Click "Confirm"

Note: See A/B Testing » Code (Glossary) for more info

  1. Set the Stat

• Open "Player Administration"
• Open "Stats"
Enter the DBID of the active player from the Unity Console Window and the namespace of game.private.player
• Click "Add Player Stat"
• Enter Name of PLAYER_LEVEL and a Value of 1 or 2. Each returns a dataset via TrialDataService

Note: The trial in this example depends on a stat value. However, other types of trials do not. Choose the best criteria for the needs of the project.


Step 3. Load Data

Here the game client will load the TrialTrial - An experiment to group players into buckets either by percentage or according to a player stat criteria data. If the current player qualifies for the criteria of the TrialTrial - An experiment to group players into buckets either by percentage or according to a player stat criteria created in Step 2, the player will receive overridden data. Otherwise the player will receive the default data created in Step 1.

3a. Load Manifest

GetCloudDataManifestResponse playerManifestResponse =
     await _trialDataService.GetPlayerManifest();

3b. Load Data

Loop through the meta. There will be 0 or more values. The implementation here depends on the needs of the project. In this example, each loop iteration is parsed as Json and tested for compatibility with the MyPlayerProgression data type.

foreach (CloudMetaData cloudMetaData in playerManifestResponse.meta)
{
    string path = $"http://{cloudMetaData.uri}";

    // Load the data, respecting GZip format
    string response = 
        await ExampleProjectHelper.GetResponseFromHttpWebRequest(path);

    MyPlayerProgression myPlayerProgression = 
        JsonUtility.FromJson<MyPlayerProgression>(response);

    // Store the data
    if (myPlayerProgression != null)
    {
        _data.MyPlayerProgression = myPlayerProgression;
    }
}

Examples

📘

Beamable SDK Examples

This and all examples are available for download at GitHub.com/Beamable_SDK_Examples

Flowchart

Here is a flowchart demonstrating an example of one A/B Testing TrialTrial - An experiment to group players into buckets either by percentage or according to a player stat criteria.

The game front-end loads the player manifest from the TrialDataService. The PlayerProgression data returned depends on a player-specific StatStat - Active data entity used for read/write of player state value.

Json Data Files

To follow along with the steps above using the example data download these Json data files which matches the MyPlayerProgression data type. Or create and choose your own custom files and custom data type.

  • base_cohort.json - For use with Step 1.3 above
  • player_level_1_cohort.json - For use with Step 2.4 above
  • player_level_not1_cohort.json - For use with Step 2.4 above

Code

Here are examples which cover common programming needs.

The TrialDataServiceExample.cs loads the MyPlayerProgression object with appropriate values. The related A/B Test trial uses the player's StatStat - Active data entity used for read/write of player state of PLAYER_LEVEL to determine the appropriate values.

using System.Collections.Generic;
using System.Threading.Tasks;
using Beamable.Common.Api;
using Beamable.Common.Api.CloudData;
using Beamable.Examples.Services.CloudSavingService;
using Beamable.Examples.Services.ConnectivityService;
using Beamable.Examples.Shared;
using UnityEngine;
using UnityEngine.Events;

namespace Beamable.Examples.Services.TrialDataService
{
    /// <summary>
    /// Represents the data in
    /// the A/B Testing trial.
    /// </summary>
    [System.Serializable]
    public class MyPlayerProgression
    {
        public int MaxHealth = 100;
        public int MaxInventorySpace = 10;
    }

    /// <summary>
    /// Holds data for use in the <see cref="ConnectivityServiceExampleUI"/>.
    /// </summary>
    [System.Serializable]
    public class TrialDataServiceExampleData
    {
        public bool IsUIInteractable = false;
        public string DataName = "MyPlayerProgression";
        public MyPlayerProgression MyPlayerProgression = null;
        public List<CloudMetaData> CloudMetaDatas = new List<CloudMetaData>();
        public bool IsInABTest { get { return CloudMetaDatas.Count > 0; } }
    }

    [System.Serializable]
    public class RefreshedUnityEvent : UnityEvent<TrialDataServiceExampleData>  { }

    /// <summary>
    /// Demonstrates <see cref="TrialDataService"/>.
    ///
    /// NOTE: This demo uses other concepts
    /// too. See <see cref="CloudSavingServiceExample"/>
    /// for more info.
    /// 
    /// </summary>
    public class TrialDataServiceExample : MonoBehaviour
    {
        //  Events  ---------------------------------------
        [HideInInspector] public RefreshedUnityEvent OnRefreshed = new RefreshedUnityEvent();

        //  Fields  ---------------------------------------
        private TrialDataServiceExampleData _data = new TrialDataServiceExampleData();
        private ICloudDataApi _trialDataService;
        private IBeamableAPI _beamableAPI;

        //  Unity Methods  --------------------------------
        protected void Start()
        {
            Debug.Log($"Start() Instructions...\n" +
                      " * Setup AB Testing in Portal per https://docs.beamable.com/docs/abtesting-code\n" +
                      " * Run The Scene\n" +
                      " * See onscreen UI for results.\n" +
                      " * If IsInABTest is false, something is incorrect. Repeat these steps.\n" + 
                      " * If IsInABTest is true, everything is correct. Visit the portal to change " +
                      "the `PLAYER_LEVEL` stat value, then repeat these steps see load other data.\n");

            SetupBeamable();
        }


        //  Methods  --------------------------------------
        private async void SetupBeamable()
        {
            _beamableAPI = await Beamable.API.Instance;

            Debug.Log($"beamableAPI.User.id = {_beamableAPI.User.id}");

            _trialDataService = _beamableAPI.TrialDataService;

            await LoadTrialData();
        }

        public async Task<EmptyResponse> LoadTrialData()
        {
            // Load any trials
            GetCloudDataManifestResponse playerManifestResponse =
                await _trialDataService.GetPlayerManifest();
            
            // Loop through trials
            _data.MyPlayerProgression = null;
            _data.CloudMetaDatas = playerManifestResponse.meta;
            foreach (CloudMetaData cloudMetaData in _data.CloudMetaDatas)
            {
                string path = $"http://{cloudMetaData.uri}";

                // Load the data, respecting GZip format
                string response = 
                    await ExampleProjectHelper.GetResponseFromHttpWebRequest(path);

                MyPlayerProgression myPlayerProgression = 
                    JsonUtility.FromJson<MyPlayerProgression>(response);

                // If trial is related, store data
                if (myPlayerProgression != null)
                {
                    _data.MyPlayerProgression = myPlayerProgression;
                }
            }

            _data.IsUIInteractable = true;
            Refresh();
            return new EmptyResponse();
        }


        public void Refresh()
        {
            string refreshLog = $"Refresh() ..." +
                                $"\n * IsInABTest = {_data.IsInABTest}" +
                                $"\n * CloudMetaDatas.Count = {_data.CloudMetaDatas.Count}" +
                                $"\n\n";

            //Debug.Log(refreshLog);

            // Send relevant data to the UI for rendering
            OnRefreshed?.Invoke(_data);
        }
    }
}

Verify Success

After running the scene containing the TrialDataServiceExample.cs per above, you can further verify success.

Visit the Portal and change the value of any stat related to the trial for the currently logged-in DBIDDBID - The database identification. Beamable generates an anonymous account for the player when the project first runs. See the Unity console window for the DBIDDBID - The database identification. Beamable generates an anonymous account for the player when the project first runs.

Steps

  • Change the value of the PLAYER_LEVEL stat to 1. Run the scene and see the result.
  • Change the value of the PLAYER_LEVEL stat to 2. Run the scene and see the result.

If the onscreen UI shows different text in each case, then all is setup properly.

Advanced

This section contains any advanced configuration options and workflows.


Did this page help you?