A/B Test Setup Guide: InstaFind App Load Impact on E-commerce Metrics

This guide will walk you through configuring your website and Google Analytics (GA4) property to conduct an A/B test. The goal of this test is to measure how loading the InstaFind application impacts key e-commerce metrics on your website. We will set up the test so that 50% of your users will have InstaFind loaded, and 50% will not.

Instafind Configuration

Your development team will need to implement or adjust a JavaScript configuration object on your website. This object, window.InstaFindConfig, controls the A/B test. For this specific test, it should be set up as follows:

window.InstaFindConfig = {
    // ... other InstaFind configurations if any ...

    // Configuration for A/B testing the loading of InstaFind
    "abTestInstafindLoad": {
        "active": true, // Set to true to run this A/B test.
        "testId": "instafind_load_vs_noload_v1", // Unique ID for this test.
        "variations": {
            "loadInstafind": { // 50% of users will get this version (InstaFind loads)
                "weight": 50
            },
            "noLoadInstafind": { // 50% of users will get this version (InstaFind does not load)
                "weight": 50
            }
        }
    }

    // ... other InstaFind configurations if any ...
};

Talk with the Instafinfd support to ensure the above abTestInstafindLoad configuration is correctly enabled within your website's configuration.

Google Analytics 4 Setup

To analyze the results of this A/B test, you need to configure your Google Analytics 4 property.

Prerequisites: * You must have an active Google Analytics 4 (GA4) property implemented on your website using gtag.js.

Register Custom Dimensions

These custom dimensions will allow GA4 to capture and report the A/B test details.

  1. Navigate to your GA4 Property.
  2. In the bottom-left corner, click on Admin (the gear icon).
  3. In the Property column, find and click on Custom definitions (this might be under "Data display").
  4. Select the Custom dimensions tab.

  5. Click the Create custom dimensions button.

    You need to create two custom dimensions:

    Custom Dimension 1: Test Identifier * Dimension name: InstaFind Test ID * Scope: Select Event. * Description: (Optional) "Identifier for the A/B test run by the InstaFind application, e.g., instafind_load_vs_noload_v1." * Event parameter: Enter exactly instafind_test_id. * Click Save.

    Custom Dimension 2: Variation Name * Click Create custom dimensions again. * Dimension name: InstaFind Variation Name * Scope: Select Event. * Description: (Optional) "Name of the variation a user was exposed to in an InstaFind A/B test, e.g., loadInstafind or noLoadInstafind." * Event parameter: Enter exactly instafind_variation_name. * Click Save.

    Please note: It can take up to 24-48 hours for these custom dimensions to be fully processed and available for use in GA4 reports and audiences.

Verify Event Reception

Once the A/B test is active on your site and the InstaFind configuration is live:

  1. In your GA4 Property, go to Reports > Realtime.
  2. Look for an event named instafind_ab_exposure in the "Event count by Event name" card.
  3. Click on this event name. You should see the parameters instafind_test_id (with the value instafind_load_vs_noload_v1) and instafind_variation_name (with values loadInstafind or noLoadInstafind) being reported.
  4. For more detailed inspection, you can use Admin > DebugView (requires enabling debug mode for your browser).

This step confirms that the A/B test data is being sent correctly to GA4.

Create Audiences

Creating audiences will make it easier to compare the behavior of users in each test group.

  1. Navigate to Admin.
  2. In the Property column, click on Audiences.

  3. Click New audience, then Create a custom audience.

    Audience 1: Users with InstaFind Loaded * Audience name: A/B Test - InstaFind Loaded (instafind_load_vs_noload_v1) * Under "Include users when:", add a new condition: * Select the event: instafind_ab_exposure. * Add parameter filter: * Select parameter: InstaFind Test ID (or instafind_test_id). * Condition: is exactly (or exactly matches). * Value: instafind_load_vs_noload_v1. * Add another parameter filter (click "Add condition"): * Select parameter: InstaFind Variation Name (or instafind_variation_name). * Condition: is exactly (or exactly matches). * Value: loadInstafind. * Set membership duration as appropriate (e.g., 30 days, or the expected duration of your test). * Save the audience.

    Audience 2: Users without InstaFind Loaded * Click New audience > Create a custom audience again. * Audience name: A/B Test - InstaFind NOT Loaded (instafind_load_vs_noload_v1) * Under "Include users when:", add a new condition: * Select the event: instafind_ab_exposure. * Add parameter filter: * Select parameter: InstaFind Test ID (or instafind_test_id). * Condition: is exactly (or exactly matches). * Value: instafind_load_vs_noload_v1. * Add another parameter filter (click "Add condition"): * Select parameter: InstaFind Variation Name (or instafind_variation_name). * Condition: is exactly (or exactly matches). * Value: noLoadInstafind. * Set membership duration similarly. * Save the audience.

Analyzing Metrics

Once your custom dimensions are active and audiences are populating (allow 24-48 hours after setup):

  • Use Audiences in Reports: In your GA4 Reports (e.g., Engagement reports, Monetization reports), you can add comparisons using the audiences you created. Select "Add comparison," then choose your "A/B Test - InstaFind Loaded" audience for one segment and "A/B Test - InstaFind NOT Loaded" for another.
  • Compare Key Metrics: Analyze vital e-commerce metrics such as:
    • Conversion rates (overall and specific e-commerce events like purchase, add_to_cart).
    • Average order value.
    • Revenue per user.
    • Add-to-cart rates.
    • Checkout completion rates.
  • Use Explorations: For more detailed analysis, use the Explore section in GA4. You can build custom reports (e.g., Funnel exploration, Path exploration) and apply your A/B test audiences as segments.

Important Considerations

  • Define Success Metrics: Before starting the test, clearly define which e-commerce metrics are most important and what constitutes a "successful" outcome for a variation.
  • Test Duration & Sample Size: Run the test long enough to gather a statistically significant amount of data. The required duration depends on your website traffic. Consider using an A/B test duration calculator.
  • Data Processing Time: Remember the 24-48 hour delay for custom dimensions and audience data to become fully available in GA4.
  • testId Consistency: The testId (instafind_load_vs_noload_v1) used in your website configuration must exactly match the testId value used when creating GA4 audiences or filtering reports.
  • Accuracy of Base Analytics: This A/B test analysis relies on your existing GA4 e-commerce tracking being accurate. Ensure all relevant e-commerce events and conversions are correctly tracked.
  • External Factors: Be mindful of any other major website changes, marketing campaigns, or promotions running at the same time, as they could influence user behavior and potentially skew test results.

By following these steps, you'll be able to effectively measure the impact of loading the InstaFind application on your users' e-commerce behavior. If you have questions specific to InstaFind's functionality, please refer to your InstaFind contact. For GA4-specific questions, Google's official documentation is a valuable resource.