A/B testing for publishers: 6 ideas for programmatic and UX optimization

A/B testing is widely known among website managers as a method to test different variables on a website. However, for publishers working with programmatic advertising, it goes beyond that.

When executed correctly, A/B testing can drive up session RPM, user lifetime value and deliver crucial incremental revenue. In this article, we will explore the essential steps for conducting A/B tests, the tools required, and present five valuable A/B testing ideas that can revolutionize programmatic yield for publishers. 

Additionally, we'll touch upon some of the limitations of A/B testing in the programmatic world.

How to set up A/B testing for publishers

Before delving into A/B testing and A/B/n testing  ideas, it's essential to understand the key steps in the testing process:

What to test

Limit each test to one specific element to ensure clarity. For instance, if you're testing different ad sizes, focus on making the changes to your chosen sizes rather than altering everything.

Test setup

Consider how changing one element might impact others. Ensure that you adjust your setup for SSP placements and determine if you need to modify the SSP's settings.

Also, make sure you have a proper A/B testing tool tailored for programmatic. Many publishers don’t test their ad stack for lack of development resources and knowledge. However, with the right tools, you can A/B test your website with barely no coding involved. 

We’ll delve into A/B testing tools for publishers later in this article. 

Tracking metrics

Define the key performance indicators (KPIs) for your test. While Session RPM is often the primary KPI, you may want to delve deeper into effective RPMs, bid rates, and assess the significance of any revenue increase.

In addition to the primary KPI, keep an eye on secondary metrics like viewability, AdX share of voice, session duration, pageviews per session, and Core Web Vitals.

6 A/B test ideas for publishers

Now, let's delve into five A/B testing ideas specifically tailored for publishers:

1. Testing different layouts

Structural layout changes are common experiments for publishers. The goal is to optimize where ads should be placed and how they behave to maximize user engagement and revenue. 

For example, in the past, publishers placed most ads at the top of the page, known as "above the fold." However, this led to lower viewability because users often scroll past these ads. A/B tests can identify optimal ad placements, such as in the middle of content, to improve viewability and engagement.

2. Quantity of ads on the pages

Finding the ideal number of ads on a page is critical for maximizing revenue while ensuring a positive user experience. Too few ads can lead to insufficient revenue generation, while too many ads may drive users away. A/B testing can help discover the right balance in the quantity of ads to optimize lifetime value.

3. Personalization for diverse user segments

Users differ in behavior and sources of traffic, which can impact revenue. Personalizing ad experiences for distinct user segments can optimize lifetime value without compromising user experience.

For example, tailoring ad content and placements for direct visitors, social media referrals, and SEO traffic can enhance engagement and increase revenue from each segment. Another good idea is to test different versions for new visitors versus returning visitors, since they tend to have different navigation behaviors. 

4. Demand-based tests

Different demand sources, like SSPs or ad networks, can impact session RPM, revenue, and user experience. Evaluating these sources is essential for maximizing programmatic yield. 

For example, you can test the incremental value of adding a new SSP compared to the current setup and determining how different demand providers affect session RPM, revenue, and user experience.

5. Video player and ad provider testing

Video players and ad tech providers play a crucial role in session RPM and user experience. Analyzing their performance is vital for optimizing ad revenue and user engagement. 

Conduct A/B tests to compare different video players in terms of session RPM, user experience, and other factors. This can help identify which video player choice is most profitable.

6. Ad stack setup tests

Ad delivery configurations, such as prefetching, lazy loading, and timeout settings, impact ad loading time and user experience. It's crucial to find the right setup that optimizes revenue and viewability. 

Make sure to test various aspects of the ad stack's behavior, including lazy loading and timeout settings, and adjust settings to match user behavior and layout to ensure users see ads without delays.

Tools for publisher A/B testing

Selecting the right tools for A/B testing is crucial. While many A/B testing tools are designed for e-commerce and UX testing, such as Optimizely and VWO, testing changes in the ad stack can be challenging. 

Most A/B test software is designed for digital marketers, making it difficult for publishers to track ad clicks from ad networks or track variables such as video providers, ad sizes, and more. 

Consider using Google Analytics 4 (GA4) to complement tracking for certain tests. Also, ensure you have the proper Google Ad Manager (GAM) key-values set.  Google also used to have its own A/B testing tool, Google Optimize, which was discontinued in 2023.  Learn more about A/B testing tools in our post about  Google Optimize alternatives.

Luckily, currently there are powerful A/B testing solutions designed for publishers, where you can set up tests in just a few minutes and check results in real-time.

Yield Manager for streamlined A/B testing 

Assertive Yield's all-in-one solution, Yield Manager, simplifies the A/B testing process for publishers. It provides real-time data, session-level insights, and granular reporting on various dimensions. 

Yield Manager allows publishers to experiment with different ad sizes, placements, and dynamic configurations. It offers real-time data on multiple metrics, including impressions, winning bidders, unit performance, session-level information, device performance, browser performance, and country performance, enabling granular reporting. 

You can also test different ad sizes, ad providers and SSPs against each other and analyse revenue, session RPM, viewability, and more.

This tool empowers your ad operations team to make data-driven decisions based on the comprehensive insights it provides, ensuring that A/B testing leads to optimized programmatic yield.

A/B test your website like a pro

A/B testing is a publisher's best friend. It empowers you to fine-tune your ad stack, maximize revenue, and enhance the user experience. To harness the full potential of A/B testing, make sure you have the right tools.

To explore the endless possibilities of A/B testing and optimize your programmatic revenue, start your journey with Assertive Yield for free. If you want more information about our all-in-one programmatic solution, connect with us to learn more about our products and how they can elevate your publishing endeavors.