Skip to main content

A/B Testing with Magnolia

Posted by rah003 on February 11, 2010 at 1:01 AM PST

Is it possible to do an A/B testing with Magnolia? The question popups often in the discussions with potential Magnolia users from various e-business companies. This really depends on what you want to test. The design of whole site or just best position for an ad on a single page or layout of one page or ...

 If you want to do it on the site level the easiest (with current release - 4.2.3 at the time of writing) is to:

  • create 2 themes,
  • create 2 site definitions,
  • copy the website tree
  • assign original the Site1/Theme1 and
  • Site2/Theme2 to the copy and
  • use Apache to hide the subtrees and redirect half of incoming traffic to one site copy and other half to the other copy based on whatever criteria you want.

The main disadvantage of this approach is need to edit every change twice or copy the content from one subtree to another periodically or via observation. Another one is need for reverse proxy and mod_rewrite to translate all the links if you also want to hide the fact that each site is in different subtree.


Easier yet, to setup, but not necessarily to maintain, is to:

  • use just one copy of the content and 2 public instances,
  • on each of the public instances deploy different Site/Theme so depending on from which public instance you access you would see different design/layout
  • and again use the Apache or a load balancer to divide the incoming traffic between public instances. 

Since the Site configuration is determined from the home page of your site based on the Site configuration name in the STK, make sure to use the same name for Site configurations on both public instances and on the author.

The obvious disadvantage of this approach is that on the author you would see only one design and that if you accidentally activate the Site/Theme from the author instance you would override the setting on both publics, advantage is that the editing of the content happens only in one subtree.


If you want to test just different add or image, have those in DMS and use custom DAM handler that will load the image randomly from specified dms folder (and also have tracker in the page to see which loaded image generated more clicks). Similarly you can also create a paragraph that would display different text (I would probably store such text in specific data types in the data module).

This all it going to get even easier with 4.3 where you would be able to assign site configuration based on the domain name with which the site was accessed. You might want to check the previous entry on multi site support and Philipp's entry on configuration inheritance and short videos of the features attached to both the entries.



Hi Jan, good point, maybe

Hi Jan, good point, maybe there is anothere way: You can change the way a site recognize the theme. E.g. extending and overriding the method getTheme(), maybe looking into a possible set of preconfigured themes. In this case you can adopt a 50% random choice to provide the theme. You can go further, and adopt other "theme-resolution-strategies", providing as many solution your marketing team need. Of course, if you want only to change the theme. Different would be to change also the content, the template, and so on... Matteo

In theory yes. Although, the

In theory yes. Although, the site is not foreseen to be changed so it would not be possible without custom build of STK and the theme provided by site is also used at various places like to initialize DAM handlers so it might be bit difficult to change it on the fly. But it is indeed something we might try to provide as an option in one of the future releases.


But if you add (in current release) a "class" = "my.package.MySite" under Site Configuration, the content2bean loads another site implementation. So, it seems to be possible nowadays, isn't it?

 Yes that piece works since

 Yes that piece works since C2B is used to load the site, but as I mentioned for example DAM support is initialized only once in the init method and there might be other assumptions. Still you are right that if anyone wants to go this way, extending the Site is one of the options.