D-Optimal Design for Network A/B Testing

A/B testing refers to the statistical procedure of experimental design and analysis to compare two treatments, A and B, applied to different testing subjects. It is widely used by technology companies such as Facebook, LinkedIn, and Netflix, to compare different algorithms, web-designs, and other online products and services. The subjects participating in these online A/B testing experiments are users who are connected in different scales of social networks. Two connected subjects are similar in terms of their social behaviors, education and financial background, and other demographic aspects. Hence, it is only natural to assume that their reactions to online products and services are related to their network adjacency. In this paper, we propose to use the conditional auto-regressive model to present the network structure and include the network effects in the estimation and inference of the treatment effect. A D-optimal design criterion is developed based on the proposed model. Mixed integer programming formulations are developed to obtain the D-optimal designs. The effectiveness of the proposed method is shown through numerical results with synthetic networks and real social networks.

A/B测试是指一种实验设计和分析的统计过程，用于比较应用于不同测试对象的A和B两种处理方式。它被脸书、领英和网飞等科技公司广泛使用，以比较不同的算法、网页设计以及其他在线产品和服务。参与这些在线A/B测试实验的对象是在不同规模社交网络中相互关联的用户。两个相互关联的对象在社会行为、教育和财务背景以及其他人口统计学方面具有相似性。因此，很自然地假设他们对在线产品和服务的反应与他们的网络邻接性有关。在本文中，我们提议使用条件自回归模型来呈现网络结构，并将网络效应纳入处理效应的估计和推断中。基于所提出的模型，制定了D - 最优设计准则。开发了混合整数规划公式以获得D - 最优设计。通过合成网络和真实社交网络的数值结果展示了所提方法的有效性。