One-Way Mirrors in Online Dating

One-Way Mirrors in Online Dating: A Randomized Field Experiment
Ravi Bapna, Jui Ramaprasad, Galit Shmueli, Akhmed Umyarov1
1. Motivation and Background
According to the United States (US) Census2, 46% of the single population in the US uses online
dating to initiate and engage in the process of selecting a partner for reasons ranging from
finding companionship in a lonely world to marrying and conceiving children, and everything in
between. Finding the optimal dating and ultimately marriage partner is one of the most important
socio-economic decisions made by humans. Yet, such dating markets are fraught with frictions
and inefficiencies, often leading people to rely on choices made through happenstance – an
offhand referral, or perhaps a late night at the office (Paumgarten 2011). As is often the case, the
Internet not only replicates the physical-world processes of human interaction, but also extends
them, affording newer capabilities that are next to inconceivable in the physical world. These
capabilities range from algorithmic matching to predictive modeling of mate recommendations
(Gelles 2011), a science perfected for books and movies, now being deployed to what might be
the ultimate experience good (Frost et al. 2008). In this research we focus our attention on the
proverbial one-way mirror, an IT enabled feature unique to the online world of dating. This
feature, which we formally call semi-anonymous weak signaling a) allows individuals to view
profiles of potential mates anonymously, without leaving a trace, while retaining the ability to
view who visited their profiles, and b) gives the individual the choice of leaving their trace, a
weak signal, selectively, on the pages of selected users’ profiles they visit. It should be noted
that most online dating sites default to allowing their users to view the lists of those who visited
them, what we call bi-directional non-anonymous profile viewing. In this study, we seek to
determine in a causal manner, through a large-scale randomized field experiment conducted on
one of the largest online dating sites, whether semi-anonymous weak-signaling can affect the
matching levels, the matching efficiency, and perhaps, even more dramatically, the mate
preferences of individuals.
Our motivation for studying matching efficiency as an outcome is motivated by Piskorski (2012)
who documents that dating markets are fraught with frictions ranging from high search costs to
asymmetric societal norms that often lead to social failures. Akin to a market failure, which
implies an economic exchange that did not take place, but had it taken place would have made
everybody better off, a social failure is a human connection that should have taken place (in that
it would have increased the welfare of both sides), but for one reason or another did not. In the
context of heterosexual dating, these matching inefficiencies arise both due to physical
constraints of time and space, the costliness of the initial information acquisition, as well as
societal norms, such as those inhibiting women from making the first move (Piskorksi 2012).
Our focus on studying the effect of anonymity on possible changes to preference structures stems
from the emerging, but largely understudied, literature on the disinhibition effect of the Internet,
where a user’s behavior changes once she can behave anonymously. This online disinhibition
literature has its roots in social psychology (Joinson 1998, Suler 2004). Kling et al. (1999)
review social behavior on the Web, and state that “People say or write things under the cloak of
anonymity that they might not otherwise say or write.” Such anonymity induced changes have
1
2
Author names in alphabetical order
www.ft.com/intl/cms/s/2/f31cae04-b8ca-11e0-8206-00144feabdc0.html?#axzz1TbHiT1Xv
1
been observed in contexts ranging from adult film and books (Holmes et al. 1998) to, more
recently, ordering pizza (McDevitt 2012). The conceptualization of anonymity in these studies is
usually situated in an offline-online contrast, wherein an online transaction eliminates personal
interaction, freeing consumers to purchase products online that they would be uncomfortable
buying in person. McDevitt (2012) exploits the exogenous launch of a website by a physical
world pizza company to examine whether ordering pizza behind the cloak of a website (as
opposed to face-to-face at a physical store) drives same-customer differences in ordering
patterns. In doing this, he determines causally that by allowing anonymous purchases the Internet
potentially lowers consumers' inhibitions and, consequently, now duly emboldened, the average
consumer chooses a different, more long-tail and guilt-laden calorie-heavy set of pizzas. While
such arguments imply that IT may enable an increase in the diversity in revealed preferences,
with forces such at the disinhibition effect at play, it may also homogenize, or decrease the
diversity in revealed preferences (Fleder and Hosanagar 2009). The latter may occur given that
recommender systems, such as those recommending potential matches on online dating sites, are
often designed so that users are directed to recommendations based on their past behavior.
Additionally, with a larger online market to choose from and no inhibitions in browsing
extensively, individuals may search more and be more likely to find people more similar to them.
We identify our effect causally using a large-scale randomized trial, similar in spirit to Aral and
Walker (2011) and Bapna and Umyarov (2012), in partnership with one of the largest online
dating sites in the world. Our experiment involves treating a randomly chosen subset of 10,000
bi-directional non-anonymous users with the ability to use semi-anonymous weak-signaling for a
month. This puts our work in contrast to extant online disinhibition literature that relies on
exploiting natural experiments based on online-offline channel differences to identify the
anonymity effect. In the context of dating, it is hard to imagine large-scale anonymous viewing
or ‘checking out’ of target mates in a singles bar, without the counter-party being unaware of the
inspection.
We seek to answer the following research questions:
• Are there any significant differences between the propensities of men and women in
making the first move?
• Does semi-anonymous weak-signaling improve matching levels and efficiency?
o Given known gender asymmetries in mating markets (Fisman et al. 2006), does
the effect of this feature differ across genders?
• Are women more likely to avail the (IT induced) ability to leave a weak signal, thereby
overcoming known social barriers that limit them from making the first move?
• Does semi-anonymity affect individuals’ matching preferences leading them to exhibit
different revealed mate preferences?
Our work complements the economics literature devoted to measurement of mate preferences
(Fisman et al 2006, Hitsch et al 2010), a topic also of interest to scholars in sociology and
psychology (Buss 1995). Similar to Hitsch et al. (2010), our measurement of mate preferences
relies on data from one of the largest online dating sites in business. Where we depart from this
stream of literature is in our use of a randomized treatment to identify the effect of a unique IT
enabled artifact — anonymity—that could potentially alter individuals’ mate preferences and
2
increase their matching efficiency. Both these effects, if significant, will lead to welfare gains in
mating markets and will result in lower levels of social failures (Piskorski 2012).
At the time of writing, we are in the process of calibrating our experiment using secondary data
from the same online dating site, wherein we simulate a quasi-treatment using differences in
behavior between those who buy a premium package that enables them to browse semianonymously with those who do not. These results do show evidence that those with semianonymous browsing abilities have a larger number of matches and match more efficiently than
those without, supporting the notion that IT-enabled features may reduce social failures. We
fully realize that these results are confounded due to the other features included in premium
membership (such as advanced premium search features) and self-selection of premium users.
Yet, similar to Bapna and Umyarov (2012), the quasi-experiment helps motivate and calibrate
our randomized field trial, the results of which we will present at WISE, given the opportunity.
We have scheduled the full randomized trial to run for a month starting October 1, 2012, giving
us enough time to analyze and present our results at WISE 2012.
2. Experimental Design
Consider a focal user on monCherie.com (name changed due to our privacy protection
agreement), looking for a date, who we will call the target. The default setting on
monCherie.com is that of bi-directional non-anonymous profile viewing. If the focal user visits
the profile of the target, the target knows through her ‘visitor page’ that the focal user checked
her page out. Prior literature suggests that known societal norms, such as women’s reluctance to
make the first move (Piskorski 2012), as well as other social inhibitions (Joinson 1998, Suler
2004) will lead to frictions and resulting inefficiencies in matching in the bi-directional nonanonymous setting. The focal user might search sub-optimally, either in quantity, or via an
inhibited set of mate preferences, and as a result have weaker matching outcomes. Based on an
adaptation of Bapna and Umyarov (2012), our field experiment relies on lowering the social
stigma and/or inhibition by randomly assigning (through a free one month gift) the feature of
semi-anonymous weak-signaling to a randomly selected group of 10,000 monCherie.com users.
We then compare the matching levels and efficiency as well as the distribution of revealed mate
preferences to a control group of another 10,000 randomly selected users who have similar
observable and unobservable characteristics to the treatment group. It should be noted that
monCherie.com currently allows users to purchase a premium membership. This premium
membership includes the semi-anonymous browsing feature along with several other premium
features, such as advanced search and filtering, which could generate similar outcomes
(McDevitt 2012) to the effect of semi-anonymity. Thus, if we purely adopted Bapna and
Umyarov (2012) and used the gifting of premium membership (in its whole) our results would be
confounded by these other additional features that improve search and filtering. In our
experiment, we worked with our partner research site to eliminate the issues of confounds by
only gifting the semi-anonymous weak-signaling feature. Needless to say, the random
assignment of the treatment rules out myriad problems of endogeneity and alternative
explanations that would confound any analysis of such a question based on observational data.
We expect that semi-anonymous weak-signaling, i.e. the ability not to leave a trace or to
selectively leave a trace, can have a significant positive impact in overcoming some known
social failures by lowering inhibitions that can cause social failures and increasing both the
3
absolute number of matches and matching efficiency per message sent. We are also interested in
understanding the impact of semi-anonymity on preference alteration, looking at whether the
uninhibited individuals will reveal a latent, possibly more diverse, set of mate preferences.
3. Empirical Regularities
As mentioned earlier, prior to running the field experiment, we examine a set of secondary data
from our research site that allows us to characterize some empirical regularities, answer a subset
of our research questions and derive some initial insights based on a proxy, the full premium
membership adoption, for our treatment. This dataset includes basic demographic, as well as
individual-level viewing and messaging behavior (but not the actual content of the messages) of
100,000 users of the site over a period of 30 days. Our analysis here is based on a set of basic
demographics that were made available at the time of submission. We will have a full set of
demographic information for a period before, during and after our field experiment, which will
allow us not only to understand our outcome of number of matches and matching efficiency
better, but also to analyze preference diversification across a large number of dimensions
including attractiveness, income, physical characteristics, personal habits (e.g., smoking), etc.
Summary statistics (Table 1) indicate that, on average, men initiate more messages (7.3 vs. 1.5).
In particular, women are close to 5 times less likely to initiate a conversation than men, reflecting
the gender based social norm that contributes to the occurrence of social failures (Piskorski
2012). If women, who would have benefitted from initiating contact with men, do not do so, they
relegate themselves to be responders to moves made by men, who might not be the ones most
closely aligned with their preferences. This could also influence and explain why men, who are
aware of this tendency of women, message more, inundating women, who perhaps would then
resort to sub-optimal heuristics for whom to respond to. At the same time, women on average
have a higher number of matches per person than men (2.95 vs. 2.18). Men are also more likely
to use the semi-anonymous browsing (proxied by the full premium feature) than women. This
difference was particularly acute for a sub-sample of men who are who are at the extreme high
end of the messaging distribution, the ‘players’ who are five times more likely to be premium
users relative to the average male. The regression results presented in the next section explore
these relationships in more detail.
4. Results
At the time of writing, as discussed, we have a proxy for the treatment, which is the feature that
allows users to pay a premium and buy semi-anonymous weak signaling, as well as a slew of
other features such as improved search and receiving no advertising on the site. This is what we
label as proxyTreatment to present the results below. Following our detailed conversations with
the senior management of our research site, we adopted the industry standard definition of a
match used by online dating sites, including ours:
“A match occurs when three messages are exchanged between two users: a sender to a
target, the target back to the sender, and another message back to the target.”
We find that both the number of matches and matching efficiency increase in the proxy for semianonymous weak signaling. We plan on measuring the disinhibiting impact of semi-anonymity
on mate preferences, as well as delve deeper into the adoption of weak signaling especially by
4
women using the (more detailed and causal) data from the field experiment. To reiterate, we
recognize that there are confounding features and selection issues in proxyTreatment, but if we
have the opportunity to present at WISE, we will have results from the randomized experiment.
Tables 2a and 2b present the results from our analysis that examine the relationship of variable
proxyTreatment with both the count of matches per person and a person’s matching efficiency
respectively. Given that the count of matches is discrete and non-negative with a large number of
zeroes, we use a zero-inflated Poisson model. The results of this model suggest that users who
received the proxy ‘treatment’, which includes the semi-anonymity weak-signaling feature, have
a larger number of matches compared to their counterparts who do not. The interaction effects
demonstrate an average increase of 50% in matches per person for men and 30% for women who
received the ‘treatment’ compared to those who did not, holding age constant. Matches per
person also decrease in Age but at a lower rate for ‘treated’ group.
We define matching efficiency as the ratio of the number of matches made by a person to the
total number of messages sent out by that person. In Table 2b we see that matching efficiency is
on average higher for those receiving the proxy ‘treatment’ but that this relationship is moderated
by gender, where men benefit from the treatment but not women. Efficiency in Age behaves
similarly to that observed for matches per person.
In the next phase of our analysis, armed with richer demographic data that we will receive as a
part of our randomized field experiment, our plan is to replace the proxy treatment with the real
randomized treatment and also examine the effect of disinhibition on the distribution of
preferences. Under the cloak of semi-anonymity will individuals relax same-race preferences, or
be more exploratory with respect to orientation, or tradeoff an attribute such as attractiveness
with age, moving perhaps to more substitutable preferences as exhibited by a less convex
indifference curve?
Discussion
Our early results provide motivation for running a large-scale randomized field experiment in
one of the world’s foremost online dating sites to causally examine the relationship we have
explored above. Thus far, we have established and quantified the extent to which women are less
likely to make the first move in online dating, a likely cause for social failures (Piskorski 2012).
We also find that our proxy measure of semi-anonymous weak-signaling does indeed mitigate
social failures and increase the likelihood of a match over a shorter period of time. The online
dating context is unique and interesting in many ways. At one level it allows us to understand
human behavior around a fundamental social, economic and emotional decision. Further,
matching two individuals is a complex task, relative to, say, matching a buyer with a product in
product markets. In dating there are two sets of individual preferences that have to be taken into
account in order to produce a successful match. Matching two humans is not only something that
applies to dating and marriage, but also to new models of distributed work and crowdsourcing.
Thus, we expect this study and our associated methodology to be the basis of a stream of work
on how the Internet and social media are changing some of the fundamental activities we carry
out as humans.
5
References: Are available at https://docs.google.com/document/d/1Q6J8LSNRuNP0ABGe4jndjlLB2dTfCFg_pPdetbESBQ/edit
Table 1. Summary Statistics
Gender N Obs
Variable
Mean
Std Dev Bottom 25%
Median
Top 25%
Min
Max
Male
56534
Age
Semi-anonymity
Orientation
Msg initiated
Msg sent
Msg received
Views sent
Views received
Matches
29.62
0.0077
1.13
7.30
25.68
15.41
205.35
8.03
2.18
10.33
0.0874
0.41
37.79
93.20
52.44
648.28
21.29
6.76
22.67
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
26.75
0.00
1.00
0.00
0.00
1.00
16.00
0.00
0.00
33.42
0.00
1.00
2.00
12.00
7.00
181.00
8.00
1.00
13.17
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
110.67
1.00
3.00
2637.00
4054.00
3039.00
97004.00
1355.00
259.00
Female
43466
Age
Semi-anonymity
Orientation
Msg initiated
Msg sent
Msg received
Views sent
Views received
Matches
29.46
0.0043
1.23
1.55
24.44
40.78
104.54
19.42
2.95
10.48
0.0658
0.60
7.42
78.98
102.69
248.62
40.74
7.97
22.09
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
26.66
0.00
1.00
0.00
0.00
2.00
7.00
2.00
0.00
33.67
0.00
1.00
0.00
16.00
38.00
105.00
23.00
2.00
13.17
0.00
1.00
0.00
0.00
0.00
0.00
0.00
0.00
110.67
1.00
3.00
347.00
5058.00
4251.00
8454.00
1312.00
332.00
Table 2a: Zero Inflated Poisson Model for Count of Matches per Person
(data are limited to those who sent at least one message; the zero model is omitted for brevity)
Parameter
Estimate
Std Err
p-value
Estimate
Std Err
p-value
Intercept
1.9317
0.0029
<.0001
1.9303
0.0029
<.0001
proxyTreatment
0.3823
0.0135
<.0001
0.4038
0.0172
<.0001
Gender (1=female)
0.1753
0.004
<.0001
0.1778
0.0040
<.0001
-0.0081
0.0002
<.0001
-0.0083
0.0002
<.0001
-0.1372
0.0290
<.0001
0.0054
0.0013
<.0001
Age (normalized)
proxyTreatment * Gender
proxyTreatment * Age
Table 2b: Linear Regression Model for Matching Efficiency per Person
(data are limited to those who sent at least one message)
Parameter
Estimate
Std Err
p-value
Estimate
Std Err
p-value
Intercept
0.0966
0.0006
<0.0001
0.096450
0.0006
<0.0001
proxyTreatment
0.0123
0.0040
0.0018
0.016620
0.0050
0.0009
Gender (1=female)
0.0494
0.0009
<0.0001
0.049731
0.0010
<0.0001
-.00003
0.00005
0.5436
-.000037
0.00005
0.4337
-.02475
0.0087
0.0043
0.0007
0.0004
0.0839
Age (normalized)
proxyTreatment * Gender
proxyTreatment * Age
6