Lessons From Google’s Data-Driven Design Experiments

A recent Guardian article (http://www.theguardian.com/technology/2014/feb/05/why-google-engineers-designers) describes Google’s successful use of data to determine the right shade of blue for hyperlinks in the Gmail product, resulting in a $200M in additional revenue per year. According to Dan Cobley:

In the world of the hippo (highest paid person’s opinion), you ask the chief designer or the marketing director to pick a blue and that’s the solution. In the world of data you can run experiments to find the right answer.

This radical approach seems to be a direct challenge to the authority and expertise of designers. The article highlights that this experiment with data-driven design optimisation led to the resignation of Doug Bowman, Google’s then Visual Deign Lead. A defense of the authority of designers comes in this article from Google Ventures http://www.gv.com/lib/design-instinct-vs-data, which seeks to find a role for “design instinct” in making particular design decisions.

In reality, the data vs. design instinct opposition is a false one. It is impractical to propose that all design decisions are determined by A/B testing in the way that the optimal link colour was chosen. It is difficult to imagine how such a design process could work in practice. Therefore, designers will always have a role to play in making decisions about the way products look. But designers should not see validation of design choices by data as diminishing their authority. According to Daniel Kahneman (c.f. Thinking, Fast and Slow), all expert judgements (regardless of profession) should be subject to empirical validation to prevent errors caused by flawed heuristics or the “overconfidence effect” (http://en.wikipedia.org/wiki/Overconfidence_effect)

The Google Ventures article’s defense of design instinct is at best half-hearted as it also highlights the danger of over-reliance on instinct alone and cites the Segway and the Apple round USB mouse as examples of where this goes wrong. This is because designers are not able to sufficiently anticipate the needs of real end-users. The author suggests that designers can improve their “instincts” by observing real users and conducting research – all of which weaken the case for pure design instinct.

The real conclusion that should be drawn is that while data is essential to validate the expert judgements of designers and avoid the pitfalls of overconfidence, it is not practical to suggest that a data-driven process could entirely supplant the judgement of design professionals. Design professionals should not see data as an affront to their “instincts”, but as a valuable tool to improve their work – just as professionals do in other practices.