Aaron Louie – Blog

(continued from Part 1 – Evolution and Design)

Remember the old metaphysical question: “if a tree falls in the forest, and no one’s around to hear it, does it make a sound?”

Well, here’s an equally unanswerable question: “if someone designs a web site, and no one’s around to measure its performance, was it a success?” If you designed or manage that web site, your job depends on being able to answer that question. However, many user experience designers have historically avoided the question with the enigmatic and frustrating non-answer: “it depends.” Well, the time is overdue for us to take ownership of measuring the performance of our designs. We can start by understanding what performance means.

“Performance” implies that someone is observing the performance – the audience. And there’s an implication of critique, or measurement of the performance. In order to guide the evolution of the design to fulfill some strategy, the design’s performance must be measured.

Performance implies an audience.

The performance of any creative effort is subject to interpretation by the people observing the performance, and those people are called “the audience”. To reiterate the argument I made in Part 1, user experience designers aren’t creating art for self-expression. We’re creating art for use. My design is made to perform a particular function, made useful by having an audience that values the design’s function.

If I design a web application behind a firewall that no one ever finds or uses, what have I accomplished, other than self-expression? By definition, user experience design requires an other (the user) to experience the work. My design must be used in order to fulfill its destiny.

Performance implies measurement.

The audience is neither monolithic nor homogeneous. Each audience member will have their own unique set of values, motivations, cultural filters, and personality traits, which will affect how they view the performance. They will each bring their own criteria by which they will judge the performance, whether they know it or not.

In the movie theater, I might judge a performance based on the quality of the story, the set design, or the acting. A film critic might judge the character development, the narrative structure, or the impact on the world of filmmaking. A marketer might focus on the product placements, the brand opportunities, or the potential for a companion video game and a line of toys. Conversely, my mom might judge the performance on how comfortable the seats were, the temperature of the air conditioning, or the cleanliness of the restrooms.

Thankfully, there are some criteria that are common amongst audience members. We can group people together who have common attributes and common ways of judging performance. In the digital marketing business, they are called segments. In user experience design, these groups are called personas (I’ll save the discussion of the difference between marketing segments and personas for another day). For each audience segment or persona, the user experience designer can identify which criteria they will design for.

Who you perform for determines what you measure.

By choosing an audience to please, the designer must deal with the accompanying criteria that audience uses to judge performance. The criteria you choose to measure determines how you will design for performance.

For a typical web site, here are just a few examples of the different types of audiences and their different measures of performance:

You might notice that these last two are more cross-disciplinary and cross-audience. This is because they’re based on values, rather than on roles. Value-centered design (see Jess McMullin’s Boxes & Arrows article) and value-sensitive design (see Batya Friedman’s work at UW) are approaches to design that define success by how well the design aligns to commonly-held definitions of worth or ethics, respectively (I’m oversimplifying).

Some of the above approaches are better than others for different contexts. A user experience designer might gravitate toward a completely user-centered approach, but their job might depend on business- or politics-centered measurements. For a company whose entire business is conducted through their web site, depending solely on user-centered measurements of success might have disastrous consequences. Especially if the web server is crashing due to some poorly tested code.

As an online user experience designer, I must deliver a design that the users love, but that is also on time, within budget, functional, revenue-generating, and adheres to the values of the company and its customers. I also need to create an experience that will evolve as those values and the market landscape change over time.

How you perform determines how you EVOLVE.

Which brings us back around to evolution. Adapting a design to the changing environment will ensure its survival. Even if you don’t intentionally vary the design, your site or product WILL change over time. HOW it changes depends on how you measure its performance. What do you value? The user? The money? The environment? Design for the audience(s) you must satisfy to survive. Measure what matters to that audience. Keep what works, toss out what doesn’t, explore new approaches, measure again, and repeat.

Next up: Part 3 – Process & Deliverables