Extra! Extra!

Every company wants better insights into their customers – so how do you get to the holy grail of customer insights? Ethnographic research is one best practice to getting to customer insights. According to a study (download) Data Driven Design: Digital Experience Teams Are Focused On Website Metrics That Don’t Demonstrate Business Value (April 2012), a commissioned study conducted by Forrester consulting on behalf of Extractable, the #1 tool that companies are not using but that they believe could provide new information they do not have today is ethnography.

Despite this recognition of the value of ethnography, interviews with customers are used as a replacement method for ethnographic research because of the expense associated with field research. It’s important to understand the differences between interviews and ethnographic research to understand what kind of customer insights you are missing out on if you only do interviews with customers.

• What is an Interview?
  • Duration: 30 minutes to several hours
  • Format: Sitting down with a customer and talking through a pre-made list of questions

• What is Ethnographic Research?
  • Duration: 2 to 6 hours
  • Format: Observing customer at work in their environment, occasionally interrupting the customer to ask clarifying questions that come as a result of what you observe

• Why Interviews are not Sufficient
  • Interviews don’t tell you what you don’t know you don’t know. In other words, interviews are designed by a moderator around certain topics. If a topic is not covered in the interview but is important to the customer you are interviewing, you may never ever know about its existence. Asking open questions can help to mitigate this problem, but it does not solve it.
  • People leave out what they consider routine details when recounting past events. People are good at telling you what they did in the past but not as good at telling you what they desire in the future. However, while customers can give you feedback about past experiences, they will leave our details that don’t seem important to them. These details may indeed be the new seeds of innovation.
  • Interviews can restrict what customers share due to political or relational concerns. Customers sometimes are hesitant to give negative feedback in the presence of a company they have a close relationship with. Customers can sometimes better demonstrate their frustration than articulate it in words.
  • Interviews do not capture the context of the interaction. Customers can recount past experiences, but it is hard to understand the environment in which these events happened. What does a customer’s desk look like? Who does a customer talk to every day? When a customer receives a package from your company, how does that package come to them? An interview cannot answer these questions.
• Why Your Company can Benefit From Ethnographic Research
  • Ethnographic research offers insight into context and culture. Field research requires that the design team is on-site at the place where the customers are doing their work. This allows us to understand what physical and cultural implications affect the workplace. It allows the design team to see how customers model their workplace for optimal work flow. We see what other tools the customers use in their workplace in addition to the website and how the website is used in conjunction with other tools.
  • Ethnographic research helps the design team understand what we don’t know. Because of the more exploratory nature of ethnographic research, it helps us understand what work or needs are not currently supported on the website. This identifies opportunities to surpass competition and do something that no one else is doing in the market today.
  • Ethnographic research provides guidance for emotional design. Design and usability are fundamentally about ensuring that a customer can accomplish the tasks they need to on the site. However, emotional design holds that emotions also have a crucial role in the human ability to understand the world and learn new things. By understanding how customers setup their own workplace, we can use metaphors to setup the website in the same way to emotionally resonate with them. We want to build on the natural way that people work.
  • Ethnographic research expresses the whole process, no just bits and pieces. Being in context with the customers doing work, we will be able to see the work as a continuous flow of tasks instead of discrete and artificial tasks. In interviews, customers tend to abstract information about their work. We want to know the concrete details of their work, and we can find that out by watching them do their work.
  • Ethnographic research informs beyond the website. Because of the thorough exploration of how customers do their work and what culture they do their work in, the data from ethnographic research can inform businesses processes and strategies beyond the digital strategy.

The next time your company is evaluating strategies for identifying customer insights, consider the value that ethnography brings to the table above and beyond customer interviews.

A design process that includes user testing is one of the most powerful and effective ways to collect qualitative data, which can dramatically improve business. In a new study (download), Data Driven Design: Digital Experience Teams Are Focused On Website Metrics That Don’t Demonstrate Business Value (April 2012), a commissioned study conducted by Forrester consulting on behalf of Extractable, almost half (47%) of participants didn’t conduct Usability Testing because of a lack of Infrastructure, skills, or it being too expensive.

User testing need not be elaborate or even expensive to get crucial results for businesses. It is simply a matter of baking it into the process and keeping focused on business outcome.

User testing helps explain metrics to tell a complete story

Focused user testing begins with the qualitative method of finding the biggest areas of opportunity on sites through analytics and metrics. At Extractable we often look to analytics and metrics to identify areas worth looking into more closely. The qualitative methods of being able to explain them can significantly affect business outcome.

Combining quantifiable with qualitative approaches yield the highest return by telling a complete story of why something is happening. Evaluating opportunities with this combined effort of understanding complex metrics is crucial for creating successful strategies for design with the biggest impact on business outcome. For example, Analytics may be telling you that visitors are bouncing from the homepage after only 5 seconds. A trained eye with a qualitative data skill set would be able to dive deeper into this identified area of opportunity. Investigation of many different pieces of information such as the product offered is crucial.

It leads to really crucial theories such as “The business proposition and offering is presented at such a high level that it pertains to everyone but essentially talks to no one.” From this, underlying concepts develop into specific design solutions aimed at solving problems and challenges that directly affect business outcome. In this example, a concept could be “Users are unable to identify and relate to the product and therefore abandon the site.” As design takes this further and develops a prototype addressing this issue, the theory can be proven by seeing positive patterns form across multiple users with various quotes, completion rates, etc.

Customers validate solutions and give user centric insights in testing sessions.

Listening to customers in usability testing can help to solve particular problems but more importantly can validate solutions and make designs better. In user testing, participants are encouraged to talk out loud as they interact with your site, ultimately giving you insights into their mental models through the language they use to describe it.

Through this comes confidence that you are building features that are wanted, needed and will be used. Getting at the heart of the matter of course is important to enhance your experience and induce a positive emotive response in people. Users notice many things in web experiences and directly relate them as a reflection of the company itself, making a big impact.

Baking User testing into the design process keeps it closely tied to project goals

Writing scripts and moderating sessions as part of the design process, keeps user-testing results focused and relative as well as keeping extraneous information to a minimum. This approach to testing targets specific areas of the site that related directly to the business outcome. Whether it be measuring engagement or gaining insights into expectations, this type of qualitative information is essential to determining improvements with conversion rates and other business metrics.

The benefits of stopping a project before it goes into production in order to do user testing are significantly out weighed. If, for example, you find out early that no one is seeing a horizontal scrollbar and missing hidden content in over half of the interface, you can tweak this bit and get it right before it goes into development instead of doubling development time which is much more costly.

Sometimes tweaks aren’t so clearly isolated and can create a domino effect on the entire Information Architecture ecosystem maximizing one feature and impacting another negatively. Sketching a few solutions while taking into considerations the tradeoffs for each can also be better understood in iterative testing. Inviting participants to evaluate the implications of various changes objectively assures the continued improvement based on real customer experience.

Summary

In order to keep user testing costs low and benefits high, focus on combining quantitative and qualitative methods, validate with customer insights while keeping it a part of the design process and closely tied to business outcome.

There are various ways companies learn about their customers. They fall generally into two kind of activities: quantitative and qualitative activities. Quantitative activities focus on measurable behaviors and include:

• Web-site analytics
• Sales numbers
• CRM reporting
• Surveys

Qualitative activities focus on understanding why these behaviors happen and include:

• User interviews
• Focus Groups
• Eye-tracking
• Ethnography
• Usability Testing

According to an independent study (download) Data Driven Design: Digital Experience Teams Are Focused On Website Metrics That Don’t Demonstrate Business Value (April 2012), a commissioned study conducted by Forrester consulting on behalf of Extractable, “Many firms report that they don’t use techniques like card sorting (46%), ethnography (44%), website session recording and playback tools (44%), eye tracking studies (44%), and usability lab testing (30%) (see Figure 8). Instead, they focus on techniques like search engine search term analysis (75%), behavioral web analytics (72%), and online survey and feedback tools (67%).”

At Extractable, we believe that the best customer insights come from the intersection of the quantitative data and qualitative data. Why is that?

At a high level, these are the differences in strengths between quantitative data and qualitative data:

How can the strengths of these different methods reinforce each other and yield better results?

• Quantitative identifies areas of opportunity for qualitative. With a large sample size, quantitative can detect anomalies and patterns in customer behaviors. With this knowledge, qualitative analysis can focus on identifying why this behavior is happening at the core.  For example, in a recent project, analysis of web analytics showed that customers were visiting two pages back and forth in succession on the website. The two webpages were two different product pages. The short-term solution was to combine the information on these two pages. However, the question remained: what was driving people to want to see this information together? This was a question that we took into qualitative methods.
• Qualitative explains quantitative. Qualitative data focuses on why the customer is behaving in a certain way. Once fundamental needs are discovered through qualitative methods, longer-term solutions can be proposed and tested. In our example, the long-term solution that qualitative research set out to find was why people were connecting these two product pages and what information in particular they were associating. Through qualitative usability testing observing prospective customers using the website, we found that fundamentally people did not understand the differences between the product offerings. A better explanation of the product offerings was needed at a higher level of the website, like the homepage.
• Quantitative validates qualitative. Because qualitative is focused on a small representative sample set, quantitative can give the number results to quantify the success of a change. Quantitative data can help companies understand how changes have affected other customer behavior. In our example, after we added a better comparison and overview explanation of the product offerings of the company on the homepage, we saw less switching between pages and a more focused path that allowed the engaged visitor to get one level deeper into the site on average.
• Quantitative allows for maintenance and optimization. Quantitative data can continue to monitor and optimize customer behavior in real-time to understand how customer behavior is changing and to understand when qualitative testing would be useful. In our example, we continued to monitor the company’s analytics. Over time as the company’s industry became better known, the need for explanation of products became less important. The new challenge was how to differentiate its products from other competitors.

There are many reasons companies don’t pursue qualitative data. Chief among them are the large expense and the specialized skills needed. Because qualitative data requires sitting down with customers and listening to them as individuals, it can be time-consuming to moderate. The moderator must be sure not to lead customers to any conclusions but impartially guide the session. In addition, individuals’ feedback must be analyzed for patterns in order to understand trends about how customers think.

However, there is no substitute for getting inside your customers’ minds. Qualitative insights allow us to not only create a website that is usable and meets a customer’s short-term goals, but a web experience that resonates with customers at a deeper emotional level and that meets their long-term goals for how it should feel to do fulfill the needs that your business offerings meet. Dig deeper and enhance your quantitative practices with qualitative methods.

Often we see that corporate websites may not be performing because the management structure for the web team is incorrect.  Supporting these experiences were two core data mistakes were uncovered in the research study we just undertook with Forrester (download it in full, here).

The two key data mistakes uncovered in the study were:

1. The most common data points that websites are measured on are Traffic (66%), Brand Recognition (47%),  and Time-on-Site (46%)
2. The most common data points that web teams are measured on are project timelines (65%) and project budgets (64%)

As we started to look at this data we were puzzled.  Here are some reasons why these data points are flawed:

• Traffic:  If a site manager spends half of the annual budget on driving unqualified/non-converting traffic to their site, chances are that the budget is wasted.  This happens a lot more often than you think (as the research proves).
• Brand Recognition: In May of 2010 in the United States, British Petroleum had excellent brand recognition.  The problem was that the brand was perceived as the company responsible for gulf oil spill.  If brand recognition is high, but the positive attributes of the brand are not understood by customers – nothing helpful has been achieved.
• Time-On-Site:  This is just as likely a data point for visitor frustration as it is for the satisfactory engagement of visitors.  If you want to test this out and you have a Windows laptop – try to update the drivers for you hardware on the manufacturer’s website (i.e. Dell, IBM, HP, Sony, etc).  It’s likely that you will have a high time-on-site and an absolutely maddening experience.
• Budget and Timelines:  Web teams can easily take away important research and functionality (and often do) from a project in order to hit timeline and budgetary goals without actually improving the business performance of a website.  Businesses need to constantly improve and make money – so timelines and budgets are important, but viewing budgets/timelines in a silo can distract a team from the most important goals that ultimately improve the business.

While websites can serve many different goals – the most common goals for the web channel are to increase sales/revenues, increase profits, and/or increase customer loyalty.   While these goals can vary, the import aspect is that they impact the business.  Let’s look at a simple process of establishing important goals for a web team and thus important ways to measure the success of a website/web team.

1. Develop business goals:  First define and prioritize the business goals for having and improving the website.   These do not need to tie directly to a default metric reported in your web analytics platform.  It should be agreed upon by the team that improving these goals will qualitatively help the business.
2. Create benchmarks:  Dive into the data and report on historical achievement of these goals. The time period for the benchmarks (i.e. 1 month or 16 quarters) will depend on the goals, the organization, and the visitors.
3. Create forecasts:  Create agreeable forecasts that demonstrate a lift in the prioritized goals.   We typically create forecasts based on identified opportunity, data from previous engagements, and industry standards.
4. Gain consensus and Buy-In:   Discuss the business goals, benchmarks, and forecasts with the executive team to gain organizational consensus on what the web channel can and should achieve.
5. Define reporting:  Document the data sources for all reporting and establish owners for each report to ensure accountability, consistency, and accuracy.

Once you have performed these steps, you now have an effective set of qualitative data points to gauge the effectiveness of the web channel AND the web team that manages it.  Auxiliary attributes such as  budget, ability to hit timelines, website traffic, and time on site may be a part of the reporting.  But the important goals that will determine how the web channel positively influences the business should be the focus to determine a successful web channel and web team.

Here at Extractable, we are strong advocates of the data-driven design approach to creating powerful, enjoyable and successful web and digital experiences for users. You’ve heard us talking about our experiences with data-driven design on this blog, at conferences and in person, but we wanted to go further and learn about how firms across America, both b2b and b2c, used data in their design processes, what tools they used and what outcomes they generated.

So we commissioned a study from Forrester Consulting to do just that, and the results are now in!

The full study: “Data-Driven Design”, a commissioned study conducted by Forrester consulting on behalf of Extractable, April 2012 is now available for download on our site.

We have also created a great infographic of the key findings, available here.

In this blog post, we will focus on a few key findings and what we at Extractable have learned from the study. Over the next few days, we will dissect the study further with a series of deeper posts on key findings.

We saw two key findings in the study.

Firstly, some 60% of firms surveyed had seen improvements in their website due to use of data. And, if the company also reported they had a repeatable design process, the numbers reporting improvement grew to 71%. This is a powerful result. To us as advocates of incorporating data in the design process, the result is a key validation that the process can produce measurable business outcomes. On a daily basis the projects we undertake for our clients using data-driven design are seeing positive results which we are now seeing across a larger and diverse sample size.

However, the second key finding is that many firms are struggling with data: measuring the wrong kinds of data; missing out on key inputs; and even ignoring key data points.

Some specific examples from the study include:

• Companies don’t know how or can’t apply the right tools and processes to optimize their sites. Only 28% of companies are happy with the tools and techniques they use to measure their websites today. As high as 52% believe there are other tools that could provide them with better insight.
• Many firms are measuring the wrong kinds of data. Sites are often measured on metrics that don’t show business value. For example, 46% of respondents indicated they used “time on site” as a key measure. This doesn’t always indicate a positive experience—it could also mean that users are lost trying to navigate the site.
• Some key data is being ignored. 37% report ignoring data that is uncovered and 34% that they gather data but do not use it. Sometimes this seems to be a factor of the ‘Highest Paid Person’s Opinion’ overriding the insights drawn from key data.

Based on the study and our own experiences, it seems that many firms understand the value of data, are looking for the best ways to use it, but fall short in terms of creating a strong data-driven design process, supported by the right skills, teams and tools to be effective.

After reviewing the study, the strategy team here at Extractable synthesized our thoughts into some core recommendations. To make the most of data-driven design, companies should look to:

• Define the metrics for the site based primarily on the central business goals/outcome of the site (or other digital asset), such as sales, leads, customer service efficiency, etc.
• Measure the right set of data that will allow you to see the effect of the changing elements of the experience on that goal/outcome.
• Include a wider set of data/tools, including behavioral tools as well as data from non-web systems such as financial/sales and customer service tools, to ensure you understand why users are doing what they are doing.
• Apply these insights to the design process. Test the updated designs, analyze the results and repeat.

As a reminder, the full study can be downloaded here:

Additionally, Extractable is hosting a Webinar on the study’s findings featuring guest speaker, Forrester Research Inc. Analyst, Adele Sage. Sign up here.

Local Edition is a hip new San Francisco bar located in the basement of the Hearst Building where the San Francisco Examiner used to operate. Local Edition pays homage to the newspaper culture of the 50’s and 60’s.

Upon stepping off Market Street and into Local Edition, the design aesthetic immediately transports you into a world resembling a posh speakeasy. Old newspapers on display, vintage typewriters strewn about, red velvet booths lining the walls, sultry lighting, bar keeps dressed in newsie garb, all evoke the feelings of nostalgia for this historical era.

The cocktail menu is a vintage newspaper with innovative cocktails names such as “The Chief,” “The Eagle” and “Fidel & Che.”  Major kudos to the mixologist who developed this eclectic mix of trendy yet old school concoctions. Using interesting ingredients such as homemade root beer, dill, and all-spice simple syrups create a truly unique experience.

The Extractable team ventured to Local Edition to celebrate a new addition to the Extractable family. Scott Briskman is joining Extractable as the Chief Creative Officer. We are thrilled to have this design visionary on the team.

Here are some pictures from the night!

In this digital age, it is fascinating to remember the printing press and the way we used to share information. Local Edition is awesome on many levels and a must see San Francisco hot spot!

What is your favorite cocktail at Local Edition??

Local Edition
691 Market Street, San Francisco CA 94105
415-795-1395

“So you’re interested in marketing?” was the response I got the other day when I told a colleague that I was interested in how businesses use social media.

My answer: an emphatic “NO.”

What I meant was, “I’m not JUST interested in marketing.” When 67% of B2C and 41% of B2B companies have reported acquiring a customer through Facebook (source), it’s hard to ignore its importance to marketing efforts. However, as Terri Maxwell laid out in her article, “Social Media is More than Just Marketing,” I believe that social media encompasses much more than just reaching out to new customers. All too often, I see this important channel being ignored by companies, or worse, abandoned after a few feeble attempts at engaging with customers.

If social media doesn’t just market your company, what does it do?

• Extends your brand
• Generates organic traffic to your website
• Provides a feedback and support channel
• Establishes credibility in your industry
• Acts as a touchpoint in a customer’s experience with your company

Practically said, your customers expect that:

• The way you talk on Twitter and Facebook is the way you talk on your website or in your call center
• You will respond to them when they have something to say to you on social media
• You are active in producing content on social media channels and engaging with other industry professionals
• You reward them for connecting with them on social media

It’s not enough to just design a website for your company anymore. You have to conscientiously design a full digital experience, which includes social media channels. How exactly do you design for social channels?

• Adjust messages for the channel-specific needs. For example, learn how to say the same message in 140 characters on Twitter.
• Make your company findable, discoverable, and recognizable on social networks. For example, personalize your LinkedIn url to be “company/yourcompanyname.”
• Real-time channels require real-time responses. Respond to your customers on social channels promptly.
• Don’t be scared of people’s public criticism on social media channels. Treat support requests on social channels as opportunities to improve your company’s image and relationship to customers. You can read more about this paradox of service recovery here.
• Ask, just don’t tell. If a customer has connected with you on a social channel, they are ready to interact with you. Engage them with conversational questions. For example, use a poll on Facebook to help understand how they like your new line of products.
• Learn from others: Jakob Nielsen has already conducted research about what users expect from companies’ social media efforts. Read about it here.

A full social media strategy is not for the faint of heart. Start simply and start by dipping your toes in several different social networks. Then, get a tool that wil help you quantify and understand your social media efforts like the ones listed here. Measure, measure, measure. Then optimize: how often you post, the nature of your posts (marketing blasts, questions, sharing of others’ content), and how you integrate social and your website. That’s what data-driven design is all about.

Extractable has been named an Official Honoree by The Webby Awards 2012. We received not one, but two honors in the IT Hardward/Software category. The honored sites include cloud security solutions leader, www.webroot.com and supply chain, workforce, and retail solutions pioneer, www.redprairie.com.

This is a huge honor for us, as The Webby Awards is the leading international award honoring excellence on the Internet. The Webbys is presented by The International Academy of Digital Arts and Sciences, which includes an Executive 1,000-member body of leading Web experts, business figures, luminaries, visionaries and creative celebrities.

We are extremely proud of these two sites and feel they are great examples of the cutting edge work we produce.

Webroot provides the fastest and lightest security solutions that protect personal information and corporate assets from online threats. Extractable redesigned the site’s user experience, developed clearer calls to action and new interaction methods, introduced Webroot’s updated brand, and supported a revolutionary new product launch. The redesigned site has positioned Webroot to achieve their aggressive growth targets.

RedPrairie is a leading provider of supply chain, workforce, and retail technology solutions. Extractable developed a new corporate website in 5 languages, generated double-digit growth in sales leads, and implemented a new content management system that enables the RedPrairie team to change content elements without engineering support. The new site has helped RedPrairie realize a global online presence representative of its industry leadership.

To find out more, check out the The Webby Award Page!

There are many values of sketch ideation on paper with a sharpie or pen. It not only saves so much time and digital effort but also is one of the most useful communication tools when thinking through interface designs. Here are five specific values to this highly integral part of the process here at Extractable:

Brainstorming

As a communication tool amongst team members, sketching is a quick way to brainstorm on the fly and work through use-case scenarios. The strategy team here is positioned in front of two expansive walls of white boards and they are rarely white!

Communicating

Sketching out interfaces is useful as a communication tool amongst team members. Sketching out high-level concepts can help with brainstorming but we also communicate more solidified ideas between teams. Presenting a strategy or functionality to visual design or technical is much less intimidating in the form of a sketch as opposed to a formal deliverable. This is very useful when designs need to be tweaked for minor improvements.

A pre-final-wireframe

Sketch wireframes alleviates the need to go back and retrofit digital wireframes, which is not only costly but also not very exciting. Start wire framing initially with sketches and take them to the computer only once you have thought through all the use case scenarios, functionality, information hierarchy, goals, content etc. Once a final sketch is prepped it will serve well as a compelling piece to digitize.

Storyboarding

Interactions are hard to sketch, however, main states of interfaces can easily be represented on sketches and presented as a storyboard. For example, we have done paper cut outs for hover over states, drawing while explaining an interaction in real time on a white board, and multiple sketched wireframes in succession to represent the different states of navigation through an experience.

Client discussions

Effectively communicating ideas to clients doesn’t always have to be a long slick presentation. Sometimes it is the simple piece of paper as the main discussion point that will get your point across.

Steven Johnson discusses the slow hunch in his book Where Good Ideas Come From. A summary of his thoughts about innovation from the book is found in this TED Talk video. The main premise behind the slow hunch is that innovative ideas are not developed in some definable “a-ha moment” but instead evolve over time.

As I listened to Steven talk about his book this morning at a conference I’m attending, it reminded me of what one of my mentors at Carnegie Mellon University imparted: “Trust the process.” The user-centered design process is inherently iterative and relies on testing and refining hunches.

Let’s face it: the design process itself is scary. As researchers and designers, we must be comfortable with confusion, ambiguity, and the unknown. We live for hunches that we develop on the fly listening to customers and immersing ourselves in the experience of those we design for. A mixture of careful listening (to both words and behaviors), empathy, and relating complex processes to ones we are familiar with allow us to create hunches about what the underlying problems – not just the symptoms – are. Iterative design allows us to vet a large quantity of hunches early and fast. By failing fast, we refine and discard our hunches. Eventually, our hunches start to accurately encapsulate the design challenge, and we can move from hunches to implications and eventually action. But it doesn’t happen overnight.

As user experience experts and designers, we must trust the process. We must be comfortable with slow hunches. We must be comfortable working with hand-drawn sketches and not refined digital interfaces. We have to let go of our pride in our ideas and suspend judgment. We can’t be afraid of being wrong – but must be led by the desire to fully understand needs. If we try to shortcut the slow hunch, we will end up with unfulfilling solutions.

I’ll close with this quote from one of my favorite books, The Alchemist by Paulo Coelho:

“When someone makes a decision, he is really diving into a strong current that will carry him to places he had never dreamed of when he first made the decision.”

Following the design process is like diving into a strong current – our hunches can lead us to designs and experiences we never dreamed of. Dive in!