Issue 12 \\ Videodrome

Issue 12 \\ Videodrome

In Lou Rosenfeld's recent (and wonderful) article for A List Apart, he referenced the fable of the blind men and the elephant. Long fable short, no one could see the big picture, and everyone drew faulty conclusions.

Seeing the big picture is necessary on all sides of UX. For our Design and Research teams, it means we really need to watch our customers use our products in order to learn where we fall short or succeed. For our Knowledge Base team, it means seeing how customers search for and learn about our application features.

In this issue, we're going to use Videos to explore how we see the big picture. First, UX Researcher Jenn Downs describes our video viewing sessions and how they're breaking down walls. Then, our Video Documentation team explains the process behind creating and measuring the success of our educational videos. We finish with some links of interest.
Editing: Gregg Bernstein, Aarron Walter
Illustration: Caleb Andrews
Photography: Jason Travis
Hey, did you miss our last issue?
Forward to Friend

Video Party Time

by Jenn Downs
Over the past two years Team Research at MailChimp has grown from a team of Jenn to a team of four amazing full-time researchers. As a result we’re able do a lot more user interviews and usability tests than we used to, which produces lots of data and hours of video footage. Instead of just emailing everyone in the company with video links, hoping they have time to watch them and discuss with colleagues, we instead try to stay true to our enduring spirit of collaboration by inviting people to watch recordings of usability tests together.

In the past two years we’ve also added nearly 100 additional employees across the company. 910+ pizzas ago we were able to bribe Chimps across every department to attend video pizza parties. But Pizzanomics shows us it’s no longer sustainable to invite the whole company to every video party, every time. They’ve got customers to help, stuff to make, people to hire, and emails to write!

Every month we invite the UX, Marketing, DesignLab, Engineering, Knowledge Base, and Quality Assurance teams to a video party where we show the best (and sometimes the most painful to watch) examples of our customers struggling with our apps. Our goal isn’t to provide warm, fuzzy validation of our work. We want to spark conversation about the ways we can make MailChimp better, and expose the most pressing usability issues we need to resolve. Nothing can create empathy for users better than watching them struggle.

This month we’re actually hosting two parties—one with usability test videos and one with a very brave person who will use MailChimp for the first time with our UX team looking on. Inevitably, after every event we see groups of designers and developers huddled together, scratching their heads as they ponder ways to solve the problems they’ve witnessed.

These videos are powerful. They inspire action. We want share them across the entire company, but we’re careful to avoid overwhelming our colleagues with too much information, which can cause people to tune out.

Include, But Don’t Overwhelm

While we only have a small group watching and discussing usability testing videos together, we still share choice videos over internal networks. This way, anyone can watch them whenever they have time or are just curious what struggles our users are currently facing.

This newsletter you’re reading is something we encourage our fellow Chimps to subscribe to so they can learn about the design, development, and research processes and what we’re doing as a team.

We share the UX love by attending meet-and-greets with new MailChimp Support team members, and by presenting on our history and goals to help long-time support team members learn about UX. We also encourage people outside the UX group to post to and research issues in our ever expanding Evernote database. This way we can see issues from all angles and know what people are working on.

As a company grows and succeeds, silos can threaten communication. As researchers, we can knock them down by moving valuable data from our heads and computers into action-inducing conversations.

As our UX team has grown and our work has diversified, we’ve learned that research isn’t enough. We have to turn data into stories and share them with our teams. Telling the stories of our users incites action that leads to better products. Better products make happier customers, and that makes us happy too.

By the way, we're looking for a talented UI designer to join the MailChimp UX team.
Is this you?

Some Words About Videos

by Maribeth Coulter & Ariana Hargrave
MailChimp makes it easy to send emails, but we also pack in some powerful features. Some folks learn how to use these features by reading instructions in our Knowledge Base, while others like to see and hear their lessons in a video.

The video side of our documentation team caters to that latter group by creating brief, step-by-step tutorials that walk users through our app. The average video runs about a minute and a half—short enough to hold someone's attention and concise enough to provide the information they need.

So far, this approach has proved effective—our top 20 most recent videos have 21,252 total plays, which is huge considering that a majority of those videos are less than a month old. Those same videos show an average engagement rate of 77%, which means that roughly 77% of our viewers watch all the way through to the end of a video.

We thought we’d open the curtains a bit and explain how we identify video subject matter, how we map out our video content, and how we iterate based on viewership and feedback.

Our Challenge: An Overview

Because we cater to a diverse audience with a wide range of needs, our video documentation is a balancing act.

We strive for simplicity in covering complex material
We want to get our customers up the magic escalator of acquired knowledge. We try to make it easy to get in, get acclimated, and move on up, and we do this by teaching the basics with the assumption that a little bit of powerful knowledge will inspire our customers to learn more. To achieve this, we look for the path of least resistance to achieve a goal (and the fewest clicks to get there).

We cater to various learning styles
Everyone learns in different ways—by seeing, by hearing, or by doing. Considering these styles, we show (visually) and describe (audibly) while keeping video “pausers” in mind. The pausers can pause and re-watch to try the actions on their own and then continue on.

We also aren’t too prescriptive, as not everyone likes to be directed. Each video uses first-person phrasing such as “I’ll click” or “I can.” This lets users see how we perform a task, but leaves the steps open to experimentation or customization. This phrasing also comes across as if we’re showing them a lesson one-on-one. We took inspiration from This Old House—those guys use plain language spectacularly to explain what a doohickey is, how it’s used, and what they’ve accomplished by using doohickey A vs thingamabob B.

We work within time-constraints based on releases and customer interest
Every few weeks, MailChimp is updated—something gets added, refined, or removed. We prioritize our video workload by what has changed the most and what videos indicate the highest viewership.

We strive for accessibility to a global audience
MailChimp is growing around the world. When we create a video, we keep this in mind by using common—not slang—language, adhering to the naming conventions used throughout our documentation (including in closed captions in each video), and being deliberate and specific in each description of action.

Selecting a Topic

We choose topics from the top 250 to 500 searched Knowledge Base articles. There’s a bit of subjectivity involved here, as we rely on our tech support experience to correlate the top searched topics to familiar pain points for our customers. A high number of searches for an article could mean people are really interested in a feature, or they’re really confused by the feature. We also review specific article steps and screenshots to determine if the topic requires a walk-through to better understand it.

We work with our Analytics team to determine which high-traffic articles, like Getting Started with Merge Tags, would benefit from having additional instructional content. For instance, we learned that folks have trouble with formatting merge tags since the “pipe” key is not frequently used, which made us decide to show in our video the “merge tag drop down” function in the editor, which automatically adds the formatted merge tag to the content.


Once we select a topic, we think about how customers ask about a feature. That leads us to the specific angle we adopt for a topic—we list out the points to cover and draft a script for each topic. It’s always helpful for us to state the video’s goals at the start of a tutorial, and again at the end to reinforce the message—the clearer the message in a video, the easier for users to follow it. Our intention is to empower users to figure things out on their own, and reduce the number of  customers entering our Customer Support queue.

For our scripts, we walk through the ways each process could be performed in the app. We then identify the path that is most sensible and involves the least amount of steps. This helps us figure out where the transitions should be and how to explain the process. There may be multiple ways to use a feature, so we look for the path that gets customers to each step the fastest with the fewest number of clicks. The longer a set of directions is, the more likely we are to lose someone.

With our path determined, we get to the storyboard phase, map out the critical scenes, and begin making our videos with Screenflow.

Using Video Analytics to Gain Insight

The analytics provided by our video host, Wistia, give us an unbiased view of how our videos are performing with customers all over the world. As people watch, we gain insight into how people are interacting with each video.

We look for folks who scrub back through the video—they might be struggling with specific words or visuals. Once we discover a trend, we discuss making adjustments for clarification. 

When people exit the video around the same point, it’s often an indicator that they just wanted to see where to access a feature. Once they’ve seen where it is, they’ll figure out how to use it on their own.

However, common exit points can also indicate the video is too long or delivers too much information. Short videos hold users attention well, but long videos can explain complicated concepts in detail. We look for a balance when we create new videos.

When we see people are repeatedly watching passages of a video, we listen to the voice-over for words or phrases that might confuse people, and check the visuals to make sure the transition clearly displays how to reach the current step. Before the redesign of MailChimp, we found the phrase “gear icon” (a reference to an old settings menu) received an exceptional number of re-watches from viewers in the United Kingdom and Canada, but in other countries where English is not the primary language it was passed over.

When we see a lot of skipping around in videos it’s a signal that people are looking for a speedy overview of a topic. To solve the problem we look at cutting frequently skipped content to focus on what’s most relevant.


Engagement is based on many factors. Wistia provides heat maps of each video, which include color-coded, individual viewer details including total watch time, skip-throughs, complete stops, and rewatches of specific areas.

If customers aren’t engaged with a video, we look into possible explanations and ask ourselves a lot of questions.

What was difficult to grasp when we first learned MailChimp?
Terminology is always tricky. When we describe buttons and actions in MailChimp, we think of how the name will be perceived by customers and try to use intuitive terms. As mentioned previously, we encountered measurable confusion with the term “gear icon.”

What type of tutorial content could have helped us better understand concepts?
To better understand our own products, it helps to learn new features ourselves with personal MailChimp accounts. This way we learn our product features and discover where we need further explanation.

Are we too instructional at times?  Are we not instructional enough?
We try to avoid a large amount of “click here to make this happen” without additional explanation in our videos. Each video opens with an explanation of a feature and how we’ll use it. From there, we show how it functions and reiterate what we just did. We also have to ensure we’re not spending too much time on all the feature can do versus explaining how to best utilize it.

Is our language too American?
MailChimp is growing internationally, and we’re cognizant of this. Sometimes it’s all too easy to slip into familiar turns of phrase which don’t translate across borders.

If the timeline heatmap shows a drop off, was it because the user got the info they needed so quickly that they left early?
We tend to see more drop-offs when we move features around. With the recent redesign, we found a high number of people accessing the videos to see how to find a feature, but as soon as we showed them where, they left. This indicated that they were already familiar with how the feature works.

Are people more informed than we think?  
We have to assume some level of understanding of our app when we create videos. However, we also structure the videos to accommodate new users who might only be familiar with concepts and not specifics.

The Job Doesn’t End

If a video shows really high engagement, we pat ourselves on the back for a job well done. At the same time, we wonder if the topic is fairly difficult to grasp, which could lead to a higher percentage of viewers watching to the end. Each video’s stats come from complex conditions, ranging from language barriers to level of previous knowledge of a topic.

We’re constantly learning and iterating based on our viewership. We dissect each video's stats from multiple angles to figure out if our script and visuals need improvement, or if the content is doing what we intended. And this goes on daily to keep our videos relevant and our customers empowered.

UX Around The Web

Ask Us Anything

We want this newsletter to be a dialogue. If you have any questions for our team about pattern libraries, design research, or even Caleb's sock collection, send them in. Seriously: hit reply and ask us anything. We'll try to answer every email and maybe even share our conversation in future newsletters.

© 2001-2013 All Rights Reserved.
MailChimp® is a registered trademark of The Rocket Science Group

view in browser   unsubscribe   update subscription preferences   

Email Marketing Powered by Mailchimp

Love What You Do