|
|
|
|
Issue 16 \\ Open Ended
In this issue, UX Developer Jason Beaird shares some of the history behind the MailChimp UI and how our pattern library has evolved over the course of 2 major redesigns. Next, Laurissa Wolfram presents the joys and challenges of working with open response survey questions and provides tips for organizing them in SurveyMonkey.
|
|
Hey, did you miss our last issue?
On Twitter: @MailChimpUX
|
|
|
Life in the web industry always seems to clip by at a pace that feels a bit like dog years. While time does fly when you're having fun, the trends, techniques, and technologies change so fast that a 2 year old site or application design really can feel like it's a decade old. When I started at MailChimp in early 2010, this is what the application dashboard looked like.
|
|
|
|
MailChimp as it looked way back in 2010. “How you livin?â€
|
|
|
As you can see from this week's newsletter header, It's a giant leap from "MAiL CHiMP" circa 2001, but there were some things that still worked the same way. Up until that point in MailChimp history, the markup and styles for every component of the application had been — quite lovingly — coded from scratch. Sure, we used common classes for things like buttons and grids, but those auburn tones, font styles, and gradients in the design above were baked in all over the CSS. In short, it was far from what our friend Brad Frost would call Atomic Design. When we started talking about a full redesign in mid-2010, we knew things had to change.
|
|
|
We started toiling away months before we ever saw a single screenshot of what DesignLab was conjuring up for MailChimp's new look. Our primary focus for this "pre-design" was to reduce repitition in our CSS to make the actual redesign process easier. We did this by combining similar interface elements into reusable patterns. The metric we used to gauge our success was total CSS size. Over the six months before we released the 2011 redesign, we managed to cut about 120k from our CSS. Here's what the dashboard looked like after we launched.
|
|
|
|
The MailChimp dashboard from February 2011 until June of 2013.
|
|
|
As we were creating these reusable patterns, we started archiving them in an internal "cheat sheet" for a couple reasons. First, we wanted to create a style guide that anyone writing front-end code for the app could use as a shortcut. Second, we also wanted a test page to show whether or not the changes we made to our pattern CSS would break things in the app.
|
|
|
The first version of the pattern library which we relied on for over 2 years.
|
|
Fast-forward to November 2012
Knowing that we were going to be embarking on another big redesign, we started crafting a brand new pattern library. We already had a very modular approach to working on the app, so our focus the second time around was making things responsive, more user friendly, and even more flexible. As with the previous redesign, our team was working for several months before the actual design was settled on.
|
|
|
|
The current MailChimp dashboard at desktop, tablet, and phone widths.
|
|
|
By the time we launched the (new) New MailChimp, we had a much more extensive collection of common interface patterns. Unlike our old pattern library, though, we made this one public. While we haven't posted about it anywhere yet, it's been available at http://ux.mailchimp.com/patterns since the 2013 redesign launched in June.
|
|
|
Now, you may be asking: why would we publish something like this when it probably won't benefit anyone outside of MailChimp? Well, it's for the same reason that we keep posting to this newsletter. We want to be transparent, share our challenges, and show our work. As with just about everything on the web, our pattern library was inspired by the work of many brilliant people and we hope it'll inspire others as well. It's not perfect and you might even find it broken at times, but it's served us well for the last 6 months—which is really like 3½ years in web industry and dog years.
|
|
By the way, we're looking for a talented UI designer to join the MailChimp UX team.
Is this you?
|
|
|
|
In Issue 4, fellow MailChimp researcher Fernando talked about the importance of identifying patterns by looking at open survey responses holistically. Tagging and categorizing responses helps us see trends much more clearly than if we try to identify them by reading responses in isolation.
|
|
|
But, as Fernando mentioned, this can be challenging when you have thousands of responses to filter. How do we dig through all of that data and find meaning in it?
|
|
|
While there are many different approaches to analyzing open ended survey responses, I’m going to share a few tips for how I move this process along in SurveyMonkey.
|
|
Figure out what you really want to learn
Before you even read the very first response, remind yourself why you conducted this survey. You’ll look at your responses very differently if your goal is understanding how many of your customers express frustration about a particular feature, versus identifying commonly used workflows or features. Regularly refreshing yourself on what you’re trying to learn will help you stay focused and move through data more quickly.
|
|
Filter and categorize
Open-response questions are a great way to gather in-depth answers… but when you have several hundred of them, they can be a little tricky to dig through.
|
|
|
I’m one of those people who can’t wait to see what our customers have to say, so I usually start by skimming through the responses, just to ease my curiosity. But once I’ve gotten a feel for what people are saying, my first priority is to filter all the responses that aren’t particularly helpful. For example, I usually get a big handful of “n/aâ€s from folks who either have nothing to say or just want to skip to the next question. I do a quick search for “n/a†and all of its variations (none, na, n.a., nada, nope) and categorize them with something like “Unanswered†or “Nothing.â€
|
|
|
|
Searching for and categorizing “n/a†and “none†can help you quickly eliminate some of the “noise†so you can focus on finding patterns in your responses.
|
|
|
After the “n/aâ€s are cleaned out, I start skimming through the responses for obvious patterns and keywords that I can use as search terms. Depending on what I need to learn, this can be a MailChimp feature (template, user profile, autoresponder), common task (importing, segmenting, syncing), or software (Excel, EventBrite, CRMs). SurveyMonkey does all the hard work for me by searching for responses with these keywords, listing all the responses together, and highlighting the search term in each. From there, I can select multiple responses and categorize them all at once.
|
|
|
|
Search for keywords in the responses, then use the checkboxes to categorize in bulk.
|
|
|
The Text Analysis tab can also identify common themes or patterns—it pulls out the most common words used across responses and displays them as a word cloud. These identified words aren’t always very useful, but sometimes Text Analysis uncovers a few gems.
|
|
|
|
Text Analysis collects the most commonly used words in the responses and lists them as a word cloud. Sometimes these words can help you spot patterns.
|
|
|
I can usually categorize several hundred responses fairly quickly by using the Search and Text Analysis tools. Once I’ve sorted and categorized all of these responses, I begin the more tedious task of reading through and categorizing everything else that’s left.
|
|
Review and adjust categories
Once I have all of my responses tagged, I review the responses in each category. Sometimes I find that a category is too broad, so I split it into multiple categories. I also might move responses from one category to another that seems more appropriate. For example, I might have initially identified a response as a “UI Design†problem, when it actually belonged in the category for “Content/Copy.†There’s always some shuffling that goes on before I feel like everything is categorized accurately.
|
|
Identify prospective interviewees
Surveys are great for helping us understand what a group of users might be struggling with, but they don’t always help us understand why. Our research team usually follows our surveys with a round or two of customer interviews. As I’m going through survey responses, I try to tag people I want to reach out to later. Usually I’m looking for an interesting problem or scenario that I want to know more about. Flagging potential interviewees while I’m sorting data saves me from having to search for them again later.
|
|
|
|
Keep an eye open for folks you want to follow up with. Filter them into an “Interview†category so you can find them easily when you’re ready to conduct interviews.
|
|
Keep track of outliers
Surveys can help us see overall trends across large groups of users, but they’re also good at drawing attention to users who really stand out from the crowd. It’s particularly useful to flag these “outliers†and reach out to them individually for a follow up conversation. Our UX team is always on the lookout for interesting use cases, for people who are pushing the limits of our app, or for interesting workarounds. These are often people we can learn from and are well worth the time to chat.
|
|
Learn to let go and move on
This is a hard one. I used to feel like a negligent researcher when I didn’t explore every single pattern, trend, or finding that came up in my surveys. But eventually I realized that this didn’t make me a better researcher—it just made me a busier researcher.
|
|
|
If you come across something that’s interesting but outside the scope of your current research goals, great! Start a running list of “things to explore†and come back to it later.
|
|
Final thoughts
Your job doesn’t end once you’ve filtered all of your responses into categories. Sure, you might be able to say “22.31% of respondents mention they use mobile tools.†But this statistic doesn’t tell you who these users are, which mobile tools they use, the tasks they are trying to accomplish with these tools, or why they prefer to work with a mobile device. It’s up to us as researchers to find meaning from this data by putting it into context or digging deeper for more information. Will organizing and categorizing data do this for us? No, but it does give us a very good place to start.
|
|
Ask Us Anything
We want this newsletter to be a dialogue. If you have any questions for our team about what conferences we're looking forward to, what candy is still left in the office this week, or even Freddie's Halloween Costume, send them in. Seriously: hit reply and ask us anything. We'll try to answer every email and maybe even share our conversation in future newsletters.
|
|
|
|
|
|
|
