The Project

A 6-week contract project to analyze consumer survey responses about ClassPass’s market space.

My Role

UX Researcher, working in a 2-person team, with direction from one of ClassPass’s lead UX researchers.

When 2 heads are better than one…

In this contract research project for ClassPass, I discovered that I had a very important user to consider - me!

My project partner and I had 6 weeks to develop a coding scheme and then read and code 700 short-answer market survey responses. Sounds easy, but navigating between data sets and spreadsheets took longer than we expected. Working individually to go through all the responses was incredibly slow, and we were frustrated with our progress.

With our deadline coming fast, we iterated on our process to see where the slowdowns were and how we could improve our productivity. We came up with creative solutions (like working together to ease our cognitive loads and code faster), got buy-in from our supervisors, and worked with the ClassPass team to deliver the data on time.

In order to track the general fitness market space and overall consumer experiences, the ClassPass UX Research team launched a 700-response qualitative survey with Qualtrics.

My colleague and I transformed these short answers into quantitative data by reading each response and coding the information numerically using a Google spreadsheet. ClassPass’s data and analytics team will then create analytic formulas to process the data into numeric insights about commonalities and general trends in behaviors and attitudes.

Here’s a look at the final data format (with mocked-up info as the real insights are proprietary to ClassPass):

Setting up the coding scheme

In order to create a coding system for the large survey that we would be analyzing, we first reviewed a smaller 120-person study and created codes based on the answers found there.

The survey was broken down into sections, so we did small chunks at a time - we would read about 20 responses in one section, come up with the most common answers and use those as codes for the rest of the data set in that section. If a respondent mentioned a code in their answer, we marked it with a “1”, if not, then a “0”.

If we found that a code wasn’t working quite as we expected, we changed it. If another answer started popping up, we added that as a code.By the time we had finished coding all of the 120 responses, we felt that we had the coding scheme set up to accommodate most of the anticipated answers. Of course, for each section, we had place marked “other”, where we could collect meaningful outliers.

Getting started with the Coding process

One discovery we made about the business of surveys is that it’s not easy to get great responses. As we started to code responses from the larger survey, we found many submissions that had “trash data” - responses that were either nonsense, minimal or sometimes even offensive. We had to do several rounds of correspondence with the survey company to weed out the bad data and get it replaced with meaningful responses.

We initially started coding by going from one spreadsheet to another, reading from the data set and then moving to the coding sheet to input the data. The physical process of moving from one sheet to another and the cognitive load required to remember data as we moved to the coding sheet made the process pretty excruciating. We tried using 2 monitors in hopes that more digital real estate would help, but our progress was still very slow.

UX’ing the Process

One morning, in an effort to break up the monotony of the slow coding process, I suggested to my colleague that we might try coding together. I had an idea that if we didn’t have to take the time to go back and forth between spreadsheets, we could code more efficiently. We tried for about an hour and found that our speed had doubled. We kept going and found that in addition to coding more quickly, we could code for longer stretches without needing breaks.

When we told the lead UX Researcher about our discovery, she wasn’t convinced at first. With her years of experience, she had mastered the tricks needed to go more effortlessly between spreadsheets. And while she was doing some coding herself, her work was naturally broken up by the tasks she had to complete for other projects. However, once we presented the time-savings we had found, she agreed to our co-working plan.

We also discovered another efficiency boost when the Head of UX offered to have all of his staff give us time to code together. Coding with a variety of people added even more cognitive flex to the process, making the work less tiring. As an added benefit, the whole UX team was introduced to the rich data that the survey had collected and could start thinking about their own hypotheses to test. Empathy all around!

AT THE FINISH LINE

Once we had UX’ed the process, we were able to complete all the coding by the end of our contract period. We had time to go back to check for and clean up errors, and we left the research team with some big picture hypotheses that they could explore in the data analytics phase of the work

My Contribution

In an effort to UX our coding process, I came with the idea to code together rather than on our own, which unlocked great time-savings for our process. I also developed hypotheses which will be tested in the data analysis phase of the project.

THE TAKEAWAYs

• This was my first project after pivoting to UX, and it confirmed that I really do love the research aspect of the work.

• Empathy and a passion for people is key to this work - in order to create a coding scheme that works, we had to analyze the test survey without bias or assumptions

• I learned to trust my own instincts about the best way to work and to feel comfortable changing things up to make the process go better. The Lead UX Researcher had instructed us in the process that worked best for her, but we had to make adjustments if we were going to succeed. Happily, she was open to our ideas and trusted us to implement our recommended changes.