Incorporating user research on live projects — Part 3

The story was kindly provided by Liliia Piddubna (https://www.linkedin.com/in/liliia-piddubna-2148a6167/), the Product Designer at Valor Software.

This article is part of a series:

• Part 1(https://medium.com/incorporating-user-research-on-live-projects-part/incorporating-user-research-on-live-projects-part-1-afdfa1c3d5d6): Context of the Project, User Persona, Search for Participants.
• Part 2 (https://valorsoftware.medium.com/incorporating-user-research-on-live-projects-part-2-399477634aed): Preparation for the Interview, Competitors Research.
• Part 3: Context of Pandemic, Transcribing, Affinity Diagraming, Report.
• Part 4: Conclusion, Lessons Learned.

Conducting contextual inquiries during the pandemic

Ok, so what about user research in times of a pandemic? Is it even possible? Thanks to technological progress, people now are much better technically equipped, and video calls are not a surprise anymore. The key is preparation and moderation skills while running an interview or a contextual inquiry.

We did the following to conduct user research effectively:

1. Set up video calls. In our case, we had 2 calls in Google Hangouts and 1 in Zoom.
2. Chosen screen and audio recorders.
3. Arranged meetings in advance, sent invitation letters with a short explanation of the purpose.
4. Asked participants to be ready to show some of their notes, the software they use in their workflow, to be ready to share their screens.
5. Prepared a cheatsheet with topics to discuss.

Looking ahead, I want to say that all 3 arrangements went well and productive. On average, inquiries lasted 1–2 hours. They were productive and gave lots of insights. The structure of each session was the following:

1. Introduction. Thanking for giving your time, and asking for consent to record audio and video. Telling more details about the purpose of the study and its structure (1st one — interview, second — observation).
2. Starting with simple questions, such as: What is your role? What’s the first thing you do when you come to the office? What are your daily activities?
3. Moving to more complex topics: start talking about frustrations and challenges.
4. Going deeper: discussing specific spheres of work, for example: how do you forecast any budget/workforce losses? What helped you in preventing high turnover? What the lack of data interferes with?
5. Asking to show the process of gathering and analyzing data about employees.
6. Observing and making tips.
7. Repeating.

Transcribing

The next step before we can go to the data analysis is to transcribe our inquiries. For me, honestly, it was the most boring part of the study, but still crucial 🤷🏻‍♀️. While making a study, we can’t manually record all the communication — it’s better to take some notes and clarifications to deepen the conversation later. So after we are done with an inquiry, we make a cup of coffee/tea, sit down, play the recording, and start typing. Absolute concentration.

On average, each interview was 5–8 pages long. We really got a lot of information — especially in combination with video recordings of participants’ workflows.

But to find any patterns and tendencies, so that we can form design suggestions, we have to work out the data — in our case, qualitatively. How can we do that?

Qualitative data: processing

Affinity Diagramming

Affinity Diagramming is a method we used while working with qualitative data. The essence of the method is uniting observations, ideas, quotations — everything we have got during our studies into clusters on a basis of similarity and affinity. So we basically make groups of information pieces that have something in common. Then we label these clusters.

So what was the process in our case?

After transcribing the interviews, I wrote down all the key observations, thoughts, frustrations, and tricks on hundreds of post-it notes. To make the process of sharing deliverables easier (in terms of total remote work with peers), I have chosen the Miro tool.

Each color defined a specific participant. I processed each interview separately, and as a result, had huge masses of information gained from inquiries with respondents.

It was a real mess, and the next step I should have done — is clustering all these pieces into groups so that we can analyze them, and find common patterns, problems, and needs.

The result we had:

Uncovered key problems and needs

It turned out, that the most crucial challenges our potential users faced in their everyday activities were the following:

1. Having data for different types of people analytics in different tools and spreadsheets causes a time increase in processing, comparing, and finding correlations.
2. Too many spreadsheets and different tools make knowledge sharing and transferability between HR department team members complicated.
3. The complexity of storing information (too many steps because of many tools) causes incorrect data about employees.
4. Budget losses caused by lack of the data, such as absenteeism, burnout, correlating quantitative performance, and engagement, etc.
5. Low level of turnover prediction.
6. It is hard to track the effectiveness of recruitment, organizational, and educational programs.

What are the key needs discovered within the research?

1. Be valuable among other departments with the help of data-driven approaches and metrics
2. Have data to get resources — budget, employees, programs
3. Keep turnover and retention on a benchmark
4. Measure effectiveness of the programs
5. Keep teams engaged and well-performed
6. Meet forecast goals and revenues

Brainstorming

Based on a quick review of the clusters, I brainstormed ideas we could bundle into the feature. Note — it is much better to try this activity with your team to collect diverse approaches and solutions. Such workshops help teams be on the same page while working on the discovery and ideation stage of the product/feature/enhancement development, and increase engagement.

Report preparation

After this activity, I have moved further to create a detailed report with Key Insights and Design Ideas. In the end, it turned out as an 8-page document that concluded the following sections:

1. Motivations and Frustration Summary
2. Overview of key topics and functionality ideas for each of them (headcount, data correlations, revenue (quantitative performance), qualitative performance, talent acquisition, turnover, promotions, 1on1s, etc.)
3. Design recommendations to predefined dashboards views for each role (HR, Manager, Regular Employee)
4. Design recommendations to dashboard functionality

Here you can read some pieces of the report:

Based on the insights we have worked on the informational structure of the feature so that we can implement it later on in the wireframes.

Informational structure:

The report with supplying materials, as competitors’ analysis, initial hypothesis, informational structure, and widget ideas was presented to the product owner and QA lead of the team and then sent to other team members. The next step was wireframes creation. Mostly we focused on such features, as:

– Adaptive grids;

– Multiple dashboards;

– Customizable widgets;

– Actionable insights forming.

I can give you a small glance at the following work we have done:

But wireframing is another big part of the process, going apart from the research stage of the project.

Conclusion

Ok, if you’re about to run a research, nothing can stop you. And when the pandemic comes — comes help from the technologies. Video calls that you schedule in advance might be even more efficient than live meetings since people do get ready for those and even come on time. Maybe that’s because communication became even a more lively need these days :-)

If your questions for the interview are ready, prepare yourself for another role on top of an interviewer. You need to moderate the conversation in order to pick up the most valuable grains for your research. This requires attention, so make a cheatsheet with topics to discuss. When ready with this part, you need to transcribe the requirements you gathered. You record, then you transcribe taking all the time you need because this takes time.
‍
When all the key observations are at hand, you can ease the process of sharing deliverables by going with the Miro tool. You realize the most crucial challenges of potential users and formulate the key needs. Now it’s just a detailed report with Key Insights and Design Ideas left to complete the mission. Good job!