Improve your knowledge by combining qualitative research with data science
As a data science practitioner, think about all the data you have. Then think about the latest data science methods you know. Now try to answer these two questions: Why are your users adopting or rejecting your products? And how do they use these products?
Both may sound simple, but they are trick questions. You can much more easily determine what characteristics of products are adopted, when and where they are adopted, and perhaps even by whom.
Answering “why” and “how” questions with analytics is, however, much more complicated because you need to better understand your users, the contexts in which they operate, their considerations and their motivations. Although aThe answers to these questions are essential, reAta science teams can’t always rely solely on assumptions, models and numbers to understand the choices users make and the decisions that lead them to use a product the way they do.
The point of this little brainstorming exercise is to suggest that while analytics has many benefits, it also has its limitations. Recognizing these limits will help you broaden your vision and become more innovative.
But to get there, you’ll need to embark on an exploration where parameters and variables are less known, assumptions are mostly absent, and curiosity abounds. You will need to think in a way that diverges from how you were trained, and you will need to use fundamentally different data and research methods than you usually rely on.
In short, consider incorporating qualitative research into your analysis process.
Qualitative research can answer ‘why’ and ‘how’ questions
Typical analyzes involve obtaining data from users’ devices, their recorded activities, or through user experiences such as A / B testing. But to answer “why” and “how” you will have to learn user perspectives, meanings and considerations directly from them.
Instead of top-down analysis, collect data by going to the “field” – the contexts in which your users operate. Rather than relying on known assumptions and existing variables to do deductive work (by far the most common form of analysis), immerse yourself in an inductive process of qualitative research.
The data collected in qualitative research is different from what you are used to in data science. This is how it works:
- Notes: Field observations are made while the researcher takes notes on users and their behaviors, which may take place in the natural environment or in the laboratory. Viewing a user using a product (a website, an app, a gadget) are common examples.
- Literature review: Because the observations are made to study the choices and behaviors made on the spot, to obtain longitudinal data, the researcher will often rely on documents generated by the user. There are two types of these documents: community registers and newspapers. The first refers to content generated by community members (think Reddit on a topic of interest). In the second, the researcher asks users to systematically record their views over time in the form of a diary.
- Interviews: In-depth interviews are conducted with informants participating in observational studies and based on the knowledge the researcher has obtained from documents. Rather than closed questionnaires, these are conducted with open-ended questions to gain more information from the participants’ perspective. (for example, “Why did you use the app the way you did? “).
Note that unlike traditional data used in data science, qualitative data is layered and complex. Field observations are linked to notes, the notes are then connected to interviews, both are then connected to documents.
It is not a linear process, and by going back and forth between these interconnected layers of data, patterns emerge, research questions are refined, new behaviors and characteristics are identified and information is acquired. And because the data is intentionally collected mostly in an unstructured way (i.e. without answers to specific, closed-ended questions), the contribution reflects different perspectives.
All of this can lead to refined questions and hypotheses to be pursued using data science tools.
Qualitative data science work is scarce, but shows promising results
In business contexts, qualitative research is mainly reserved for studies of user experience (UX), product and UX design, and innovation. This intensive research work is largely disconnected from the work done by data science teams.
However, if your data context involves people, you should consider bridging this disconnect.
Take, for example, a engineering team at Indeed who realized they needed to create a new lead quality metric, but didn’t have enough information about leads and how to rate their properties. They therefore spent time observing, interviewing and analyzing the documents of their account managers. By analyzing this data, they identified characteristics they had not considered before and developed the metric they were looking for.
Being able to collect data on new features and design machine learning models added significant value. But they realized that as the market, user needs, and their platform continued to evolve, it was important to come back from time to time to collecting qualitative data to further inform their pipeline. analysis. This continuous integration of qualitative data and big data has generated millions of dollars in additional revenue.
Or consider the results of qualitative research and data science teams working together at Spotify. Despite a plethora of user data on the online streaming service, the company still had to make sense of user behavior when they received advertisements. The data science team followed the standard approach and performed an A / B testing (intervention being skippable ads and control being the standard ad experience). The results led the data science team to identify distinct patterns of behavior.
Interestingly, the company also commissioned qualitative researchers to study users directly. Their conclusions were fundamentally different. for example, they found that some of these profiles had nothing to do with inherent choices, but were actually more the result of confusion about the characteristics and information presented.
Learning from this experience, the company began to adopt a mixed methods approach (where qualitative data is integrated with more structured big data) to leverage the benefits of both approaches. They established a common research query, designed a process in which researchers constantly communicated, and then “triangulated” their ideas with both qualitative and quantitative data.
The result was more comprehensive research data, where business and design decisions, such as explicitly notifying users of ad skip limits, were based on information obtained from users and data about users.
How to start incorporating qualitative research into analysis
There are several ways for a data science team to engage with qualitative data:
- Create curiosity by challenging your analysis team. Start by generating a list of your key data characteristics, hypotheses, and general research questions. Next, ask your team for the assumptions and contexts related to the list. This is where asking some of those “why” and “how” questions is likely to be helpful.
- Educate about the immersive research process and the types of data collected in this type of work. This book is a great example for general applications that involve both more structured (quantitative) traditional research, more immersive (qualitative) research, and suggestions on how to combine them. This book teaches how to draw insight into the data from the ground up.
You should also become familiar with the three data collection methods (observations, interviews and document analysis) that are at the heart of qualitative research:
Finally, connect with researchers who have immersive research experience, inside and outside your business. They can help you in your thinking about how to collect and analyze qualitative data. And maybe you can offer to help them with some parts of the tedious analysis of the more quantitative part of their data.