Evaluative Research: What It Is and Why It Matters

Evaluative research can shed light on your product's performance and how well it aligns with users' needs. From the early stages of development until after your product is launched, this research method can help bring user insights to the forefront. 

In this comprehensive guide, we will demystify evaluative research, delving into its significance, comparing it with its counterpart—generative research, and highlighting when and how to use it. 

We will also explore some of the most widely used tools and methods in evaluative research such as usability testing, A/B testing, tree testing, closed card sorting, and user surveys.

Why is evaluative research important?

Evaluative research, also referred to as evaluation research, primarily focuses on assessing the performance, usability, and effectiveness of your existing product or service. Its importance stems from its capacity to bring the user's voice into the decision-making process. 

It allows you to understand how users interact with your product, what their pain points are, and what they appreciate or dislike it, therefore being a key element of UX research strategy.

By providing this feedback, evaluative research can help you prevent costly design mistakes, improve user satisfaction, and increase the overall success of a product.

Furthermore, evaluative research is important for understanding the context in which your product or service is used. It offers insights into the environments, motivations, and behaviors of users, all of which can influence the design and functionality of the product.

Evaluative research can also guide decision-making and prioritization and show which areas require immediate attention, making it an important part of your user research process.

Evaluative vs. generative research

While both evaluative and generative research are integral to product development, they serve different purposes and are conducted at different stages of the product life cycle.

Generative research, also known as exploratory or formative research, is typically conducted in the early stages of product development. Its main purpose is to generate ideas, concepts, and directions for a new product or service. 

This type of research seeks to understand user needs, behaviors, and motivations, often using techniques such as user interviews, ethnographic field studies, and contextual inquiry.

On the other hand, evaluative research is often performed at later stages of product development, often once a product or feature is already in place. It focuses on evaluating the usability and effectiveness of the product or service, identifying potential issues and areas for improvement.

While generative research is more qualitative and exploratory, evaluative research tends to be more quantitative and focused, employing methods such as usability testing, surveys, and A/B testing. 

However, both types of research are vital for the success of your product, complementing each other in providing a comprehensive understanding of user behavior and product performance.

When should you conduct evaluative research?

The timing of evaluative research depends on the product life cycle and the specific needs and goals of the project. 

However, it's typically conducted at several points:

Prototype stage

You can carry out evaluative research as soon as a functional prototype is available. It can help you assess if the design is moving in the right direction and whether users can accomplish key tasks. The issues you identify at this stage can be relatively inexpensive to fix compared to later stages.

Pre-launch stage

You can conduct evaluative research prior to launching a product or a major update to ensure there are no critical usability issues that might negatively affect user adoption.

Post-launch stage

Even after a product has been launched, you should conduct evaluative research periodically. This allows you to monitor user satisfaction, understand how your product is being used in real-world scenarios, and identify areas for further enhancement.

One of the factors that sometimes gets overlooked post-launch is the language used for labels, features, and descriptions within a product. With this short product copy clarity survey, you can assess whether the wording you use makes sense to your users:

Evaluative research tools and methods

There are several tools and methods used in evaluative research, each with its unique benefits and appropriate scenarios for use. Here we discuss five main methods: usability testing, A/B testing, tree testing, closed card sorting, and surveys.

Surveys 

Owing to their versatile nature, user surveys can serve multiple functions in evaluative research. 

They are used to gather both quantitative and qualitative data, facilitating the analysis of user behaviors, perceptions, and experiences with a product or service, as well as creating user personas. 

Surveys allow you to collect a large amount of data from a broad spectrum of users, including demographic information, user satisfaction, self-reported usage behaviors, and specific feedback on product features.

The resulting data can inform decisions around product improvement, usability, and the overall user experience.

When using a survey tool, you can opt for standardized surveys, such as the universal Net Promoter Score below, or create your own templates with questions that suit your needs. 

You can also use surveys post-interaction to capture immediate user responses and identify potential problems or areas for improvement. The template below, for instance, allows you to gauge user satisfaction with your checkout process:

Surveys can also be used to perform longitudinal studies, tracking changes in user responses over time. This can provide valuable insight into how product changes or updates are affecting the user experience.

Usability testing

Usability testing involves observing users as they engage with your product, often while they complete specified tasks.

It allows you to identify any difficulties or stumbling blocks users face, making it a robust tool for uncovering usability issues.

You can apply usability testing at various stages of product development, from early prototypes to released products.

When used early, it can help you catch and rectify issues before a product goes to market, potentially saving time and resources. In the case of existing products, usability testing provides invaluable insights into areas needing improvement, offering a roadmap for updates or new features.

Additionally, usability testing can provide you with context and a deeper understanding of quantitative data. 

For example, if analytics data shows users are dropping off at a specific point in a digital product, usability testing could reveal why this is happening. 

A/B testing

A/B testing consists, in short, of comparing two versions of a product or feature to determine which performs better. 

Each version is randomly shown to a subset of users, and their behavior is tracked. The version that achieves better results—as determined by predefined metrics such as conversion rates, time spent on a page, or click-through rates—is typically considered the more effective design.

The primary advantage of A/B testing is its ability to provide definitive, data-driven results. Unlike other methods which can sometimes rely on subjective interpretation, A/B testing delivers clear, quantifiable data, making decision-making simpler and more precise.

This form of testing is particularly beneficial for fine-tuning product designs and optimizing user experience. Whether it's deciding on a color scheme, the positioning of a call-to-action button, or the content of a landing page, A/B testing gives you direct insight into what design or content resonates best with your user base, leading to more effective design choices and, ultimately, better business outcomes.

Watch this short video for tips on how to incorporate A/B testing into your user research:

Tree testing

Tree testing can help you understand how well users can find items within a product's structure, essentially testing the "findability" of information.

In a tree test, users are presented with a simplified version of the product's structure, often represented as a text-based tree. They are then given tasks that require them to navigate this tree to find specific items. 

Their journey through the tree, including the paths they take and any difficulties encountered, provides valuable insight into the effectiveness of the product's information architecture.

Evaluative research through tree testing can highlight potential issues such as confusing category names, poorly structured paths, or misaligned user expectations about where to find certain items. 

Closed card sorting

Closed card sorting involves users organizing items into pre-existing categories, thereby shedding light on how they perceive and classify information.

In a closed card sort, participants are given a set of cards, each labeled with a topic or feature, and a list of category names. The task for the users is to sort these cards into the provided categories in a way that makes sense to them. 

As users engage in this activity, researchers can gather insights into how they group information and understand their logic and reasoning.

This type of research can identify patterns in how users categorize information, highlight inconsistencies or confusion in the current categorization or labeling, and suggest improvements. 

Boost your evaluative research with surveys 

Evaluative research plays a vital role in the product development cycle. While it is typically associated with later stages of product development, it can and should be conducted at various points to ensure the product's continued success. 

Surveys, one of the key tools used in evaluative research, can turn a spotlight on areas of your product that users love, as well as highlight those that could benefit from refinement or improvement. 

With Survicate, you can keep your finger on the pulse of user feedback and maintain a competitive edge. Simply sign up for free, integrate surveys into your evaluative research, and watch as they provide the fuel you need to drive your product's success to new heights.