The Unique Insights and Practical UX Recommendations Shared by Participants with Cognitive Disabilities

In the summer of 2024, a working group of expert researchers convened to address a critical gap in user experience research: how to effectively conduct accessibility testing with individuals who have cognitive disabilities. This initiative, undertaken by Fable where the author currently serves as VP of Innovation, aimed to refine methodologies for a demographic that represents the most prevalent disability in the United States. Cognitive disability, an umbrella term encompassing conditions that affect information processing, memory, focus, and learning, impacts approximately 13.9% of the U.S. population, with a concerningly rapid increase noted in recent studies. The working group established four key objectives to guide their exploration: to understand the unique challenges faced by this user group, to develop best practices for recruitment and engagement, to refine testing methodologies for actionable insights, and to create resources for broader adoption within the UX community.

To achieve these goals, the group developed a specialized screener to identify participants who self-identified with challenges in memory, focus, and learning. Concurrently, they conducted a thorough review of existing academic literature on studies involving cognitive testers, extracting best practices and common challenges. This foundational research informed a pilot study involving 25 testers. Through an iterative process of testing and refinement, the team developed a comprehensive guide for conducting user interviews with cognitive testers, alongside a survey designed to quantify their digital product experiences. The culmination of this effort was a detailed documentation of their findings, providing a valuable resource for future research.

Following the pilot study, a hypothesis emerged: participants with cognitive disabilities might uncover a richer vein of usability insights compared to those in general population (gen pop) user research. To rigorously test this assertion, a joint study was initiated in collaboration with Fable’s partners at the University of California, Irvine, with contributions from researchers Syed Fatiul Huq, Pranav Pidathala, Ali Brown, and Michael Fagan.

The Cognitive Usability Study: A Comparative Approach

The core of the validation study involved the creation of three distinct websites using an AI prototyping tool. These websites were designed to represent a variety of user goals and content types, allowing for the testing of a diverse range of tasks.

Table 1: Websites and Tasks Tested

Website	Strong Snacks	Turning Pages	Crown & Comb
Description	A website featuring three-ingredient, high-protein recipes, categorized by type (e.g., vegan, muscle building). It also includes blog posts on protein and contact information.	A curated bookstore with an extensive catalog. Features include genre filtering, a book-swiping mechanism for personalizing preferences, custom book lists, a shopping cart, and a checkout process.	A hair salon website enabling online appointment and consultation bookings. Offers a VIP program and various special package purchases.
Design	Simple, brutalist, bright, with abundant imagery.	Moody, classic, dark, featuring numerous book cover images.	Bold, clean, black and white with vibrant color accents.
Content	Recipes, blog posts.	Books and curated reading lists.	Services, an experience guide, and membership information.
Key Functionality	Filter by category, newsletter subscription.	Shopping cart, book matching, book lists, personalized recommendations.	Appointment booking, online consultations.
Tasks	1. Browse recipes by category. 2. Read a blog post. 3. Subscribe to the newsletter.	1. Find a specific genre of book. 2. Add a book to the cart. 3. Create a custom book list. 4. Complete the checkout process.	1. Book a standard haircut. 2. Find information on the VIP program. 3. Purchase a bridal package.

A single screener, incorporating questions about memory, focus, and learning, was used to recruit participants. These individuals were then divided into two groups: those who self-identified as having cognitive challenges and those who did not. It is important to note that the term "cognitive disability" encompasses neurodiversity, which describes individuals whose brains process information and learn differently. This commonly includes people with learning disabilities like dyslexia, ADHD, and autism.

The study comprised 30 user interviews, with 10 sessions dedicated to each website. An equal split of five cognitive participants and five general population participants was maintained for each website. During each session, participants were tasked with completing all assigned tasks for a single website, facilitated by a researcher. Following their sessions, all participants completed the Accessible Usability Scale (AUS) survey, a 10-question, Creative Commons-licensed tool for evaluating the usability of digital products.

Data Analysis Approach: Quantifying User Experience

The analysis of the study data involved a meticulous review of all recorded sessions and transcripts. Researchers documented every instance where a participant expressed a concern, question, difficulty, or sought clarification on functionality. Each of these instances was counted as an "issue." Furthermore, instances where a participant missed a crucial element of a task, even if they did not explicitly recognize the omission, were also noted. Crucially, every suggestion for improvement made by a participant was recorded.

Issues and suggestions were counted on a per-participant basis, meaning a single participant’s repeated mention of the same issue was counted only once. This approach acknowledges that recurring issues across multiple participants signal universal challenges within the user interface or experience.

Examples of issues identified included:

• Difficulty locating specific information or features.
• Confusion regarding the purpose or functionality of certain interface elements.
• Overwhelm due to excessive options or cluttered layouts.
• Challenges with navigation and understanding the site structure.
• Unclear calls to action or button labels.

Examples of participant suggestions for improvement included:

• Simplifying navigation menus.
• Using clearer and more concise language.
• Implementing visual cues to indicate interactive elements.
• Reducing the number of steps in key processes.
• Providing more context or explanations for complex features.

Findings of the Cognitive Usability Study: A Significant Impact

Across the three websites tested, a clear pattern emerged: participants with cognitive disabilities identified a substantially higher number of usability issues and proposed more suggestions for improvement compared to their general population counterparts. Specifically, cognitive participants uncovered 1.8 times more issues and made 1.8 times more suggestions than the gen pop group. This finding strongly supported the initial hypothesis.

The Accessible Usability Scale (AUS) scores further illuminated these differences, with lower scores indicating poorer perceived usability.

Table 2: Strong Snacks – Performance Comparison

	Total Issues	Average Issues	Median Issues	Total Suggestions	Average Suggestions	Median Suggestions	Average AUS	Median AUS
Gen Pop	32	6.4	6	13	2.6	2	90.5	97.5
Cognitive	49	9.8	9	24	4.8	4	76.8	73.0

The "Strong Snacks" website, characterized by its simple design and content, yielded the lowest overall number of issues and the highest median AUS scores. This aligns with expectations for an easily navigable and straightforward website. However, even on this less complex site, cognitive participants identified an average of 3.4 more issues and made 2.2 more suggestions than the gen pop group. Their average experience score was also 13.7 points lower.

Table 3: Turning Pages – Performance Comparison

	Total Issues	Average Issues	Median Issues	Total Suggestions	Average Suggestions	Median Suggestions	Average AUS	Median AUS
Gen Pop	55	11	10	26	5.2	4	78.0	80.0
Cognitive	86	17	15	42	8.4	6	60.8	58.0

The "Turning Pages" bookstore, with its more varied functionality and a higher number of tasks, unsurprisingly generated the most issues. Cognitive participants on this site found an average of 6 more issues and made 3.2 more suggestions. They also rated the overall experience 17.2 points lower than gen pop participants, indicating a more significant impact on their perception of usability.

Table 4: Crown & Comb – Performance Comparison

	Total Issues	Average Issues	Median Issues	Total Suggestions	Average Suggestions	Median Suggestions	Average AUS	Median AUS
Gen Pop	26	5	4	15	3	3	49.5	35.0
Cognitive	62	12	11	27	5.4	2	63.8	68.0

The "Crown & Comb" hair salon website was intentionally designed with complexity, including a particularly challenging task: locating the bridal package. Intriguingly, on this site, cognitive participants identified an average of 7 more issues and made 2.4 more suggestions. Their average score for the overall experience was notably 14.3 points higher than that of the gen pop participants.

A notable divergence in AUS scores was observed between the "Turning Pages" and "Crown & Comb" websites. While cognitive participants rated "Crown & Comb" higher than "Turning Pages," the gen pop group had the opposite perception. This discrepancy may be attributed to how identified issues impacted each group’s overall perception of usability. For cognitive participants, the sheer volume of issues encountered on "Turning Pages" might have disproportionately influenced their perceived usability more than for the gen pop group.

Further analysis of issue categories revealed distinct patterns. Cognitive participants encountered significantly more issues related to content, buttons and links (affordances and function), icons or visual elements, and media (video, animations). They nearly matched gen pop participants in navigation issues.

Table 5: Number Of Issues By Category

Issue Category	Strong Snacks (Gen Pop)	Strong Snacks (Cognitive)	Turning Pages (Gen Pop)	Turning Pages (Cognitive)	Crown & Comb (Gen Pop)	Crown & Comb (Cognitive)
Content	11	22	11	30	23	36
Navigation	18	22	25	17	2	7
Buttons and Links	0	5	7	20	3	0
Icons or Visual Elements	3	16	2	3	4	23
Media	0	2	0	1	0	0

Qualitative data further enriched these findings. When comparing the verbatim feedback from participants who provided the lowest AUS scores within their respective groups for the "Crown & Comb" website, distinct differences emerged. The gen pop participant described the experience as "frustrating and not engaging." In contrast, the cognitive participant reported feeling "drained and less able to focus," highlighting a more profound impact on their overall well-being. This suggests that usability issues can have a deeper psychological and emotional toll on individuals with cognitive disabilities.

In summary, the study revealed that participants with cognitive disabilities identified a total of 197 usability issues across the three websites, compared to 113 identified by gen pop participants. They also provided 93 suggestions for improvement, versus 54 from the gen pop group. This translates to cognitive participants finding 1.8 times more issues and making 1.8 times more suggestions.

How Cognitive Participants Benefit UX Research: Uncovering Hidden Usability Barriers

The inclusion of cognitive participants in UX research consistently surfaces issues related to cognitive load – a factor that extends its impact beyond individuals with diagnosed cognitive disabilities. These issues can significantly affect:

• Older adults: As the population ages, cognitive function naturally declines, making interfaces with high cognitive load more challenging.
• Users under stress or in time-sensitive situations: High cognitive load can impair decision-making and task completion even for neurotypical individuals.
• New users of a product or service: Unfamiliarity with a digital product inherently increases cognitive load.
• Individuals experiencing temporary cognitive impairment: Factors like fatigue, illness, or multitasking can temporarily reduce cognitive capacity.

The question then becomes: what critical insights might have been missed without the participation of cognitive testers, and how could this have negatively impacted the business outcomes for these websites?

Strong Snacks: Enhancing Clarity and Trust

On the "Strong Snacks" website, cognitive participants highlighted several key areas:

• Content clarity: They identified instances where recipe instructions could be misinterpreted or where nutritional information was not immediately apparent.
• Visual Hierarchy: They noted that the layout did not consistently draw attention to crucial elements like ingredient lists or cooking times.
• Animation: The presence of continuous animation, while visually appealing to some, was flagged as a distraction that could impede focus.

These are improvements that would foster greater trust in the content for all users, while simultaneously making it easier to read and skim for essential information. The findings align with established design principles, such as avoiding constant animation and strategically using layout to guide user attention, principles that even experienced designers would advocate for.

Turning Pages: Addressing Interaction Ambiguity

Without cognitive participants, crucial, albeit subtle, usability issues on the "Turning Pages" website might have been overlooked. These included:

• Confusing interactions: The functionality of the "Add to book bag" button, for example, led to confusion regarding its immediate effect.
• Unclear provenance of information: Participants questioned the source of reviews and recommendations, potentially eroding user trust in the platform’s curated content.

While all participants noted the difficulty in locating the book-matching feature, cognitive testers emphasized a deeper problem: the inconsistency of interactions, which hindered their ability to predict and understand system behavior, thereby diminishing their confidence. For any user seeking to purchase a book, a clear and unambiguous understanding of the process – from adding items to the cart to completing checkout – is paramount. Over time, a lack of clarity in such a critical flow can translate into lost revenue, impacting hundreds or thousands of potential customers.

Crown & Comb: Bridging Usability Gaps to Accessibility Barriers

The "Crown & Comb" website particularly underscored the benefits of including cognitive participants, who raised concerns about:

• Information Overload: The sheer volume of services, packages, and membership tiers presented a significant cognitive burden.
• Ambiguous Navigation: Finding specific services, like the bridal package, required navigating through multiple layers of information, often with unclear labeling.
• Inconsistent affordances: The visual cues for interactive elements were not always clear or consistent, leading to uncertainty about what could be clicked or acted upon.

The difficulty in locating the bridal package was a challenge for all participants. However, the cognitive group demonstrated how this usability issue escalated into an accessibility barrier. When combined with:

• Complex task flows: Requiring multiple steps and decisions.
• Dense information presentation: Overwhelming users with too much text or too many options.
• Unclear pathways: Lacking direct or intuitive routes to desired outcomes.

This creates a high cognitive load that can prevent some individuals from completing the task. This is precisely where usability issues can transform into significant accessibility barriers, overwhelming certain users and rendering a digital experience inaccessible.

Key Takeaways for Inclusive UX Research

The findings from this study offer several critical takeaways for UX professionals aiming to create more inclusive digital experiences:

• Cognitive participants provide disproportionately high-value insights: Their ability to identify usability issues that impact cognitive load makes them invaluable for improving clarity, predictability, and ease of use.
• Usability issues can become accessibility barriers: When cognitive load becomes too high, even seemingly minor usability flaws can prevent users from completing tasks.
• Focusing on cognitive load benefits everyone: Improvements made to address cognitive load, clarity, and predictability enhance the experience for all users, not just those with diagnosed cognitive disabilities.
• Cognitive inclusion serves as an on-ramp to broader accessibility research: By addressing cognitive load first, teams can build a stronger foundation for future accessibility testing with assistive technology users.

Start Incorporating Cognitive Insights Now: The Future of UX

The demographic landscape of the United States is shifting, with the population aged 65 and older projected to constitute 25% of Americans by 2060. This demographic trend is directly linked to increased prevalence of cognitive decline. As this segment of the population expands, businesses will face a growing imperative to design for more complex user needs.

Individuals with cognitive access needs represent a natural starting point for inclusive research because they consistently identify usability issues that UX teams are already accustomed to addressing. This can make cognitive inclusion a more accessible entry point into broader accessibility research and testing. While insights from assistive technology users remain crucial, many teams struggle with how to initiate such research.

Cognitive accessibility serves as a powerful gateway into more comprehensive accessibility initiatives. By prioritizing cognitive load, clarity, and predictability, UX teams can establish research methodologies that simplify future work with screen readers, screen magnifiers, and alternative navigation users. As one UX Manager at Bell Media observed, "Two sessions with cognitive users feel like 200 because of the volume of insights we get."

In this exploratory study, participants with cognitive disabilities identified 1.8 times more issues and proposed 1.8 times more suggestions than general population participants. This level of impact has been observed in research conducted by Fable’s clients on non-AI-generated websites as well.

Cognitive inclusion in UX research is no longer optional; it transcends mere accessibility compliance. It is a strategic approach that empowers UX teams to enhance research efficiency, cultivate clearer content, simplify user flows, and ultimately deliver superior products for a wider audience.

Study Limitations and Future Directions

This study, while providing significant insights, had certain limitations. The relatively small sample size means the findings are more qualitative than quantitatively validated. Furthermore, testing was conducted on two different platforms: Fable Engage for cognitive participants and UserFeel for general population participants. Variations in platform features and participant panels can influence the quality and depth of insights, as well as participants’ comfort levels with the research process.

The study also involved different researchers facilitating the user interviews. While efforts were made to maintain consistency through identical task structures and discussion guide templates, the facilitator’s approach can inevitably influence the dynamics of a session. However, to ensure consistency in data analysis, all issue and suggestion counts were performed by a single individual, applying the same criteria across all websites and participants.

Looking ahead, continued research and the development of standardized methodologies for recruiting and engaging cognitive participants are essential. As the understanding and acceptance of cognitive inclusion grow within the UX community, the potential for creating truly universally designed digital experiences will be significantly amplified.

Resources for Cognitive Inclusion

For those embarking on their cognitive inclusion journey, the following resources offer valuable guidance and support:

• Fable: A platform dedicated to inclusive user research, offering tools and insights for testing with diverse user groups.
• Accessible Usability Scale (AUS): A free, Creative Commons-licensed survey for evaluating website and mobile app usability.
• Cognitive Accessibility Guidelines: Resources from organizations like the W3C and others that provide best practices for designing accessible digital content.
• Neurodiversity in UX: Communities and publications focused on integrating neurodivergent perspectives into the design process.