Using Eye-Tracking to Improve the Discoverability, Readability, and Learnability of the Cooper Hewitt API Website.
UX Research
Eye-Tracking
Task
Laiba Sarwar (Me)
Kato Oppenheim
Phil Garip
Team
2024 (3 months)
Duration
Tobii Pro
Excel
Figma
Tools
My Role
UX Researcher
Project Overview
The Cooper Hewitt API provides its users with access to the data of 215,000 items in their collection, along with information about the people, companies, and other organizations that were associated with them.
This project tested how novice and expert API users interact with the Cooper Hewitt API website by conducting a usability study and eye-tracking. We provided several recommendations focused on improving how people locate the API website (discoverability), find API examples (readability), and learn to run a query (learnability).
Our pitch meeting to the Cooper Hewitt developers and managers.(Left to Right: Kato, Phil, Laiba (Me))
THE CHALLENGE
Stakeholders worry about how the platform can teach users how to create projects from various skill levels because of the lack of discoverability.
METHODOLOGY
TASK CREATION
What kinds of tasks can we give the participants to fulfill the objective?
Since the client's ask was to check how the users found the API documentation and how they could understand it, we focused on creating tasks that can individually help understand each user' process.
Which is why in order to track how they found the documentation, the first task was to get on the API page that they currently have linked. For testing how the documentatiion is read, the second task of finding an example and inspiration for projects lets the participants be able to explore and learn more about the platform. Lastly, the third task is created to see how the participants actually go about creating their queries and changing them so that we can see what they have learned from the documentation and if they know can start creating a project of their own.
Task 1 (Discoverability)
Find the API Documentation from the Cooper Hewitt home page.
Task 3 (Learnability)
Consult the API documentation and run a query. Now modify it!
Find an example of how the Cooper Hewitt API has been used in other projects and explore (i.e.: creatively, business application, etc.).
Task 2 (Readability)
PARTICIPANT RECRUITMENT
What kinds of users did we recruit and test for?
One of the main motives of the documentation from Cooper is that the platform is used and understood by not only advanced users who have used and created work from APIs before but also by new or novice users who don't use APIs often.
Therefore, we recruited both types of audiences so that we could see if there were any patterns and insights within the two groups and also how each participant individually learns the platform. We wanted to cover each base which is why there is a variety of experience and knowledge within each group.
Novice Users (N=3)
Aged 18 - 35
1-5 API knowledge
Less than 6 months of API experience
Advanced Users (N=3)
Aged 35 - 64
7-9 API knowledge
4+ years of API experience
EYE TRACKING STUDY
So, how did we conduct the test?
THE RESULTS
3 - Neutral
Tested By
Post Test Questionaire
Efficiency
58.5/100
(Average is 68)
Tested By
System Usability Scale
Satisfaction
14% Increase
Tested By
Pre & Post Test Questionaire
Effectiveness
Saccades, Fixation, Duration
For us to understand each user's journey. We used the Tobii platform that let us conduct the eye-tracking tests seamlessly with the participant.
We analyzed how each participant went through the platform by focusing on...
Saccades
The path of the center of vision from one part of the visual field to another.
Fixation
Locations on the page where participants stay for at least 200 ms.
Duration
The time it takes each participant to complete the task.
Retrospective Thinking Aloud
During the eye-tracking test, we encouraged the participants to conduct each task to the best of their ability without guidance.
Once each participant is done with the tasks, we review the recording with the eye-tracking movements and ask follow-up questions to better understand the participant' thought process.
After presenting to the Cooper Hewitt designers, developers and managers, they expressed how grateful they are for the extensive feedback and the thoughtful designs.
They asked follow-up questions on the two types of users we tested and the recruitment process for them which we were able to clarify on the levels of experience they had.
Interests
The recommendations were also well received and laid out for their own pitches. We also recommended more research and uset testing as well as better resources for A/B Testing and Data Analysis which could further help their discoverability and learnability efforts.
Next Steps
Final Presentation with Clients
Pre-Test
When recruiting our participants, we asked questions about their experience, knowledge, and technical skills.
Post-Test
After the eye-tracking test, we provided each participant with questions on how they can self-reflect their knowledge.
System Usability Scale (SUS)
The SUS 10-Question questionnaire offers a way to evaluate the usability of a website by a scale from 1-5 (Strongly Disagree to Strongly Agree)
1.
Focus On New Users
3.
Understand what users are making with the API
2.
Learn If Visitors Can Comprehend The API
THE ASK
What do the client' want to focus on?
THE OBJECTIVE
Identify the user behaviors of different types of users of the API documentation in order to improve site accessibility and learnability.
CHALLENGE 1
Users Found it Hard to Navigate to the API Page
5/6 Users
Had Trouble Finding The API
"
If they really want people to get to that page, I don’t know why they would make it so small at the bottom?” “If I wanted someone to look at it, I’d want to have it [in the global navigation].
- Participant 2 (Novice User)
OVERALL FINDINGS
Participants did increase their knowledge of APIs but had difficulty understanding the code and the layout.
RECOMMENDATION 1
There are 3 different ways the API Page can be incorporated within the navigation.
Include The API Page In The Navigation
SURVEYS
In what ways did we test participants increase in knowledge?
2/6 Users
Did Not Notice The Search Field
"
As a museum, I'm not expecting them to surface their API docs immediately on search.
- Participant 4 (Advanced User)
"
Because there wasn’t as much detail provided in it, as well as the spatial area; it’s so narrow that it’s so easy to look past. If there was some color added to it or some descriptive copy – it’s so easy to look past.
- Participant 2 (Novice User)
CHALLENGE 2
The Search Feature Should Be More Noticable
RECOMMENDATION 2
Provide Search Its Own Field
"
It would be great if there was a caption that's just like, 'The Cooper Hewitt's API is blah blah blah of x-number of items in the collection,' or something like that would have been cool but also probably not how they tell search results apart.
- Participant 6 (Advanced User)
CHALLENGE 3
The Search Results Should Include Thumbnails and Descriptions
Task 2 - Reliability
RECOMMENDATION 3
Add Images/Details On Search Results
Task 1 - Discoverability
CHALLENGE
Users were confused on the location of where to find example projects
4/6 Users
Wanted More Project Examples
"
Sorta hoping there were were more projects, but, you know, c 'est la vie.
- Participant 6 (Advanced User)
"
What I think I was probably assuming is that there might be a sort of hero example on the homepage. Here's our API, here's what it could do. Here are some examples in action that would be relatively straightforward.
- Participant 4 (Novice User)
Where Were They Looking?
Task 3 - Learnability
Home Page
User’s gazes were fixated towards the top of each page when looking for project examples.
Heatmap of all six participants gazes on the API Hompage, for Task 2.
Showcase
With three projects on the page, the first project received heavy visual engagement.
Heatmap of all six participants gazes on the Showcase Page, for Task 2.
API Page
Only one participant went to the bottom of the API documentation to look at the example queries.
Participant Q4's eye movements on the API Page, for Task 2
5/6 Users
Needed More Guidance On Writing Queries
Snippet of Participant 1 behavior while attempting to run a query
"
I was looking to see if there were queries that were already there...I can run off an example.
- Participant 1 (Novice User)
"
I opened up [The API page] where I need examples and then a second tab where I'm actually executing things in the editor.
- Participant 4 (Novice User)
RECOMMENDATION
Consolidate and Feature Projects In One Place
CHALLENGE
Users Needed More Help Getting Through The Platform
RECOMMENDATION 1
Add A Panel To The Editor
RECOMMENDATION 2
Provide A Video Tutorial
THE CONCLUSION
FEEDBACK
Well, What Did The Clients Say?
TAKEAWAYS
What Did I Get From This Project?
What did I learn throughout the project?
Eye-Tracking, Eye-Tracking, Eye-Tracking!
Learning the eye-tracking method is so important and valuable to UX researchers and I was able to see exactly why.
Collaboration
Working in a team can test a project, but with this team, I was grateful to continuously improve my collaboration skills by communicating and being more confident in my insights.
What would I have done differently if I had another chance?
Data Analysis
Since the API site is fairly recent, there was no real data we could look at to see user drop-offs on Google Analytics or Hotjar. This research could be very vital to the insights and should be included in all projects.
A/B Testing
Providing recommendations on specific changes and their impact could also have helped with insights that would strengthen the project even more.
What were some challenges that I saw and how did I fix them?