Role
Researcher
Recruiter
Moderator
Designer
Editor
Recruiter
Moderator
Designer
Editor
Timeline
4 Weeks
Client
Wikimedia
Foundation
Foundation
Team
4 UX Consultants
Type
Academic
This study was conducted In association with the Wikimedia Foundation. The study focused on assessing the effectiveness of Wikipedia's search page from the perspective of experienced editors.experienced editors. The main objectives for our team were to identify areas that require improvement to enhance the user experience and simplify information retrieval for editors. Through Moderated User Testing, we uncovered significant issues with the current search page and its functions. Our team also developed actionable recommendations to make it more usable.
This usability study aimed to assess the effectiveness of Wikipedia's search page and function from the perspective of experienced Wikipedia editors*. The client was interested in knowing the types of tasks editors use the search function.
*Experienced editors were defined as individuals that have edited multiple articles in the past, and log in to their personal accounts to make edits.
The assumptions provided to us by the Wikipedia team during our initial meeting, based on their previous reseach were:
These foundational assumptions served as the cornerstone for our exploration. Our objective was to study the navigation patterns of editors and their interactions with the search page. We aimed to identify potential usability issues or deficiencies in functionality that may impede editors' capability in completing their tasks effectively.
We collected the required data using moderated remote usability testing. It can be broken down to 4 parts:
Developing questions and interview script based on the focus areas of the research.
Selecting participants that matched the persona of experienced Wikipedia editor.
Conducting interview sessions to get insights into usability challenges.
Creating a report with research findings and actionable recommendations.
We created a screening questionnaire to help identify suitable participants for the study. This screener form was distributed through multiple online channels such as email lists, the Reddit, Discord, and Wikipedia's own community forums.
Out of 56 people that responded to the screener form, 7 were chosen to take part in the moderated testing sessions. The main criteria used for selecting participants was their level of experience with editing Wikipedia articles.
The user testing sessions were scheduled as remote interviews conducted via Zoom, with each session lasting between 45-60 minutes. During the interviews, participants were asked to complete a series of tasks while thinking aloud, allowing us to gain insights into the participants' thought processes and any challenges they faced.
Two researchers worked with the participant for each remote user testing session. One researcher guided the participant through the tasks, while the other researcher took notes.
Participants in our study struggled to locate the search page. One way to reach the search page is to run an empty search which was not known to our participants. All of the participants needed assistance reaching the search page.
Our proposal involves adding a dedicated button on the homepage that leads users directly to the search page. This button would be prominently placed and easily visible to users and we suggest labeling it as "Advanced Search". By positioning it in close proximity to the default search bar, we aim to provide users with convenient and efficient way to access more detailed search options,
Testing reveals that fewer than half of the participants used advanced search filters and capabilities available in the advanced search section. Many users indicated that this was because advanced features were initially hidden within a dropdown menu. Moreover, a significant number of users reported difficulties in comprehending the explanations and language used on the page.
Our proposed solution involves expanding the dropdown menus by default and rearranging the advanced search features within the available on-screen area to eliminate the need for scrolling. We also provided more visible and explicit instructions on how to use these features, so users can easily understand how to utilize these features.
Approximately half of our participants attempted to input natural language queries directly into the search bar, resulting in error messages or irrelevant search results. Some of these participants expressed disappointment with the search functionality as they expected similar queries to yield relevant results and links similar to what they would find on search engines like Google.
Our recommendation here is to incorporate an advanced search algorithm to enhance the system's ability to comprehend natural language queries. This algorithm could potentially scan the Wikipedia Database and swiftly provide the user with desired information, along with a link to the most relevant article for further reading.
According to feedback from participants, there is a desire to filter search results after the initial search is carried out. Currently, the platform does not offer the option to refine results without having to go back to the top, adjusting the search parameters and re-running the search. This process is tedious and time-consuming.
Our proposal is to introduce a faceted search panel with the various categories related to the search term. The proposed faceted search feature would enable the users to narrow down their search results based on different facets including year, content type, category and more. This would provide users with the ability to filter their search results efficiently without having to re-run the search.
The client was appreciative of our efforts in conducting the usability testing and agreed with the critical problems we identified in their interface. They were eager to look through the detailed report we provided and were open to working with us to further.
If we had more time, we would conduct more user testing sessions as it was difficult to find experienced editors who are well-versed with the search page to verify our findings and potentially discover more problems.
Working on this study helped me improve my ability to conduct usability interviews. Also learned how collaborating and organizing efforts with the team members through clear communication are vital to facilitate the research study.
