An example of recommendations that IE Lab user testing can provide to platforms like Nexus@Mizzou
Usability Study for MU Extension Nexus@Mizzou Program
In spring of 2018, the IE Lab worked with MU Extension’s Nexus@Mizzou team and conducted a study on their recently updated website, https://extension.missouri.edu. The goal of the project was to evaluate the usability of the website and provide recommendations for improvement of the customer experience.
The study has been planned with two phases; the first phase, the IE Lab was asked to analyze a set of ten defined tasks and successful pathways users must take to complete them. Based on the tasks, IE Lab assembled a list of general users between the ages of 22 and 60, with various levels of technology proficiency to navigate the website, using different electronic devices while completing these tasks and talking through their process (task-based usability study with Think Aloud method). After observing users’ chosen pathways and post interviews, the team analyzed data and developed recommendations, which summarized the users’ actual experience with the system.
Phase II of detailed user testing will take place in 2019 after Extension has completed the consolidation of all program websites, content, online courses and events onto the platform.
Mobile Applications for Microlearning: A Review of Academic and Industry Implementations
In 2017 – 2018, The IE Lab worked with the Reynolds Journalism Institute (RJI) and conducted a study for RJI Fellow, Linda Austin, Project Director for NewsTrain. The main purpose of this project was to explore technical systems for microlearning and specifically provide design recommendations for mobile microlearning applications for journalists.
The project was organized into two project phases. In Phase 1, IE Lab evaluated best practices and ideas for microlearning approaches that can be used on smartphones. Three research methods used were academic literature review, popular trade/press review, and a platform cross-comparison matrix of app features for microlearning purposes. In Phase 2, IE Lab conducted interviews for understanding the successes and failures of microlearning in journalism.
Based on the Phase 1 results, the deliverables included a summary report of the academic literature review, a summary report of the industry (popular/trade press literature) review, a matrix of best practices/characteristics embodied in the Unleesh mobile app versus other six mobile learning systems. Based on the interview results (Phase 2), IE Lab identified types of skills that are best taught in mobile microlearning as well as how to teach, structure and present micro-lessons for journalists.
Actionable techniques recommended for designing mobile microlessons:
- How to teach/design microlearning lessons: interactivity, feedback, personalization
- How to structure courses: independent micro-lessons with content segmentation, learning paths, multi-formats
- Technical settings: push notifications, learning analytics, big data analytics, dashboard design
- Short micro-lessons and active assignments: brevity, granularity, learner assignments / problem-solving
More Information about our client:
LexMizzou Winning Team
LeXMizzou – a Learning Expedition@Mizzou
From 2015 to 2017, the IE Lab supported the LexMizzou project, a team led by Dr. Isa Jahnke, Associate Professor in the School of Information Science and Learning Technologies, which engages students and other faculty researchers with the goal of creating meaningful learning that goes beyond classroom learning and supports learners to explore and connect topics and theoretical concepts, augmenting information with physical objects in the real world (LeXMizzou, 2015).
The overall goal of the LeXMizzou project is to give students access to become game designers for learning. Students design, develop, and evaluate a location-based augmented reality mobile game around the campus of the University of Missouri-Columbia.
The LeXMizzou project started in September 2015 and ended December 2017. Throughout the entire project lifecycle, IE Lab continuously provided usability study support for the project, covering UX fundamentals, design lifecyle, instruments, tools, presentations and training on usability study.
On October 26, 2017, IE Lab researchers provided a presentation on how to conduct usability studies for the LeXMizzou student teams. The presentation included two parts: a presentation of some fundamentals of usability/UX study (presented by IE Lab manager Neeley Current and researcher Hao He), and a working session that helped teams develop their usability plan (hosted by IE Lab researcher Minh Pham).
LeXMizzou. (2018). About. Retrieved from Learning Expedition at Mizzou – LeXMizzou Renewal : https://lexmizzou.wordpress.com/
Comparative Systems Review
In 2017, the IE Lab conducted a Comparative Systems Review for RJI Fellow, Dr. Michelle Ferrier, Associate Professor, Scripps College of Communication, Ohio University. The overall goal of the study was to assess whether a current system(s) exists that may support the learning, community, and social needs of the Media Innovation Space tool under development by Dr. Ferrier. The system intended to offer a Grow in Place model that may consist of a distributed, hybrid online/face-to-face offerings in which users may create profiles, showcase their work, access resources for Just-In-Time learning, and foster strategic partnerships. The primary audience for the Media Innovation Space is student entrepreneurs and the space will serve as a Virtual Accelerator.
Two phases were conducted for the study:
Phase 1. Technology/System Review & Needs Assessment:
IE Lab compared Dr. Ferrier’s Media Innovation Space system requirements with existing systems functionality, interaction design, navigation, layout and aesthetics.
Phase 2. Sitemap & Wireframe Prototype Development
With direct input from Dr. Michelle Ferrier, IE Lab:
- Created workflows for the site: graphical representation of actions and activities of users on site
- Developed a basic sitemap for the site: important for understanding the content organization
- Developed low fidelity wireframe prototypes for the site: generated wireframes to show page structure and content requirements for individual pages. (Design tools: AxureRP and Balsamiq software)
In Phase 1, a report provided sample interaction design, features and functions that may meet the design requirements for the Media Entrepreneurship system. In Phase 2, the workflow and basic sitemap for the site were provided. Also, a low fidelity wireframe in AxureRP and Balsamiq software for five main pages were created.
More Information about our client:
PUSH: An open source mobile platform for media organizations
In Oct 2017, the Information Experience (IE) Lab at the University of Missouri, Columbia worked with the 2017-2018 Reynolds Journalism Institute Fellow, Christopher Guess to conduct a usability study of the Push app. Push is an open-source mobile platform for media organizations to build their own mobile news apps for news distribution. The IE Lab conducted a task-based analysis, a heuristics design review, and a mobile device test on the MeydanTV and the Bivol apps – two apps designed and developed on the Push platform – to thoroughly identify the usability and the issues of the apps built on the platform. Additionally, the IE Lab translated the Push app into five languages, Vietnamese, Farsi, German, Chinese and Arabic to support media organizations in building their mobile news apps in these languages..
The findings from the study of the apps revealed that the Push platform has great potential to help small media organizations build their own quality mobile news apps. The apps designed and developed on the Push platform include fundamental functions required for mobile news apps. Also, media organizations, especially small startup publishers, can forget about the hard-coding normally required to build apps and easily and quickly develop their mobile news apps on Push. The study also identified functionality, design, navigation, and user control issues of the apps. Recommendations for the further enhancement of the user-friendliness of the apps were also provided based on Neilsen’s heuristics principles and the cross-comparison of the other mobile news apps.
For further information of Christopher Guess and his Push app stories, click here:
Usability Research for St. Charles City County Library Website
In May 2017, the IE Lab conducted a usability evaluation of the St. Charles City-County Library (SCCCL) website (http://www.youranswerplace.org) during its prototype stage of development. Eighteen library patrons participated in the study. The eighteen participants were divided into six groups, based upon library user personas developed by SCCCL. Specific tasks were created for each persona type in order to evaluate user interactions and identify design problems of the prototype website.
A member of the IE Lab team met with each participant individually for up to an hour during which participants performed tasks on the prototype of the library website. Each participant completed six to eight task prompts that were based upon the persona group to which they were assigned. The sessions were recorded with Morae – an usability software that allows the capture of participants’ mouse clicks and cursor movements on the prototype interface, as well as the audio and video recording of the participant.
The prototype of the SCCCL website has a lot of potential; however, the study with participants indicated that there is still room for improvement. Some of the key recommendations the IE Lab made to improve the website include the findability of resource content, the robustness of footer to aid users in the site navigation, and the choice of terminology in navigation bars.
Evaluation of ElevatEd; K-12 Strategic Planning Platform
ElevatEd is a web-based knowledge management system designed for school administrators and teachers. It was designed and created by MU Hook Center intended for K-12 schools with the main goal to improve workflow related to strategic planning in schools, e.g., creating and executing data driven strategic improvement plans.
The IE Lab started with user evaluation of the ElevatEd system in Fall 2015. The main goal of the study conducted by the IE Lab focused on evaluating the usability of ElevatEd. The IE Lab studied whether the users were able to use the tool, in particular, whether it is usable for creating and conducting strategic actions more efficiently, and whether the users were satisfied (ISO Norm 1998/2015; Sheehan, & Lucero, 2015). These goals guided the overall usability study. Through the evaluation of the web-based application, IE Lab was able to recommend improvements.
IE Lab applied the following methods to study the usability of ElevatEd.
- Pilot Design Review
- Task Analysis Pilot Study
- Interviews and Prototype Analysis
- Modelling Workflows with Sociotechnical Walkthrough (STWT), SeeMe Models (Herrmann, 2009)
The final recommendations were concluded based on the data collected from both participants and IE Lab experts and the data analyzed. Some key recommendations were:
- Keep the system as simple as possible; it shouldn’t require too many clicks.
- It should be accessible with tablets and laptops without major alteration to page structure.
- The system should be able to sync the existing school calendar to the new app and/or to Google/Outlook Calendar.
- Terminology and labeling needs some revision.
- The system should consider integrating a student-centered view that focuses on progressing individual students. In addition, the system should consider enabling a user to visualize their roles within the system.
- It would be better for teachers to see graphical display at a glance of the results once they have input the data into the system.
Herrmann, T. (2009). Systems Design with the Socio-Technical Walkthrough. In Handbook of Research on Socio-Technical Design. New York: Hershey, pp. 336-51.
Hook Center. (2018). About. Retrieved from Hook Center for Educational Leadership and District Renewal : https://education.missouri.edu/hook-center/about/
ISO 9241-11 (1998), Standard Usability Framework, International Standard Organization, revised 2015, www.iso.org
Sheehan, B. & Lucero, R. (2015). Initial Usability and Feasibility Evaluation of a Personal Health Record-Based Self-Management System for Older Adults. In eGEMs (Generating Evidence & Methods to improve patient outcomes), Vol. 3, Issue 2, Article 3, 2015 DOI: http://dx.doi.org/10.13063/2327-9214.1152, http://repository.edm-forum.org/egems/vol3/iss2/3.
Time Up and Go Test (TUG test): Kinect-based Fall Risk Evaluation and Feedback System
The TUG test is a Kinect-based fall risk and evaluation feedback system for elderly users developed by Dr. Phillips Lorraine and Mizzou engineering students. The TUG test times an individual to rise from sitting in a standard armchair, walk 3 meters, turn, return to the chair, and sit down. It is a measure of functional balance and a sensitive and specific measure of fall risk in community-dwelling balance-impaired older adults. The test used Kinect depth cameras to compute gait parameters.
In Fall 2016, the IE Lab was hired to conduct usability testing and offer refinement of the Kinect-based fall risk evaluation and feedback system. In this project, the tools and methods we applied for the evaluation of the TUG test include the pilot test, instruction slides, interview guide and System Usability Score survey (SUS), observation note table, and Morae recorder software.
The findings from the evaluation show that the TUG test system still needs improvement in three aspects: instruction slides, walking test, and the result sheets. Some main recommendations to the TUG test system are that the duration of the instruction slides should be optimized, the walking test should give participants clear signals and indicators as for when to start and when to stop, and the interpretation text should be placed at a more obvious location on the result sheet.
Reading patterns on the Panacea’s Cloud index page, using Tobii as the eye-tracking tool
Heat map for the Panacea’s Cloud index page
Panacea’s Cloud: An augmented reality (AR) triage and communication tool for mass casualty disaster incidents
Funded by the Wallace H. Coulter Foundation, Panacea’s Cloud is a collaboration between MU researchers Prasad Calyam, assistant professor of computer science, and Sal Ahmad, assistant professor of surgery and medical director of the Trauma and Surgical Intensive Care Unit. Panacea’s Cloud is an augmented reality communication and coordination tool for mass casualty disaster situations intended to “coordinate resources, information sharing, and two-way communication between medical incident commanders (ICs) and first medical responders (paramedics)”. In multiple concurrent disaster situations, task completion time and task completion success rates are critically important to save people’s lives (Demir et.al, 2017). Any system built to respond to mass casualty incidents (MCI) sites should be able to have easy-to-use interface which can effectively analyze and present information and foster critical decisions to reduce casualties (Demir et.al, 2017). Usability research is therefore indispensable to improve the system and to make the best solution available in terms of technology and user-friendly interface design.
The IE Lab was hired to conduct the usability research for Panacea’s Cloud. The usability research was conducted between March 1 – May 5, 2016 in order to evaluate the initial proof-of-concept product of Panacea’s Cloud. This system has been iteratively developed and refined based on user experience research driven methodology that employed a mixed methods approach, including the views of clinical experts. Specifically, the IE lab initially conducted the design review to test and revise the system; then, task-based eye-tracking studies; and post-task surveys and interviews were conducted to derive further design recommendations.
Main design recommendations for the system include incorporating Situational Awareness features, a Synchronous Map View system, a Hands-Free Communication service with AR and smart glasses, Digital Notes, and resilient Wi-Fi networks (Demir et.al, 2017).
For further information about Panacea’s Cloud system and its use as a disaster solution, please read more at:
Demir, F., Ahmad, S., Calyam, P., Jiang, D., Huang, R., & Jahnke, I. (2017). A Next-Generation Augmented Reality Platform for Mass Casualty Incidents (MCI). Journal of Usability Studies, 12(4).
Division of IT
The Division of IT at the University of Missouri recently updated their website to include more options on their homepage as well as dropdown menus for improved information architecture. The IE Lab conducted a think-aloud usability study in order to evaluate the new website and learn the perspectives of current University of Missouri students, faculty members and staff. The goal of the study was to evaluate the Division of IT website by looking at how users located commonly sought information such as email accounts and the password manager. During the study, users frequently commented on the usability of the website, which gave the researchers an insight into the overall perceived functionality, navigation and aesthetics. A findings report was delivered to the Division of IT and the site went live in the summer of 2014.
Cerner Health Wins
The IE Lab was hired by Cerner to help them develop Cerner Heath Wins. Health Wins is a Web based application designed to coordinate and motivate weight loss competitions at the individual or organizational level. The lab provided expert reviews of the site, opening and closing surveys for their pilot launch, and participant driven usability tests of their application. We were able to identify functionality, navigation, and information architecture issues that helped them improve the project.
University of Missouri Libraries
According to their about section: “The MU Libraries serve a student body of 34,748 plus a faculty of 2,121, and have a collection of 3.1 million print volumes, 53,400 journal titles (in print or online), and over 7.5 million microforms. With an annual budget of $16.8 million, the Libraries support the instruction, research, service, and economic development missions of the University of Missouri. By acquiring scholarly resources, developing innovative services, and applying new information technologies, the MU Libraries fulfill their primary purpose: to serve our users.” (http://library.missouri.edu/about/) The IE Lab helped the MU libraries fulfill their commitment “to serve our users” by making a more user-centric homepage.
The IE Lab assisted in four phases of the MU library website redesign process that implemented various aspects of user centered design principles. These phases included: a card sort, verifying information architecture, prototype testing, and a summative evaluation of the redesign. These phases were conducted in an iterative manner with each subsequent phase building upon the findings of the previous phase. After each phase, members of IE Lab met to discuss the findings and to provide recommendations for the Library Website Redesign Committee.
- Simplified the appearance by adding more white space and reducing the amount of content on the homepage
- Went from 7 navigation menus to 5
- Re-worded the navigation menus based on user feedback to make the navigation menus more in line with how users
- Added a Quick Links section
- Added 4 interactive icons
The opportunity to work with MU Libraries allowed the IE Lab to implement innovative usability techniques and help create a redesign that will better serve the MU community.
Newsy.com is an innovative online news service that connects users with ‘Multiple sources to help provide the real story.’ Newsy.com was established as a start-up company with strong ties to the Missouri School of Journalism at MU. The IE Lab worked with Jim Spencer, President of Newsy.com, and his staff to determine what usability studies should be conducted before Newsy.com went public.
A rapid user-centered iterative approach was used in the study. A two-part study with potential users conducting tasks and providing verbal feedback was recorded and analyzed. In a one week period, the study was designed, data was collected & analyzed, and a findings report was delivered to Newsy.com with actionable feedback. Newsy.com took the feedback, made immediate changes to the website, and a second study was conducted in similar fashion soon thereafter.
Below is the list of the clients the IE Lab have served since its establishment.
1) University of Missouri
- College of Education
- College of Engineering
- Division of Extension
- Division of Information Technology
- Human Resource Services
- International Center
- MU Extension
- MU Hook Center
- MU Family & Community medicine
- MU Library
- Office of Admissions
- Office of Research
- Office of the Registrar
- RJI Fellows, School of Journalism
- School of Nursing – Tiger Place
- Cerner Corporation
- East Central College of Missouri
- Fontbonne University Library
- International Society for Performance Improvement, St. Louis Chapter (ISPI)
- Missouri State University Library
- MOBIUS Consortium
- LANIT Consulting, Inc.
- St.Charles City-County Library
- Tiger Institute
- Shelter Insurance
- East China Normal University Distance Education College
- Taiwan Office Missouri Department of Agriculture
- Tamkang University Libraries
Tamkang University (TKU) – Taiwan
Our first international collaboration is operated by the College of Education at Tamkang University (TKU), in Damsui, Taiwan. The E-learning Usability Testing Lab, coordinated by Dr. Chun-Yi Shen and staffed by TKU graduate students, provides services to clients at TKU and other educational institutions and companies throughout Taiwan. Faculty and staff in Missouri and Taiwan work together on joint service and research projects, focusing on cross-cultural topics.
Thailand Cyber University (TCU)
In cooperation with the IE Lab, Thailand Cyber University (TCU) works to improve the usability of existing and new online courseware being developed at TCU. A consortium of 27 universities, TCU was created to serve as a means to centralize and distribute online courses offered in Thailand.
East China Normal University (ECNU)
The IE Lab is partnering with the Distance Education College at East China Normal University (ECNU) to develop and undertake collaborative research and service projects. In 2008, ECNU opened the ECNU-MU Joint Lab of Usability Engineering. The laboratory has been used for a number of usability projects related to distance education by faculty and staff.
Nanyang Technological University (NTU) – Singapore
NTU’s Centre for Educational Development (CED) has developed a new usability laboratory and assigned two staff members whose responsibility is to identify and create usability studies for faculty online course content and distance education courses. IE Lab faculty and graduate students have the opportunity to collaborate with NTU peers to develop joint research projects using data collected in the CED laboratory. CED and the IE Lab are sharing operational procedures and processes for continuous quality improvement.
E-Knowledge NGO (MeK) – Mongolia
In 2009, the IE Lab entered into its fifth partnership with the Mongolian e-knowledge NGO (MeK). This partnership initiated with a presentation from the Chairperson of MeK, Ms. Tsetseg-Ulzii Yadamsuren who provided to informational presentations on the general campus and a needs assessment on the e-knowledge projects for the campus. MeK has just begun their first offering of elearning courses in Instructional Design and initiates a collaborative research project with MU on e-learning usability evaluation