Client Stories

 

All:

 

Time Frame:

 

Methods:

 

A User-Experience and Efficacy Study of the Mobile-Microlearning Course, “The 5 C’s of Writing News for Mobile Audiences”

Time Frame:

Fall, 2018

Method:

  • Pre- and post-tests
  • Survey

Goal: The goal of the IE Lab work was to study usability and learning efficacy of Linda Austin’s mobile-microlearning course. The study sought to discover whether those who took the course showed improved skills in writing news for a mobile audience.

Client:
Reynolds Journalism Institute Fellow Linda Austin, project director for Associated Press Media Editors’ NewsTrain

Goal:
The goal of the IE Lab work was to study usability and learning efficacy of Linda Austin’s mobile-microlearning course. The study sought to discover whether those who took the course showed improved skills in writing news for a mobile audience.

Process:
The study involved three phases:
In Phase 1, data-collection instruments were developed and pre-tested. These included before- and after-course surveys, after-lesson surveys and before- and after-course tests.
In Phase 2, the actual study was conducted with 35 target users who were in the field of journalism.
In Phase 3, the IE Lab analyzed the collected data and delivered both qualitative and quantitative results, as well as recommended improvements in the 5 C’s micro-lessons.

Findings:
Eighty percent of the learners obtained higher test scores after completing the course, and that increase in scores was statistically significant. Before the course, participants had an average score of 56. After the course, participants had an average score of 73, an increase of 17 points. If we exclude guesses, their scores went up 56 points. The greatest number of points that an individual learner gained was 90 points. (The maximum number of points on the test was 100.)
When participants completed the course, they gained more knowledge and learned what the course developer intended them to learn.
Users’ confidence level in writing news headlines and stories for mobile audience also increased in a statistically significant way.
In addition, all of the 35 journalists who took the course said they would recommend it to other journalists interested in the subject.

More Information about our client and project results:

A User-Experience and Efficacy Study of the Mobile-Microlearning Course, “The 5 C’s of Writing News for Mobile Audiences”

A User-Experience and Efficacy Study of the Mobile-Microlearning Course, “The 5 C’s of Writing News for Mobile Audiences”

Usability Study for MU Extension Nexus@Mizzou Program

Time Frame:

Spring, 2018

Method:

  • Interview
  • SUS
  • Task-based analysis
  • Think-aloud

Goal: Evaluate the usability of the website and provide recommendations for improvement of the customer experience.

Client:
MU Extension’s Nexux@Mizzou

Goal:
Evaluate the usability of the website and provide recommendations for improvement of the customer experience.

Process:
The study has been planned with two phases.

  1. The IE Lab was asked to analyze a set of ten defined tasks and successful pathways users must take to complete them. Based on the tasks, IE Lab assembled a list of general users between the ages of 22 and 60, with various levels of technology proficiency to navigate the website, using different electronic devices while completing these tasks and talking through their process (task-based usability study with Think Aloud method). After observing users’ chosen pathways and post interviews, the team analyzed data and developed recommendations, which summarized the users’ actual experience with the system.
  2. Detailed user testing will take place in 2019 after MU Extension has completed the consolidation of all program websites, content, online courses and events onto the platform.

Findings:
  1. Users age 60+ with various computer skills found the website difficulty to use (SUS below 50%).
  2. Users prefer multiple accesses (e.g.through mobile devices, laptops, or desktop computers) to the website.

An example of recommendations that IE Lab user testing can provide to platforms like Nexus@Mizzou

An example of recommendations that IE Lab user testing can provide to platforms like Nexus@Mizzou

Mobile Applications for Microlearning: A Review of Academic and Industry Implementations

Time Frame:

Fall, 2017

Method:

  • Academic literature review
  • Cross-comparison matrix
  • Industrial literature review
  • Interview

Goal: Explore technical systems for microlearning and specifically provide design recommendations for mobile microlearning applications for journalists.

Client:
Linda Austin, a Reynolds Journalism Institute (RJI) Fellow and the Project Director for NewsTrain

Client URL:
None

Goal:
Explore technical systems for microlearning and specifically provide design recommendations for mobile microlearning applications for journalists.

Process:
The project was organized into two project phases.

  1. IE Lab evaluated best practices and ideas for microlearning approaches that can be used on smartphones. Three research methods used were academic literature review, popular trade/press review, and a platform cross-comparison matrix of app features for microlearning purposes.
  2. IE Lab conducted interviews for understanding the successes and failures of microlearning in journalism.

Findings:
  1. Based on the Phase 1 results, the deliverables included a summary report of the academic literature review, a summary report of the industry (popular/trade press literature) review, a matrix of best practices/characteristics embodied in the Unleesh mobile app versus other six mobile learning systems.
  2. Based on the interview (Phase2) results, IE Lab identified types of skills that are best taught in mobile microlearning as well as how to teach, structure and present micro-lessons for journalists.

Actionable techniques were recommended for designing mobile microlessons:

  • How to teach/design microlearning lessons: interactivity, feedback, personalization
  • How to structure courses: independent micro-lessons with content segmentation, learning paths, multi-formats
  • Technical settings: push notifications, learning analytics, big data analytics, dashboard design
  • Short micro-lessons and active assignments: brevity, granularity, learner assignments / problem-solving

Evaluation Criteria: Background information; Interface design; Course/lecture length; Costs and benefits; Device Application (mobile/smartphone/app/offline version); Subscription service; Instructional design; Overall assessment

Evaluation Criteria

LeXMizzou — a Learning Expedition@Mizzou

Time Frame:

Fall, 2017

Method:

  • Presentation

Goal: Create meaningful learning that goes beyond classroom learning and supports learners to explore and connect topics and theoretical concepts, augmenting information with physical objects in the real world (LeXMizzou, 2015)

Client:
LeXMizzou Project, led by Dr. Isa Jahnke, Associate Professor in the School of Information Science and Learning Technologies

Goal:
Create meaningful learning that goes beyond classroom learning and supports learners to explore and connect topics and theoretical concepts, augmenting information with physical objects in the real world (LeXMizzou, 2015)

References:
LeXMizzou. (2018). About. Retrieved from Learning Expedition at Mizzou — LeXMizzou
Renewal: https://lexmizzou.wordpress.com/

Process:
The overall goal of the LeXMizzou project is to give students access to become game designers for learning. Students design, develop, and evaluate a location-based augmented reality mobile game around the campus of the University of Missouri-Columbia.

The LeXMizzou project started in September 2015 and ended December 2017. Throughout the entire project lifecycle, IE Lab continuously provided usability study support for the project, covering UX fundamentals, design lifecycle, instruments, tools, presentations and training on usability study.

On October 26, 2017, IE Lab researchers provided a presentation on how to conduct usability studies for the LeXMizzou student teams. The presentation included two parts: a presentation of some fundamentals of usability/UX study (presented by IE Lab manager Neeley Current and researcher Hao He), and a working session that helped teams develop their usability plan (hosted by IE Lab researcher Minh Pham).

LexMizzou Participants

LexMizzou Participants

Media Innovation Space: A Comparative Systems Review

Time Frame:

Spring, 2017

Method:

  • Comparative review
  • Rapid prototyping

Goal: Assess whether a current system(s) exists that may support the learning, community, and social needs of the Media Innovation Space tool.

Client:
RJI Fellow, Dr. Michelle Ferrier, Associate Professor, Scripps College of Communication, Ohio University

Goal:
Assess whether a current system(s) exists that may support the learning, community, and social needs of the Media Innovation Space tool.

Process:
Media Innovation Space is a tool that is intended to offer a Grow-in-Place model that may consist of a distributed, hybrid online/face-to-face offerings in which users may create profiles, showcase their work, access resources for Just-In- Time learning, and foster strategic partnerships. The primary audience for the Media Innovation Space is student entrepreneurs and the space will serve as a virtual accelerator for startups run by journalist students.

Two phases were conducted for the study:
Phase 1. Technology/System Review & Needs Assessment:
IE Lab compared Dr. Ferrier’s Media Innovation Space system requirements with existing systems functionality, interaction design, navigation, layout and aesthetics.

Phase 2. Sitemap & Wireframe Prototype Development
With direct input from Dr. Michelle Ferrier, IE Lab:

  1. Created workflows for the site: graphical representation of actions and activities of users on site
  2. Developed a basic sitemap for the site: important for understanding the content organization

Developed low fidelity wireframe prototypes for the site: generated wireframes to show page structure and content requirements for individual pages. (Design tools: Axure RP and Balsamiq software)

Findings:
In Phase 1, a report provided sample interaction design, features and functions that may meet the design requirements for the Media Entrepreneurship system.

In Phase 2, the workflow and basic sitemap for the site were provided. Also, a low fidelity wireframe in Axure RP and Balsamiq software for five main pages were created.

Prototyping

PUSH: An open source mobile platform for media organizations

Time Frame:

Fall, 2017

Method:

  • Heuristics design review
  • Mobile device test
  • Task-based analysis

Goal: Identify the usability and the issues of two apps built on the PUSH platform.

Client:
2017-2018 Reynolds Journalism Institute Fellow, Christopher Guess

Goal:
Identify the usability and the issues of two apps built on the PUSH platform.

Process:
IE Lab conducted the usability study for the Push app. Push is an open-source mobile platform for media organizations to build their own mobile news apps for news distribution. The IE Lab conducted a task-based analysis, a heuristics design review, and a mobile device test on the MeydanTV and the Bivol apps — two apps designed and developed on the Push platform — to thoroughly identify the usability and the issues of the apps built on the platform. Additionally, the IE Lab translated the Push app into five languages, Vietnamese, Farsi, German, Chinese and Arabic to support media organizations in building their mobile news apps in these languages.

Findings:
The Push platform has great potential to help small media organizations build their own quality mobile news apps. The apps designed and developed on the Push platform include fundamental functions required for mobile news apps. Also, media organizations, especially small startup publishers, can forget about the hard-coding normally required to build apps and easily and quickly develop their mobile news apps on Push. The study also identified functionality, design, navigation, and user control issues of the apps. Recommendations for the further enhancement of the user-friendliness of the apps were also provided based on Neilsen’s heuristics principles and the cross-comparison of the other mobile news apps.

Examples of mobile news websites

Examples of mobile news websites

Usability Research for St. Charles City County Library Website

Time Frame:

Spring, 2017

Method:

  • Prototype analysis
  • SUS
  • Task-based analysis
  • Think-aloud

Goal: Identify the usability and user experience issues in the prototype for the new SCCOL website.

Client:
St. Charles City-County Library (SCCCL)

Goal:
Identify the usability and user experience issues in the prototype for the new SCCOL website.

Process:
In May 2017, the IE Lab conducted a usability evaluation of the St. Charles City-County Library (SCCCL) website during its prototype stage of development. Eighteen library patrons participated in the study. The eighteen participants were divided into six groups, based upon library user personas developed by SCCCL. Specific tasks were created for each persona type in order to evaluate user interactions and identify design problems of the prototype website.

A member of the IE Lab team met with each participant individually for up to an hour during which participants performed tasks on the prototype of the library website. Each participant completed six to eight task prompts that were based upon the persona group to which they were assigned. The sessions were recorded with Morae – an usability software that allows the capture of participants’ mouse clicks and cursor movements on the prototype interface, as well as the audio and video recording of the participant.

Findings:
The prototype of the SCCCL website has a lot of potential; however, the study with participants indicated that there is still room for improvement. Some of the key recommendations the IE Lab made to improve the website include the findability of resource content, the robustness of footer to aid users in the site navigation, and the choice of terminology in navigation bars.

St. Charles City-County Library website

St. Charles City-County Library website

Evaluation of ElevatEd: a K-12 Strategic Planning Platform

Time Frame:

Fall, 2015

Method:

  • Heuristics design review
  • Interview
  • Prototype analysis
  • Sociotechnical walkthrough
  • Task-based analysis

Goal: Evaluate the usability of ElevatEd, and more specifically, study:

  • whether the users were able to use the tool;
  • whether it is usable for creating and conducting strategic actions more efficiently; and l whether the users were satisfied.
Client:
MU Hook Center

Goal:
Evaluate the usability of ElevatEd, and more specifically, study:

  • whether the users were able to use the tool;
  • whether it is usable for creating and conducting strategic actions more efficiently; and l whether the users were satisfied.
Process:
ElevatEd is a web-based knowledge management system designed for school administrators and teachers. It was designed and created by MU Hook Center intended for K–12 schools with the main goal to improve workflow related to strategic planning in schools, e.g., creating and executing data driven strategic improvement plans.

The IE Lab started with user evaluation of the ElevatEd system in Fall 2015. The main goal of the study conducted by the IE Lab focused on evaluating the usability of ElevatEd. The IE Lab studied whether the users were able to use the tool, in particular, whether it is usable for creating and conducting strategic actions more efficiently, and whether the users were satisfied (ISO Norm 1998/2015; Sheehan, & Lucero, 2015). These goals guided the overall usability study. Through the evaluation of the web-based application, IE Lab was able to recommend improvements.

Findings:
The final recommendations were concluded based on the data collected from both participants and IE Lab experts and the data analyzed. Some key recommendations were:

  • Keep the system as simple as possible; it shouldn’t require too many clicks.
  • It should be accessible with tablets and laptops without major alteration to page structure.
  • The system should be able to sync the existing school calendar to the new app and/or to Google/Outlook Calendar.
  • Terminology and labeling needs some revision.
  • The system should consider integrating a student-centered view that focuses on progressing individual students. In addition, the system should consider enabling a user to visualize their roles within the system.

It would be better for teachers to see graphical display at a glance of the results once they have input the data into the system.

References:
Herrmann, T. (2009). Systems Design with the Socio-Technical Walkthrough. In Handbook of Research on Socio-Technical Design. New York: Hershey, pp. 336-51.

Hook Center. (2018). About. Retrieved from Hook Center for Educational Leadership and District Renewal : https://education.missouri.edu/hook-center/about/

ISO 9241-11 (1998), Standard Usability Framework, International Standard Organization, revised 2015, www.iso.org

Sheehan, B. & Lucero, R. (2015). Initial Usability and Feasibility Evaluation of a Personal Health Record- Based Self-Management System for Older Adults. In eGEMs (Generating Evidence & Methods to improve patient outcomes), Vol. 3, Issue 2, Article 3, 2015 DOI: http://dx.doi.org/10.13063/2327-9214.1152, http://repository.edm-forum.org/egems/vol3/iss2/3.

Example of ElevatEd webpage

Example of ElevatEd webpage

Time Up and Go Test (TUG test): Kinect-based Fall Risk Evaluation and Feedback System

Time Frame:

Fall, 2016

Method:

  • Interview
  • Observation
  • Pilot test
  • SUS

Goal:

  • Identify the usability of TUG test.
  • Offer refinement of the Kinect-based fall risk evaluation and feedback system.
Client:
Dr. Phillips Lorraine and Mizzou engineering students

Client URL:
None

Goal:
  • Identify the usability of TUG test.
  • Offer refinement of the Kinect-based fall risk evaluation and feedback system.
Process:
The TUG test is a Kinect-based fall risk and evaluation feedback system for elderly users developed by Dr. Phillips Lorraine and Mizzou engineering students. The TUG test times an individual to rise from sitting in a standard armchair, walk 3 meters, turn, return to the chair, and sit down. It is a measure of functional balance and a sensitive and specific measure of fall risk in community-dwelling balance-impaired older adults. The test used Kinect depth cameras to compute gait parameters.

In Fall 2016, the IE Lab was hired to conduct usability testing and offer refinement of the Kinect-based fall risk evaluation and feedback system. In this project, the tools and methods we applied for the evaluation of the TUG test include the pilot test, instruction slides, interview guide and System Usability Score survey (SUS), observation note table, and Morae recorder software.

Findings:
The TUG test system still needs improvement in three aspects: instruction slides, walking test, and the result sheets. Some main recommendations to the TUG test system are that the duration of the instruction slides should be optimized, the walking test should give participants clear signals and indicators as for when to start and when to stop, and the interpretation text should be placed at a more obvious location on the result sheet.

Time Up aand Go Test (TUG) example screenshot

Time Up and Go Test (TUG) example screenshot

Panacea’s Cloud: An augmented reality (AR) triage and communication tool for mass casualty disaster incidents

Time Frame:

Spring, 2016

Method:

  • Expert review
  • Heuristics design review
  • Interview
  • Post-task survey
  • Task-based analysis

Goal: Evaluate the initial proof-of-concept product of Panacea’s Cloud.

Client:
Prasad Calyam, MU researchers, assistant professor of computer science

Sal Ahmad, assistant professor of surgery and medical director of the Trauma and Surgical Intensive Care Unit

Goal:
Evaluate the initial proof-of-concept product of Panacea’s Cloud.

Process:
Funded by the Wallace H. Coulter Foundation, Panacea’s Cloud is a collaboration between MU researchers Prasad Calyam, assistant professor of computer science, and Sal Ahmad, assistant professor of surgery and medical director of the Trauma and Surgical Intensive Care Unit. Panacea’s Cloud is an augmented reality communication and coordination tool for mass casualty disaster situations intended to “coordinate resources, information sharing, and two-way communication between medical incident commanders (ICs) and first medical responders (paramedics)”. In multiple concurrent disaster situations, task completion time and task completion success rates are critically important to save people’s lives (Demir et.al, 2017). Any system built to respond to mass casualty incidents (MCI) sites should be able to have easy-to-use interface which can effectively analyze and present information and foster critical decisions to
reduce casualties (Demir et.al, 2017). Usability research is therefore indispensable to improve the system and to make the best solution available in terms of technology and user-friendly interface design.

The IE Lab was hired to conduct the usability research for Panacea’s Cloud. The usability research was conducted between March 1 – May 5, 2016 in order to evaluate the initial proof-of-concept product of Panacea’s Cloud. This system has been iteratively developed and refined based on user experience research driven methodology that employed a mixed methods approach, including the views of clinical experts. Specifically, the IE lab initially conducted the design review to test and revise the system; then, task-based eye-tracking studies; and post-task surveys and interviews were conducted to derive further design recommendations.

Findings:
Incorporate Situational Awareness features, a Synchronous Map View system, a Hands-Free Communication service with AR and smart glasses, Digital Notes, and resilient Wi-Fi networks (Demir et.al, 2017).

References:
Demir, F., Ahmad, S., Calyam, P., Jiang, D., Huang, R., & Jahnke, I. (2017). A Next-Generation Augmented Reality Platform for Mass Casualty Incidents (MCI). Journal of Usability Studies, 12(4).

Reading patterns on the Panacea’s Cloud index page, using Tobii as the eye-tracking tool

Reading patterns on the Panacea’s Cloud index page, using Tobii as the eye-tracking tool


Heat map for the Panacea’s Cloud index page

Heat map for the Panacea’s Cloud index page

Evaluation of the Website of the Division of IT at MU

Time Frame:

Summer, 2014

Method:

  • Think-aloud

Goal: Evaluate the usability and user experience of the Division of IT website.

Client:
The Division of IT at the University of Missouri

Goal:
Evaluate the usability and user experience of the Division of IT website.

Process:
The Division of IT at the University of Missouri recently updated their website to include more options on their homepage as well as dropdown menus for improved information architecture. The IE Lab conducted a think- aloud usability study in order to evaluate the new website and learn the perspectives of current University of Missouri students, faculty members and staff. The goal of the study was to evaluate the Division of IT website by looking at how users located commonly sought information such as email accounts and the password manager. During the study, users frequently commented on the usability of the website, which gave the researchers an insight into the overall perceived functionality, navigation and aesthetics. A findings report was delivered to the Division of IT and the site went live in the summer of 2014.

Division of IT website versions

Division of IT website versions

Evaluation of the Cerner Health Wins website

Time Frame:

None

Method:

  • Expert review
  • Task-based analysis
  • Think-aloud

Goal: Evaluate and improve the usability and user experience of the Cerner Health Wins website.

Client:
Cerner

Client URL:
None

Goal:
Evaluate and improve the usability and user experience of the Cerner Health Wins website.

Process:
Health Wins is a Web based application designed to coordinate and motivate weight loss competitions at the individual or organizational level. The lab provided expert reviews of the site, opening and closing surveys for their pilot launch, and participant driven usability tests of their application.

Findings:
We were able to identify functionality, navigation, and information architecture issues that helped them improve the project.

Cerner Health Wins website

Cerner Health Wins website

Evaluation of the University of Missouri Libraries Website

Time Frame:

None

Method:

  • Card sort
  • Information architecture verification
  • Prototype analysis
  • Summative evaluation

Goal: Evaluate and improve the usability and user experience of the University of Missouri Libraries website.

Client:
MU Libraries

Goal:
Evaluate and improve the usability and user experience of the University of Missouri Libraries website.

Process:
The MU Libraries serve a student body of 34,748 plus a faculty of 2,121, and have a collection of 3.1 million print volumes, 53,400 journal titles (in print or online), and over 7.5 million microforms. With an annual budget of $16.8 million, the Libraries support the instruction, research, service, and economic development missions of the University of Missouri. By acquiring scholarly resources, developing innovative services, and applying new information technologies, the MU Libraries fulfill their primary purpose: to serve our users. The IE Lab helped the MU libraries fulfill their commitment “to serve our users” by making a more user-centric homepage.

The IE Lab assisted in four phases of the MU librarywebsite redesign process that implemented various aspects of user centered design principles. These phases included: a card sort, verifying information architecture, prototype testing, and a summative evaluation of the redesign. These phases were conducted in an iterative manner with each subsequent phase building upon the findings of the previous phase. After each phase, members of IE Lab met to discuss the findings and to provide recommendations for the Library Website Redesign Committee.

Findings:
  • Simplified the appearance by adding more white space and reducing the amount of content on the homepage
  • Went from 7 navigation menus to 5
  • Re-worded the navigation menus based on user feedback to make the navigation menus more in line with how users
  • Added a Quick Links section l Added 4 interactive icons

MU Libraries Website versions

MU Libraries Website versions

Evaluation of Newsy.com

Time Frame:

None

Method:

  • Task-based analysis
  • Think-aloud

Goal: Evaluate and improve the usability and user experience of the Newsy.com website.

Client:
Jim Spencer, President of Newsy.com

Goal:
Evaluate and improve the usability and user experience of the Newsy.com website.

Process:
Newsy.com is an innovative online news service that connects users with ‘Multiple sources to help provide the real story.’ Newsy.com was established as a start-up company with strong ties to the Missouri School of Journalism at MU. The IE Lab worked with Jim Spencer, President of Newsy.com, and his staff to determine what usability studies should be conducted before Newsy.com went public.

A rapid user-centered iterative approach was used in the study. A two-part study with potential users conducting tasks and providing verbal feedback was recorded and analyzed. In a one-week period, the study was designed, data was collected and analyzed, and a findings report was delivered to Newsy.com with actionable feedback.

Newsy.com took the feedback, made immediate changes to the website, and a second study was conducted in similar fashion soon thereafter.

Below is the list of the clients the IE Lab have served since its establishment.

  1. University of Missouri (Regional)
    • College of Education
    • College of Engineering
    • Division of Extension
    • Division of Information Technology
    • Human Resource Services
    • International Center
    • MU Extension
    • MU Hook Center
    • MU Family & Community medicine
    • MU Library
    • Office of Admissions
    • Office of Research
    • Office of the Registrar
    • RJI Fellows, School of Journalism
    • School of Nursing — Tiger Place
  2. Missouri (State wide)
    • Cerner Corporation
    • East Central College of Missouri
    • Fontbonne University Library
    • IconoPsych/HomePsych
    • International Society for Performance Improvement, St. Louis Chapter (ISPI)
    • Missouri State University Library
    • MOBIUS Consortium
    • MOREnet
    • LANIT Consulting, Inc.
    • St.Charles City—County Library
    • Tiger Institute
    • Shelter Insurance
  3. Global/International
    • East China Normal University Distance Education College
    • Taiwan Office Missouri Department of Agriculture
    • Tamkang University Libraries