WHO EQUIP - digital mental health support assessment

Project goals

The goal was to deliver an application that could run on a simple tablet device and allow WHO-accredited mental health workers assess trainees via a series of roleplay assignments, the results of which could be scored for performance in different categories of mental health support competency. A simple UI allows the user to start an assessment after selecting the site they are operating in and the competencies they want to assess. Then, the app randomly selects one from a set of pre-defined “vignettes” which determines the hypothetical session the trainer and trainee will be acting out. Throughout the assessment, the trainer can freely select the competency that is being assessed, the level at which the trainee is performing that competency (1 to 4, the latter being the best) and with the option of selecting items from a list of specific remarks for each of those levels to facilitate scoring.

There are three general tiers of users: trainers, raters and organization administrators (orgadmins). Raters can assess a trainee via the app and have a specific site and competency tool assigned to them, the title and competency attributes of which corresponds to the training that was received. Trainers can assess trainees as well from several trainings and have the ability to access a dashboard representing results for every trainee. Orgadmins can add and amend backend content like users (trainers & raters), competency tools with their respective attributes and finally administer sites where the training sessions and subsequent assessments are held.

Dynamic UI & UX

For this project, we needed an assessment app, connected to an LMS (learning management system), and an accompanying website with static content. The assessment app had to be tablet-first and intuitive, importantly allowing for left-to-right and right-to-left language displays interchangeably. Our UX (user experience) specialists and UI (user interface) designers ran workshops and collaborated closely with the academic and social impact partners on the project to develop prototypes. These were then field-tested with potential end-users in Uganda and Dubai before being further refined. We iterated on the designs throughout development using results of testing at mid-fidelity level, before creating the final iterable designs in high-fidelity.

​

Because we focused on identifying the features with highest user value, we were able to concentrate our design efforts in tight incremental loops. Development was done alongside design in most cases, with UX refinement and functional prototyping occurring in parallel.

Most modern web app projects are designed to be mobile-first, with desktop (and sometimes tablet) versions being significantly different UI experiences, which adds development overhead. Modern frontend frameworks like Bootstrap allow for seamless transition of the UI between screen sizes without ever needing to reload the app.

Different here was that we knew that the app would only ever be initially used on a tablet inside a specific range of screen resolutions, so we were able to focus all design and development efforts on a single version of the UI and UX. This came at the expense of edge cases that some users were quick to discover: usability issues on small mobile devices and desktop screens with awkwardly placed and cramped UI components. Overall, it was key to keep the design overhead low so that development could progress quickly with enough room for adjustment and extensions in the future should the need arise, hence the need for a modern reactive frontend framework.

“Offline-first” PWA

One of the basic assumptions by the WHO EQUIP team when developing a digital tool for assessing mental health workers was that users would rarely have any internet connection at all when assessing trainees. The project was designed to work in remote regions of low and middle-income countries with limited access to the internet. A connection would be required when uploading data to servers for processing, but anything running as a web page would be prone to corruption from outages of the device and connectivity.

As such, we focused on developing an offline-first progressive web application (PWA), which allows the installation of the frontend application via the browser and subsequent usage without needing to be online to load the pages. The app is available on multiple devices, including desktop, tablet, and mobile. It doesn't need to be re-installed to receive updates, which offers greater accessibility to its users.

A traditional web application would not have been an adequate solution for this project, since the required usage patterns would not allow for standard asynchronous data transfer to the backend. However, leveraging the advantages of PWA, we developed an intuitive and seamless web app that allows users to install it from their browser onto their tablets and not have to care about if and when they are online while using it for its core functionality of assessing mental health workers.

Translator’s workflow

Key to success of the platform was the ability to provide easy-to-use tools to quickly make available large sections of the assessment app and its associated competency tools. Non-technical users of very different cultural and linguistic backgrounds would need access to a system that would allow them to easily translate English assessment content into their native tongue and cultural localization.

WHO is a worldwide organization and it’s very important for content and tools to be available in different languages. Currently we have support for five languages (including right-to-left Arabic) and are scaling to support a dozen more over the next year or so.

An interface was needed that would be easy to use for translators who are contractors and not always technical people. Drupal has a rich amount of features for content translations. We created an interface to allow translators to add translations in different languages, save them in draft state and pass them to administrators for review and publishing.

The UI itself is self-explanatory, translators only have access to add or edit translations in permitted languages and can’t change the content added by content editors. Any actions are tracked with revisions, so we can easily see the history of changes or restore any previous state if necessary.

The EQUIP platform saves lots of time and efforts related to data entry of the competency assessments. Before we used to take days to enter the data, which is not always feasible when you have many study participants and little time.

Frederik Steen, technical officer & Platform Administrator 

Development

Technical development started towards the end of 2019 after the UX inception sessions were held with senior stakeholders. The MVP was completed and released by March 2020. The project is still in active development, with bug fixes, new features and new operational sites being added regularly as of December 2020.

React & Redux

A key component of allowing a PWA to flourish offline is state-based data management. Redux is a native React library that allows for deterministic and predictable behaviour when storing and processing data inside a JS app environment. The data transparently persists through page reloads and is only expunged via a manual reset of localStorage, which requires technical knowledge in a browser and is usually inaccessible inside a PWA.

A trainer or rater can complete assessments in any way to any completion level, hopping between users and trainees in radio-silent areas with the app managing all data as expected. When that device eventually connects back to the internet, the data is automatically synchronized in the background via Redux Saga calling the Drupal REST API that ensures the backend is up to date with any new and in-progress assessments.

ES6 generator functions allow writing synchronous code for asynchronous operations.

Throughout, the app is constantly monitoring for connectivity and alerts the user if and when they are online and offline. Any conflicts between a local assessment and the remote equivalent is flagged as soon as a connection is available and the user is prompted to solve this discrepancy with whatever version is the most up to date according to them. This happens for example when an assessment is in progress while online, then the assessment is discarded locally while offline and is then re-started, essentially creating a new version. Automatically synchronizing the more complete version or having the user select the local or remote as the source of truth are additions in the pipeline and are straight-forward to implement robustly using Redux.

Devops & testing

Integral to any application of this scale, making sure devops are as streamlined and scalable as possible is a must. Furthermore, due to the technical complexity of many of the base requirements, automated testing needed to be present from day one.

All of our projects at SystemSeed use a set of standard tools to facilitate cooperation and integration: Github, CircleCI, Basilio (our own AWS hosting solution) and NewRelic all with their appropriate Slack integrations. Every feature branch is deployed into its own HTTP-gated test environment via CircleCI once pushed to Github.

While all developers each have their own IDE preferences (PHPStorm, Visual Studio and Atom to name a few), a common framework for testing across PHP, JS and Twig is Codeception. It allows us to write acceptance and API tests with ease for the whole stack using a unified syntax and architecture. These tests run on every deployment to test or live environments and some of them are batched to run regularly on the production site. They are scheduled and triggered by CircleCI, so there is no manual effort in testing the site thoroughly before and after every change.

To help in ascertaining user satisfaction, we installed Hotjar on the frontend. Any trainer or rater navigating the assessments or dashboard has access to a quick and easy feedback option allowing them to rate their experience and leave optional feedback alongside a screenshot. Rollbar allows us to automate JS errors with a Slack integration to notify of any unexpected user errors in real-time.

The hosting required for the EQUIP project is challenging in that it must be equally performant, secure and receive data over multiple protocols. SystemSeed managed to deliver exactly what we were looking for with room for customisation on top of their default setup.

Elise West, Product Owner on WHO EQUIP

Agile methodology

The Agile methodology in software development prioritizes iterative results and values constant feedback among all involved in the project. It is crucial to always adapt to change, ultimately producing working results faster and more efficiently than traditional methodologies that involve managerial hierarchies and waterfall planning.

The WHO EQUIP team hadn’t worked inside an Agile project framework before, but we were able to quickly ease all stakeholders into its most important rituals: daily standups, 2-week sprints and a sprint review with a demo and retrospective. The product backlog grew steadily throughout the 4 months of minimum viable product (MVP) development and every sprint had plenty of work prioritized for all involved.

UX workshops

Before beginning development in earnest, we held several inception sessions with senior WHO stakeholders to identify all priorities and goals and take a first pass at the overall UX. While development of core functionality was underway, design of the UI was taking place in parallel facilitated by the product owner (PO). While this presented challenges at times not directly implementing vetted and finalized designs, we nurtured tight feedback loops and weren’t afraid to groom the backlog at every opportunity, so the development team was able to deliver polished visuals and solid functionality for every sprint.

Throughout the project, full transparency of the product was prioritized along with its development and planning so that every measure could be evaluated at all times, keeping efficiency at optimal levels. We aimed to impress stakeholders at every sprint demo by always going that extra mile and admitting to any deviations openly at the earliest opportunity, which kept client satisfaction high.

Responsiveness

When the full extent of the COVID-19 pandemic hit and the world started going into full lockdown, we were also quick to respond to WHO EQUIP’s needs for education and training around more specific subject matter. We set up our Anu LMS solution for them in a very short period of time for their organization admins to start offering e-learning courses on mental health topics tuned towards COVID-19 subject matter. We were able to quickly hire subject matter experts who helped write in-house content for these courses and illustrate them for an appealing e-learning experience.

Many times the tight feedback loops we had with real end-users in test sites running various stages of the prototype MVP brought in sudden and unexpected change requests. The PO fed the backlog and the dev team a constant stream of prioritized stories and it wasn’t uncommon for a story to require change midway through or needing substantial follow-ups to meet the end requirements. Nevertheless, the dev team was always responsive to adjustments in feature delivery, which saved time on development overall and was met with praise by the stakeholders who were always guaranteed a steadily improving and fully functional increment after every sprint.

Outcome & feedback

The WHO EQUIP suite of web applications (pages, assessment app and LMS) are still under active development. While the first MVP trials were held in live locations in March 2020, the application has now scaled over dozens of sites worldwide and is available in ten languages.

Overall the WHO EQUIP team are very satisfied with the project which came in on time and budget, being used to help real people from first release. The project’s goal was to overcome the slow and inefficient process of assessing mental health caregivers at various stages during their training, which was plagued by non-standard paperwork in each region and coordination issues due to the global distributed nature of the program. This digital assessment tool using modern web technology helped standardize the process and could be easily and quickly scaled to various sites over the world so that assessment results could become reliable. This is exactly what happened.

9 organizations are now regularly using the EQUIP digital platform in their trainings in the field. 1500 weekly active users are learning through the EQUIP courses, in 29 different countries.

SystemSeed has been an outstanding partner to the EQUIP project and produced on time and to budget all expected deliverables.

Alison Schafer, Technical Officer, World Health Organization

Furthermore, the free & open-access LMS suite we developed for WHO EQUIP at https://whoequipremote.org based on Anu is seeing steadily increasing users across all courses as operations expand.