Human Factors Quarterly Newsletter Fall 2013

User-Centered Design: Creating Health Information Technology with the Veteran in Mind

Erin Schwartz, PhD., Research Psychologist, Health Informatics Initiative, Office of Informatics and Analytics

User-Centered Design (UCD) has increasingly become the gold standard of practice in the development of Health Information Technology (HIT). By including end-users…

Read More

User-Centered Design: Creating Health Information Technology with the Veteran in Mind Design

Erin Schwartz, PhD., Research Psychologist, Health Informatics Initiative, Office of Informatics and Analytics

User-Centered Design (UCD) has increasingly become the gold standard of practice in the development of Health Information Technology (HIT). By including end-users throughout the development process, UCD increases the likelihood that effective and desirable products will be created. In addition by considering the user from the beginning of design, post-development costs can be managed. UCD provides developers with the opportunity to discover the specific needs, wants, and characteristics of users and balance that information against their own requirements and limitations. UCD provides the opportunity to eliminate a portion of the guesswork in technology development.

In the summer of 2012, the Patient Team (Workstream B) of the Health Informatics Initiative (hi2) collaborated with VHA’s Human Factors Office to implement a UCD process by conducting a series of focus groups with Veterans in Clarksburg, WV, Columbia, MO, and Nashville, TN. Because the Patient Team of hi2 is currently tasked with the development of Web and mobile-based tools designed to support Veterans and their families in the management of their health care, connecting directly with this group of users to better understand their needs and preferences is absolutely essential.

Our primary goals for the focus groups were to better understand the perspective of Veterans about the types of information that they would like VA to know about them, the various ways they want to engage with the VA, barriers and facilitators to that engagement, and what role, if any, technology might play in optimizing their experience with the VA. Ultimately, Veterans who are more engaged with the VA system and their providers are more likely to experience improved health outcomes that benefits patients on an individual level and the healthcare system as a whole.

As a member of hi2’s Patient Team, I had the privilege to be present for the second round of group meetings at each of the sites. During focus group sessions, I was struck by both the candor and affability of the participants. Although each of the sites differed in terms of areas of focus and tone, there were many commonalities in both the process (the ways in which Veteran participants interacted with each other and with the facilitators) and the content across all three focus groups. For example, in terms of the process, participants were very willing to express their opinions even if they conflicted directly with those expressed by others in the group. But, they also clearly communicated respect for the perspectives of others as they did this.

The Veteran participants in all of the focus groups were asked to create a user profile for a returning Operation Enduring Freedom/Operation Iraqi Freedom Veteran named “Ann”. They were tasked with describing Ann’s military experiences in Iraq and Afghanistan, as well as the person she might be outside of the military. Participants were also presented with several scenarios in which Ann might connect with VA through technology including a kiosk at the point of care, a home computer, and a mobile device. In each example, Ann was asked to provide information about herself, and was provided with relevant information and VA resources by the system based on her input (e.g., being provided with housing-related resources if Ann reported being homeless). After having each site complete this exercise, the hi2 Patient Team discovered several common themes in all three focus groups.

First, all three groups made the point that Veterans should be treated as individuals and their unique circumstances should be understood and respected when they interact with the VA system. Within each group, concerns were expressed about the possibility of technology limiting or replacing human interaction, but there were also instances when it was noted that a technological interface might have advantages over face-to-face interactions (e.g., feeling more comfortable revealing sensitive information via a Web application). Overwhelmingly, the focus group participants expressed the high value placed on information that is transmitted from Veteran to Veteran, as well as their desire to be of assistance to other Veterans. All three focus groups also discussed difficulties inherent in navigating the VA system and stated that they believe that the system should make it easier for Veterans to receive services that meet their current and specific needs. Finally, all of the focus groups expressed concern about who has access to their information and stated that they would like to have more control over who is able to see it.

From a UCD perspective, the themes generated from the focus groups can help serve as guiding principles in determining the development of future HIT in VA. This might include prioritization of providing Veterans with an easy way to connect with a person by telephone or face-to-face within a Web or mobile interface. Giving Veterans a greater sense of control over their information by providing them with the capability to enter their own data into their health record, as well as developing condition-specific peer-to-peer online communities and “smart” systems that can generate responses unique to the information that a Veteran inputs.

Beyond generating ideas for future HIT products, the UCD process can continue throughout a product’s development lifecycle. In the spring of 2013, hi2’s Patient Team conducted focus groups with VA clinicians to discuss their perceptions about patient-generated data, the types of patient information that they would find most useful and how they would like to receive that information. Additionally, members of the Patient Team are formally soliciting feedback from Veterans about Web and mobile products as they are being developed. By including users in the development process, we have the potential to not only generate novel ideas for future HIT development, but also to create products that can be deemed successes from the perspectives of business owners, developers, and end-users.

Mission Health App: A Design for Better Health Outcomes

Erin Borglund, Clinical Research Assistant, University of Massachusetts Medical School; Barrett D. Phillips, MA, Research Project Director, University of Massachusetts Medical School; Thomas K. Houston, MD, Director of the eHealth Quality Enhancement Research Initiative (QUERI)

By employing a user-centered design process, VHA Office of Informatics and Analytics, Web/Mobile Solutions Program is developing a game to help Veterans achieve better health. MISSION HEALTH is an application that uses elements of competition and objective setting to take small steps toward better health outcomes. Based on mission success, Veterans will be awarded with points, increased ranks, and virtual "troops."  These troops could then be given to their competitors, members of a different branch of service, if they do not complete a certain amount of their mission. By using competition and virtual items, we anticipate that Veterans who use this application will see an improvement in their health and the health of their fellow Veterans.

The Competition Web-Based Application utilizes goal setting, a component of Social Cognitive Theory, specifically Specific, Measurable, Attainable, Realistic and Timely, or SMART goal setting. In the app, the goals are referred to as missions for continuity of the military theme.  The app will utilize these SMART missions allowing for Veterans to improve their health one small step at a time. Using SMART goals and the concept of “tiny habits,” Veterans will be able to set small “missions” that over time may have a large impact on their health.

One of the main game mechanics of the Competition App is competition. Competition has been incorporated into the app by allowing Veterans to select which branch of service they would like to compete against. If the Veterans do not complete a certain percentage of their mission, the other branch of service will gain double the points accrued by the opposing service branch. We expect competition will be a motivator for the Veterans to achieve their SMART mission(s). The competition app also includes a virtual troops component. In addition to points, participants can acquire more troops under their command by completing a certain threshold of their SMART mission. Participants also go up in “rank” according to the number of points they have accumulated and troops they have. Once the participant reaches a certain rank, they will then have to keep the troops healthy. If the participant misses a self-report on their progress, their troops’ health will slightly deteriorate. If the participant misses several self-reports, some of their troops will leave and join the opposing branch of service. The virtual troops are incorporated with the element of competition. We anticipate that Veterans will be motivated to keep participating as a means to keep the other team from acquiring their troops and points.

Before Mission Health App is released to the entire population of Veterans, there will be a usability study with a sample of Veterans who are not employed by the VHA. The app will be assessed with several usability dimensions: effectiveness, efficiency, satisfaction, learnability, accessibility, operability, flexibility, usefulness, utility and ease of use. This group’s feedback will be incorporated into the development of the game. By incorporating their feedback, we expect that system integration will be improved, adoption will increase and that retention rates will increase.

Games for health or “serious games” have been created to evoke health behavioral change. The most popular and effective serious games have been in the form of video games. Video game interventions have been used for weight management, increasing physical activity and obtaining healthier eating habits. An effective serious game will have a nearly equal balance of education and entertainment. Serious games have been speculated to be an effective behavioral intervention for adults; however, the target audience for most video game interventions has been children and adolescents. There is little but increasing evidence to suggest that serious video games may be effective in improving health outcomes.

Maladaptive health behaviors are easily acquired in today’s society and they can be extremely difficult to change. When Veterans return home from deployment, they may struggle to make healthy choices while adjusting to civilian life. As health professionals, we need to devise and implement health behavior interventions that are appealing and will be utilized. These interventions need to engage participants and be usable. It is easy to educate people about health behaviors but getting people involved in making their own health decisions is a more difficult task. The Mission Health App was designed as a means to improve the health status of Veterans. The development process for the app has incorporated user-centered design, behavioral theory and gamification components in an effort to make it usable and to engage users. Our hope is that the app will help produce positive health outcomes for Veterans.

Sociotechnical Factors in the Design of AWARE: The Alert Watch And Response Engine

Michael W. Smith, PhD., Human Factors Engineer, Houston VA Medical Center, Center for Innovations in Quality, Effectiveness & Safety; Daniel R. Murphy, MD, Internist/Informatician, Houston VA Medical Center, Center for Innovations in Quality, Effectiveness & Safety; Dean F. Sittig, PhD., Professor, University of Texas School of Biomedical Informatics; Hardeep Singh, MD, Internist/Informatician, Houston VA Medical Center, Center for Innovations in Quality, Effectiveness & Safety; Houston Patient Safety Center of Inquiry, Director

Problem of Missed or Delayed Follow-Up

Prompt, appropriate follow-up of cancer-related abnormal test results is essential. Despite VA’s many efforts to improve test result notifications through automation and electronic tracking, key diagnostic lab and imaging test results do not always receive timely follow-up. Delayed or absent follow-up on tests leads to care delays and patient safety risks and is a “major cause of malpractice suits for VHA” per the VHA Directive 2009-019.

Like most patient safety issues, it is due to complications between the functioning of different parts of the healthcare delivery system. A primary care provider has to manage large numbers of abnormal test results and other alerts on a daily basis. Keeping track of the results and the associated follow-up tasks, particularly in the context of managing multiple patients in different stages of diagnosis, is a challenging task. This poses a substantial burden on the provider’s prospective memory, and appears to be ripe for an Electronic Health Record (EHR)-based method of mitigation. However, the View Alerts function in Veterans Health Information Systems and Technology Architecture (VistA)/Computerized Patient Record System (CPRS) is designed to simply notify the provider about new information regarding their patients. It does not support patient follow-up tracking or administrative oversight or evaluation of the alert management and follow-up process.

AWARE Prototype

To address these challenges, we designed a software tool to serve as a prospective memory aid for providers. It is called AWARE, an acronym for “Alert Watch And Response Engine”. With support from VA’s Office of Innovations, we developed and conducted formative evaluations of the prototype tool at the Michael E. DeBakey VA Medical Center (VAMC) in Houston, TX. With additional support from the Office of Innovations, the tool will soon undergo beta testing at the Louis Stokes VAMC in Cleveland, OH and other sites in preparation for possible national implementation.

The sociotechnical systems framework is commonly used in human factors studies to describe an organizational unit of analysis as it relates to software interface design. Because there are multiple, interacting components of the sociotechnical system involved in alert management and follow-up (e.g., technical, human factors, workflow, organizational), there is a risk that the software intervention might not fit well with the other system components. To ensure good integration at multiple levels of the sociotechnical system, we designed and evaluated the tool following six specific design goals:

1. Mitigate strain on providers’ prospective memory by serving as an external memory aid to track the need for follow-up, and when necessary, remind providers.



When a provider closes a patient’s chart, the software tool evaluates whether the provider has received any new abnormal test result alerts for that patient related to the cancers being monitored (e.g., lung, colorectal, breast, & prostate). If so, it checks to see if any recommended follow-up actions (e.g., specialty consults, diagnostic evaluations) have been ordered. If no follow-up order is detected, the software will present a pop-up reminder notification to the provider. The associated recommended follow-up actions can be customized to accommodate a facility’s local coding and practices. Furthermore, a facility can select its own set of alerts depending on its top Quality Improvement (QI) priorities.

2. Minimize disruptions in the provider’s attention and workflow caused by the appearance of the reminder.



The reminder only appears when no follow-up is detected for one of the specific abnormal test result alerts. Thus, the provider can develop and document the patient’s care plan without ever being impacted by the reminder. When it does appear, it is when the provider is closing the chart, so it does not disrupt any immediate, ongoing task. Importantly, the reminder pop-up contains a prominent “Close and Address Later” button for when the provider chooses to defer action until a more convenient time. The software will also prevent the alert from disappearing from the View Alerts window, which has been found to be a problem by many providers.

3. Facilitate the process of ordering appropriate follow-up, or documenting reasons for not ordering follow-up.



If the provider selects the “Address Now” option, he or she is prompted to select a specific CPRS template, which presents a set of recommended follow-up actions for that abnormal test result and a set of common reasons why follow-up may not be indicated. There is also a free-text entry field for situations not addressed by the set of available options. Based on the provider’s selections, a template-based progress note and selected orders are automatically generated for review, edits and signature.

4. Enable an additional layer of safety by tracking alerts at the clinic or facility level.



It is possible that workload or coordination problems may result in abnormal test results that still do not receive timely follow-up despite the AWARE reminder function. The software also includes a QI Tool that maintains a database with information on clinical alerts, including the identity of patients who are overdue for follow-up. Safety or clinical managers could use the tool to detect patients at risk. Furthermore, the tool can help quality managers identify patterns within the different types of alerts, and if there are trends suggesting problems with rate of, or time to, follow-up.

5. Minimize changes to and interdependencies with underlying VistA/CPRS code.



As a field-developed innovation, this software was designed to limit the impact to the VistA/CRPS code. By keeping the software as independent as possible, it reduces the risk of causing any problems with other VistA/CPRS functionality, and potentially facilitates adoption at facilities using different versions or modifications of VistA/CPRS.

6. Ensure maintainability and configurability of software for a given facility, and fit with Information Technology staff.



If the software is difficult to configure and maintain, it will not be adopted or sustained. There is also the risk that configurations will fail to be updated as priorities and practices regarding alert follow-up change.

User Research with Various Stakeholders

To help reach these design goals, a range of human factors methods were used throughout the conceptualization and development of AWARE, including user research, usability testing, and heuristic evaluation. Interviews were conducted with primary care providers and other stakeholders (e.g., quality/safety managers) to identify requirements regarding workflow, clinical processes, and other needs. Multiple rounds of usability testing were conducted with providers and other stakeholders. Heuristic usability evaluations were also conducted. Providers were able to successfully use the functions of the software, and identified usability problems were addressed in design revisions. Quality/safety managers found the QI Tool to be useful.

These design goals and user research and evaluation methods ensured that we considered the interactions between the software systems and the human systems at multiple points. We designed the tool to act as an external cognitive aid to help with prospective memory for follow-up tasks. To keep this tool from causing disruptions, we designed it to accommodate providers’ patterns of attention and workflow. We also included features to support recognition of recommended follow-up practices and to facilitate the ordering and documentation process. To make sure the software is not imposing demands about how and when to practice, the software has options to defer action and to document contraindications to follow-up or other relevant information. Furthermore, as an extra layer of safety we included the QI Tool to support quality/safety managers in detecting overdue cases or other problems. To ensure maintainability, we involved Clinical Application Coordinators in the design and evaluation.

The problem of missed or delayed follow-up for abnormal test results is complex and multi-faceted. Addressing it requires participation from various stakeholders, and explicit recognition of the ways software interventions affect and depend on the different dimensions of the complex sociotechnical healthcare system.

  Back to Top

Improving Provider-Patient Interactions Through Human Factors Research: An Interview with Dr. Zia Agha 

Christopher Petteys, Veterans Health Administration (VHA), Human Factors Engineering (HFE),  Program Coordinator, Managing Editor

Dr. Zia Agha is a primary care staff physician and serves as the Director of Health Services Research & Development for the VA San Diego Health Care System. Additionally, he serves as an Associate Professor for the University of California, San Diego. Dr. Agha’s research is funded by the Department of Veteran Affairs, National Institute of Health, and Department of Defense, and is primarily focused on outcome evaluations of health care technologies. He is specifically interested in the effect of Electronic Health Record (EHR) use on verbal and non-verbal patient-physician communication behaviors, patient centered care and related health outcomes. This research has caused him to value and appreciate the science of human factors, and how the application of the human factors principles to the design of Health Information Technology (HIT) and its integration into clinical workflow can positively impact both provider and patient satisfaction.

1. How were you introduced to the concept of Human Factors?



My introduction to Human Factors grew from my interest in health Information Technology. My first research grant focused on physician–patient communication and how we can use technology to improve those communications. We investigated how frequently HIT tools were used in these interactions and we began to see issues crop up. For example, we found doctors were more engaged with the EHR and less so with the patient. In these instances, the use of HIT was negatively impacting the communication between providers and patients. The study enabled us to view, in real-life, the difficulties that can occur when having to take care of patients and use technology. Currently, I’m working with a team on designing new user interfaces for the notes function in the EHR. The current version only allows for flat text entry. The idea is to introduce an active functionality that allows for enhanced documentation and mark-up, data organization though meta-tagging, and links referencing other information in the EHR. Improving the usability of Information Technology (IT) can help ensure more time is focused on the patient and not on navigating IT tools.

2. How do you use/employ Human Factors in your work?



In order to attain a more detailed view of provider’s time engaging in EHR activities, we employ the use of usability software that tracks and logs mouse clicks, key clicks, and eye movement. The eye-tracker is particularly helpful in pinpointing which fields or elements physicians are looking at most frequently. For example, in a radiological review, is the doctor immediately scrolling down to view the impression? The eye-tracker could reveal which information is most important and should be quickly and easily accessible without unnecessary scrolling. Microsoft Connect Cameras are also utilized for physical movement tracking of physicians and voice capturing. This helps identify, automatically, how much interaction is taking place with the patients, physically and orally as compared to the computer. Time focused on the computer and not on the patient is user unfriendly for both the doctor and patient. It detracts us from being user or patient-centered and ultimately can lower satisfaction. Understanding how to increase face-to-face interactions is critical to preventing this; and the software and cameras help us do that.

3. What Human Factors–based project do you feel made the biggest impact in VHA? Why?



The Health Informatics Initiative and their work on redesigning the EHR has been monumental in addressing usability as a key metric in technology effectiveness. The broader healthcare industry has concentrated on patient-safety related improvements to technology, while the VA is ahead of the curve in introducing usability as an additional measure of successful improvement.

4. Where do you see the greatest potential for the application of Human Factors principles in VHA?



VA and the health care industry are constantly challenged by the introduction and adoption of new technologies into existing workflows. Without proper attention to the aspects of human factors and human-computer interactions, these workflows can be interrupted and negatively impacted. As a result, effectiveness and efficiency can decrease. Human factors studies engage users to help ensure the synergy between technology and work processes is maintained and these undesirable side-effects are avoided.

5. Have you seen changes in the interest in or awareness of human factors in VA? What are they?



I have seen an increased interest from both the end-user community and leadership. A lot of users are asking for more usable interfaces and simply asking more questions in general. "Why do autocorrect functions exist on my mobile phone but not on the EHR?" Such questions push us to investigate making better, smart user interfaces and IT tools. Leadership wants more suitable technologies that integrate with workflows effectively. Fortunately the informatics community is realizing the importance of usability and user-centered design in achieving successful adoption of IT. They understand human factors is more than just creating a pretty interface with pleasing images; it’s creating software and functions that are smarter and more helpful to the users.

Building a Better Electronic Health Record Through a User-Centered Design

Russell Carlson, RN, MHA, Nursing Informatics Specialist, Applied Informatics Service, Office of Informatics and Analytics

Poorly designed software results in lower productivity and efficiency and has negative impacts on quality, safety, and satisfaction. User-Centered Design (UCD) offers critical advantages in terms of developing easy-to-use products that satisfy customers, decrease expenditures on technical support and training, and ultimately increases adoption because the software is designed to be intuitive and support end-user workflow.

UCD is a method for designing ease of use into the total user experience with software products. It is a structured approach that requires a multidisciplinary team to identify end-users' needs and pain points, and then incorporate these insights into the development process. The result is the design and development of software products that support end-users' tasks, and objectives while working with the system. The result of UCD is consistent development of engaging software that is easy to set up, easy to learn, and easy to use. UCD principals help to ensure delivery of software that is immediately “familiar” to an end-user.

The Health Informatics Initiative (hi2) is a transformative effort between the Veterans Health Administration (VHA) and the Office of Information & Technology (OIT) to improve health care delivery to Veterans by shaping the future of VHA clinical information systems, communications, and patient care through deliberate application of health informatics and Health Information Technology (HIT). This future will be patient-centered, nimble, and mission-oriented, allowing collaboration between developers and clinicians, as well as between VA, VHA, private entities, and communities of care. A specific outcome of the initiative will be the modernization of the electronic health record into a Health Management Platform (HMP) consisting of modular, browser based applications supporting health care teams, patients, and VA’s health care system delivery of care. The HMP products will incorporate principles of user-centered design, so the applications will be engaging to learn, intuitive to use, and helpful for performing job duties and responsibilities.

Products and applications designed and developed using UCD methods, tools, and processes are preferred by and more quickly adopted by users, resulting in higher end-user satisfaction rates. It is essential for the contributors to the hi2 HMP applications, developers, designers, and business owners to know and understand UCD principles. The hi2 UCD Core Team is charged with developing a strategy to build and integrate a UCD model across the hi2 agile development of the HMP, which after piloting could be adopted by the rest of the organization. The Return-On-Investment will increase and the value of these methods, tools, and processes can be used throughout the organization by incorporation into software design and development models.

The objectives of the hi2 UCD Core team are:

• Evaluate all current UCD, User Interface (UI), usability, and User Experience (UX) efforts, workgroups and resources within hi2 to determine overlaps and relationships to ensure coordination of all related efforts.
• Establish a coordinated approach to UCD that ensures end-user involvement through all stages and steps of all hi2 HMP software product/UI development. This is to include at a minimum: planning, collecting data from users, developing prototypes, and conducting usability testing.
• Establish hi2 HMP design principles to guide development across all HMP components.
• Develop strategic and tactical plans that identify short-term (2-3 months), mid- term (6 months), and longer-term (12-18 months) actions that ensure interaction between end-users and the hi2 HMP occur during the design process.

Current efforts include:

• Heuristic reviews - a usability inspection method for identifying usability problems,
• Refinement of an agile development model incorporating UCD throughout the agile process,
• Participation in UCD training for the HMP component teams,
• Clinical business owners employing user stories, developers, designers, and clinical health information technology subject matter experts to provide hi2 staff with the necessary skills and understanding to incorporate UCD methods, tools, and processes into HMP software development.
• A cooperative partnership between the hi2 UCD team and the Veterans Engineering Resource Center to   implement a work plan for deploying a user-centered design study and gap analysis report and to identify performance measures for evaluating HMP. A second aim is to conduct functional capability assessments (requirements analysis) supported by market research for a usability analytics platform. The usability analytics platform is intended to be part of the emerging HMP and is envisioned to provide near real-time reporting of HMP usage and usability. The priority of this effort is to lay a framework to support ongoing quality measurement and improvement of HMP in order to better support efficiency, effectiveness, and improved user satisfaction.