A framework for evaluating non-health technology for applicability within public health Open Access
Lee, Brian Alexander (2014)
Published
Abstract
The existing literature has many valuable frameworks for evaluating public health informatics systems. The existing frameworks provide a useful set of valid attributes and measures for determining the value and impact of a system within a public health organization or community. However, emerging tools and technologies are developed outside of public health that may be of value within public health informatics systems or as stand-alone tools used by public health practitioners. These tools need to be evaluated as stand-alone components to be included, if useful, within public health information systems and practice.
This research seeks to extend the available methods for public health informaticians to evaluate particular tools, approaches, practices or non-public health systems for their viability of use within public. While much software exists outside of the public health and health information technology domains, it is important to evaluate potentially useful tools based on the characteristics important for public health. This study performed a systematic review of the public health literature to determine commonly used attributes for evaluation, harmonized them into a common set of attributes appropriate for evaluating technology and created a simple tool for performing and documenting evaluations.
Thirty-two attributes are commonly occurring within the health literature, analyzing frequency and harmonizing these attributes for generalizability in evaluation of technology outside of health and public health led to the selection of nine attribute concepts that are most useful to evaluations: Acceptability, Usefulness, Accuracy, Architecture, Data Quality, Timeliness, Costs, Organizational, and Supportability. Thirteen additional attributes may be useful depending on the judgment of the evaluator: Data Management, Data Analysis, Documentation, Performance, Flexibility, Stability, Data Dissemination, Standardization, Security, Simplicity, Portability, and Ethical. Additional attributes can be included based on the judgment and determined need of evaluators using the tool within their organizations.
Using these attributes this study developed a method for evaluating technology, a template for use by informaticians, an example evaluation using the template and a method to represent evaluation results in a reusable JSON data structure.
Table of Contents
CHAPTER 1: INTRODUCTION ................................................................................................................................... 8
INTRODUCTION AND RATIONALE.......................................................................................................................................................... 8
PROBLEM STATEMENT ........................................................................................................................................................................... 9
THEORETICAL FRAMEWORK...................................................................................................................................................................11
PURPOSE STATEMENT .............................................................................................................................................................................11
STUDY OBJECTIVES....................................................................................................................................................................................11
SIGNIFICANCE STATEMENT......................................................................................................................................................................12
DEFINITION OF TERMS .............................................................................................................................................................................12
CHAPTER 2: LITERATURE REVIEW..........................................................................................................................17
PUBLIC HEALTH INFORMATION SYSTEM EVALUATION FRAMEWORKS ............................................................................................18
HEALTH SYSTEM INFORMATION EVALUATION FRAMEWORKS ..........................................................................................................29
SYSTEMATIC REVIEW FRAMEWORKS ......................................................................................................................................................34
GENERAL TECHNOLOGY ASSESSMENT FRAMEWORKS..........................................................................................................................37
SUMMARY OF CURRENT PROBLEM ..........................................................................................................................................................38
CHAPTER 3: METHODS .................................................................................................................................................39
INTRODUCTION ...........................................................................................................................................................................................39
POPULATION AND SAMPLE ........................................................................................................................................................................39
RESEARCH DESIGN ......................................................................................................................................................................................39
Stage 1: Systematic review........................................................................................................................................................................40
Stage 2: Harmonization of evaluation attributes ...........................................................................................................................42
Stage 3: Tool development project ........................................................................................................................................................42
PROCEDURES.................................................................................................................................................................................................43
Stage 1: Systematic review.........................................................................................................................................................................43
Stage 2: Harmonization of evaluation attributes ............................................................................................................................46
Stage 3: Tool development project .........................................................................................................................................................46
INSTRUMENTS ...............................................................................................................................................................................................47
PLANS FOR DATA ANALYSIS.........................................................................................................................................................................47
LIMITATIONS AND DELIMITATIONS............................................................................................................................................................47
CHAPTER 4: RESULTS.......................................................................................................................................................49
INTRODUCTION ..............................................................................................................................................................................................49 FINDINGS..........................................................................................................................................................................................................49
Stage 1: Systematic review results............................................................................................................................................................49
Stage 2: Harmonization of evaluation attributes .............................................................................................................................60
Stage 3: Tool development project ..........................................................................................................................................................63
CHAPTER 5: CONCLUSION................................................................................................................................................72
SUMMARY OF STUDY.......................................................................................................................................................................................72 CONCLUSION.....................................................................................................................................................................................................72
IMPLICATIONS .................................................................................................................................................................................................73
RECOMMENDATIONS ......................................................................................................................................................................................73
REFERENCES ........................................................................................................................................................................ 74
APPENDIX A: FULL EVALUATION TEMPLATE ............................................................................................................76
APPENDIX B: FULL EXAMPLE EVALUATION OF TABLEAU DESKTOP USING EVALUATION TEMPLATE..85
APPENDIX C: LIST OF INCLUDED PUBLICATIONS IN SYSTEMATIC REVIEW ....................................................95
About this Master's Thesis
| School | |
|---|---|
| Department | |
| Degree | |
| Submission | |
| Language |
|
| Research Field | |
| Keyword | |
| Committee Chair / Thesis Advisor | |
| Committee Members | |
| Partnering Agencies |
Primary PDF
| Thumbnail | Title | Date Uploaded | Actions |
|---|---|---|---|
|
|
A framework for evaluating non-health technology for applicability within public health () | 2018-08-28 13:20:44 -0400 |
|
Supplemental Files
| Thumbnail | Title | Date Uploaded | Actions |
|---|