41
Category:
Logic Model Component:
Count, proportion, qualitative
Measures the extent to which intended users apply knowledge to improve practice guidelines, program design and management, training curricula, or research practice
This indicator measures the extent of the use, and the outcomes of the use, of knowledge gained from KM outputs to improve practice guidelines, program design and management, training curricula, or research practice, resulting in better service delivery, more efficient programs, better training and education of health care personnel, or stronger research designs.
Description of knowledge from KM outputs that was used, approximate timeframe of use, organization(s) involved, how programs or practice benefited from applying the information, and any further outcomes associated with use
To obtain a quantitative data, evaluators can count the instances of use of knowledge gained from a KM product or group of products. Alternatively, evaluators can calculate the percentage of respondents to a survey who said that they used knowledge gained from the KM product.
User surveys (online, mail, telephone), usually distributed after the product has been disseminated; informal (unsolicited) feedback; in-depth interviews (telephone or in-person); guidelines or protocols referencing or incorporating information/knowledge from KM outputs
Annually
The purpose of this indicator is to trace how knowledge has been specifically used to enhance practice, programs, training, education, or research. One difficulty with measuring effect on practice is that audiences may not recall which particular piece of knowledge gained from what specific KM output was used and how it contributed to a defined outcome, particularly in a case-control approach, which begins with a change in practice and looks for factors that contributed to the change.
The information in national guidelines is more likely to be adopted when it is disseminated through educational or training interventions than when guidelines are simply distributed in their original written form (NHS Centre for Reviews and Dissemination, 1999). When training and information resources are necessary components of the trainee’s education or where training is necessary to use an information resource, the training and the information resources constitute a package that should be evaluated as a whole. Anecdotal reports on use are valuable, particularly given the challenge of capturing and quantifying the use of information and outcomes of its use. It is helpful to collect in-depth stories from users of products or services, including reports on improvements, achievements, or problems that result from using a product or service.
Please indicate whether or not you have used information from the [Web product] for the following purposes. (Select all that apply.)
o To make management decisions (either personal or organizational)
o To design or improve projects or programs
o To develop or improve policy or national service delivery guidelines
o To develop training programs or workshops
o To assist in designing education materials
o To guide research agenda or methods
o To put research findings into practice
o To promote best practices
o To write reports/articles
o To develop proposals
o To increase public awareness
o To increase my own knowledge
o Other, please specify __________
Please give an example of how you have used specific information from the [Web product] in your work. (Open-ended.)
Please rate the following statements about performance areas affected as a result of using the [Web product]:
(1-Strongly disagree, 2- Disagree, 3-Not sure, 4-Agree, 5-Strongly agree)
o Based on something I have learned in it, I have changed the way I perform my job.
o I have used information from it to improve my skills.
o It has helped me to be more competent and effective at my job.
o It has helped me to perform my job more efficiently.
o It has helped to improve the performance of my organization.
Please give a specific example of how the [Web product] has improved your own performance or your organization’s performance. (Open-ended.)
In the 2011 K4Health website users’ online survey, majorities of respondents (n=224) used the information obtained from the K4Health website to improve their knowledge (72%), to design or improve projects or programs (55%), and to promote best practices (52%).
In the survey about the LeaderNet webinar on blended learning, when asked for examples of how they applied or plan to apply their new knowledge to their work, participants stated they will apply the ADDIE model (consisting of 5 phases—analysis, design, development, implementation, and evaluation), set SMART objectives (consisting of 5 criteria—specific, measurable, attainable, relevant, and time-bound), thoroughly analyze the target audience, measure learning interventions beyond Kirkpatrick’s Level 1 and 2 (reaction and learning), apply blended learning strategies to their current learning challenges, and engage in Global Health eLearning courses.
58-59
Published Year:
- 2013
Wednesday, September 6, 2017