Title:
Medical rehabilitation quality management of the German statutory accident insurance: Evaluating the validity and applicability of a modified checklist screening for potential quality problems
Project management at the University of Würzburg:
Participating scientists:
Abstract:
In the context of the German statutory accident insurance?s rehabilitation quality management efforts, a criteria-based screening checklist is evaluated that is intended to screen for specific rehabilitative cases with potential quality problems.
Based on a preceding project (?Testing of a criteria-based screening instrument in the context of accident insurance quality management efforts?), the screening checklist was modified and evaluated in three regional accident insurance funds from June until November 2005. During the survey period, checklists were filled out by accident insurance administrators who were instructed to evaluate all current rehabilitation cases (n = 189) with regard to potential quality problems.
For validation of the instrument, a case review by a physician was used. All rehabilitative cases that were screened as potentially faulty as well as a comparable number of cases without deficiencies were submitted to the accident insurances? consulting physicians who rated the medical and administrative quality aspects of each case. To conclusively evaluate the acceptance and practicability of the screening checklist, accident insurance administrators were asked to fill out an evaluation form.
About a fifth of all cases reviewed (22%) were classified as potentially problematic in terms of quality by the administrators. There were no differences between ambulant and in-patient rehabilitation. Absence of relevant documents, complications during treatment and an insufficient improvement of symptoms were cited as the most frequent problems.
Consulting physicians rated 21% of the cases reviewed as defective in medical and/or administrative terms. 18% of cases showed serious administrative problems, 14% were evaluated as insufficient in respect of medical and therapeutic aspects. There were significant differences between out-patient and in-patient rehabilitation insofar as in-patient cases showed significantly less quality problems.
Concordance between the screening checklist and physicians? ratings concerning the quality of rehabilitative cases was moderate. Sensitivity, specificity, and positive predictive power were insufficient. Negative predictive power was high (between 79% and 90%).
Using physicians? ratings as a validation criterion, an accurate assessment of potential quality problems in rehabilitation by means of a screening instrument seems difficult. The criterion-based validity of the screening checklist is thus insufficient. This shows the need for further research which shall concentrate on methodological aspects of the instrument as well as its purposes. Potential starting points are: modification of item choice and formulation; modification of answer categories; use as a complementary instrument in the context of a multi-method quality management approach.
Accident insurance administrators rated the screening checklist as a useful and valuable instrument. This evaluation demonstrates the practical applicability of this approach which introduces the accident insurance administrators´ expertise and knowledge into medical rehabilitation quality management strategies.
Key words:
accident insurance
Quality management
Rehabilitation
Screening
Projekt period: from 06.2005 to 11.2005
Funding institution:
Sonstige öffentliche Mittel ( Bundesverband der Unfallkassen )
Preceding project:
Testing of a criteria-based screening instrument in the context of accident insurance rehabilitation quality management efforts