- George A. Mashour, MD, PhD*†,
- Roy K. Esaki, MS, MD*,
- Kevin K. Tremper, PhD, MD*,
- David B. Glick, MD‡,
- Michael O'Connor, MD‡ and
- Michael S. Avidan, MBBCh§∥
+Author Affiliations
- Address correspondence and reprint requests to George A. Mashour, MD, PhD, Department of Anesthesiology, University of Michigan Medical School, 1H247 UH/SPC-5048, 1500 East Medical Center Drive, Ann Arbor, MI 48109-5048. Address e-mail to gmashour@umich.edu.
Abstract
BACKGROUND: Intraoperative awareness with explicit recall occurs in approximately 1–2 cases per 1000. Given the rarity of the event, a better understanding of awareness and its sequelae will likely require the compilation of data from numerous studies. As such, a standard description and expression of awareness events would be of value.
METHODS: We developed a novel classification instrument for intraoperative awareness events: Class 0: no awareness; Class 1: isolated auditory perceptions; Class 2: tactile perceptions (e.g., surgical manipulation or endotracheal tube); Class 3: pain; Class 4: paralysis (e.g., feeling one cannot move, speak, or breathe); and Class 5: paralysis and pain. An additional designation of “D” for distress was also included for patient reports of fear, anxiety, suffocation, sense of doom, sense of impending death, or other explicit descriptions. We reviewed 15 studies of the incidence of awareness that provided specific information about awareness reports. Five anesthesiologists at three institutions who developed the categories independently classified the events. An additional 20 individuals (attending anesthesiologists, anesthesiology residents, nurse anesthetists, medical students, and ancillary staff) not involved in the development of the categories also independently classified the events. Fleiss's kappa statistic was used to evaluate inter-observer agreement.
RESULTS: One hundred fifty-one cases of intraoperative awareness in adults were identified as valid for analysis. The overall kappa value was 0.851 (0.847–0.856, 95% confidence interval) for the basic Classes 1–5. Including additional designations of emotional distress, the overall kappa value was 0.779 (0.776–0.783, 95% confidence interval).
CONCLUSION: We report a novel classification instrument for intraoperative awareness events that has excellent inter-observer agreement and that may facilitate the study of intraoperative awareness.
Intraoperative awareness with explicit recall is a complication that is concerning to patients, anesthesia providers, and the general public. A multicenter study in the United States estimated an incidence of awareness with explicit recall of approximately 0.13%,1 which is consistent with a prospective European study demonstrating awareness in 1–2 of 1000 cases.2 Other studies have reported a much lower3 or much higher4 incidence. Thus, the magnitude of the problem is as yet unclear. Awareness is associated with long-term psychological consequences, including posttraumatic stress disorder.5–8
Because intraoperative awareness is rare, a full understanding of its incidence and sequelae will likely require the compilation and analysis of data from multiple patient populations, institutions, and studies. As such, a standard method for describing and reporting awareness events would be of value to investigators. Here, we report a novel classification instrument for intraoperative awareness events with excellent inter-observer agreement.
METHODS
We reviewed articles from 1961 (the first incidence study of awareness) to the present time using “intraoperative awareness,” “anesthesia awareness,” and “awareness during general anesthesia” as key words for a PubMed literature search. Based on the awareness reports in the literature, classification categories for intraoperative awareness events were developed (Table 1). All types of awareness events reported in the large studies by Sebel et al.1 and Errando et al.4were included, supporting content validity of the instrument.
View this table:
Table 1. Michigan Awareness Classification Instrument
Class 0 was constructed to reflect patient complaints about events that have a high likelihood of occurring in the preoperative or postoperative period, unconfirmed awareness, or dreaming. Because the vast majority of the reports in the literature only describe cases that have already been determined to be definite awareness, Class 0 was not a focus of this study. Classes 1–4 reflect commonly reported awareness experiences, whereas Class 5 reflects a combination of experiences (paralysis and pain) that may potentially be more distressing to the patient than either event alone. Because emotional distress was the only known predictor of late psychological symptoms in a study of awareness patients,7 we created a separate designation (“D” for Distress) that is independent of the reported experience.
As noted by Sebel et al.,1 more than one description of an awareness event may occur for a given case. In this situation, we graded an event based on the feature associated with the highest class. For example, if a patient described only an auditory perception, the event would be a Class 1. If, however, a patient reported an auditory perception and the feeling of paralysis, then the event was considered Class 4.
The reports of intraoperative awareness identified in the literature were reproduced exactly in a spreadsheet format and were distributed to evaluators. Five anesthesiologists at three different academic medical centers developed the categories (GAM, KKT, DBG, MOC, and MSA) and independently classified all cases valid for analysis. Additionally, 20 other individuals (attending anesthesiologists, anesthesiology residents, nurse anesthetists, medical students, and ancillary staff) at the three institutions who were not involved in the development of the categories also independently classified the events.
SAS v.9.1 (SAS Institute, Cary, NC) was used for statistical analysis. The magree SAS macro (available at: http://support.sas.com/kb/25/006.html) was used to test the agreement among multiple evaluators, using methodologies described by Fleiss et al.9,10 The algorithm outputs a generalized kappa coefficient, which is an adaptation of Scott's pi for multiple raters. The algorithm also calculates kappa coefficients for individual evaluations, based on a consideration of the proportion of all assignments, which went to a particular category. The overall kappa represents the degree to which the observed agreement exceeds chance agreement. Fleiss's kappa values were obtained for both the basic instrument (Classes 1–5) and the one including emotional distress (Classes 1, 1D, 2, 2D, 3, 3D, 4, 4D, 5, and 5D). The scale from Landis and Koch11 was used to assess agreement based on the kappa value.
RESULTS
We identified 15 studies that provided specific information about cases of intraoperative awareness.1–4,7,12–21 From these studies, we identified 151 cases valid for analysis. The 25 independent classifications for 151 cases had an overall kappa value of 0.851 for the basic Classes 1–5; the expanded instrument including the distress modifier had an overall kappa value of 0.779 (Tables 2 and 3). Determination of emotional distress decreased agreement in all five classes.
View this table:
Table 2. Inter-Observer Agreement for the Basic Classification Categories
View this table:
Table 3. Inter-Observer Agreement for the Classification Categories Including the Distress Modifier (“D”)
DISCUSSION
The development of classification systems has been beneficial in the study of rare perioperative events. For example, Han et al.22 first reported a scale that was subsequently used to elucidate the incidence and predictors of difficult and impossible mask ventilation.23 We have developed a novel classification of intraoperative awareness events in the hope of facilitating the future study of awareness incidences, risk factors, prevention strategies, and predictors of long-term psychological consequences. To our knowledge, this is the first classification instrument for intraoperative awareness events reported in the literature.
We acknowledge that it is difficult to standardize the individual and subjective experiences of patients who have suffered intraoperative awareness. Despite this inherent limitation, the inter-observer agreement of our instrument was consistently excellent for the basic categories of intraoperative awareness events. The determination of emotional distress was associated with less agreement, perhaps because of the difficulty in distinguishing the explicit patient report of distress from what the evaluator presumed to be distressing. It is important to note, however, that the event descriptions in the literature were not intended to be classified; based on the “almost perfect agreement” for the five classes, we anticipate that the prospective use of our classification will have improved agreement. The current analysis also suggests that patients should be explicitly asked about associated “emotional distress,” especially given its value in predicting long-term psychological sequelae.7
Although classes 0–5 would appear to be a rating scale, validation was based solely on categories. Until future validation is established, the instrument should be used and interpreted as categories rather than a ranked end point. We cannot assert that the classification is necessarily predictive of adverse outcomes in its current form; its predictive validity cannot be established based on awareness reports found in the literature dating back to 1961 with incomplete follow-up. Prospective study is ongoing in patients at high risk for awareness (MSA, MOC, and DBG—clinical trial NCT00682825) and the general surgical population (GAM and KKT—clinical trial NCT00689091). The reliability of the instrument determined in this study is an important first step.
In conclusion, we have developed a novel classification instrument for intraoperative awareness events with excellent inter-observer agreement. We believe that future studies of intraoperative awareness and its psychological sequelae will benefit from a common and standard description of awareness cases.
ACKNOWLEDGMENTS
We thank the many individuals from the University of Michigan, University of Chicago, and Washington University who participated as evaluators in this study
Commentaires
Enregistrer un commentaire