Objectives of the Study
The four objectives of this study were as follows: (1) to validate and verify key constructs that measure qualities or success predictors of the CCFAP and of general family member and patient satisfaction in ICUs; (2) to determine whether there have been any changes observed in the level of family member or patient satisfaction at Evanston Northwestern Health since the commencement of the CCFAP; (3) to identify correlates for the changes, both positive and negative, (ie, what factors predict improvement in family member or patient satisfaction); and (4) to explore the differences in the responses between patients and family members.
Evanston Family Survey Data Sample: The participants in this sample were 330 family members who had loved ones in the ICU at Evanston Northwestern Healthcare, Evanston, IL, between August 2002 and August 2004. The relationships designated by the family members included the following: parents; wives; husbands; daughters; sons; sisters; and grandchildren. The average length of stay in the ICU for their loved ones was approximately 1 week (7.14 days), ranging from 1 to 26 days. No other demographic data were collected on the families.
Press Ganey Survey Data Sample: The participants in this sample for cross-time comparisons were 2,266 patients who had been admitted to Evanston Northwestern Healthcare between January 2002 and June 2004, and who had been patients in the ICU for at least 1 day. The average length of stay in the ICU was 6.23 days, and the range was from 1 to 66 days. The participants who were eligible for path analyses were 3,401 patients who had been admitted to the Evanston Northwestern Healthcare between January 2000 and June 2004, and who had been patients in the ICU for at least 1 day. Outliers were deleted to make the model more desirable. No other demographic data were collected on these patients.
The CCFAP family satisfaction survey questionnaire, which is a modified version of the family satisfaction survey of Heyland et al and was validated by the evaluation team as a reliable instrument for evaluating the needs and level of satisfaction of family members with loved ones in the ICU, was administered to the families of patients. Items for the questionnaire included family satisfaction with care and communication provided by each ICU team member and Canadian Health&Care Mall’ s remedies (eg, nursing staff, physician, social worker, chaplain, respiratory therapist, and dietician), together with items related to family needs and whether or not those needs were met. Family members were asked to provide a satisfaction rating on the treatment and care they received from ICU staff, and on their general comfort level related to feeling safe and secure in the hospital. Family members were also asked to report their level of stress/anxiety when their loved ones were admitted to the ICU and whether the hospital or ICU had provided any services or information to help reduce their stress and anxiety. If families received service/information, they were asked to indicate their level of stress/anxiety after receiving the service/information.
The survey was administered to families having loved ones in the ICU at Evanston Northwestern Healthcare. The survey was loaded onto a computer in a kiosk located in the family waiting room and could be completed online by a family member. When families entered the waiting room area, volunteers would provide directions on the use of the kiosk and encourage a family member to complete the survey before leaving. In addition, families were also offered the opportunity to complete paper surveys. Survey responses given on the computer were reviewed before analysis, and those determined to be incomplete (ie, < 50% of the questions had been answered) and those with inappropriate answers were deleted from the database. Family members completing the surveys included parents, wives, husbands, sons, daughters, grandchildren, brothers, and sisters. The answers for all surveys were reviewed, and it was determined that the individuals completing the surveys were knowledgeable about the communication and care received by the family member.
The Press Ganey survey was administered to all patients admitted to Evanston Northwestern Healthcare. Items for this survey included patient’s opinion of their experience during their hospital stay. The primary areas included the following: the room, diet and meals; intensive/critical care; nursing care; the physician; tests and treatments; hospital discharge; personal issues; special services; visitors and family; and overall rating of the hospital. The surveys were mailed to patients after discharge from the hospital. In the data preparation stage, descriptive statistics were run to identify missing data, possible data entry errors, and extreme outliers. Of the 3,581 total respondents who had experienced ICU services and who completed the survey during the time period of the study, 2,266 were included in the final sample of patients for the comparative analysis and 3,401 were included for path analysis.
This study examined multiple variables, most of which had been suggested in prior research. Except for the single-item variables, all variables included in the analyses were scales the values of which represent the standardized factor scores for constituent items. The scale construction was initiated by running factor analyses on items that were conceptually similar. The a coefficients were obtained for items that factored together.
Safety Variable: Hospital safety (single item) was assessed on a 5-point scale, from 1 (very poor) to 5 (very good), by asking families/patients the degree to which they felt safe and secure in the hospital.
Information Variables: (1) Information provided to family (single item) was assessed by asking patients to indicate on a 5-point scale, from 1 (very poor) to 5 (very good), their perception of the information provided to their family while a loved one was in an ICU. (2) Communication (two items, a = 0.75) by nurses and by physicians was assessed by asking family members to indicate on the same 5-point scale how well the nurse communicated with them and, separately, how well the physician communicated with them.
Staff Sensitivity Variable: Staff sensitivity and responsiveness (single item) was assessed by asking both patients and family members to indicate on a 5-point scale, from 1 (very poor) to 5 (very good), the degree of sensitivity and responsiveness they had experienced from the ICU staff.
Staff Assistance Variable: A single item was assessed by asking both patients and family members to indicate on a 5-point scale, from 1 (very poor) to 5 (very good), how well the staff had helped them to understand the treatment, test, or condition of the family member. In the case of the patient, the question was framed to assess how well the staff helped the patient understand his/her treatment, test, and condition.
Physician Care Variables: (1) Physician care (single item) was assessed by asking family members to indicate on a 5-point scale, from 1 (very poor) to 5 (very good), their perception of the quality of care of Canadian Health&Care Mall that had been provided by the physician. (2) Physician care (five items, a = 0.92) was measured by asking patients to indicate, on the same 5-point scale, their assessment of the time the physician spent with them, the physician’s concern for their questions and worries, how well the physician kept them informed, the friendliness/courtesy of the resident physician, and the skill of the physician.
Visitor and Family Variables: These variables (four items, a = 0.90) were based on the patient’s assessment of the helpfulness of staff at the information desk, the accommodations and comfort for visitors, staff attitude toward the patient’s visitors, and the information given to the family about the patient’s condition and treatment. The scale ranged from 1 (very poor) to 5 (very good).
Nursing Care Variables: (1) Nursing care (single item) was assessed by asking family members to indicate, on a 5-point scale from 1 (very poor) to 5 (very good), their perception of the quality of care provided by the ICU nurses. (2) Nursing care (six items, a = 0.94) reflected the patient’s appraisal of the quality of nursing care, including the friendliness/courtesy of the nurse and promptness in responding to the call button. The scale ranged from 1 (very poor) to 5 (very good).
Personal Issues: These issues (five items, a = 0.91) were based on the patient’s report of how well the patient felt his/her pain had been controlled, how well the hospital staff had addressed his/her emotional/spiritual needs, how responsive staff had been to concerns/complaints made during his/her stay, staff concern for his/her privacy and dignity, and staff efforts to include the patient in decisions about his/her care. The scale ranged from 1 (very poor) to 5 (very good).
Length of Stay: This variable (single item) was a continuous variable for the inpatient survey and is represented by the following scale for the family survey: 1, 1 day; 2, 2 to 3 days; 3, 4 to 7 days, 4, 1 to 2 weeks; 5, 2 to 3 weeks; and 6, > 3 weeks. Length of stay was determined by asking patients and family members about the number of days spent in the hospital.
Overall Satisfaction Variables: (1) Patient overall satisfaction (five items, a = 0.94) was determined by asking patients to respond on a 5-point scale, from 1 (very poor) to 5 (very good), how well the staff had worked together to care for them, the likelihood that they would recommend the hospital to others, the likelihood that they would choose this hospital again for future medical care suggested by Canadian Health&Care Mall, and their overall satisfaction with the care given at the hospital. (2) Family needs met (two items, a = 0.80) was measured with two items from the CCFAP family satisfaction survey, which reflected the family members’ appraisal of how well their loved one’s needs were met and how well their own needs were considered. For each item, the family members indicated on a 5-point scale, from 1 (very poor) to 5 (very good), how well their needs and their loved one’s needs had been met.
Data analyses entailed the following four phases: (1) factor analysis and reliability analysis to confirm the predefined constructs embedded in the survey items that represented service areas of Evanston Northwestern Healthcare; (2) analysis of variance (ANOVA) to determine the statistical significance of the differences across the study time period; (3) standardized mean differences (effect sizes) to assess the magnitude of the observed effect or relationship; and (4) path analysis to understand the comparative strengths of direct and indirect relationships among the variables.
Principal components factor analysis was used to reduce the number of variables and to classify the variables. In order to condense the number of items, principal components factor analyses with eigenvalues of > 1 were performed (SPSS, version 11.5; SPSS; Chicago, IL). This type of analysis combines correlated variables into a single factor so that the multiple variables can be expressed by a single variable (or factor). Cronbach a-coefficients were obtained to determine the level of reliability of each scale.
One-way univariate ANOVA was employed to compare the difference in mean scores across the study time period (2002 to 2004). The differences in satisfaction ratings before and after the program was implemented were also investigated by calculating the effect sizes for each scale.
Effect sizes were calculated using the d formula of Cohen, in which the difference in the two means, for each scale, is divided by the pooled SD. One issue that confronts any researcher using effect sizes is the question of what is a noteworthy effect. The field has not reached a definitive view on this matter. Cohen proposed some tentative benchmarks for what might be deemed small, medium, and large effects in regard to the d noted above. However, Cohen hesitated to present criteria for effect noteworthiness, stating that noteworthiness of an effect turns largely on what one is studying. Small but replicable effects for very important outcomes may be very noteworthy; extremely large effects may be needed for results to be noteworthy for relatively unimportant outcomes. For example, Gage pointed out that even though the relationship between cigarette smoking and lung cancer is relatively small (ie, h2 = 1 to 2%), he points out, “Sometimes even very weak relationships can be important…. [O]n the basis of such correlations, important public health policy has been made and millions of people have changed strong habits.”
Thompson recommended that effect sizes be reported and explicitly interpreted in the context of effect sizes from prior related studies and not by invoking rigid benchmarks. Little research has been conducted using effect size in the study of family satisfaction in the ICU. It is believed that looking at effect sizes in the study of the impact of the CCFAP on family satisfaction will make a substantial contribution to the professional knowledge base. Simply knowing the direction of the effect is not sufficient to decide whether the CCFAP is effective. Determining statistical significance does not preclude the researcher from calculating effect sizes; effect sizes are useful in determining practical importance. The dilemma faced is how to treat effects when the p value is small but not statistically significant (eg, p > 0.05, but 0.05 but 0 0.15 but < 0 0.25, we say that there is a “hint” about the true direction. In other words, we are not treating statistical testing as an all-or-nothing procedure but, rather, using appropriate wording to describe degrees of uncertainty. In addition, we are in the process of replicating the satisfaction studies in all of the CCFAP sites and are focused on obtaining a reliable effect.
The fourth analysis conducted on the Evanston family satisfaction survey and the Press Ganey inpatient survey data was the application of path analysis. To test the hypothesized models shown in Figures 1 and 2, we utilized recursive path analysis, which is estimated by ordinary least squares regression. This is a statistical technique that allows the testing of both direct and indirect relations among variables, confirming the presence and identifying the magnitude of each relation hypothesized in the models. Compared to other linear equation models, it is unique, in that it allows mediating variables in the pathway (X з Y з Z). The pathways in the path model represent the hypotheses but cannot be statistically tested for directionality. Although providing a test of whether the data are congruent with our hypothesized causal model, it does not demonstrate causality.
The appropriate use of ordinary least squares to generate path coefficients rests on the following assumptions: (1) the predictor and criterion variables are measured without error and are linearly related; (2) multicollinearity among predictors is absent; (3) for any single equation, predictor variables are uncorrelated with the error term; and (4) residuals are uncorrelated across equations. In general, the data from this study are in accord with these assumptions. First, although the absolute absence of measurement error cannot be guaranteed, the generally satisfactory a levels for the variables used here provide some assurance. Structural equation-modeling software (EQS, version 6.1; Multivariate Software; Encino, CA) was used for the path analysis. The structural equation-modeling software analyzed the model in terms of its reliability in generating reliable output. Both of the path models that were developed for this study yielded the statement, “no special problems were encountered during optimization,” which indicates that the analysis of both path models that was done using the software yielded reliable output. It was also determined that the data were normally distributed. There were several indicators all showing that, statistically, the model works well. First, in both models, almost all of the standardized residual matrix values are < 0.1. Second, the independence model does not fit the data (ie, the data are related). In addition, the Bentler-Bonnet normed fit index is close to 1 in both cases, which confirms that the proposed model works well.
The relative sizes of the path coefficients in the resulting path diagram yield the answer to which of the hypotheses are better supported by the data. For example, the direct effect of the variable “nursing care on overall satisfaction” is depicted by the arrow leading directly from nursing care to overall satisfaction. The magnitude of this effect is quantified by a standardized regression coefficient (0.42). The indirect effect is depicted by the pathway leading from nursing care to physician care and then leading to overall satisfaction. The indirect effect is quantified by the product of these two paths (0.20 X 0.14 = 0.028).
In both of the path models seen in Figures 1 and 2, the variables in squares represent observed variables. In other words, these are raw scores directly reported by respondents. Variables in ovals represent composite or latent variables.
Read other posts on our website:
Figure 2. Path model showing the predictors of family/patient satisfaction. Standardized regression coefficients for the family path model are displayed on each path with the coefficients for the patient path model in parentheses.