Measuring patient perspectives on community pharmacy quality: psychometric evaluation of a brief survey

BACKGROUND: Measuring patient perspectives of the quality of health care delivery is an essential component of building a patient-centered model of care, which has garnered increasing emphasis under value-based payment models. Although measurements of patient perspectives of physician practices are common, few validated measures are currently available to assess patient perspectives of pharmacy quality. OBJECTIVE: To assess the reliability and validity of a brief measure that assesses patient experiences with the quality of community pharmacy services, referred to as the “patient experience” measure. METHODS: This article focuses on the second phase of a 2-phase survey development study. In the second phase, a convenience sample of 400 adult patients with chronic health conditions completed the final 7-item version of the patient experience measure via an online survey. An exploratory factor analysis was performed to determine the number of factors underlying the patient experience measure. Internal consistency reliability was assessed with Cronbach’s alpha, and construct validity was evaluated by examining relationships between the patient experience measure, a quality of pharmacy services item, and patient demographic variables. RESULTS: Eigenvalues and scree plots suggested a 1-factor solution, with this single factor explaining 64% of the variance in the patient experience measure. Cronbach’s alpha indicated excellent reliability (α = 0.90). Individuals with more positive patient experience scores rated the quality of their pharmacies’ services significantly higher (Pearson r = 0.73, P < 0.001). Mean patient experience scores for those who had gone to their pharmacies for 7-11 months (mean = 3.66) were significantly lower than for those who had been going to their pharmacies for 5 or more years (mean = 4.22). CONCLUSIONS: The patient experience measure demonstrated excellent reliability and was correlated with patient perceptions of pharmacy quality. The measure also differentiated between individuals based on the length of time they had been going to their pharmacies. The scale could be used to quickly assess patient experience with community pharmacy services.


METHODS:
This article focuses on the second phase of a 2-phase survey development study. In the second phase, a convenience sample of 400 adult patients with chronic health conditions completed the final 7-item version of the patient experience measure via an online survey. An exploratory factor analysis was performed to determine the number of factors underlying the patient experience measure. Internal consistency reliability was assessed with Cronbach's alpha, and construct validity was evaluated by examining relationships between the patient experience measure, a quality of pharmacy services item, and patient demographic variables.

RESULTS:
Eigenvalues and scree plots suggested a 1-factor solution, with this single factor explaining 64% of the variance in the patient experience measure. Cronbach's alpha indicated excellent reliability (α = 0.90). Individuals with more positive patient experience scores rated the quality of their pharmacies' services significantly higher (Pearson r = 0.73, P < 0.001). Mean patient experience scores for those who had gone to their pharmacies for 7-11 months (mean = 3.66) were significantly lower than for those who had been going to their pharmacies for 5 or more years (mean = 4.22).

What is already known about this subject
• Given the movement toward accountability and value-driven payment for health care, it is paramount that quality measures used in alternative payment models are developed and validated before implementation.
• Although numerous measures have been developed to assess patient experiences with the quality of physician and hospital services, little attention has been given to developing measures to assess patient experiences with the quality of pharmacy services.
• As pharmacists expand their roles to offer more clinical care services, the ability to measure the quality of those services will take on greater importance.

What this study adds
• This study describes the development and evaluation of a brief measure to assess patient experiences with the quality of pharmacy services, referred to as the "patient experience" measure.
• The 7-item patient experience measure was valid, reliable, and differentiated patients based on their perceived quality of pharmacy services.
Increasingly, the U.S. health care system has adopted alternative payment models (APMs) for provider reimbursement. APMs provide incentives to providers for providing highquality and cost-efficient care. In 2018, approximately 36% of U.S. health care payments were tied to some form of APM, including the use of bundled payments, shared savings, or population-based payments. 1 Given the movement toward accountability and value-driven payment for health care, it is paramount that quality measures used in APMs are developed and validated before implementation. Although significant attention has been given to the measurement of quality for physician and hospital services, less attention has been given to measuring the quality of community pharmacies and the services they provide. As pharmacies expand clinical service offerings, the ability to measure the quality of those services will take on greater importance. Different types of measurements are commonly used to assess the quality of health care, including structural, process, and outcomes measures. 2 Structural measurements assess the setting, administrative systems, and qualifications of clinicians to assess the resources available to patients at a practice site. Process-based measurements assess the components of care delivery to examine how well clinicians provide specified health care services or procedures. Outcomes measures assess the ultimate outcome of providing care and reflect improvements in patient health.
In addition, the Centers for Medicare & Medicaid Services (CMS) Star Ratings Program emphasizes the measurement of patient experience. 3 Better patient experience is predictive of higher quality process and outcomes measures commonly used in APMs; this correlation may result from an emphasis by providers in higher performing organizations on the technical aspects and interpersonal components of care. 4 Under the Star Rating Program, patient experience is currently measured using the Consumer Assessment of Healthcare Providers and Systems (CAHPS) survey, 5 which captures how patients experience or perceive key aspects of their health care, including communication with their physicians and perceptions of coordination of their health care needs.
With the shift toward greater use of APMs, pharmacies are increasingly being assessed using quality measurement standards. In fact, many existing quality measures under programs such as the star ratings system emphasize medication quality, which can be influenced through pharmacist medication management. In addition, demonstration models emphasize the use of APMs to promote the expansion of clinical services in community pharmacies. Prominent examples of this are incorporation of networks of community pharmacies to provide enhanced clinical services through initiatives such as the Community Pharmacy Enhanced Services Network (CPESN) USA and OutcomesMTM. [6][7][8][9] CPESN is a clinically integrated network focused on coordinating patient care and providing value to the health care market, while OutcomesMTM helps pharmacies administer medication therapy management services to meet CMS requirements. This shift toward clinical services and valuebased payment in community pharmacies highlights a need for validated measures to assess the quality of care that community pharmacies provide.
Although several medication management quality measures assess pharmacy performance, no measures have been specifically developed to assess patient perceptions of the quality of community pharmacy services. 10 Additionally, no patient experience measures have been endorsed to measure the quality of community pharmacies operating under APMs, and the application of instruments validated in ambulatory care settings do not appear appropriate for use in community pharmacy settings. 11 Therefore, our objective was to develop and assess the reliability and validity of a brief patient-reported measure to assess patient experiences with the quality of community pharmacy services.

PREVIOUS PHARMACY QUALITY SURVEY
The 7-item patient experience measure was developed over 2 years, using an iterative scale development process. 12 In brief, an initial pharmacy quality survey was developed and piloted in 2017 to examine patient perspectives related to 3 constructs (patient activation, medication understanding, and patient satisfaction). The survey included 6 items that measured patient activation (i.e., patient knowledge, skills, ability, and willingness to manage their health and health care 13 ); 9 items related to patient medication understanding (i.e., medications used, indications for use, instructions for use, and potential risks and benefits); and 8 satisfaction items that measured the ability of a pharmacist to meet the needs of a patient. 14 The survey also included items specific to comprehensive review of medication services and CONCLUSIONS: The patient experience measure demonstrated excellent reliability and was correlated with patient perceptions of pharmacy quality. The measure also differentiated between individuals based on the length of time they had been going to their pharmacies. The scale could be used to quickly assess patient experience with community pharmacy services.

CURRENT STUDY SAMPLE
This study received an exemption from the Institutional Review Board at the UNC-Chapel Hill. Patients were recruited through the Qualtrics XM chronic disease panel. 15 After registration, chronic disease panel participants enter basic data about themselves, including demographic information, and self-report whether they have any of 41 chronic diseases, ranging from AIDS to cancer to arthritis.
A convenience sample of 400 participants were recruited. A sample of approximately 10 subjects per item is recommended to yield reliable factor analysis results 16 ; thus, a sample of 400 was deemed sufficient to examine the psychometric properties of the 7-item pharmacy satisfaction measure. Participants were eligible if they (a) read English; (b) were aged at least 18 years; (c) took a prescription medication to treat a chronic health condition (e.g., diabetes, high blood pressure, or asthma); (d) did not fill their prescriptions through mail order or through the VA (Veterans Affairs); (e) went to the pharmacy to get their prescriptions filled at least once a month or once every month or 2; and (f) managed their own medicines.

DATA COLLECTION PROCEDURES
Qualtrics sent an email to panel members inviting them to participate in an anonymous, 5-minute survey. Interested panel members first read a description of the survey and then advanced to the survey's eligibility screening questions. Eligible participants advanced to the main survey and received an incentive worth approximately $5. All data were collected in March 2019.

MEASURES
Patient Experience. The 7-item patient experience measure assessed the experiences of patients with their pharmacies, pharmacists, and the support and quality of medicationrelated services that their pharmacists provided. Items were assessed on a 5-point Likert scale with response options ranging from 1 = strongly disagree to 5 = strongly agree.
Pharmacy Quality. Patients rated the quality of services that they received from their pharmacies in the previous 12 months on a sliding scale from lowest-quality pharmacy services possible (0) to highest-quality pharmacy services possible (10).
Demographic Questions. Six items assessed age (in years), gender, level of education, length of time using current pharmacy, frequency visiting the pharmacy, and number of prescription medications taken. demographic questions. This initial 56-item pilot survey was administered via telephone to 406 patients who used CPESN pharmacies in North Carolina. 7,8 A sample of 400 respondents was recruited in order to balance the need for a representative sample and meet budget restrictions.
Analysis of the initial survey demonstrated 2 key limitations. First, most survey items demonstrated significant ceiling effects, which correlated strongly with patient satisfaction. This suggested that patients were unable to differentiate between their general satisfaction with pharmacies and the quality of services received. Because of the high ceiling effects, the survey could not distinguish pharmacies with high-quality services from those with lower-quality services. Second, an exploratory factor analysis revealed that items did not load onto their respective patient activation, medication understanding, and patient satisfaction constructs, indicating that the items were not uniquely capturing variance in the target constructs.

COGNITIVE INTERVIEWS
Given the initial survey's limitations, we conducted 10 cognitive interviews with a purposive sample of chronic medication users to better understand how they understood survey items and response options. Individuals were recruited via Community Care of North Carolina and community list-serve postings to capture variability in region, race, gender, and pharmacy type (independent vs. chain).
Interviews revealed that patients had difficulty rating pharmacy quality, especially for clinically oriented items, such as the ability of pharmacists to improve medication understanding and patient activation. Interviewees expressed a lack of expectations from pharmacists to provide health services, such as medication adherence services, as well as a perception that providing these services was not the role of pharmacists. For example, in response to a medication effectiveness question, 1 participant said, "No, my doctor can do that. They are just pharmacists; they aren't doctors." Additionally, cognitive interview findings helped explain the initial survey's ceiling effects. Namely, patients expressed that they were highly satisfied with their pharmacies and that this satisfaction drove their responses to individual quality items. Given these observations, the cognitive interviews asked patients to reflect on their own personal judgements of pharmacy quality to guide the construction of the 7-item patient experience measure that was evaluated in the current study. Specifically, patients were provided with a paper copy of the survey, and the interviewer used a semistructured interview guide to ask how they would respond to survey items based on their personal pharmacy experiences.

SAMPLE CHARACTERISTICS
A total of 1,253 individuals accessed the survey, of which 843 were ineligible and 10 were randomly excluded due to panel oversampling (Supplementary Figure 1,

PATIENT EXPERIENCE ITEMS
Means for patient experience items ranged from 3.67 ("My pharmacist treats me like more than just a customer") to 4.50 ("I trust my pharmacist to fill my prescriptions safely"; Table 2). Three items had means lower than 4.0, which indicates that, on average, participants had more neutral opinions related to those statements.
Eigenvalues and scree plots suggested a 1-factor solution (Supplementary Figure 2 and Supplementary Table 1, available in online article). Only 1 factor had an eigenvalue that exceeded 1. This factor explained 64% of the item variance. A principal component analysis revealed that all 7 items loaded at 0.75 or higher on this single factor, providing additional evidence for a single factor solution.

RELIABILITY AND VALIDITY
Cronbach's alpha for the satisfaction scale was excellent (alpha = 0.90).
Patient experience scores strongly correlated with participant perceptions of pharmacy quality (Pearson r = 0.73, P < 0.001); individuals with higher patient experience scores rated the quality of their pharmacies' services higher. Patient experience scores weakly correlated with age (Pearson r = 0.12, P = 0.02), with older individuals reporting more positive patient experience. One-way ANOVA revealed a significant difference in patient experience by length of time that patients had been going to their current pharmacies (F (6,393) = 3.16, P = 0.005). The mean patient experience score for those who had gone to their pharmacy for 7-11 months (3.66) was lower than that of those who had been going to their pharmacies for 5 years or more (4.22). Patient experience was not significantly associated with

DATA ANALYSIS
Data were analyzed in SPSS Statistics 25 (IBM, Armonk, NY). After examining item-level descriptive statistics, including factor loadings (i.e., the relationship of each item with the underlying factor), an exploratory factor analysis with promax rotation was conducted. We first analyzed scree plots and eigenvalues to determine the number of factors underlying the solution. We calculated Cronbach's alpha to assess internal consistency reliability. To assess construct validity, we examined bivariate associations between the patient experience measure and (a) demographic characteristics and (b) the overall pharmacy quality item. Specifically, Pearson's correlations, t-tests, and 1-way ANOVAs with Tukey HSD tests were used as appropriate.

Frequency of pharmacy visits
At least once a month 320 80 Once every month or two 80 20 Time to complete survey (in seconds) 206 117

TABLE 1 Sample Demographics (N = 400)
Changing norms around expectations for the level of care pharmacists can provide could lead to greater acceptance of pharmacists as providers of clinical services. This may also provide some explanation as to why, despite there being several surveys to capture patient perceptions, no measures have garnered endorsement for use in APM models implemented in the community pharmacy setting. 17,18 Patient experience scores were also significantly associated with the amount of time patients had been going to their pharmacies. Patients who have positive experiences with their pharmacies may be more likely to keep going to those pharmacies. More qualitative work is needed to understand which aspect of patient experience measured is most valued by patients and leads them to stay with their pharmacies long term.
The patient experience measure may yield greater variability in scores if administered to individuals who are using a new pharmacy for the first time, since individuals do not have previous experiences with the pharmacy to influence their responses. Longitudinal studies that determine how patient experience scores change over time would provide insight into whether experience scores vary by visit or are based on a gist interpretation of one's overall impression of the pharmacy.

LIMITATIONS
This study has some limitations to consider. Use of a convenience sample could limit generalizability. Although Qualtrics panel members had a self-reported chronic the number of prescription medications taken, gender, education, or prescription fill frequency.

Discussion
The 7-item patient experience measure performed well among a sample of individuals with chronic disease using community pharmacies to fill their prescription medications. The current measure has several attributes that are desirable in measuring pharmacy performance under APMs. Specifically, every response option from "strongly disagree" to "strongly agree" was endorsed by participants, and 3 items had mean Likert responses less than 4, the response option for "agree." This improves the ability of the survey to distinguish between pharmacies and addresses the limitation of a strong ceiling effect.
There are significant challenges to measuring community pharmacy quality from the perspective of patients. Chief among these is that low patient expectations of the services provided by community pharmacies hinder the ability to measure the quality of specific clinical services. Specifically, patients in the cognitive interview phase of the study stated that physicians, not pharmacists, should be the point person for medication understanding and assessment of medication effectiveness. In order for pharmacies to successfully transition to a more clinical model of care, there is a need for the profession to increase patient awareness of the services that pharmacies can offer and the level of clinical care patients should expect from their pharmacies.  disease and filled an average of 4 prescriptions at community pharmacies, they may not be representative of the larger population of community pharmacy clients. Approximately 40% of the sample had at least a bachelor's degree, which also may not be representative of the general population. Future work should administer the measure with more representative samples to see if the results can be replicated and determine the correlation between the patient experience measure and measures of pharmacy satisfaction to examine the magnitude of association between these 2 constructs.
Many survey items asked specifically about the pharmacist and not the pharmacy or other pharmacy staff. This decision was made because several cognitive interview participants were unaware of whether the staff members who they interacted with were pharmacy technicians or pharmacists.
Finally, we did not ask what type of pharmacy (e.g., chain or independent) that individuals frequented. Future work should determine whether pharmacy quality ratings vary by pharmacy type.

Conclusions
In this sample, the patient experience measure had excellent reliability and good construct validity, as evidenced by the measure's ability to differentiate between individuals based on their perceived quality of pharmacy services, as well as the length of time they had been going to their pharmacies. Future work should examine whether the measure performs well in other samples and further explore relationships between the patient experience measure and other constructs, such as pharmacy satisfaction.