Dr Gina Di Malta, Senior Lecturer in Psychotherapy and Counselling, Open University

The field of mental health and psychotherapy has seen significant advancements over the years with a constant quest to improve therapeutic outcomes. One critical aspect of this quest, and particularly from a pluralistic standpoint, is understanding the client’s perspective. At the heart of our programme of research—comprising four separate studies—lays the question: How do patients perceive the helpfulness of the outcome measures used in their treatment? We’re delighted that the results of our research have now been published in one of the leading international journals in the field, Assessment (see Open Access article here).

The journey of the development and validation of a unique self-report measure, the Patient-Perceived Helpfulness of Measures Scale (ppHMS), started back in 2016 at the CREST Clinic, University of Roehampton, which offered up to 24 weeks of pluralistic therapy. The scale was designed to gauge patients’ perspectives on the helpfulness of the measures used in psychological treatment. An endeavour that promised to transform the way psychotherapy is personalised and delivered.

The Birth of the ppHMS

The journey begins with the realisation that understanding the client’s perspective is essential to enhancing therapeutic outcomes. But how do you start measuring something as subjective as the patient’s perception of helpfulness of measures in psychological treatment?

We started by asking our patients about their experiences of using measures. In the first study, qualitative interviews with patients paved the way for item development. Patients’ experiences and insights provided a foundational understanding of the construct of ‘patient-perceived helpfulness of measures’. This approach revealed the nuances and intricacies of the patients’ perspective, guiding the development of items that accurately represented this perspective.

From an extensive item pool, a 10-item scale emerged. The goal was to make the scale as concise and comprehensive as possible, keeping the patient’s experience at the forefront. Further psychometric exploration in both clinical and online samples of psychotherapy patients led to the creation of a more streamlined six-item scale.

Stability Across Nations

The following steps in this journey were to test the ppHMS on diverse and representative groups of clients. A stratified sample of UK patients participated, providing insights into the usefulness of the scale. At this stage a five-item model emerged. The model was confirmed in a new US stratified sample, showcasing the stability of the ppHMS across the two nations. The ppHMS proved itself to be a generic scale capable of assessing the helpfulness of psychotherapy measures in different settings.

Reliability and Validity

Any measurement tool needs to meet certain criteria based around reliability and validity. The ppHMS excelled in this regard. It displayed excellent internal consistency (i.e., the different items aligned with each other) and test-retest reliability (people gave similar answers at a subsequent time point), reinforcing its reliability as a measurement tool. Furthermore, it demonstrated convergent and divergent validity (showing that it was statistically related to similar measures, and different from measures of other things). Convergence with the ‘Delighted-Terrible scale of product satisfaction’ was particularly strong, underlining the ppHMS’s ability to capture the patient’s perception of form helpfulness effectively. However, satisfaction with psychotherapy showed lower convergence, emphasizing the unique nature of the ppHMS in assessing the helpfulness of measures.

Invariance Across Measures

An essential aspect of any measurement tool is its invariance, which shows the stability of an instrument across different uses and contexts. In our studies, the ppHMS demonstrated invariance over time periods and countries.

A unique feature of the ppHMS was for the scale to be used to assess the helpfulness of a range of therapy measures, and demonstrate invariance across all measures. In our studies we used the ppHMS with the most commonly used outcome and feedback measures in the UK and US. Invariance was observed in all cases. This adds to the scale’s versatility and robustness as a tool that can be used to assess helpfulness across outcome and feedback measures.

Limitations and Future Directions

As with any research, there are limitations to consider. The studies that led to the development and validation of the ppHMS had their own unique constraints. For instance, the initial qualitative interviews were limited to patients without any serious mental health conditions, which may have created a positively biased representation of patients’ experiences of using measures. Additionally, the studies focused on Western samples and would benefit from further validation in diverse and non-Western cultures. Future research can also explore the predictive validity of the scale on psychotherapy outcomes.

Unlocking the Potential of the ppHMS

The development and validation of the Patient-Perceived Helpfulness of Measures Scale offer a transformative tool for the field of psychotherapy. The ppHMS can be applied in four ways to benefit patients.

1. Service-Level Decision-Making: The ppHMS can provide patient-centric evidence to inform the selection of measures as part of Routine Outcome Monitoring (ROM) and Clinical Feedback (CF) systems in services. It supports the decision-making process around choosing the most helpful measures.
2. Research with Patient Groups: The ppHMS can be used in research to identify which measures are most helpful for specific patient groups. This data can be used to tailor ROM and CF systems for different demographics or diagnoses, enhancing the quality of care.
3. Individual Patient Level: Clinicians can use the ppHMS to identify the most helpful measures for individual patients. This personalised approach can enhance patient engagement and improve outcomes by selecting measures that align with a patient’s specific issues.
4. Enhanced Dialogue: The ppHMS can also prompt dialogue between patients and psychotherapists, encouraging shared decision-making and discussions about psychotherapy processes.

Looking Ahead

As we continue on this journey to harness the power of patient feedback to inform evidence-based practice, it seems the ppHMS has the potential to revolutionise the way in which routine outcome monitoring in psychotherapy is personalised and delivered. It emphasizes the importance of understanding and respecting the patient’s perspective, leading to more effective and meaningful therapeutic experiences.

The ppHMS, with its reliability, validity, and versatility, stands as a beacon of hope in the ever-evolving field of psychotherapy. It is a reminder of the pluralistic principle that progress is made when we listen to the voices of those at the heart of it all—the clients. This scale has the potential to change the landscape of mental health treatment and usher in a new era of personalised care, where each client’s unique needs are at the forefront of their therapeutic journey.

Reference

Di Malta, G., Cooper, M., Bond, J., Raymond-Barker, B., Oza, M., & Pauli, R. (2023). The Patient-Perceived Helpfulness of Measures Scale: Development and validation of a scale to assess the helpfulness of using measures in psychological treatment. Assessment. https://doi.org/10.1177/10731911231195837

Measure

The ppHMS items can be seen below, with PDF and Word versions for download. The measure is free to use under Creative Commons License CC-BY-NC-ND 4.0 . Please do let us know how you get on with it and any findings.

---

---

Patient-Perceived Helpfulness of Measures Scale (ppHMS)

For form ____________, please select a box to indicate how much you agree or disagree with the statements below. There are no right or wrong answers.

1. I got something out of using this form

1 Strongly disagree

2 Disagree

3 Neutral

4 Agree

5 Strongly agree

2. The form was a useful addition to what we did in therapy

1 Strongly disagree

2 Disagree

3 Neutral

4 Agree

5 Strongly agree

3. This form got me thinking about what matters to me in therapy

1 Strongly disagree

2 Disagree

3 Neutral

4 Agree

5 Strongly agree

4. Using this form made the therapy better

1 Strongly disagree

2 Disagree

3 Neutral

4 Agree

5 Strongly agree

5. I learnt something from using the form in therapy

1 Strongly disagree

2 Disagree

3 Neutral

4 Agree

5 Strongly agree