Abstract

Background

The effectiveness of the NSW Health 鈥淕et Healthy Information and Coaching Service^庐鈥�(Get Healthy) to facilitate weight loss on a population scale has been documented, but this was based on self-reported measures. Our study aims to test the effectiveness of the Get Healthy Service on objectively measured weight, BMI, waist circumference, and changes in other health behaviours, including nutrition, physical activity and alcohol intake.

Methods

Men and women aged 40鈥�70 years (n鈥�=鈥�154) with pre-diabetes (5.7% < HbA1c鈥�<鈥�6.5%) were referred from GP Practices to the Get Healthy Service, NSW Health. A subset (n鈥�=鈥�98) participated in the 鈥淶inc In Preventing the Progression of pre-Diabetes鈥� (ZIPPeD) trial (ACTRN12618001120268).

Results

The self-reported outcomes showed a statistically significant improvement from baseline to 12 months in weight (mean 2.7听kg loss, p鈥�<鈥�0.001), BMI (mean 1 unit reduction, p鈥�<鈥�0.001), and waist circumference (mean 4.3听cm reduction, p鈥�<鈥�0.001). However, in the objectively measured outcomes from ZIPPeD, the differences were more modest, with point estimates of 0.8听kg mean weight loss (p鈥�=鈥�0.1), 0.4 unit reduction in BMI (p鈥�=鈥�0.03), and 1.8听cm reduction in waist circumference (p鈥�=鈥�0.04). Bland-Altman plots indicated that discrepancies were due to a small number of participants who dramatically underestimated their weight or BMI. There were minimal changes in nutrition, physical activity, and alcohol.

Conclusions

The potential benefits of Get Healthy should be interpreted with caution as we have shown significant differences between self-reported and objectively measured values. More valid and objective evidence is needed to determine the program鈥檚 effectiveness and cost-effectiveness.

Background

Weight loss is an important strategy for people living with overweight or obesity with, or at risk of developing, type 2 diabetes and other chronic conditions. Among these individuals, as little as 5% body weight loss can improve glycaemic control and reduce the need for diabetes medication [1]. Aimed at population health promotion, NSW Health offers the Get Healthy Information and Coaching Service (Get Healthy), supporting adults to make positive lifestyle changes and reduce excess weight, through healthy eating and physical activity.

Launched in 2009, [2] Get Healthy is a free, telephone-based service in which participants receive 10鈥�13 individually tailored phone coaching, generally over a six-month period, delivered by university qualified health professionals, called coaches. The service was initially designed for self-referral, with a complementary clinician referral pathway subsequently added. Get Healthy covers multiple distinct programs, tailored for diabetes, pregnancy, or Aboriginal participants, which differ in content and intensity. Get Healthy is a key initiative in the State鈥檚 Healthy Eating and Living (HEAL) Strategy and the NSW Ministry of Health has invested significantly in the provision of this service [3].

There is evidence that the Get Healthy programs are effective for reducing weight and improving diet over 6 months, although these evaluations largely relied on self-reported information obtained via telephone by the coaches as part of the standard service [4]. One validation sub-study has been performed for Get Healthy, on a convenience urban sample of 38 participants. This validation compared self-reported and objectively measured height, weight, and waist circumference at six months follow-up [5]. While the study found acceptable validity for self-reported height, weight, and waist circumference, and modest reliability in the nutrition and physical activity measures, these validation results require replication.

Previously, we reported a placebo-controlled, double-blind, randomised trial testing whether zinc supplementation could improve glucose handling parameters in participants with pre-diabetes: the Zinc In Preventing the Progression of pre-Diabetes (ZIPPeD) study [6]. Although the trial did not show any effect of zinc on glucose handling, all participants (both control and intervention) were referred to the Get Healthy Service for the Type 2 Diabetes Prevention. The secondary outcomes in the ZIPPeD study (weight, BMI, and waist circumference) coincided with the outcomes targeted in Get Healthy, which provided an opportunity to independently assess the effectiveness of the Get Healthy Service for improving health markers and behaviours associated with diabetes risk, using a longitudinal cohort study design. In addition, the collection of objective measurements at participants鈥� General Practitioner鈥檚 (GP) office provided an opportunity to directly compare the subjectively reported and objectively measured weight, BMI, and waist circumference for the same individuals, before and after participation in Get Healthy. Self reported nutrition and physical activity data were also collected in ZIPPeD, providing the opportunity to assess the concurrent validity of these measures.

Given our randomised controlled trial (RCT) detected no effect of zinc supplementation on any outcomes, the current study combined the ZIPPeD intervention and control groups and reports the change in outcomes from baseline to 12 months for the Get Healthy measures (subjectively reported) and our ZIPPeD study measures (objectively measured). The primary aim was to test the effectiveness of the Get Healthy Service on weight, BMI, and waist circumference, and health behaviours including alcohol intake, nutrition, and physical activity. Secondly, we tested the concordance between the Get Healthy and trial data. Thirdly, we tested the effectiveness of Get Healthy on blood pressure, glycaemic control, and blood lipids at 12 months follow-up.

Methods

Study design and sample

The ZIPPeD randomised placebo-controlled trial (ACTRN12618001120268) was conducted within the Hunter New England Local Health District (HNELHD) from 2018 to 2021. Study recruitment was impacted by COVID-19. The ZIPPeD protocol [7] and the results [6] have been published previously. In brief, ZIPPeD included 98 participants, testing the effectiveness of 30听mg zinc gluconate daily supplementation over 12 months for improving glycaemic control in those with pre-diabetes.

Potential participants were screened by a research nurse at GP practices and those found to have pre-diabetes were offered referral to the Get Healthy Type 2 Diabetes Prevention Service (Supplementary Fig.听1). Participants who consented to Get Healthy were also invited to join the ZIPPeD trial, which included men and women aged 40鈥�70 years with pre-diabetes and a body mass index (BMI)鈥夆墺鈥�28听kg/m².

The Get Healthy team collected data for referred participants throughout the ZIPPeD study. These data were shared with the study team at the individual level for those who consented and participated in ZIPPeD and at the aggregate level for those who were screened and consented to Get Healthy, but not ZIPPeD. Participants who progressed to type 2 diabetes during the trial were withdrawn from the study but were encouraged to continue with Get Healthy and referred to their GP for standard care. These participants remained in the Get Healthy evaluation but did not have an objective measure of weight or other outcomes at 12 months.

This study was approved by the Hunter New England Human Research Ethics Committee, NSW, Australia (18/07/18/3.03) and registered with the University of Newcastle Human Research Ethics Committee (H-2018-0302-959).

Data collection

The primary outcomes were weight, BMI, and waist circumference, while the secondary outcomes were serum metabolic markers (Low Density Lipoprotein (LDL), High Density Lipoprotein (HDL), Total Cholesterol, Triglycerides, Fasting Blood Glucose, Haemoglobin A1c (HBA1c) and Insulin), dietary measures, physical activity, and alcohol intake. Participants in ZIPPeD had baseline measurements performed and were followed up at 1, 6, and 12 months. Pathology request forms were mailed to participants who then presented to their nearest community pathology laboratory. At the baseline and 12-month follow-up appointments, anthropometry was collected by a trained Registered Nurse in the General Practice setting, blinded to allocation group, at the time of their appointment. Participants enrolled in Get Healthy were asked to self-report their height and weight at enrolment, and their weight at 6 and 12 month follow up, via phone calls. Instructions were to measure height without shoes, and to measure weight with only underclothes on; no other standardisation was performed.

The Dietary Questionnaire for Epidemiological Studies (DQES v2), a validated, semi-quantitative food frequency questionnaire [8] was collected at baseline and 12-month follow-up in the ZIPPeD study. The DQES v2, from the Cancer Council of Victoria is a 74-item food frequency questionnaire developed specifically for Australian adults. Data were analysed by the Cancer Council, using previously described and validated methods [8].

Physical activity data were collected in ZIPPeD using questions from the Active Australia survey, [9] including items to assess the frequency (number of sessions) and duration (minutes per week) of walking, moderate, and vigorous physical activities over the previous week. The Active Australia Survey is an established questionnaire for monitoring physical activity levels at the population level, [9] although its criterion validity against accelerometry data is limited [10, 11].

All participants enrolled in Get Healthy were asked about their alcohol consumption at baseline and completion of the program. Alcohol intake was assessed using the modified Alcohol Use Disorders Identification Test (AUDIT-C), a 3-item alcohol screen [12, 13].

Data analysis

Characteristics of those who did and did not participate in ZIPPeD were summarised using frequency with percent (ordinal or dichotomous measures) or median with interquartile range (Q1, Q3; continuous measures) and compared using Chi-squared test or Wilcoxon/Mann-Whitney tests, respectively. The effect of Get Healthy on each outcome was estimated as the mean change from baseline to follow-up (on the scale of the link function; see below) with 95% confidence interval (CI).

Generalised linear mixed models were used for estimation, including a random intercept to account for repeated measures on participants and when appropriate, a fixed effect indicating participation in ZIPPeD (yes/no). For continuous outcomes (weight, BMI, waist circumference, and laboratory values) a normal response distribution and identity link function was used, with effects estimated as the least squares mean change from baseline to follow-up with 95% CI. For count outcomes (number of serves of fruit and vegetables, number of physical activity sessions), a Poisson response distribution with log link function was used and effect estimates were exponentiated to incidence rate ratios (IRR) with 95% CI. Regression diagnostics for the linear models included inspection of the histogram of the residuals and residual versus fitted plots. For the count outcomes, the Poisson regression assumption of equal mean and variance of the response was assessed by comparing likelihood with a negative binomial model that accommodated overdispersion of the outcome (variance鈥�>鈥塵ean).

To compare Get Healthy self-reported data to ZIPPeD study measures, Spearman correlation coefficients were calculated. Systematic bias between the two methods of ascertainment was explored using Bland-Altman plots and paired t-tests. A sensitivity analysis including only completers, including those who graduated early, was also performed. Statistical programming and analyses were performed using SAS9.4 (SAS Institute, Cary, NC), with results declared statistically significant at the conventional 5% level.

Given this was a secondary analysis of a previous RCT, sample size calculations were not performed a priori. However, our total sample size (n鈥�=鈥�154) should be sufficient to detect a Cohen鈥檚 d of 0.2 at 80% power and p鈥�=鈥�0.05, and our ZIPPeD sample (n鈥�=鈥�98) should be sufficient to detect a Cohen鈥檚 d of 0.3; both of these effects may be considered small but clinically relevant.

Results

Participants

Figure听1 describes the flow of participants who were included in this study. Overall, 1,212 people with pre-diabetes were identified through GP practices; of these 251 were eligible for the Get Healthy program and ZIPPeD trial, of whom 143 consented to ZIPPeD but only 98 completed the baseline questionnaires and were randomised. The Get Healthy referral form was lost for one ZIPPeD participant and they were not enrolled. A further 57 participants who were referred to Get Healthy but were either not eligible for ZIPPeD, or were eligible but dropped out between consent and completion of baseline questionnaires, were also included, leading to 154 participants being included in analyses.

While there were 22 early completions and 32 graduations from the program, most participants (100/154, 65%) withdrew or were lost to follow-up. On the other hand, 11 people re-enrolled in a repeat Get Healthy program.

Fig. 1

Flowchart of patient recruitment and inclusion (GH鈥�=鈥塆et Healthy)

The characteristics of those who did and did not participate in ZIPPeD were largely similar, although the trial participants tended to be younger, chose the brief intervention more often compared to the full program, and were more likely to withdraw or be lost to follow-up (Supplementary Table 1).

Change over time in weight outcomes

Using the self-reported outcomes from Get Healthy, there was a statistically significant improvement from baseline to 12 months in weight (mean 2.7听kg loss, p鈥�<鈥�0.001), BMI (mean 1 unit reduction, p鈥�<鈥�0.001), and waist circumference (mean 4.3听cm reduction, p鈥�<鈥�0.001) (Table听1). However, using the objectively measured outcomes from ZIPPeD the differences were more modest, with point estimates of 0.8听kg mean weight loss (p鈥�=鈥�0.1), 0.4 unit reduction in BMI (p鈥�=鈥�0.03), and 1.7听cm reduction in waist circumference (p鈥�=鈥�0.04). Comparing the self-reported and objectively measured outcomes at baseline suggested a consistent underestimate of weight and waist circumference in self-reported data, and this discrepancy appeared to grow over time.

Table 1 Change over time in weight variables using self-reported (get healthy) outcomes and objectively measured (GP office) outcomes

Change over time in alcohol, nutrition, and physical activity

Repeat AUDIT scores indicated a statistically significant reduction in alcohol consumption (0.67 reduction in AUDIT score (0.50, 0.90), see Table听2). There was no improvement from baseline to 12 months in the number of serves of vegetables and fruits, using either self-reported measures or from the food frequency questionnaire (Table听2).

For physical activity, the Get Healthy measures showed an improvement in the number of self-reported episodes per week of overall physical activity and moderate to vigorous physical activity by 12 months. However, this was not concordant with the physical activity data from ZIPPeD (obtained via validated instrument), which showed no change in physical activity over time (Table听2).

Table 2 Change over time in nutrition and physical activity using self-reported (get healthy) outcomes and objectively measured (GP office) outcomes

Concordance between self-report and objective measures

We directly compared concordance between self-reported and objectively measured weight outcomes. Spearman correlation coefficients were very high at 0.98 for weight, 0.89 for BMI, and 0.92 for waist circumference. Bland-Altman plots were used to explore systematic bias in self-reported data (Fig.听2). Although most participants reported accurately, with most points falling on the line of no difference (i.e. no discrepancy between the two methods), for each measure there was a small number who had highly discordant measures of weight or waist circumference, which skewed the average difference. This led to an average underestimate of 0.5听kg weight, 0.6 units BMI, and 1.58听cm in waist circumference in self-reported data. Extreme minimum values indicate some large underestimates, including a 5.5听kg discrepancy in weight, 21-unit discrepancy in BMI, and 27听cm discrepancy in waist circumference (Table听3). All values were within the plausible range and hence remained in the analyses.

Fig. 2

Bland-Altman plots for weight, BMI and waist circumference. A. Bland Altman plots for weight. B. Bland Altman plots for BMI. C. Bland Altman plots for waist circumference

Table 3 Bland-Altman plots for weight outcomes. The mean differences (GetHealthy 鈥� Zipped) are all negative which indicates that, on average, participants underestimated their values in the self-reported GetHealthy variable compared to the objectively-measured zipped variable

Change over time in metabolic outcomes (ZIPPeD only)

There was no difference in any of the metabolic (lipids and glycaemic markers) or physical measures (blood pressure) after the Get Healthy program, except for a 3.7 mmHg reduction in mean diastolic blood pressure (Supplementary Table 2).

Sensitivity analysis for completers only

Analyses of the subset of participants who completed the Type 2 Diabetes Prevention Program, including early completions, showed very similar results as the main analysis (see Table听4).

Table 4 Sensitivity analysis of change over time in weight variables using self-reported get healthy data for completers (early completion or graduated); ZIPPeD and non-ZIPPeD participants combined

Discussion

The findings of this study suggest the participants who engaged with and completed the Get Healthy Type 2 Diabetes Prevention Program showed small reductions in weight, waist circumference and alcohol intake and increases in moderate to physical activity at the 12 month follow up. However, the 65% drop-out rate and loss to follow-up for the cohort referred to Get Healthy was high, albeit consistent with other weight loss interventions [14]. Self-reported improvements in weight, BMI, and waist circumference in the Get Healthy Type 2 Diabetes Prevention Program were very similar to previous reports [4, 15] but larger than those obtained when these parameters were objectively measured. It is important to note that some of this study occurred during the worst of the COVID-19 lockdowns in NSW. Weight maintenance during this time may actually represent a win given data suggest that weight tended to increase more generally in the Australian population [16]. However, potential to either gain or lose weight during COVID isolation, depending on context, has also been described [17].

The potential benefits of Get Healthy should be interpreted with caution as we have shown differences stemming from some participants underestimating their weight and waist circumference, with the magnitude of underreporting growing over time. When measured objectively in ZIPPeD, weight loss did not reach statistical significance; a post-hoc sample size calculation indicates that we had 80% power to detect a Cohen鈥檚 d of 0.3, so a difference smaller than this may have been missed. Underreporting weight and overreporting height has been observed previously in other studies compared to objective measures [18, 19]. Yoong et al. have reported on discrepancies between self-report and objective measurements, ascribing it to social desirability bias [20]. Others have found that the underestimation occurs more with those at higher BMI [21]. People generally wish to appear to be adhering to the advice and it is possible that the rapport that developed with Get Healthy coaches during follow-up accentuated this tendency. The small proportion of extreme values was sufficient to skew the average and, at best, overstate the effect of Get Healthy, or at worst, create a positive effect where there was none. It is therefore difficult to determine whether the apparent effectiveness of Get Healthy Type 2 Diabetes Prevention Program was completely negated by this self-report bias, or whether it was reduced in magnitude but still important at a population level. Larger objective validation of the GHS over longer time frames is required. The pattern of underreporting is worth noting. For example, the 3 individuals with the greatest degree of underreporting weight are all on the lower range of weight (70鈥�90听kg), and have virtually no difference in reported vs. measured BMI, while the one with the greatest discrepancy in BMI (22 units) accurately reported weight. The 3 with the greatest discrepancy (underreporting) in waist circumference were all in the middle of the range, and showed virtually no discrepancy in weight. This pattern suggests that individuals are not consistently misrepresenting their values, e.g. social desirability bias, but rather that the discrepancies are more likely due to error. However, given the spread of discrepancies is not even, i.e. more likely to underreport than overreport compared to measured values, this suggests a mild degree of bias.

While it is possible that this discrepancy between self-report and objectively measured values occurred only in the subgroup enrolled in ZIPPeD due to selection bias, and would not be seen in those normally enrolling in Get Healthy, this seems unlikely for 2 reasons: (a) the characteristics of those who participated in ZIPPeD were largely similar to those who participated in Get Healthy only (with some minor differences, see Supplementary Table 1); and (b) those participating in a trial would be expected to be more motivated, health conscious and compliant.

It is interesting to note that for both the nutrition and physical activity outcomes, the ZIPPeD measures were consistently higher than the self-reported Get Healthy values. Given that neither set of values was objectively measured it is difficult to know which is the more valid estimate. We saw a similar pattern of self-reporting in physical activity, with people overstating the number of episodes per week and even more so for the number of episodes of moderate to vigorous activity.

Given the tendency to favourably report outcomes, it is also important to question whether a similar pattern may be seen with alcohol. Get Healthy has previously been modified to target alcohol consumption, [22] and AUDIT scores in this study did show a significant improvement over time. Because the AUDIT depends on self-report, it will be important to assess whether this effect was again driven by social desirability bias and bias in self-report, or whether objective measures confirm this apparent beneficial effect.

The high rate of drop-out (65%) is not encouraging but could have been related to COVID-19. Nevertheless, it would be important to explore other ways to make the program more engaging, perhaps through principles of gamification, having an accompanying app, or via partnering with others who are doing the program to add a social dimension [23].

The statistically significant findings in this study should be interpreted cautiously in the context of multiple comparisons and high rates of missing data. While this evaluation uses pre and post data from a relatively small cohort referred to the Get Healthy Type 2 Diabetes Prevention Program, it represents a larger sample than the previous Get Healthy validation study (for Standard Coaching) [5] and there have been no RCTs or evaluations of the Get Healthy Type 2 Diabetes Prevention Program, [24] although a combination of study designs may be necessary to determine real-world effectiveness of this scaled intervention [25]. Given the likelihood that missing data may be in those who are least likely to participate in the program, it is worth noting that the direction of bias is towards the null, i.e. the discrepancy between self-reported and objectively measured values may be even higher than stated here.

Given the significant investment in operating the Get Healthy service, it is important that further, high quality evidence be generated about its effectiveness. The service is well established, has good reach with culturally and linguistically diverse, and disadvantaged populations, [26, 27] has been adapted for Indigenous populations, [28, 29] and can be easily integrated into GP practice [30]. More valid and objective evidence is needed to determine the program鈥檚 effectiveness.