Asia Pacific J Clin Nutr (1994) 3, 193-200

 

Evaluation of a diabetes knowledge and behaviour (DKB) questionnaire

D. Simmons FRACP, C. Mandell BHB, C. Fleming RGN, B. Gatland and L. Leakehe

South Auckland Diabetes Project, Middlemore Hospital, Otahuhu, Auckland, New Zealand.

The primary prevention of type 2 (non-insulin-dependent) diabetes is now considered possible through adopting lifestyle changes. Population strategies for preventing diabetes are now being developed. The South Auckland Diabetes Project has developed a questionnaire to assess the impact of a diabetes awareness, exercise/healthy eating programme in the local communities. The questionnaire was evaluated among local adult Europeans (n=127), Maori (n=103) and Pacific Islands people (n =167). The questionnaire is interviewer-directed and takes approximately 30 min to administer. Diabetes knowledge was assessed using four open questions and 31 closed true/false questions which had good reliability (Cronbach's a range: 0.59 0.90), reproducibility (Pearson's r range: 0.39-0.74) and external validity (r range: 0.28 0.56) among all ethnic groups. Median scores increased by 7-13% on re-testing. The open and closed question scores were 7-13% and l>26% higher respectively among those with diabetes or a family history of diabetes (n = 78).

Important dietary habits were assessed using four tools: (1) a seven-item food preparation/fat content 'fat index'; (2) a four-item high-fat/high-refined-carbohydrate score had good reliability (Cronbach's α 0.51-0.74), reproducibility (r = 0.37-0.70) and external validity when compared with a dietetic assessment (total fat r = 0.44-0.90); (3) a 12-item food frequency questionnaire based on standard portion sizes also shared good reproducibility (Pearson's r = 0.45-0.52) and correlated well with the dietetic assessment of total calories (r = 0.48 0.64) and of calories due to fat (r = 0.41-0.65), and (4) a simple question related to the frequency of fruit consumption correlated negatively with the fat index in Europeans (r = 0.25, P<0.05) and Maori (r = -0.33, P<0.01). While the questionnaire does not give a quantitative assessment of nutritional habits, it does offer a speedy tool for evaluating population-based lifestyle and diabetes awareness interventions directed at the prevention and control of type 2 (non-insulin-dependent) diabetes.

Introduction

The population of South Auckland (303 513 in 1991) includes a high proportion of Polynesians of both Maori (50 589) and Pacific Islands (47 436) descent¹. These populations have an age-adjusted prevalence of diabetes that is up to fourfold higher than that of New Zealand Europeans^2,3. In view of this, a diabetes awareness and education programme is being developed in an attempt to control the predicted increase in numbers of people with diabetes and its tissue damage⁴. To evaluate the programme and its pilot s 1000 tudies, a rapid and simple tool was needed to assess the impact of any interventions on both diabetes knowledge and lifestyle among the general population and target groups. Of particular importance was the need to ensure that the final validated questionnaire was understandable, easy to use and culturally appropriate.

We now describe the design, reliability and validity of the diabetes knowledge, nutrition, television watching habits and alcohol sections for use in a population with a high prevalence of diabetes.

Method

Development of the questionnaire

The questionnaire included eight components: demographic and socioeconomic details, diabetes knowledge, key dietary habits, exercise, smoking and alcohol habits, perceived weight and ideal weight, and anthropometric measures (weight, height, waist, hip). After the evaluation of the questionnaire, one further measure relating to 'happiness' was added. This had been validated elsewhere⁵. The exercise questions were replaced by a separate set that had previously been validated in another New Zealand population⁶. These are not reported here.

The questionnaire was interviewer-directed. The assessment of diabetes knowledge was piloted with five open questions (including those previously described⁷), a four-point and a five-point Likkert scale and 47 closed questions derived from other published diabetes knowledge questionnaires^8-12. Questions were grouped under 10 stems with 3-6 true/false responses. The suitability of these sets of questions was discussed with members of the Maori and Pacific Islands communities prior to testing.

The final questionnaire included four open questions relating to the nature, symptoms, complications and treatment of diabetes⁷, together with a four-point Likkert scale question relating to the degree of damage (and death) caused by controlled diabetes. The coding for the open questionnaire has been described previously⁷. A further 31 closed questions from seven stems covered the nature, symptoms, complications, risk factors and prevention of diabetes, and included identification of high fat and high sugar foods (see Appendix 1). In the development stage, an obvious tendency for guessing the closed questions was observed, in spite of reassurance that the response 'don't know' could be used. In view of this, for the definitive application, incorrect answers were assigned a negative score of -1. Correct answers were scored +1, and 'don't know' was scored 0. The total score was converted into a percentage with all negative totals scored as 0%.

The dietary questions were intended to identify changes in the type, cooking, general preparation, and frequency of a number of key foods. The initial section described qualitative habits associated with a high-fat diet, while the second section was a 1 2-item food frequency questionnaire designed to reflect not only the total caloric intake but also the degree to which a low-fat, low-refined-carbohydrate diet was consumed vs a high-fat, high-refined-carbohydrate diet. Important high-fat foods and other key foods were identified through discussion with four dietitians and two nutritionists with extensive experience in the clinical assessment and management of the dietary practices of the local Polynesian communities. The items were further discussed with Maori and Pacific Islands community workers. Certain foods were important only for one or two ethnic groups.

Cheese, while a major source of dietary fat among Europeans, was rarely consumed by Pacific Islands people. The complex carbohydrates consumed also differ greatly between ethnic groups (eg kumara among Maori, green bananas and taro among Pacific Islands people). Key indicators of fat consumption, likely to be of use in the evaluation of interventions were assessed with seven questions. A simple ranking system was adopted for each item ranging from 0 (recommended ie lowest fat content) to 2 (ie high fat) and a 'fat index' calculated as the sum of the ranks expressed as a proportion of twice the total number of items included. (The denominator was doubled because the top rank in the numerator items was 2). As a number of subjects did not eat some of the foods, their 'fat index' was calculated by excluding the item from both denominator and numerator. The 'fat index' was expressed as a percentage.

From the 12 items chosen for the food frequency questionnaire, three measures were derived: (1) pieces of fruit eaten per week; (2) a high-fat/high-refined-carbohydrate score (HFHRC score) taken as the number of times per week any one of the following four foods were consumed: chocolate bars, servings of ice cream, biscuits, small cake or piece of cake, and (3) relative total calories consumed per week, taken from the estimated calorific value of these eight items and from food frequency questions for eggs, chops, potatoes and slices of bread. Questions relating to alcohol consumption were developed from previous questionnaires¹³. The question relating to the number of hours of television watched was developed by the authors.

Item selection

The resulting questionnaire was piloted among 55 European, Maori and Pacific Islands volunteers from the local community, including students (n=14), diabetic patients (n=12) and clinic staff (n = 8). Following the pilot, the questionnaire was found to take longer than the 20-30 min intended. Questions felt to be of least importance were therefore deleted. Problem areas within the questionnaire were identified and final modifications made. The draft and final versions were both assembled into a 'user friendly' booklet form, which included a comprehensive patient explanation, consent form and assessment of the need for an interpreter (never needed).

Validation studies

Volunteers for the reproducibility, reliability and internal validation studies were recruited either from the domestic staff (ie cleaners, orderlies) of a local hospital or through acquaintances of 28 previously unemployed local residents on work-based training courses at the local Polytechnic. Questions were asked by a trained interviewer using the first language of the subject. Subjects were asked not to read anything about diabetes until after the second visit which was at least 3 months after the first visit. Volunteers for the external validation of the dietary questions were recruited from the workforces of two local manufacturing companies. Volunteers were randomly assigned to receive either a dietary assessment or the DKB questionnaire first. The dietetic assessments were made by a registered dietitian using the method of Burke and Stuart¹⁴.

Statistics

Analyses were carried out using SPSS-PC (SPSS Inc, Chicago, II, USA). Internal consistency was assessed using Cronbach's alpha and stability by Pearson correlations. Comparison of categorical variables was by X², and normally distributed continuous variables by analysis of variance. Non-normally distributed variables (eg knowledge scores) were compared by Mann-Whitney test (unpaired) or Wilcoxon's test (paired). For allocation of calorific v 1000 alue to the items in the short food frequency questions, standard portions were assumed according to New Zealand Dietetic Association guidelines¹⁵. Internal validity for the subjects' responses was assessed by comparing scores from those with and without diabetes or a family history of diabetes. The hypothesis tested was that the former would know more about diabetes.

Results

Subjects

Characteristics of the 350 subjects involved in the reliability studies are shown in Table 1. Overall, 69% agreed to complete a repeat questionnaire. The proportion agreeing to repeat the questionnaire was lowest among Maori and Pacific Islands subjects. Within each ethnic group, subjects repeating and not repeating the questionnaire were similar with regards to sex, age, educational level achieved, family history of diabetes and language at home. Only 0.9% of subjects ate no meat, 2.7% ate no bread and 5.2% no potatoes.

Reliability and validity of knowledge questions

Table 2 shows the characteristics of the closed and open knowledge questions. Good reliability and stability are demonstrated. Reproducibility of both the open and closed questions was acceptable, although the repeat score was always significantly higher than the initial score. This was particularly the case for the open questions among Pacific Islands subjects. While Maori and Europeans had similar closed scores, Maori had lower open scores. The scores for the seven stems in the closed questions correlated significantly (P<0.001) with the total scores in each ethnic group (Pearson's r: European 0.46-0.83; Maori 0.53-0.84; Pacific Is 0.57-0.80). Similarly, the score for each of the four open questions correlated significantly (P<0.001) with the total open score (Pearson's r: European 0.68-0.83; Maori 0.54-0.81; Pacific Is 0.67-0.76).

 

Table 1. Characteristics of subjects in reliability/reproducibility and dietary external validation studies.

	Reliability/reproducibility studies				Dietary studies
	European	Maori	Pacific Is	Comparison across ethnic groups: P<
Number seen: seen twice/seen once (%)a	91/112 (81%)	39/77 (51%)	112/161 (70%)	0.001	47b
Age	41±17	35±14	36±13	0.01	36±10
% male	40%	34%	22%	0.01	28%
Only English Spoken at home	100%	79%	24%	0.001	1000 81%C
Education
Secondary or less	50%	64%	76%	0.001	72%
Form 6 and above	50%	36%	24%		28%
% Family history of diabetes	25%	30%	24%	ns	40%
% diabetes	30%	27%	17%	0.05	42%

^a Relates to number of subjects seen on two occasions and those only seen on one occasion. ^b 15 Europeans, 26 Maori, 6 Pacific Islands. ^c Maori and Pacific Islands subjects only.

 

Table 2. Test characteristics of diabetes knowledge questions.

	European (n=112)		Maori (n=77)	Pacific Is (n=161)	All (n=350)
Reliability (Cronbach 's α)
Open (n=4 questions)		0.74	0.59	0.66	0.73
Closed (n= 31 questions)		0.87	0.90	0.90	0.89
Both (n= 35 questions)		0.89	0.88	0.90	0.89
External validity (Pearson's r)
Open vs closed score		0.56	0.28	0.42	0.42
(Totals) sig:		P<0.001	P<0.05	P<0.001	P<0.001
Reproducibility (Pearson’s r)		(n=91)	(n=39)	(n=112)	(n=242)
Open scores		0.74	0.39	0.56	0.68
sig:		P<0.001	P<0.01	P<0.001	P<0.001
Closed scores		0.69	0.38	0.39	0.50
sig:		P<0.001	P<0.05	P<0.001	P<0.001
Reproducibility (Median (range))
Open scores: initial		33%(0-100%)	20%(0-73%)	7%(0-87%)	20%(0-100%)
repeat		40%(0-87%)	27%(0-80%)	20%(0-93%)	27%(0-93%)
Closed scores: initial		55%(0-90%)	58%(0-94%)	48%(0-97%)	52%(0-97%)
repeat		65%(0-97%)	68%(0-100%)	55%(0-97%)	61%(0-100%)
Scores by diabetes (self or family) (1-tailed)
Open scores: no diabetes		27%(0-80%)	13%(0-67%)	0%(0 80%)
diabetes		37%(0-100%)	20%(0-73%)	13%(0-87%)
sig:		P<0.05	P =0.07	P<0.001
Closed scores: no diabetes		45%(0-90%)	39%(0-90%)	48%(0-94%)
diabetes		58%(0-90%)	65%(0-94%)	58%(0 97%)
sig:		P<0.02	P<0.02	P<0.03

Table 2 shows that the open and closed scores were generally able to detect this. Neither open nor closed knowledge measure was correlated with age within any ethnic group. Among Europeans, those with education above form 6 had a higher 'knowledge index' than those with less education (29% vs 43%, P<0.01).

Using the Likkert scale question, diabetes was seen as 'very damaging and can kill you' (the highest score) by the great majority of people (European, Maori, Pacific Is respectively: initial 79%, 83%, 68%; repeat - 90%, 90%, 70%). As very few subjects had lower scores, Pearson's correlations could not be calculated. However, there was no significant difference in score between initial and repeat testing. Among Pacific Islands people, those with diabetes or a family history of diabetes had a significantly higher score than the others (P<0.02).

Reliability and validation of a 'fat index '

Table 3 shows that within each of the seven items, the 'fat index' was ranked highest in the 'high-fat' group and lowest in the 'low-fat' group. This correlation held true both overall and within each ethnic group. Reliability and reproducibility of the questions were acceptable both overall and within ethnic group as shown in Table 5. Over 90% of subjects from all ethnic groups ate a high-fat spread. The item relating to this was least reliable with an increased Cronbach's a on its removal (increasing overall to 0.62, Europeans to 0.54, Maori to 0.63, Pacific Islands to 0.63). The external validation of the 'fat index' (Table 4) demonstrated a significant correlation with the total fat consumed overall and within ethnic group.

 

Table 3. Qualitative indicators of fat consumption: the 'fat index'.

	High-fat (2)	Medium-fat (1)	Low-fat (0)
How do you usually use/cook:
Milka	Dark blue/silver	Light blue	Green/trim
Eggs	Fried/scrambled		Poach/microwave/boil
Chops	Fry/pan roast	1000	Boil/ grill/ microwave
Chicken	Fry/pan roast		Roast on rack
Do you:
Cut the fat off meat	Never/sometimes	Usually	Always
Cut the skin off chicken	Never/sometimes	Usually	Always
When you eat bread, what spread do you usually use	Dripping/ butter/ margarine	Low fat margarine	None

The numbers in brackets indicate the 1000 score for each item. If two or more options are chosen, that containing the highest fat is coded. The 'fat index'=Σ (7 scores)/(No. of items x 2). If the item is not eaten (not the spread question), that item is omitted from the denominator. a Colour signifies fat content of standard milk varieties marketed in New Zealand.

The 12-item food frequency questionnaire

The reliability of the four-item HFHRC score was acceptable as shown in Table 4. The reproducibility studies and external validation for the HFHRC score and food frequency calories were acceptable (Table 4). The fruit frequency question was reproducible (Table 4). Among Europeans in the external validation study, the fruit frequency question correlated negatively with the total fat consumed (r = 0.66, P<0.01) and total protein consumed (r = -0.62, P<0.05). A variable degree of internal validation for the 'fat index' and three food frequency scores was provided through correlations with each other (Table 4). The fat index and food frequency calories were negatively correlated with age among Maori (r = -0.23, -0.27 respectively, P<0.05) and Pacific Islands subjects (r = -0.24, P<0.01, -0.29, P<0.001 respectively) but not Europeans. The fat index and HFHRC were not correlated. Dietary measures did not differ significantly with educational level achieved or with diabetes status.

 

Table 4. Test characteristics of dietary questions

1000

	European	Maori	Pacific Is	All
Reliability (Cronbach's α): n	112	77	161	350
Fat index (7 items)	0.51	0.53	0.51	0.53
High-fat/high-refined carbohydrate (HFHRC)	0.53	0.67	0.74	0.66
Internal validity (Pearson's r)
Food frequency vs HFHRC	0.55***	0.38**	0.44***	0.41***
Food frequency vs fat index	0.23*	0.26*	0.05	0.18***
Fat index vs fruit frequency	-0.25*	-0.33**	-0.07	-0.18***
Reproducibility (Pearson's r)	(n=91)	(n=39)	(n= 112)	(n=242)
Fat index	0.61***	0.70***	0.44***	0.56***
Food frequency	0.49***	0.45**	0.52***	0.51***
HFHRC	0.64***	0.37*	0.56***	0.54***
Fruit frequency	0.54***	0.41*	0.21*	0.35***
Reproducibility (median (range))
Fat index (%) 1	58(8-100)	75(8-100)	67(8-100)	60(8-100)
Fat index (%) 2	58(0-100)	75(8-100)	58(0-100)	58(0-100)
Food frequency calories 1	4776	5468	5764	5308
(calories/week)	1682-18886	1450-19686	1146-17674	1146-19686
Food frequency calories 2	4141*	5633	6092	5092
(calories/week)	1656-14093	996-13540	236-24954	236-24954
HFHRC (d/week) 1	1.8(0.0-5.0)	1.5(0.0-4.5)	1.3(0.0 6.0)	1.5(0.0 6.0)<