Translation, cross-cultural adaptation, reliability and validity of the Turkish version of the Olerud-Molander Ankle Score (OMAS)

Abstract

Objective

The aim of this study was to translate and culturally adapt the Olerud-Molander Ankle Score (OMAS) into Turkish and to assess its reliability and validity.

Methods

The Turkish version of the OMAS (OMAS-Tr) was developed after the translation and back-translation, which included the stages recommended by Beaton. The OMAS-Tr was administered to one hundred patients (49 females, 51 males; average age: 42.3 ± 17.7; range 16–81 years) with malleolar fractures. The OMAS-Tr was completed twice by each participant at 7- to 10-days intervals to assess test-retest reliability based on the interrater correlation coefficient, whereas Cronbachs alpha evaluated internal consistency. The external validity was evaluated with correlations between the Turkish version of the Foot and Ankle Ability Measure (FAAM) and the Turkish version of the SF-12 questionnaire. The distribution of floor and ceiling effects was also analyzed.

Results

The internal consistency (Cronbachs α = 0.84) and the test-retest reliability (ICC = 0.98) were excellent. The mean interval between the two tests was 8.6 ± 1.4 days. The mean and standard deviation of the first and second assessments of the OMAS-Tr were 74.1 ± 23.7 and 75.7 ± 23.9, respectively. There was a strong correlation between the OMAS-Tr and the FAAM subscales on activities of daily living and sports (r = 0.86, r = 0.83; p < 0.001, respectively). The OMAS-Tr displayed very good to good correlation with the SF-12 physical component score and the SF-12 mental component score (r = 0.72, r = 0.60, p < 0.001, respectively).

Conclusion

OMAS-Tr was a valid and reliable tool to assess ankle fracture-related problems. Nonetheless, further studies are needed to assess its responsiveness.

Level of evidence

Level III, diagnostic study.

Keywords

Ankle fracture ; Reliability ; Score ; Validity ; Olerud-Molander Ankle Score

Ankle fractures are relatively common orthopedic injuries with an incidence of approximately 187/100,000 per year among different ages and genders.^{1  ;  2} The incidence of ankle fractures is highest in young males and middle-to older-aged women.^{3  ;  4} Malleolar fractures are one of the most common injuries in the lower extremity that require operative treatment.⁵ However, malleolar fractures may still be associated with poor clinical outcomes regarding accompanying occult intra-articular soft tissue or chondral injuries.^{6  ;  7}

Many Patient-Reported Outcomes (PROs), such as the Karlsson score, Foot and Ankle Outcome Score (FAOS), American Orthopedic Foot and Ankle Score (AOFAS), and Foot and Ankle Ability Measure (FAAM) have been developed for the assessment of foot and ankle injuries. While the Karlsson score is generally used for ligament injuries, the FAOS and AOFAS are used for ankle injuries and were developed to evaluate different foot and ankle pathologies.^{7  ;  8} The FAAM is a self-report, region-specific instrument that has displayed the ability to distinguish individuals with different levels of functional performance.^{9  ;  10} Translation, cross-cultural adaptation, reliability and validity of the FAAM into Turkish has been reported.¹¹

Developed in 1984, the Olerud-Molander Ankle Score (OMAS) is a disease-specific questionnaire specifically designed to evaluate the functions of patients with ankle fractures.^{12 ; 13 ; 14  ;  15} The concurrent validity of the OMAS was compared with the Linear Analog Scale and the Ankle Function Score and was found good regarding the floor/ceiling effects.¹⁶ The English version of the OMAS has only been translated into Swedish.¹⁷ The Turkish version of the OMAS would potentially be a PRO measure that would be useful in the clinical management of Turkish-speaking patients with various ankle fractures.

The purpose of this study was to translate and culturally adapt the OMAS into Turkish and evaluate its reliability and validity compared to the FAAM and Short Form-12 (SF-12).

Patients and methods

Translation and cultural adaptation

The guideline suggested by Beaton was preferred for cultural adaptation and translation of the OMAS into Turkish (OMAS-TR).¹⁸ Two native Turkish translators performed the initial translation. The informed translator was a surgeon and the uninformed one was a teacher. The two forward translations were synthesized after being reviewed and discussed by a committee. Two native English translators, who speak Turkish fluently, translated the pre-final Turkish version back to English. After comparing these two translations with the original OMAS, a pre-final version of the OMAS-TR was approved for the purpose of pilot test application (Table 1 ). The pilot test was applied to 20 patients who were asked to assess the comprehensibility. Questions that caused comprehension problems were noted and patients were asked for suggestions.

Table 1. Olerud Molander Ayakbileği Skoru.

	SKOR
Ağrı
Yok	25
Engebeli zeminde yürürken	20
Düz zeminde yürürken	10
Ev içinde yürürken	5
Sürekli ve Aşırı	0
Sertlik
Yok	10
Var	0
Şişlik
Yok	10
Akşamları	5
Sürekli	0
Merdiven Çıkma
Sorunsuz	10
Zorlanarak	5
Mümkün Değil	0
Koşma
Mümkün	5
Mümkün Değil	0
Zıplama
Mümkün	5
Mümkün Değil	0
Çömelme
Sorunsuz	5
Mümkün Değil	0
Destekler
Yok	10
Sarmak, Bandajlamak	5
Baston veya Koltuk Değneği	0
Çalışma, günlük yaşam aktivitesi
Yaralanma öncesi ile aynı	20
Kısıtlı	15
İş değişikliği	10
İleri düzeyde iş güç kaybı	0
Toplam	100

Participants

Prior to the study, institutional approval was obtained from the Hacettepe University Ethics Committee (GO 15/495-21). Two hundred fifty-six patients with malleolar fractures, who had undergone surgical or conservative treatments between 2010 and 2014, were recruited from the archives of the Department of Orthopedics and Traumatology at Hacettepe University. The inclusion criteria were; being 16 years of age or older, having malleolar fractures, and having been accepted to participate in both the test and re-test assessments. Patients with tibial pilon fractures, talus fractures, coexisting fractures and serious comorbidities, cognitive impairment, and lack of understanding of the Turkish language were excluded. One hundred patients (49 females, 51 males; average age: 42.3 ± 17.7, range: 16–81 years) who met the inclusion criteria were enrolled in the study. All patients' radiographs were verified by the senior author (E.T.) to depict a malleolar fracture.

Patient-reported outcome measures

OMAS

The OMAS is a disease-specific PRO, which evaluates ankle fractures. The OMAS consists of nine questions with different scorings: pain (25 points), stiffness (10 points), swelling (10 points), stair climbing (10 points), running (5 points), jumping (5 points), squatting (5 points), supports (10 points) and work/activity level (20 points). The score is calculated as the sum of each rated item. Each question was based on an ordinal rating scale with an overall score ranging from 0 (totally impaired function) to 100 (excellent or completely unimpaired function). Values from 0 to 30 were considered poor, 31–60 fair, 61–90 good, and 91–100 excellent.¹²

FAAM

The FAAM is an evaluative self-reported instrument to comprehensively assess the physical function of individuals with musculoskeletal disorders of the feet and ankle.¹⁹ It consists of activities of daily living (ADL) and sports subscales, the first containing 21 and the latter eight items. The ADL and sports subscales have a total score of 84 and 32, respectively.²⁰

SF-12

The SF-12 was developed based on the 36-item Short-Form (SF-36) with the intent of reproducing the SF-36 in a brief and more useful form.²¹ The physical component score (PCS) and the mental component score (MCS) of the SF-12 were derived by the weighted sum of 12 items' scores using the USA standard SF-12 scoring algorithm.²²

Study procedures

Administration of outcome measures

The patients were asked to complete the OMAS-TR, the previously validated Turkish version of the FAAM, and the SF-12. The research assistant distributed the three questionnaires to each patient. Difficulties during the testing regarding comprehensibility and any inconsistencies regarding patients' problems were noted. The patients were requested to complete the second assessment of the OMAS-TR within 7–10 days after their first assessment in order to determine the test-retest reliability.

Statistical analysis

Data analysis was performed using the SPSS 15.0 for Windows^® (SPSS Inc., Chicago, IL, USA). The first and second assessments of the scores with the Kolmogorov–Smirnov test confirmed that the results were distributed normally.

Reliability

Cronbachs α was used to assess the homogeneity of the questions for internal consistency within the test. A Cronbach α value ranging from 0.70 to 0.95 was considered to be adequate.²³ The test-retest reliability was assessed by completing PROs on two occasions. We assumed that there would be no significant change in the health status of the patients within an interval of 7–10 days. The test-retest reliability was calculated by an intraclass correlation coefficient using a two-way, mixed-model analysis under consistency. Values of 0.4 or greater were considered satisfactory (r = 0.81–1.00 excellent, 0.61–0.80 very good, 0.41–0.60 good, 0.21–0.40 fair and 0.00–0.20 poor). Patients who reported “no change” in their condition between the first and second assessment were included in the analysis. As an index of measurement precision, the ICC was used to calculate the Standard Error Measurement (SEM). The SEM is calculated as the standard deviation of the scores divided by the square root of 1-ICC. The Minimal Detectable Change (MDC) refers to the minimal amount of change within measurement error. The SEM was used to determine MDC at 95% confidence limit (MDC 95%) and was calculated as SEM*1.96*√2.²⁴

Construct validity

Validity is the degree to which PROs measure the structure it is supposed to measure.²⁵ Evidence for the construct validity of the OMAS-TR was provided by determining its relationship to the FAAM and SF-12. Pearson correlation coefficients and their 95% confidence intervals were calculated to assess the construct and convergent/divergent validities. Correlation values ≥ 0.4 were considered satisfactory (r = 0.81–1.00 excellent, 0.61–0.80 very good, 0.41–0.60 good, 0.21–0.40 fair and 0.00–0.20 poor).²⁶

Floor and ceiling effects

Floor and ceiling effects were assessed on the first administration of the OMAS-TR to determine content validity. Floor/ceiling effects were considered present if more than 30% of the patients achieved the highest or lowest possible scores.^{27  ;  28}

Results

Translation and cross-cultural adaptation

After preparation of the Turkish translation and administration of the pilot test, some minor changes were made to Item 1 regarding pain: ‘walking on uneven/even surfaces’ (engebeli ve düz zeminde yürüme) , Item 8 dealing with supports: ‘tape, wrapping’ (sarmak, bandajlamak ), and Item 9 dealing with work/activity levels: ‘severely impaired’ (ileri düzeyde iş-güç kaybı ). The translation committee changed or adapted the aforementioned words without compromising their meaning.

Measurement properties and testing

Table 2 illustrates the main demographic characteristics of the patients. As we mentioned in our inclusion criteria, all participants completed the test-retest assessments. The mean follow-up period of the patients was 4.3 years. The OMAS-TR was completed in approximately three minutes. The mean values of the first and second assessments of the OMAS-TR and the first assessment of the Turkish version of the FAAM and SF-12 are presented in Table 3 .

Table 2. Patient demographics.

Female/male	49%/51%
Mean age ± SD (range)	42.3 ± 17.7 (16–81)
Mean height ± SD (cm)	166.1 ± 12.6
Mean weight ± SD (kg)	73.7 ± 18.5
Mean BMI ± SD	26.7±3.2
Education level
Primary school	16%
High school	33%
University	51%
Fracture type
Lateral malleolar	39%
Medial malleolar	21%
Posterior malleolar	4%
Bimalleolar	28%
Trimalleolar	8%
Involved ankle
Right ankle	48%
Left ankle	52%
Treatment
Closed reduction/spica cast	43%
Open reduction/internal fixation	57%
Occupation
White collar worker	31%
Blue collar worker	17%
Retired	14%
Student	16%
Unemployed	22%
Sportive activity
Professional	2%
Recreational	42%
None	56%
Comorbidities
Inflammatory disease	3%
Cardiovascular disease	7%
Diabetes mellitus	6%
Hypertension	11%
Other surgery	17%

BMI: body mass index, SD: standard deviation.

Table 3. Descriptive statistics for the patient-reported outcomes.

Assessment outcomes	Mean ± SD
OMAS-TR Assessment 1	74.1 ± 23.7
OMAS-TR Assessment 2	75.7 ± 23.9
FAAM-ADL	65.8 ± 19.9
FAAM-S	19.8 ± 11.3
SF-12 PCS	47.4 ± 9.0
SF-12 MCS	55.2 ± 7.6

ADL: activities of daily living, FAAM: Foot and Ankle Ability Measure, MCS: mental component score, OMAS-TR: Turkish version of the Olerud-Molander Ankle Score, PCS: physical component score, S: sports, SF-12: Short-Form 12.

Reliability

The internal consistency of the first assessment of the OMAS-TR was very good with a Cronbachs α value of 0.84. The mean interval between the two tests was 8.6 ± 1.4 days. The test-retest reliability of the OMAS-TR was found excellent, with an ICC value of 0.98. The SEM and MDC for the OMAS-TR were 3.3 and 9.1, respectively Table 4 .

Table 4. Correlations between the OMAS-TR and the Turkish version of the FAAM and SF-12 subscales.

OMAS-TR	FAAM-ADL	FAAM-S	SF-12 PCS	SF-12 MCS
r	0.86	0.83	0.72	0.60
p	0.001	0.001	0.001	0.001

ADL: activities of daily living, FAAM: Foot and Ankle Ability Measure, MCS: mental component score, OMAS-TR: Turkish version of the Olerud-Molander Ankle Score, PCS: physical component score, S: sports, SF-12: Short-Form 12.

Validity

The correlation coefficient between the OMAS-TR and the Turkish version of FAAM-ADL and FAAM-S subscales was good to excellent (r = 0.86, r = 0.83; p < 0.001, respectively). The OMAS-TR and the Turkish version of SF-12 PCS and SF-12 MCS showed very good to good correlation (r = 0.72, r = 0.60, p < 0.001, respectively) (Table 3 ).

Floor and ceiling effects

In the overall OMAS-TR, none of the patients achieved the minimum score of 0, thus there is no floor effect. 27% of the patients in the first assessment and 29% of the patients in the second assessment reached the maximum score of 100. However, these percentages were below the cut-off point of 30%, suggesting no ceiling effect.

Discussion

The aim of this study was to translate and culturally adapt the OMAS into Turkish and provide reliability and validity data for the translated version based on a sample of Turkish-speaking patients with ankle fractures. Based on our sample population, the OMAS-TR demonstrated acceptable levels of reliability and validity to be used as a PRO questionnaire for Turkish-speaking individuals with a variety of ankle fractures.

The OMAS has been preferred by many researchers in evaluating their treatment outcomes for several years.^{6 ; 12 ; 29 ; 30  ;  31} Questions are easy to understand and all items are related with normal daily activities. The score is also researcher-friendly because it is not divided into subscales, thus, additional calculations are not required. There are some advantages of using the OMAS compared to other PROs. Pain during different weight-bearing positions is the major disabling complaint of patients who have ankle fractures. Nilsson et al found that one year after the injury, more than half of the patients still experienced pain while walking.^{13  ;  32} Stiffness is one of the crucial problems after surgical or conservative treatment. While swelling and stiffness are directly investigated by the OMAS, other PROs either evaluate them indirectly or do not at all.

Internal consistency of the OMAS-TR was very good (Cronbachs α = 0.84) and rated higher than the values previously reported by the Swedish version (Cronbachs α = 0.76). The test-retest reliability of the OMAS-TR was excellent (ICC = 0.98). The original version of the OMAS, which was published in 1984, did not include any psychometric properties in the questionnaire.¹² Therefore, we can compare our results solely with the available Swedish version of OMAS. In this version, the test-retest reliability was calculated using both the Spearman rank correlation coefficient and ICC.¹⁷ The authors reported excellent values of rho = 0.95 and ICC = 0.94, which are similar to our results. The first assessment of the OMAS-TR was completed at the clinic and the second assessment via phone interview with a total of 100 patients. However, the Swedish version only included 42 patients for the retest assessment and the first assessment was handled via mail correspondence instead. We preferred to see the patients for the first time at the clinic, in order to be able to note the difficulties they experienced while answering the questions. This allowed us to adapt the cross-cultural process more effectively.

The time interval between the test-retest must be long enough to prevent recollection of the previous answers and short enough to prevent the occurrence of a real change in the construct to be measured. Between these two test periods, the patient should not have had any treatment for the related problem.^{20  ;  27} We repeated the test within 7–10 days. In order to ensure that an individuals condition had not changed, we asked the patients if their condition was the same as during the first assessment. In the Swedish version, the time interval between the two assessments was two weeks. The MDC was 9.1, meaning that a change of less than this value on repeated administrations of the OMAS-TR should be considered a reflection of measurement error rather than a true change in the patients condition. The MDC, named the smallest real difference (SRD), was 12 in the Swedish version, which is higher than our result. This may explain the higher ICC value in our sample population.

Evidence of the validity of the OMAS-TR was investigated by determining its relationship with the FAAM and SF-12. The correlation coefficient between the OMAS-TR, FAAM-ADL and FAAM-S subscales was excellent (r = 0.86 and r = 0.83, respectively). Only the Swedish version reported the validity of the OMAS using the disease-specific questionnaire FAOS. Therefore, we cannot compare our results with any available literature. We did not prefer using the FAOS for construct validity, as the FAOS is a region-specific instrument intended to evaluate the symptoms and functional limitations in individuals with generalized foot and ankle disorders. Items for the FAOS were adapted from the Knee Injury and Osteoarthritis Outcome Score (KOOS). We believe that the ADL subscale of the FAOS deals with problems in non-weight bearing conditions, for instance ‘putting on socks’, ‘taking off socks’, ‘lying in bed’, ‘pain in bed’ or ‘sitting’. These are not major or relevant complaints of patients with ankle fractures. On the other hand, Nilsson et al¹⁷ reported that the correlation between the Swedish version OMAS and FAOS-ADL and FAOS-S subscales were 0.80 and 0.85, respectively, which is very similar to our validation results with the OMAS-TR and FAAM subscales. In addition, to determine convergent and divergent validity, we determined the level of associations between the scores on the OMAS-TR and the two summary scores for the SF-12. The OMAS-TR was more strongly related to the concurrent measures of physical function than the concurrent measures of mental function with values of 0.72 and 0.60. No floor/ceiling effects were reported in this study. The absence of a ceiling or floor effect provides support for the content validity of the OMAS. However, the ceiling effect of the OMAS-TR in the first and second assessments was very close to an unacceptable level of 30%. The study included patients who had surgery or conservative treatment with a mean follow-up period of 4.3 years. This was a long recovery time for the patients, so it may be the reason why almost 30% of the patients had reached the highest score of the OMAS-TR, resulting in a ceiling effect.

There were some limitations to this study. First, we only included patients with malleolar fractures but the OMAS has been developed for a variety of ankle fractures, therefore, reliability and validity of the OMAS-TR should be conducted for patients with different types of ankle fractures in future studies. Second, the only transcultural-adapted version of the OMAS in the literature was in Swedish,¹⁷ therefore, we could not compare our results to those of the OMAS versions in other languages and highlight the similarities or differences. Third, we could not report the responsiveness data, which includes critical measures, in order to evaluate a change in the patients health status. Assessing the responsiveness of instruments determines whether the assumption of constant variance is appropriate. Future studies are necessary to assess the responsiveness in order to determine the minimum clinically important differences for the OMAS-TR with regards to ankle fractures that commonly affect the ankle.

In conclusion, the OMAS-TR is a reliable and valid tool with values similar to those reported for the original and other translated versions. It is short and easy to administer and interpret with a minimal amount of time required for clinicians, patients and researchers. Therefore, the OMAS-TR can be used as a PRO measure for Turkish-speaking individuals with various ankle fractures.

References

1. 1 P.J. Daly, R.H. Fitzgerald Jr., L.J. Melton, D.M. Ilstrup; Epidemiology of ankle fractures in Rochester, Minnesota; Acta Orthop Scand, 58 (1987), pp. 537–544
2. 2 F. Kralinger, M. Lutz, M. Wambacher, V. Smekal, K. Golser; Arthroscopically assisted reconstruction and percutaneous screw fixation of a pilon tibial fracture; Arthroscopy, 19 (5) (2003), p. E45
3. 3 S.L. Jensen, B.K. Andresen, S. Mencke, P.T. Nielsen; Epidemiology of ankle fractures. A prospective population based study of 212 cases in Aalborg, Denmark; Acta Orthop Scand, 69 (1998), pp. 48–50
4. 4 B.R. Singer, G.J. McLauchlan, C.M. Robinson, J. Christie; Epidemiology of fractures in 15,000 adults: the influence of age and gender; J Bone Jt Surg Br, 80 (1998), pp. 243–248
5. 5 M. Bhandari, S. Sprague, B. Hanson, et al.; Healt-related quality of life following operative treatment of unstable ankle fractures: a prospective observational study; J Orthop Trauma, 18 (6) (2004 Jul), pp. 338–345
6. 6 E. Turhan, M.N. Doral, M. Demirel, et al.; Arthroscopy-assisted reduction versus open reduction in the fixation of medial malleolar fractures; Eur J Orthop Surg Traumatol, 23 (8) (2013 Dec), pp. 953–959
7. 7 J. Karlsson, L. Peterson; Evaluation of ankle and joint function: the use of a scoring scale; Foot, 1 (1991), pp. 15–19 http://dx.doi.org/10.1016/0958-2592(91)90006-W
8. 8 E.M. Roos, S. Brandsson, J. Karlsson; Validation of the foot and ankle outcome score for ankle ligament reconstruction; Foot Ankle Int, 22 (2001), pp. 788–794
9. 9 C.R. Carcia, R.L. Martin, J.M. Drouin; Validity of the foot and ankle ability measure in athletes with chronic ankle instability; J Athl Train, 43 (2) (2008), p. 179e83
10. 10 R.L. Martin, D.M. Hutt, D.K. Wukich; Validity of the foot and ankle ability measure (FAAM) in Diabetes mellitus; Foot Ankle Int, 30 (4) (2009), p. 297e302
11. 11 Melih Malkoç, Ozgur Korkmaz, Derya Çelik; Sercan Yalçın Ayak ve ayak bileği kullanılabilirlik ölçeğinin Türkçe’ye çevirisi ve kültürel adaptasyonu AOTT Supplement; (2014) 1 page 76 OP22-5
12. 12 C. Olerud, H. Molander; A scoring scale for symptom evaluation after ankle fracture; Arch Orthop Trauma Surg, 103 (1984), pp. 190–194 http://dx.doi.org/10.1007/BF00435553
13. 13 G.M. Nilsson, K. Jonsson, C.S. Ekdahl; Effects of a training program after surgically treated ankle fracture: a prospective randomised controlled trial; BMC Musculoskelet Disord, 10 (2009), p. 118 http://dx.doi.org/10.1186/1471-2474-10-118
14. 14 G.M. Nilsson, K. Jonsson, C.S. Ekdahl; Outcome and quality of life after surgically treated ankle fractures in patients 65 years or older; BMC Musculoskelet Disord, 20 (8) (2007), p. 127
15. 15 R. Wang, C.K. Thur, E.M. Gutierrez-Farewik; One year follow-up after operative ankle fractures: a prospective gait analysis study with a multi-segment foot model; Gait Posture, 31 (2010), pp. 234–240 http://dx.doi.org/10.1016/j.gaitpost.2009.10.012
16. 16 P. Wees, E. Hendriks, H. Beers; Validity and responsiveness of the ankle function score after acute ankle injury; Scand J Med Sci Sports, 22 (2012), pp. 170–174 http://dx.doi.org/10.1111/j.1600-0838.2010.01243.x
17. 17 G.M. Nilsson, M. Eneroth; Ekdahl CS the Swedish version of OMAS is a reliable and valid outcome measure for patients with ankle fractures; BMC Musculoskelet Disord, 14 (2013 Mar 25), p. 109 http://dx.doi.org/10.1186/1471-2474-14-109
18. 18 D.E. Beaton, C. Bombardier, F. Guillemin, M.B. Ferraz; Guidelines for the process of cross-cultural adaptation of self-report measures; Spine (Phila Pa 1976), 25 (24) (2000 Dec 15), pp. 3186–3191
19. 19 R.L. Martin, J.J. Irrgang, R.G. Burdett, S.F. Conti, J.M. Van Swearingen; Evidence of validity for the foot and ankle ability measure (FAAM); Foot Ankle Int, 26 (11) (2005), p. 968e83
20. 20 M. Mazaheri, M. Salavati, H. Negahban, et al.; Reliability and validity of the Persian version of Foot and Ankle Ability Measure (FAAM) to measure functional limitations in patients with foot and ankle disorders; Osteoarthr Cartil, 18 (6) (2010 Jun), pp. 755–759 http://dx.doi.org/10.1016/j.joca.2010.03.006
21. 21 J. Ware Jr., M. Kosinski, S.D. Keller; A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity; Med Care, 34 (1996), pp. 220–233
22. 22 J. Ware Jr.; Sherbourne CD the MOS 36-item short-form health survey (SF-36). Conceptual framework and item selection; Med Care, 30 (1992), pp. 473–483
23. 23 C.B. Terwee, S.D. Bot, M.R. de Boer, et al.; Quality criteria were proposed for measurement properties of health status questionnaires; J Clin Epidemiol, 60 (2007), pp. 34–42
24. 24 H.C. De Vet, C.B. Terwee; Bouter LM Current challenges in clinimetrics; J Clin Epidemiol, 56 (2003), pp. 1137–1141
25. 25 D.A. van Kampen, L.W. van Beers, V.A. Scholtes, C.B. Terwee, W.J. Willems; Validation of the Dutch version of the simple shoulder test; J Shoulder Elb Surg, 21 (6) (2012 Jun), pp. 808–814 http://dx.doi.org/10.1016/j.jse.2011.09.026
26. 26 J.J. Irrgang, R.G. Marx; Clinical outcomes in sport and exercise physical therapies; G.S. Kolt, L. Synder-Mackler (Eds.), Physical Therapies in Sports and Exercise (2nd ed.), Elsevier, Edinburgh (2007), pp. 206–219
27. 27 C.A. McHorney, A.R. Tarlov; Individual-patient monitoring in clinical practice: are available health status surveys adequate?; Qual Life Res, 4 (4) (1995 Aug), pp. 293–307
28. 28 J.C. Nunnally, I.R. Bernstein; Psychometric Theory; (3rd ed.)McGraw-Hill, New York (1994)
29. 29 N. Lash, G. Horne, J. Fielden; Ankle fractures: functional and lifestyle outcomes at 2 years; ANZ J Surg, 72 (2002), pp. 724–730 http://dx.doi.org/10.1046/j.1445-2197.2002.02530.x
30. 30 G. Day, C. Swanson, B. Hulcombe; Operative treatment of ankle fractures: a minimum ten-year follow-up; Foot Ankle, 2 (2001), pp. 102–106
31. 31 K.A. Egol, R. Dolan, K.J. Koval; Functional outcome of surgery for fractures of the ankle. A prospective, randomized comparison of management in a cast or a functional brace; J Bone Jt Surg Br, 82 (2000), pp. 46–49
32. 32 G. Nilsson, P. Nyberg, C. Ekdahl; Performance after surgical treatment of patients with ankle fractures 14-month follow up; Phys Res Int, 8 (2003), pp. 69–82