Research Article - Biomedical Research (2019) Volume 30, Issue 1
Model-based computer-aided method for diagnosis of cardiovascular disease using IR thermogram.
- *Corresponding Author:
- Jayanthi T
Department of Biomedical Engineering
SRM Institute of Science and Technology
Tamil Nadu, India
Accepted date: January 11, 2019
The human thermal system maintains a core body temperature of 36.7°C. Any deviation from this causes considerable discomfort. The arteries and veins play a significant role in transferring the temperature to maintain the thermal comfort of the human system. Thus the aim of this work was to bring out the potential of thermal imaging in diagnosing the cardiovascular disease in comparison with the standard techniques. The average skin surface temperature (SST (°C)) was compared with the standard bio-signaling method electrocardiograph (ECG) and the ECG-derived heart rate variability (HRV) time domain parameters. Standard biochemical assay and one minute ECG signal were obtained from known CVD men (n=10) and age-matched men (n=10) In each subject, infra-red (IR) thermogram of selected skin area of the body was obtained. From the ECG signal, RR interval was computed using Pan-Tompkins algorithm. HRV time domain variables were also calculated. When SST(°C) variables using static SST (°C) alone were utilized in the CAD model for evaluation of CVD, it was found that, Naive-Bayes classifier with variables ranking and selection by Wilcoxon method gave an accuracy of 90%, and its sensitivity and specificity were found to be 80% and 90% respectively. It was comparable to the obtained CAD model using ECG variables as a standard, whose accuracy was same, and its sensitivity and specificity were found to be 90% and 80% respectively. Also, a better accuracy of 95% (sensitivity-90%, specificity-75%) could be achieved in classifying the normal and CVD subjects when the static average SST (°C) was used along with the HRV time domain variables. Thus the inclusion of IR thermogram in the CAD model along with standard variables improves the accuracy of cardiovascular disease evaluation.
Static infra-red, HRV time domain variable, Classifiers, Feature selection, Cardiovascular disease
Cardiovascular diseases (CVDs) are a group of disorders of the heart and blood vessels, and they include coronary heart disease, cerebrovascular disease, peripheral arterial disease, rheumatic heart disease, congenital heart disease, deep vein thrombosis and pulmonary embolism . CVD is the leading cause of death and disability worldwide among the noncommunicable disease (NCD) (17.5 million deaths, or 46.2% of NCD deaths) World Health Organization (WHO) estimates that by 2020, CVD would prevail as the leading cause of death and disability over infectious disease globally. Hence the aim of this study was to investigate the efficacy of variables obtained from SST (°C) of selected skin area of the body regions (using IR thermogram), ECG, and HRV time domain in the evaluation of CVD in men. A computer-aided diagnostic (CAD) model was also proposed for evaluation of CVD with high accuracy using different set(s) of these measured variables by testing and comparing with various feature selection methods (Bhattacharya, t-test, ROC, entropy and Wilcoxon) and classifiers (SVM, Naive Bayes, kNN, and decision tree)
Infrared Thermal Imaging
The first use of IR thermal imaging for disease diagnosis started in 1960. The radiation emitted by the object depends on its temperature and emissivity. The human skin surface is almost equivalent to a black body with an emissivity of 0.98 regardless of race and origin . The IR spectrum can be used to quantify the skin surface temperature (SST °C) with good sensitivity . It is gaining more importance for blood vessel mapping. The vasculature underneath the skin is responsible for the SST of the body. Hence, the infrared radiation can be used for mapping the vasculature underneath . The main attraction of IR thermal imaging comes from the facts that it is entirely non-contact, therefore giving no interaction with tissues, easy to use, and easy to store the data. It is one of the simplest forms of imaging which can provide physiological information . Hence, it can be used to diagnose disorders connected with musculoskeletal conditions, peripheral/cerebral vascular conditions, diagnosis of cancer/tumor, deep vein thrombosis, and facial vasculature recognition for the biometric purpose .
Heart Rate Variability
‘HRV’ is a measure of cyclic variations of the heartbeat to beat intervals that reflect heart autonomic functions. The imbalance in the sympathetic- and vagal- activity of the heart leads to its abnormalities. Also, it is a non-invasive technique, which can monitor the status of the autonomic nervous system (ANS) Analysis of RR interval (ms), extracted from ECG, can give more potential pathophysiological information about the heart. Low HRV values are associated with first CVD events. Reduced HRV variables are found in acute myocardial infarction, arrhythmias, cardiac death, and congestive heart failure. The calculated HRV time domain variables were correlated statistically significantly with all the CVD risk factors (both modifiable and non-modifiable) . Christensen et al. showed that the measured HRV variables showed a statistically significant difference between normal and the hypercholesterolemic subjects .
Persistent hypertension is one of the risk factors for CVD. It is the early indication of dysregulation in ANS . In a study carried out by Natarajan et al., the calculated mean values of time domain HRV variables, namely SDNN, RMSSD, and pNN50 were lesser (p<0.01) in hypertensive subjects when compared to normal subjects . Urooj et al. found that the calculated time domain HRV variables were lesser (p<0.01) in both hypertensive and smoker subjects when compared to their normal counterparts . Yoo et al. studied the relationship between the HRV variables and risk score evaluated by FRS method to predict the ten-year risk of CHD in normal adults . In a report, Alonso and Gonzalez captured the impact of human thermoregulation on the cardiovascular system. Heat stress increases the blood flow to the extremities. By calculating the stroke volume, heart rate, cardiac output and the mean arterial pressure, the authors implies that the blood flow rate and its SST (°C) were related . Faezipour et al. extracted RR interval from the ECG signal using Pan- Tompkins algorithm, and it was used for automated heart beat detection .
Computer-Aided Diagnostic Model
Successful implementation of medical diagnosis requires data mining techniques like feature ranking and classifiers. The distance based ranking techniques with classifiers have been used in some studies for disease diagnosis. Dimensionality reduction was performed by different feature ranking methods like Bhattacharya, t-test, entropy, ROC, Wilcoxon, linear SVM, and other entropy based methods. This reduced feature set was used for an automatic classification of the disease. The classifiers can be implemented using neural networks (NN) Different supervised learning classifiers like SVM, decision tree, and naïve Bayes, multi-layer perceptron and unsupervised learning network like kNN were used for this purpose . The efficiency and accuracy of this automated technique in the evaluation of the disease depends on the variables used for classification.
Materials and Methods
A free screening camp for CVD patient was organized for the South Indian population at a private hospital, during March 2010. A total number of registered participants of both sexes were 150, whose age ranged from 25 to 70 y. An informed consent form was obtained from all the subjects. A detailed questionnaire prepared for the study was used to evaluate the health status of each subject, participated in the camp. Of the total number, 70 subjects were known cases of fever, neurological disorders, diabetes mellitus, thyroid abnormalities and nephropathy and thus they were excluded from this study. The remaining 80 subjects were known cases of CVD (n=10), hypertension (n=5) and normal subjects (n=65) According to the guideline for assessment of the risk of CVD [14,15], the men gender is one of the risk factors. Hence, a total number of 10 known CVD men were included in this study. The details of their CVD complications are summarized as in Table 1. Also, a total number of 10 age-matched normal men were included in the study for comparison. Remaining 60 subjects were excluded from this study analysis. This clinical trial was registered in the Clinical Trial Registry of India (Ref. No: REF/ 2012/ 06/003696) The study groups are given as follows:
Group I: Known CVD men (n=10, mean ± SD age=47.6 ± 13 y); average years since CVD complications=2.1 ± 0.3 y;
Group II: Normal, age-matched men (n=10, mean ± SD age=45.9 ± 14.9 y);
|S. no||Age (y)||Complication||Duration of the complication (y)||Other notable risk factor|
|1||43||Heart attack||3||Low healing of wound in legs|
|2||49||Heart attack, stroke||2.5||Chest pain|
|3||48||Vessel block||NA||Chest pain|
|4||55||Heart attack||2||Chest pain, Hypertension|
|6||45||Heart attack||2 .2||Family positive history of CVD|
|8||34||Myocardial infarction||2||Chest pain|
|10||34||Myocardial infarction||2.1||Family positive history of CVD|
Table 1: Duration of CVD complications in ‘known CVD’ men (Group I)
The following anthropometric variables were measured in each subject: i) Body weight (kg); ii) Body height (cm); iii) Waist circumference (cm); and iv) Hip circumference (cm) Blood pressure (mm Hg), systolic and diastolic (SBP and DBP) was recorded using the standard sphygmomanometer method. Blood samples were taken in all subjects under non-fasting conditions. The following variables were measured according to the standard biochemical analysis: i)TC (mgdl-1); ii) HDL (mgdl-1); iii) Random glucose (mg/dl); and iv) HbA1c (%)
ECG was captured from each subject in the supine position after a resting period of 2 minutes using standard ECG machine with 12 leads (Vesta 12 lead Real-time RMS ECG 121i, Recorders & Medicare Systems, RMS, India) After the placement of electrodes, the recording of ECG was done for 1 min at 25 mm/s with gain settings of 10 mm/mV.
Calculation of ECG variables
From the obtained one minute ECG signal, the following variables were measured using inbuilt software (RMS ECG version 121i) of the ECG machine: i) PR interval (ms); ii) P-Jayanthi wave duration (ms); iii) P wave amplitude (μV); iv) QRS width (ms); and v) QTc (ms) Further, RR interval (ms) was calculated from the ECG using Pan-Tompkins algorithm implemented in MATLAB Version R2014a. Calculated RR interval (ms) was employed in both HRV- and DWT- analysis.
The calculated RR interval (ms) was analyzed using the standard freeware (Kubios HRV 2.2, Biosignal Analysis and Medical Imaging Group, University of Eastern Finland, Finland) and the following time domain variables were measured: i) RR Mean (ms); ii) RR SD; iii) Heart rate mean (bpm); iv) Heart rate SD (bpm); v) RMSSD (ms); vi) NN50; vii) pNN50 (%); viii) RR triangular index; and ix) TINN.
Average SST (°C) of selected skin areas of the body
The clinical trial was conducted in special metabolic clinical trial ward constructed under Good Clinical Practice regulation, where the room temperature was maintained at 20°C. All the subjects were made to sit for 15 min in the temperature controlled waiting room, which reduces the effect of an environment temperature of the skin. The subjects were then asked to stand in the image capturing room uncovered below the knee, no metal ornaments of any kind in hand or face. The relative humidity was 50%, and emissivity was set at 0.98. The distance between camera and subject was kept 1m constant. The trial was conducted for a week in the forenoon session only, to avoid the effect of cyclic variations in the atmospheric temperature in the study.
In each subject, the static IR thermogram of selected skin area of the body regions was taken under standard conditions using an IR thermal camera (Therma Cam T400, FLIR Systems, Inc., USA) It includes the following: i) Forehead (AP view); ii) Neck (lateral view of bilateral sides); iii) Forearm (AP view of bilateral sides); and iv) Foot (both AP- and PA-views of bilateral sides) The thermograms were stored in *.jpg format on the computer and then analysed off-line. Using the software (FLIR Quick Report, version-1.2 software), supplied by the thermal camera manufacturer, an average SST (°C) of each selected skin areas of the body was measured from the corresponding acquired static IR thermogram. Average SST (°C) displayed in each ROI was considered for analysis.
Different CAD models for CVD
In each subject, the total numbers of variables measured were 64, and its breakup is summarized as follows: i) 11 (both demographic and biochemical); ii) 5 (ECG); iii) 9 (HRV time domain); and iv) 9 (average SST (°C) using IR thermogram) All these variables were ranked by different methods: i) Bhattacharya; ii) t-test; iii) ROC; iv) Entropy; and iv) Wilcoxon methods. After ranking, each feature set was run through each of the following classifiers separately: i) SVM; ii) Naive Bayes; iii) kNN; and iv) Decision tree. An exercise of framing CAD model in the evaluation of CVD with high accuracy was carried out.
After ranking and classification for each approach, the accuracy of the network is calculated and stored. From the stored ranked, feature matrix, a feature with the last rank was removed, and the procedure was repeated for next ranking and classifier method (Figure 1) The above said the procedure was implemented in MATLAB Version R2014a.
Statistical comparison of variables measured between the men with CVD and age-matched normal men
Table 2 shows the statistical comparison of variables measured between the men with CVD and age-matched normal men. In the men with CVD (Group I), the mean values of the following variables were significantly (p<0.05) higher than in the age-matched normal men (Group-II): i) SBP (mm Hg); ii) Average SST (°C) of foot posterior left-side; iii) PR interval (ms); iv) P wave amplitude (μV); v) QRS width (ms); vi) RR SD (ms)
|Measured variables||Group-I: Men with CVD (n=10, mean ± SD age=47.6 ± 13 y)||Group-II: Age-matched normal men (n=10, mean ± SD age=45.9 ± 14.9 y)||Statistical significance (p-value)|
|I. Demographic details|
|Age (y)||47.6 c 13||41.9 ± 14.9||NS|
|Body height (cm)||162.4 ± 8.5||163.6 ± 7.6||NS|
|Body weight (Kg)||78.0 ± 13.3||64.0 ± 19.8||NS|
|Waist circumference (cm)||96.5 ± 11.6||96.8 ± 12.6||NS|
|Hip circumference (cm)||103.4 ± 12.9||103.1 ± 12.4||NS|
|SBP (mm Hg)||137.0 ± 13.37||123.0 ± 11.6||<0.05|
|DBP (mm Hg)||84.0 ± 8.43||83.0 ± 8.23||NS|
|II. Biochemical variables|
|TC (mg dl-1)||198.6 ± 61.6||137.8 ± 40.6||NS|
|HDL (mg dl-1)||35.5 ± 1.7||43.9 ± 9.53||<0.05|
|(TC/HDL)||5.6 ± 1.2||4.3 ± 1.16||NS|
|Random glucose (mg dl-1)||137.45 ± 8.34||107.14 ± 5.34||NS|
|HbA1c (%)||5.35 ± 2.5||5.32 ± 0.86||NS|
|III. Average SST (°C) of selected skin areas of the body using IRT_STAT|
|Forehead||34.9 ± 0.5||35.5 ± 0.6||<0.05|
|Neck||Right||35.3 ± 0.5||35.8 ± 0.5||<0.05|
|Left||34.8 ± 1.0||35.7 ± 0.5||<0.05|
|Forearm||Right||33.2 ± 1.4||34.3 ± 0.8||<0.05|
|Left||34 ± 0.8||33.5 ± 1.4||NS|
|Anterior foot||Right||28.6 ± 2.5||27.9 ± 1.6||NS|
|Left||28.2 ± 2.6||27.9 ± 1.8||NS|
|Posterior foot||Right||28.1 ± 1.4||27.3 ± 2.2||NS|
|Left||28.2 ± 1.0||26.8 ± 1.8||<0.05|
|PR interval (ms)||170.4 ± 28.9||146.09 ± 20.14||<0.05|
|P wave amplitude (μV)||143.1 ± 30.25||117.91 ± 28.45||<0.05|
|P wave duration (ms)||135.6 ± 16.16||116.91 ± 25.14||NS|
|QRS Width (ms)||109.0 ± 39.33||104.36 ± 13.62||<0.05|
|QTc (ms)||379.8 ± 47.48||389.09 ± 26.03||NS|
|RR SD (ms)||96.8542 ± 102.1||69.2 ± 32.5||NS|
|HR mean (bpm)||66.4286 ± 16.8||77.0931 ± 9.06||NS|
|V. HRV variables with normalization (log transformation)|
|RR mean (ms)||6.86 ± 0.3||6.67 ± 0.3||NS|
|RR SD (ms)||4.11 ± 1.05||4.09 ± 1.04||<0.05|
|HR mean (bpm)||4.15 ± 0.3||4.34 ± 0.3||NS|
|HR SD (bpm)||1.36 ± 1.02||1.66 ± 1.02||NS|
|RMSSD (ms)||4.5 ± 1.06||4.43 ± 1.06||NS|
|NN50||1.81 ± 0.29||2.4 ± 0.29||NS|
|pNN50 (%)||2.9 ± 0.74||3.16 ± 0.744||< 0.05|
|RR triangular index||1.4 ± 0.66||1.68 ± 0.66||NS|
|TINN (ms)||5.17 ± 0.9||5.25 ± 0.9||NS|
Table 2: Statistical comparison of variables between men with CVD and Normal men.
CAD model for CVD
Figure 2 shows the comparison of skin temperature in the selected skin areas of the body. The variables obtained from different approaches mentioned earlier were utilized in the automated evaluation of CVD. The performance of all the ranking methods in combination with each classifier for the above-mentioned approaches was tested. When static SST (°C) was alone used in the CAD model after variable ranking and selection (5 of 9 variables) by Wilcoxon method, 90% accuracy (sensitvity-80% and specificity-90%) was obtained with the naive-bayes classifier. These results were comparable to CAD model framed with standard ECG variables alone which gave an accuracy of 90% (sensitvity-90% and specificity-80%) Also, the following approaches (feature ranking/classifier) had the highest accuracy of 95%: a) Demographic and biochemical variables (Bhattacharya/SVM): sensitivity=90% and specificity=80%; b) HRV time domain and average SST (°C) variables from the static IR thermogram (Wilcoxon/SVM): sensitivity=90% and specificity=75%; d) Demographic, biochemical, ECG and HRV time domain variables (Wilcoxon/kNN): sensitivity=90% and specificity=85%. The performance of the CAD model that gave the highest accuracy in each combination of feature ranking and classifier for all the approaches is as tabulated in Table 3.
|S. no||CAD model approach||Feature ranking method||Classifier||No. of variables selected (Total no. of variables measured)||Sensitivity %||Specificity %||Accuracy %|
|i.||Demographic and Biochemical variables||Bhattacharya||SVM||7 (11)||90||80||95|
|ii.||ECG variables||Bhattacharya, t-test, entropy, roc, Wilcoxon||SVM||5 (5)||90||80||90|
|iii.||HRV time domain variables||t-test||SVM||7 (9)||80||90||85|
|iv.||Average SST (°C) of selected skin areas of the body||Wilcoxon||Naive-Bayes||5 (9)||80||90||90|
|v.||Demographic, biochemical &ECG variables||roc||Naïve-Bayes||10 (16)||90||80||85|
|vi.||Demographic, biochemical and HRV time domain variables||t-test||Decision tree||10 (20)||90||80||85|
|vii.||Demographic, biochemical and average SST (°C) variables||Wilcoxon||SVM||12 (20)||90||80||85|
|viii.||ECG and HRV time domain variables||roc||Decision tree||9 (14)||85||80||90|
|ix.||ECG and average SST (°C) variables||Bhattacharya||k-NN||8 (14)||80||80||85|
|x.||HRV time domain variables and average SST (°C) variables||Wilcoxon||SVM||9 (18)||90||75||95|
|xi.||Demographic, biochemical variables, ECG and HRV time domain variables||Wilcoxon||KNN||18 (25)||90||85||95|
|xii.||Demographic, biochemical variables, ECG and average SST (°C) variables||t-test||Decision tree||12 (25)||80||85||85|
|xiii.||Demographic, biochemical variables, HRV time domain variables and average SST (°C) variables||Wilcoxon||Naive-Bayes||18 (29)||85||85||85|
Table 3: Performance analyses of different CAD models using different feature set in the evaluation of CVD with higher accuracy.
An asymmetrical pattern of BP, especially in the upper extremity, is a common early symptom in clinical examination of atherosclerosis. The radial artery in hand is the major artery that reflects the changes in the central blood pressure. With the increased blood flow to the skin due to vasodilators Kenney et al. have pointed out the possibility of reduced mean arterial blood pressure . Falcone et al. proposed a model for automatically predicting CVD subjects based on HRV, which gave an accuracy of 85.7% . The database the author had used for classification was 139 Holter recordings of the diseased subjects. The prediction model was analysed with two networks, SVM, and decision tree. In our study, a better accuracy of 90% (sensitivity-80%, specificity-90%) was possible in classifying the normal and CVD subjects when the static SST (°C) alone used and in combination with HRV time domain variables the accuracy was 95% (Table 3)
Various classifiers like feed forward back propagation network and cascade-eliminate backpropagation network were used by Samant et al. to classify the hypertension subjects, to classify 1168 records of 13 variables which included biochemical, hemorheological and clinical variables of different age group subjects . The highest accuracy of 92% was obtained with the feed forward back propagation network. Likewise in our study, the combinational approaches for CAD model performed better than the individual approaches . The main limitation of the study is the number of subjects considered in the study population.
In the design of CAD model for CVD, the combinational approaches using different feature sets, namely: i) Demographic and biochemical variables; ii) ECG variables; iii) HRV time domain variables; and iv) Average SST (°C) of selected skin areas of the body gave a better accuracy of 95% than compared to these feature sets used individually. Of the different feature ranking methods employed Bhattacharya distance method and t-test gave better results (12 of 24 approaches) than compared to other methods, whereas the supervised classifiers like SVM, were more suitable (9 out of 24 approaches) than compared to other unsupervised classifiers.
All procedures followed were in accordance with the ethical standards of the responsible committee on human experimentation (institutional and national) and with the Helsinki Declaration of 1975, as revised in 2000 . Informed consent was obtained from all patients for being included in the study. We declare that we have no conflict of interest.
- Global Status Report on noncommunicable disease 2014, World Health Organisation, Switzerland, 201 World Health Organization. Global status report on alcohol and health 2014. WHO 2014.
- Togawa T. Non-contact skin emissivity: measurement from reflectance using step change in ambient radiation temperature. Clin Phys Physiol Measur 1989; 10:39.
- Ring EF, Ammer K. Infrared thermal imaging in medicine. Physiol Measur 2012; 28; 33.
- Buddharaju P, Pavlidis IT, Tsiamyrtzis P, Bazakos M. Physiology-based face recognition in the thermal infrared spectrum. IEEE Trans Pattern Anal Mach Intell 2007; 29: 613-626.
- Bruning RS, Dahmus JD, Kenney WL, Alexander LM. Aspirin and clopidogrel alter core temperature and skin blood flow during heat stress. Med Sci Sports Exerc 2013; 45: 674-682.
- Falcone C, Colonna A, Bozzini S, Matrone B, Guasti L, Paganini EM, Falcone R, Pelissero G. Cardiovascular risk factors and sympatho-vagal balance. J Clin Exp Cardiol 2014; 5.
- Christensen JH, Toft E, Christensen MS, Schmidt EB. Heart rate variability and plasma lipids in men with and without ischaemic heart disease. Atherosclerosis 1999; 145: 181-186.
- Sztajzel J. Heart rate variability: a noninvasive electrocardiographic method to measure the autonomic nervous system. Swiss Med Weekly 2004; 134: 4-22.
- Natarajan N, Balakrishnan AK, Ukkirapandian K. A study on analysis of Heart Rate Variability in hypertensive individuals. Int J Biomed Adv Res 2014; 5: 109-111.
- Yoo CS, Lee K, Yi SH, Kim JS, Kim HC. Association of heart rate variability with the framingham risk score in healthy adults. Korean J Fam Med 2011; 32: 334-340.
- González-Alonso J. Human thermoregulation and the cardiovascular system. Exp Physiol 2012; 97: 340-346.
- Faezipour M, Saeed A, Bulusu SC, Nourani M, Minn H, Tamil L. A patient-adaptive profiling scheme for ECG beat classification. IEEE Trans Info Technol Biomed 2010; 14: 1153-1165.
- Acharya UR, Vidya KS, Ghista DN, Lim WJ, Molinari F, Sankaranarayanan M. Computer-aided diagnosis of diabetic subjects by heart rate variability signals using discrete wavelet transform method. Knowl Based Sys 2015; 81: 56-64.
- Greenland P, Alpert JS, Beller GA, Benjamin EJ, Budoff MJ, Fayad ZA, Foster E, Hlatky MA, Hodgson JM, Kushner FG, Lauer MS. 2010 ACCF/AHA guideline for assessment of cardiovascular risk in asymptomatic adults: a report of the American College of Cardiology Foundation/American Heart Association task force on practice guidelines developed in collaboration with the American Society of Echocardiography, American Society of Nuclear Cardiology, Society of Atherosclerosis Imaging and Prevention, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular J Am Coll Cardiol 2010; 56: 50-103.
- Samant R, Rao S. Evaluation of artificial neural networks in prediction of essential hypertension. Int J Comp Appl 2013; 81.
- Walchli C, Saltzwedel G, Krüerke D, Kaufmann C, Schnorr B, Rist L, Eberhard J, Decker M, Simoes-Wust AP. Physiologic effects of rhythmical massage: a prospective exploratory cohort study. J Alt Compl Med 2014; 20: 507-515.
- Chao AC, Chern CM, Kuo TB, Chou CH, Chuang YM, Wong WJ, Hu HH. Noninvasive assessment of spontaneous baroreflex sensitivity and heart rate variability in patients with carotid stenosis. Cerebrovasc Dis 2003; 16: 151-157.