Abdomen

- Automated CT Liver Volumetry by Use of Three-Dimensional Fast-Marching and Level-Set Segmentation

Chest

Cardiac

Machine Learning for Image Processing / Pattern Recognition

Machine-Learning Design

Pattern Recognition

Research Facilities

Automated CT Liver Volumetry by Use of Three-Dimensional Fast-Marching and Level-Set Segmentation

We developed an automated scheme for segmenting and calculating liver volume in hepatic CT by means of 3D fast-marching and level-set segmentation algorithms. Automatic liver segmentation on hepatic CT images is challenging because the liver often abuts other organs of similar density. Our purpose was to develop an automated liver segmentation scheme based on a 3D level-set algorithm for measuring liver volumes. Hepatic CT scans of eighteen prospective liver donors were obtained under a liver transplant protocol. Scans were acquired with a multi-detector CT system with a 16-, 40-, or 64-channel detector scanner (Brilliance, Philips Medical Systems, Netherlands). We developed an automated liver segmentation scheme for volumetry of the liver in CT. Our scheme consisted of five steps. First, a 3D anisotropic diffusion smoothing filter was applied to CT images for removing noise while preserving the structures in the liver, followed by a gradient magnitude filter for enhancing liver edges. A nonlinear gray-scale enhancement filter was applied to the gradient magnitude image for further enhancing the boundary of the liver. By use of the enhanced gradient magnitude image as a speed function, a 3D fast-marching algorithm generated an initial surface that roughly estimated the shape of the liver. A 3D level-set segmentation algorithm refined the initial surface so as to fit the liver boundary more accurately (see the figure for a segmentation result). The liver volume was calculated based on the refined liver surface. Automated volumes were compared to manually determined liver volumes. The mean liver volume obtained with our scheme was 1598 cc (range: 1002-2415 cc), whereas the mean manual volume was 1,535 cc (range: 1,007-2,435 cc). The mean absolute difference between automated and manual volumes was 128 (9.5%) ± 119 (9.4%) cc. The two volumetrics reached excellent agreement (the intra-class correlation coefficient was 0.89) with no statistically significant difference (P=0.13). The processing time by the automated method was 2-5 min. per case (Intel, Xeon, 2.7 GHz), whereas that by manual segmentation was approximately 50-60 min. per case. CT liver volumetrics based on an automated scheme agreed excellently with manual volumetrics and required substantially less completion time. Thus, our automated scheme provides an efficient and accurate way of measuring liver volumes in CT; thus, it would be useful for radiologists in their measurement of liver volumes.

Illustration of automated liver segmentation. (a) Original CT liver image. (b) Segmented liver obtained with our automated liver segmentation method (Red curve).

A CAD Utilizing 3D Massive-Training ANNs for Detection of Flat Lesions in CT Colonography in a Large Multicenter Clinical Trial

We developed a computer-aided diagnostic (CAD) scheme for detection of flat lesions (also called flat polyps or depressed polyps) in CT colonography (CTC) in a large multicenter clinical trial in collaboration with Don C. Rockey, M.D., at the Southwest Medical Center of the University of Texas. Flat lesions in the colon are a major source of false-negative interpretations in CTC. A major challenge in CAD schemes is the detection of flat lesions, because existing CAD schemes are designed for detecting the common bulbous polyp shape. Our purpose was to develop a CAD scheme for detection of flat lesions in CTC. We developed a CAD scheme consisting of colon segmentation based on histogram and morphologic analysis, detection of polyp candidates based on intensity-based and morphologic feature analysis, and linear discriminant analysis for classification of the candidates as polyps or non-polyps. To detect flat lesions, we developed a “tolerant�Emorphologic analysis method in the polyp detection step for accommodating the analysis to include a flat shape. For reduction of false-positive (FP) detections, we developed 3D massive-training artificial neural networks (MTANNs) designed to differentiate flat lesions from various types of non-polyps. Our independent database consisted of CTC scans of 25 patients obtained from a multicenter clinical trial in which 15 institutions participated nationwide. Each patient was scanned in the supine and prone positions with collimations of 1.0-2.5 mm and reconstruction intervals of 1.0-2.5 mm. All patients underwent “reference-standard�Eoptical colonoscopy. Flat lesions were determined under either “height�E(< 3 mm high) or “ratio�E(height < 1/2 long axis) criteria. Twenty-eight flat lesions were identified. Among them, 11 (39%) were false negatives in CTC. Lesion sizes ranged from 6-18 mm, with an average of 9 mm. Our MTANN CAD scheme detected 68% (19/28) of flat lesions, including six lesions “missed�Eby reporting radiologists in the original clinical trial, with 10 (249/25) FPs per patient. The figure shows examples of flat lesions, which are very small or on a fold (these are major causes of human misses). Some flat lesions are known to be histologically aggressive; therefore, detection of such lesions is critical clinically, but they are difficult to detect because of their uncommon morphology. It should be noted that these two cases were “missed�Eby reporting radiologists in the original trial; thus, detection of these lesions may be considered “very difficult.�EOur scheme would be useful for detecting flat lesions which are a major source of false negatives, thus potentially improving radiologists�E sensitivity in their detection of polyps in CTC.

Illustrations of flat lesions which exhibit uncommon flat morphology. (a) A flat lesion on a fold (10 mm; adenoma) in the cecum was detected correctly by our MTANN CAD scheme (indicated by an arrow). (b) A small flat lesion (6 mm; adenoma) in the cecum was detected correctly by our MTANN CAD scheme.

Polyp Detection in CT Colonography: Performance of a CAD Scheme Incorporating 3D MTANNs on False-Negative Polyps in a Multicenter Clinical Trial

We developed computer-aided diagnostic (CAD) scheme for detection of polyps in CT colonography (CTC) and evaluating our CAD scheme with false-negative polyps in a large multicenter clinical trial in collaboration with Don C. Rockey, M.D., the Southwest Medical Center at the University of Texas. A major challenge in CAD schemes for detection of polyps in CTC is the detection of “difficult�Epolyps which radiologists are likely to miss. Our purpose was to develop a CAD scheme incorporating 3D massive-training artificial neural networks (3D MTANNs) and to evaluate its performance on false-negative (“missed�E cases in a large multicenter clinical trial.  We developed an initial CAD scheme consisting of colon segmentation based on mathematical morphology, detection of polyp candidates based on intensity-based and morphologic feature analysis, and linear discriminant analysis for classification. For reduction of false-positive (FP) detections, we developed a “mixture�Eof seven expert 3D MTANNs designed to differentiate between polyps and seven types of non-polyps, including folds, stool, the ileocecal valve, and rectal tubes. Our independent database consisted of CTC scans of 614 patients obtained from a large multicenter clinical trial in which 15 institutions participated nationwide. Each patient was scanned in the supine and prone positions with collimations of 1.0-2.5 mm and reconstruction intervals of 1.0-2.5 mm. All patients underwent “reference-standard�Eoptical colonoscopy. One hundred fifty-five patients had clinically significant polyps. Among them, about 45% patients received false-negative interpretations in CTC. For testing our CAD scheme with 3D MTANNs, 14 cases with 14 polyps/masses were randomly selected from the false-negative cases where lesions were visible in both supine and prone scans retrospectively. Lesion sizes ranged from 6-35 mm, with an average of 10 mm. The initial CAD scheme detected 71.4% (12/14) of “missed�Epolyps, including sessile polyps and polyps on folds, with 18.9 (264/14) FPs per patient. The 3D MTANNs removed 75% (197/264) of the FPs without loss of any true positives; thus, the performance of our CAD scheme was improved to 4.8 (67/14) FPs per patient. With our CAD scheme incorporating 3D MTANNs, 71.4% of polyps “missed�Eby radiologists in the trial were detected correctly, with a reasonable number of FPs. Our CAD scheme would be useful for detecting “difficult�Epolyps which radiologists are likely to miss, thus potentially improving radiologists�Esensitivity in their detection of polyps in CTC.

Illustrations of polyps “missed�Eby physicians in a multicenter clinical trial.  Left: A very small polyp (6 mm) was detected correctly by our CAD system (indicated by an arrow).  This polyp was not detected in either CTC or the “gold standard�Eoptical colonoscopy in the trial.  Right: A sessile-type polyp (a major source of human false negatives) was detected correctly by our CAD system.   A patient with a small (7 mm) sessile polyp (one of the major sources of false-negative interpretations by radiologists) that was “missed�Ein a clinical trial.  (a) our CAD scheme incorporating 3D MTANNs correctly detected the polyp and pointed it by an arrow in an axial CTC image, (b) the polyp in the 3D endoluminal view, and (c) 3D volume rendering of the colon with three computer outputs indicated by yellow circles (one in the rectum is a true-positive detection and the other two are FP detections).

Ensemble Training for a Mixture of Expert 3D MTANNs for Eliminating Multiple False-Positive Sources in CAD for Polyp Detection in CT Colonography

We developed a “mixture of expert�E3D massive-training artificial neural networks (3D MTANNs) for reduction of multiple types of false-positive (FP) detections in a computer-aided diagnostic (CAD) scheme for detection of polyps in CT colonography (CTC). To train the mixture of expert 3D MTANNs consisting of several 3D MTANNs, we select training cases manually based on the visual appearance of polyps and non-polyps. Manual selection of training cases for the multiple expert 3D MTANNs, however, is time-consuming and does not necessarily provide the optimal selection for the multiple 3D MTANNs. To address these issues, we developed a method for systematically selecting training cases for multiple classifiers, which we call “ensemble training,�E for the optimal selection of training cases. We applied the proposed ensemble training to the mixture of expert 3D MTANNs. We started from a seed 3D MTANN trained with polyps and non-polyps of a seed type. We applied the trained seed 3D MTANN to polyps and various types of non-polyps to analyze the weakness of the seed 3D MTANN. We arranged the output scores of the 3D MTANN for polyps and non-polyps in an ascending order to form the score scale representing the degree of difficulty in distinction between polyps and non-polyps by the seed 3D MTANN. The score scale was divided into several segments, and ten non-polyps were sampled from the center of each segment so that sets of non-polyp samples covered diverse difficulties. We trained several 3D MTANNs with several sets of non-polyps so that each 3D MTANN became an expert for the non-polyps at a certain level of difficulty. We then combined expert 3D MTANNs with a mixing artificial neural network (ANN) to form a “mixture of expert�E3D MTANNs. Our database consisted of CTC datasets acquired from 100 patients, including 26 polyps. We applied our initial CAD scheme to this CTC database. FP sources included haustral folds, stool, colonic walls, the ileocecal valves, and rectal tubes. The mixture of expert 3D MTANNs distinguished all polyps correctly from more than 50% of the non-polyps. Thus, the mixture of expert 3D MTANNs was able reduce one half of the FPs generated by a computerized polyp detection scheme while the original sensitivity was maintained. We compared the effectiveness of ensemble training with that of training with manually selected cases. The performance of the 3D MTANNs with ensemble training was superior to that of the 3D MTANNs trained with manually selected cases.

Free-response receiver-operating-characteristic (FROC) curves for the mixture of expert 3D MTANNs obtained with ensemble training and the mixture of expert 3D MTANNs obtained with standard training for 26 polyps and 489 non-polyps (FPs). The FROC curve for the mixture of expert 3D MTANNs indicates a reduction in the FP rate from 4.9 to 2.2 per patient at a 96% by-polyp sensitivity level (100% by-patient sensitivity).

Eliminating Multiple False-Positive Sources in CAD for Polyp Detection in CT Colonography by Means of a Mixture of Expert 3D Massive-Training Artificial Neural Networks

We developed a method for eliminating multiple false-positive (FP) sources in a CAD scheme for the detection of polyps in CT colonography (CTC) by use of a “mixture of expert�E 3D massive-training artificial neural networks (3D MTANNs) in collaboration with Hiroyuki Yoshida, PhD, Massachusetts General Hospital.  One of the major challenges in CAD for polyp detection in CTC is the reduction of FPs while sensitivity is maintained.  To eliminate multiple FP sources, we developed a “mixture of expert�E3D MTANNs consisting of seven 3D MTANNs that were designed to differentiate between polyps and seven sources of FPs, among them folds, stool, colonic wall, ileocecal valve, and rectal tubes.  Each expert MTANN was trained with each of these specific FP sources and with representative polyps, together with teaching volumes represented by a Gaussian distribution for a polyp and zero for a non-polyp.  The seven MTANN experts were combined with a logical multiplication operation such that different FP sources could be removed.  Our database consisted of 146 CTC datasets obtained from 73 patients.  Each patient was scanned in both supine and prone positions with a CT scanner that had a collimation of 2.5-5.0 mm and a reconstruction interval of 1.25-5.0 mm.  Radiologists established the locations of polyps by use of optical colonoscopy reports as reference standards.  Fifteen patients had 28 polyps, 15 of which were 5-9 mm, and 13 were 10-25 mm in size.  The CTC cases were subjected to our previously reported CAD scheme consisting of centerline-based extraction of the colon, shape-based detection of polyps, and a Bayesian-ANN-based classification of polyps.  The original CAD scheme yielded 96% (27/28) by-polyp sensitivity with 3.1 (224/73) FPs per patient.  The mixture of expert 3D MTANNs removed 70% (157/224) of FPs without loss of any true positives; thus, the FP rate of our CAD scheme was improved from 3.1 to 0.9 (67/73) FPs per patient while the original sensitivity was maintained (Figure 1).

Use of a mixture of expert 3D MTANNs demonstrating a substantial elimination of false positives in CAD for polyp detection in CTC while maintaining a high sensitivity.

Reduction of False Positives in Computer-Aided Detection of Polyps in CT Colonography Using a Massive-Training Artificial Neural Network (MTANN): Suppression of Rectal Tubes

One of the limitations of the current computer-aided detection (CAD) of polyps in CT colonography (CTC) is a relatively large number of false-positive (FP) detections.  Rectal tubes (RTs) are one of the typical sources of FPs.  Radiologists can easily recognize and dismiss RT-induced FPs; thus, they may lose their confidence in CAD as an effective tool if the CAD scheme consistently generates such “obvious�EFPs due to RTs.  Therefore, removal of RT-induced FPs is desirable while maintaining a high sensitivity.  We developed a three-dimensional (3D) massive-training artificial neural network (MTANN) for distinction between polyps and RTs in 3D CTC volumetric data.  The 3D MTANN is a supervised volume-processing technique which is trained with input CTC volumes and the corresponding “teaching�Evolumes.  For distinction between polyps and non-polyps including RTs, a 3D scoring method based on a 3D Gaussian weighting function is applied to the output of the trained 3D MTANN.  Our database consisted of CTC examinations of 73 patients, scanned in both supine and prone positions (146 CTC datasets in total), with optical colonoscopy as a reference standard for the presence of polyps.  Fifteen patients had 28 polyps, 15 of which were 5-9 mm and 13 were 10-25 mm in size.  These CTC cases were subjected to our previously reported CAD scheme that included centerline-based segmentation of the colon, shape-based detection of polyps, and reduction of FPs by use of a Bayesian neural network based on geometric and texture features.  Application of this CAD scheme yielded 96.4% (27/28) by-polyp sensitivity with 3.1 (224/73) FPs per patient, among which 20 FPs were caused by RTs.  To eliminate the FPs due to RTs, we trained a 3D MTANN with ten representative polyps and ten RTs, and we applied the trained 3D MTANN to the above CAD true- and false-positive detections.  In the output volumes of the 3D MTANN, polyps were represented by distributions of bright voxels, whereas RTs and other normal structures partly similar to RTs appeared as darker voxels, indicating the ability of the 3D MTANN to suppress RTs effectively.  Application of the 3D MTANN to the CAD detections showed that the 3D MTANN eliminated all 20 RT-induced FPs, as well as 53 FPs due to other causes, without removal of any true positives.  Overall, the 3D MTANN was able to reduce the FP rate of the CAD scheme from 3.1 to 2.1 FPs per patient (33% reduction), while the original by-polyp sensitivity of 96.4% was maintained.

Illustrations of various non-training polyps and the corresponding output volumes of the trained 3D MTANN (upper figures) and non-training RTs and the corresponding output volumes (lower figures). FROC curves showing the overall performance of the trained 3D MTANN when it was applied to the entire database of 27 polyps (48 true-positive volumes) and 224 non-polyps (FPs).

Reduction of Quantum Noise in Low-Dose Double-Contrast Radiographs of the Stomach

We have developed a supervised nonlinear filter based on an artificial neural network (ANN), called a "neural filter," for reduction of quantum noise in low-dose gastric double-contrast radiographs of the stomach.  The neural filter can be trained with input images and the corresponding teaching images.  To learn the relationship between low-dose and high-dose x-ray images, we created simulated low-dose radiographs from actual high-dose radiographs by use of a model of quantum noise.  We used the simulated low-dose radiographs as the input images and the corresponding high-dose radiographs as the teaching images for the neural filter.  After training, the neural filter provided estimated high-dose radiographs.  The quantum noise in input low-dose radiographs was reduced with the trained neural filter, while the lining of the stomach was maintained.  We evaluated the performance of the neural filter based on the improvement in the signal-to-noise ratio (ISNR).  The ISNR was defined as the improvement in the quality of the output images compared with the input images.  An average ISNR of 1.9 dB was obtained with the neural filter for testing radiographs.

Low-dose gastric double-contrast radiographs of the stomach (left) and the output image of the trained neural filter for reduction of quantum noise (right).

 ↑ Go to top

Enhanced Digital Chest Radiography: Temporal Subtraction Combined with “Virtual Dual-Energy�ETechnology for Improved Conspicuity of Growing Cancers and Other Pathologic Changes

We developed a novel temporal-subtraction (TS) technique combined with “virtual dual-energy�Etechnology for improved conspicuity of growing cancers and other pathologic changes in digital chest radiography (CXR). Digital CXR makes use of advanced image-processing techniques in the radiology viewing environment. A TS technique provides enhanced visualization of tumor growth and subtle pathologic changes between previous and current CXRs from the same patient. Our purpose was to develop a new TS technique incorporating “virtual dual-energy�Etechnology to improve its enhancement quality. Our TS technique consisted of ribcage edge detection, rigid body transformation based on a global alignment criterion, image warping based on local spatial displacement vectors under the maximum cross-correlation criterion, and subtraction between the registered previous and current images. A major problem with TS was obscuring of abnormalities by rib artifacts due to misregistration. To reduce the rib artifacts, we developed a massive-training artificial neural network (MTANN) for separation of ribs from soft tissue. The MTANN was trained with input CXRs and the corresponding “teaching�Esoft-tissue CXRs obtained with dual-energy radiography. Once trained, the MTANNs did not require a dual-energy system and provided soft-tissue images in which ribs were substantially suppressed (thus the term “virtual dual-energy�Etechnology). Our database consisted of 100 sequential pairs of digital CXR studies from 53 patients. To assess the registration accuracy and clinical utility, a chest radiologist subjectively rated original TS and rib-suppressed TS images on a 5-point scale. By use of “virtual dual-energy�Etechnology, the contrast of ribs in the original CXRs was reduced to 8% while maintaining that of soft tissue; thus, rib artifacts in the TS images were reduced substantially. The registration accuracy and clinical utility ratings for TS rib-suppressed images (3.7; 3.9) were significantly better than those for the original TS images (3.5; 3.6) (P<0.01; P<0.02, respectively). Our “virtual dual-energy�Etechnology reduced rib artifacts in TS CXRs and improved the enhancement quality of TS images for the assessment of pathologic change (see the figure). Thus, our TS combined with “virtual dual-energy�Etechnology would be useful for radiologists in the assessment of tumor growth and other pathologic changes between previous and current digital CXRs.

Comparison of our rib-suppressed TS images with conventional TS images. (a) Previous chest radiographs. (b) Current chest radiographs of the same patient. (c) Rib-suppressed TS images with fewer rib artifacts. (d) Conventional TS images.

Computerized Detection of Lung Nodules in Low-Dose CT, Part I: Basic Principle of Massive-Training Artificial Neural Network (MTANN) for Reduction of False Positives

We investigated a novel pattern-recognition technique based on an artificial neural network (ANN), called a massive-training artificial neural network (MTANN), for reduction of false positives (FPs) in computerized detection of lung nodules in low-dose CT.  The MTANN consists of a linear-output multilayer ANN model, which is capable of operating on image data directly.  The MTANN is trained by use of input images together with the teaching images containing the distribution for the “likelihood of being a nodule.�Enbsp; To achieve a high performance, the MTANN is trained by use of a large number of sub-regions extracted from an input image.  The output image is obtained by scanning of an input image with the MTANN, i.e., the MTANN acts like the convolution kernel of a filter.  The distinction between a nodule and a non-nodule is made by use of a score which is defined from the output image of the trained MTANN.  In the output images of the trained MTANN, nodules are represented by bright pixels, whereas vessels are mostly dark.  The MTANN was applied to the task of reducing the number of FPs produced by our computerized scheme for lung nodule detection.  Our database contains 50 nodules, including 38 “missed�Ecancers from a lung cancer screening program.  The MTANN was trained with ten typical nodules and ten typical non-nodules and then applied to the remaining 40 nodules and 1068 non-nodules (FPs) in a validation test.  The results indicated that 66% (706/1068) of the FPs were removed without a reduction in the number of true positives, i.e., a sensitivity of 100% (40 out of 40 nodules) with 0.35 FPs per slice.  Accordingly, the false-positive rate of our original scheme was improved from 1.02 to 0.35 FPs per slice without a reduction in the overall sensitivity.  Thus, the performance of computer-aided diagnostic schemes for lung nodule detection in CT can be improved substantially by use of the MTANN.

Architecture and training of a massive-training artificial neural network (MTANN) for distinction between nodules and vessels. Output images of the trained MTANN for non-training nodules and non-training medium-sized vessels.

Computerized Detection of Lung Nodules in Low-Dose CT, Part II: Usefulness of Multiple Massive-Training Artificial Neural Networks (Multi-MTANNs)

We extended the capability of a single massive-training artificial neural network (MTANN) and developed a multiple MTANN scheme (multi-MTANN) for further removal of false positives (FPs) in computerized detection of lung nodules in low-dose CT.  The multi-MTANN consists of several MTANNs arranged in parallel.  Each MTANN is trained by use of the same nodules, but with a different type of non-nodule (i.e., FP).  Each MTANN acts as an expert for a specific type of non-nodule, e.g., five different MTANNs were trained to distinguish nodules from various-sized vessels; four other MTANNs were applied to eliminate large vessels in the hilum, vessels with some opacities, soft-tissue opacities caused by the partial volume effect, and some abnormal opacities.  The outputs of the MTANNs were combined by use of  the logical AND operation such that each of the trained MTANNs eliminated none of the nodules, but removed some of the various types of non-nodules.  In the output images of the trained multi-MTANN, three major types of nodules, i.e., pure ground-glass opacity (p-GGO), mixed GGO, and solid nodules, are represented by light distributions, whereas various types of non-nodules are mostly dark.  The trained multi-MTANN was applied to the reduction of FPs reported by our computerized scheme for lung nodule detection based on a database of 63 low-dose CT scans (1765 sections), which contained 71 confirmed nodules including 66 biopsy-confirmed primary lung cancers, from lung cancer screening.  The multi-MTANN was applied to 58 non-training true positives (54 patients) and 1726 FPs (non-nodules) reported by our scheme in an independent test; these were different from the training set.  The results indicated that 83% (1424/1726) of non-nodules were removed with a reduction of one true positive (nodule), i.e., a classification sensitivity of 98.3% (57/58).  By use of the multi-MTANN, the false-positive rate of our original scheme was improved from 0.98 to 0.18 FPs per section (from 27.4 to 4.8 per patient) at an overall sensitivity of 80.3% (57/71).

Architecture of multiple massive-training artificial neural networks (multi-MTANN) for distinction between nodules and various types of non-nodules. Output images of the trained multi-MTANN for non-training nodules and various types of non-training non-nodules. FROC curves of the single MTANN and the multi-MTANN consisting of nine MTANNs for the test set consisting of 57 true positives (nodules) and 1726 FPs (non-nodules) in an independent test.

Massive-Training Artificial Neural Network (MTANN) Trained with a Small Number of Cases for Enhancement of Nodules and Suppression of Vessels in Thoracic CT:  Phantom Experiments

A massive-training artificial neural network (MTANN) is a trainable, highly nonlinear filter consisting of a linear-output multilayer artificial neural network model.  For enhancement of nodules and suppression of vessels, we used 10 nodules and 10 non-nodule images as training cases for MTANNs.  The MTANN is trained with a large number of input subregions selected from the training cases and the corresponding pixels in teaching images that contain Gaussian distributions for nodules and zero for non-nodules.  We trained three MTANNs with different numbers (one, nine, and 361) of training samples (pairs of the subregion and the teaching pixel) selected from the training cases.  To investigate the basic characteristics of the trained MTANNs, we applied the MTANNs to simulated CT images containing various-sized model nodules (spheres) with different contrasts and various-sized model vessels (cylinders) with different orientations.  In addition, we applied the trained MTANNs to non-training actual clinical cases with 59 nodules and 1,726 non-nodules.  In the output images for the simulated CT images by use of the MTANNs trained with small numbers (one and nine) of subregions, model vessels were clearly visible and were not removed; thus, the MTANNs were not trained properly.  However, in the output image of the MTANN trained with a large number of subregions, various-sized model nodules with different contrasts were represented by light nodular distributions, whereas various-sized model vessels with different orientations were dark and thus were almost removed.  This result indicates that the MTANN was able to learn, from a very small number of actual nodule and non-nodule cases, the distinction between nodules (sphere-like objects) and vessels (cylinder-like objects).  In non-training clinical cases, the MTANN was able to distinguish actual nodules from actual vessels in CT images.  For 59 actual nodules and 1,726 non-nodules, the performance of the MTANN decreased as the number of training samples (subregions) in each case decreased.  The MTANN can be trained with a very small number of training cases (10 nodules and 10 non-nodules) in the distinction between nodules and non-nodules (vessels) in CT images.  Massive training by scanning of training cases to produce a large number of training samples (input subregions and teaching pixels) would contribute to a high generalization ability of the MTANN.

Ten nodule images (top two rows) and ten non-nodule images (bottom tow rows) including vessels used for training an MTANN. Simulated CT image (left) that contains various-sized model nodules with different contrasts and various-sized model vessels with different orientations, and the corresponding output image (right) of the MTANN trained with ten actual nodules and ten actual vessel images.

Reduction of False Positives in a CAD Scheme for Detection of Lung Nodules on MDCT by Use of 3D Massive-Training Artificial Neural Network

A major challenge in computer-aided diagnostic (CAD) schemes for lung nodule detection in multi-detector-row CT (MDCT) is to reduce false positives (FPs) while maintaining a high sensitivity level.  Our purpose in this study was to develop a three-dimensional (3D) massive-training artificial neural network (MTANN) for reduction of FPs.  To process quasi-isotropic voxels in the 3D MDCT volume, we developed a 3D MTANN with a 3D input kernel.  For distinction between nodules and non-nodules (FPs), the 3D MTANN was trained with input CT volumes and the corresponding teaching volumes that contain a 3D Gaussian distribution for a nodule, and zero for a non-nodule.  To remove nine different types of FPs reported by our CAD scheme based on selective enhancement filters with linear discriminant analysis, we developed a multiple 3D MTANN consisting of nine 3D MTANNs.  Each 3D MTANN was trained with ten typical nodules and ten non-nodules in each of nine different types of FP sources such as medium-sized vessels, small vessels, vessels with high contrast, and some soft-tissue opacities.  Nine MTANNs were combined with the logical AND operation such that nine different types of non-nodules could be eliminated.  Our database contained 62 nodules in 32 scans acquired from 32 patients with an MDCT system.  The scan consisted of an average of 186 thin-slice CT images (the slice thickness ranged form 1.0 to 2.5 mm).  Nodule sizes ranged from 5 to 30 mm.  All nodules were confirmed by chest radiologists.  With our original CAD scheme, a sensitivity of 96.8% (60/62 nodules) together with an average of 14.9 FPs per case was achieved.  The multiple 3D MTANN was applied to further reduction of FPs.  The results indicated that 62.2% (296/476) of FPs were removed with a loss of only one true positive.  Thus, the FP rate of our original CAD scheme was improved to 5.6 FPs per case at an overall sensitivity of 95.2% (59/62 nodules).  By using a multiple 3D MTANN, the specificity of our CAD scheme for lung nodule detection on MDCT can be improved substantially while a high sensitivity is maintained.

Architecture and training of a three-dimensional massive-training artificial neural network (3D MTANN) for distinction between nodules and non-nodules in MDCT. Output images of the trained 3D MTANN for various-sized non-training nodules and vessels in MDCT images.

Computer-aided Diagnostic Scheme for Distinction between Benign and Malignant Nodules in Thoracic Low-Dose CT by Use of a Massive-Training Artificial Neural Network

Low-dose helical CT (LDCT) is being applied as a modality for lung cancer screening.  It may be difficult, however, for radiologists to distinguish malignant from benign nodules in LDCT.  Our purpose in this study was to develop a computer-aided diagnostic (CAD) scheme for distinction between benign and malignant nodules in LDCT by use of a massive-training artificial neural network (MTANN).  The MTANN is a trainable, highly nonlinear filter based on an artificial neural network.  To distinguish malignant nodules from six different types of benign nodules, we developed a multiple MTANN scheme (Multi-MTANN) consisting of six expert MTANNs arranged in parallel.  Each of the MTANNs was trained by use of input CT images and teaching images containing an estimate of the distribution for a "likelihood of being a malignant nodule," i.e., the teaching image for a malignant nodule contains a two-dimensional Gaussian distribution and that for a benign nodule contains zero.   Each MTANN was trained independently with ten typical malignant nodules and ten benign nodules from each of the six types.  In the output images of the trained MTANNs, malignant nodules are represented by light distributions, whereas benign nodules are mostly dark.  The outputs of the six MTANNs were combined by use of an integration ANN such that the six types of benign nodules could be distinguished from malignant nodules.  After training of the integration ANN, our scheme provided a value related to the “likelihood of malignancy�Eof a nodule, i.e., a higher value indicates a malignant nodule, and a lower value indicates a benign nodule.  Our database consisted of 76 primary lung cancers in 73 patients and 413 benign nodules in 342 patients, which were obtained over 3 years from a lung cancer screening program on 7,847 screenees with LDCT for three years in Nagano, Japan.  The performance of our scheme for distinction between benign and malignant nodules was evaluated by use of receiver-operating characteristic (ROC) analysis.  Our scheme achieved an AUC (area under the ROC curve) value of 0.882 in a leave-one-out cross-validation test.  Our scheme correctly identified 100% (76/76) of malignant nodules as malignant, whereas 48% (200/413) of benign nodules were identified correctly as benign.  Therefore, our scheme may be useful in assisting radiologists in the diagnosis of lung nodules in LDCT.

Malignant nodules with three major types of patterns, i.e., pure ground-glass opacity (GGO), mixed GGO, and solid nodule, various benign nodules, and the corresponding output images of the trained MTANN. ROC curve for the MTANNs in distinction between 76 malignant nodules and 413 benign nodules in a leave-one-out cross-validation test.

False-Positive Reduction in Computer-Aided Diagnostic Scheme for Detection of Nodules on Chest Radiographs by Means of Massive-Training Artificial Neural Network (MTANN)

We developed a technique that uses a multiple massive-training artificial neural network scheme (multi-MTANN) to reduce false positives (FPs) in a computer-aided diagnostic (CAD) scheme for nodule detection on chest radiographs.  Our database consisted of 91 solitary pulmonary nodules including 64 malignant nodules and 27 benign nodules in 91 chest radiographs.  With our CAD scheme based on a difference-image technique and linear discriminant analysis, a sensitivity of 82.4% with 4.5 FPs per image was achieved.  We developed the multi-MTANN for further reduction of the FPs.  An MTANN is a  nonlinear filter that can be trained with input images and the corresponding teaching images.  For reducing the effects of the background levels in chest radiographs, a background-trend-correction technique followed by contrast normalization was used on the input images to the MTANN.  For enhancement of nodules, the teaching image contained the distribution for a "likelihood of being a nodule."  Six MTANNs in the multi-MTANN were trained by use of typical nodules and six different types of non-nodules (FPs).  In the output images of the trained MTANN, nodules are represented by light distributions, whereas non-nodules such as ribs are mostly dark.  By use of the trained multi-MTANN, 68.3% of FPs were removed with a reduction of one true positive.  The false-positive rate of our original CAD scheme was improved from 4.5 to 1.4 FPs per image at an overall sensitivity of 81.3%.  By use of a multi-MTANN, the false-positive rate of our CAD scheme for lung nodule detection on chest radiographs was improved substantially while a high sensitivity was maintained.

Non-training nodules and the corresponding output images of the trained MTANN and various non-training non-nodules and the corresponding output images of the trained MTANN. Free-response receiver-operating characteristic (FROC) curve of the multi-MTANN consisting of six MTANNs for 75 true positives (nodules) and 410 FPs (non-nodules).

Virtual Dual-Energy Radiography:  Image-Processing Technique for Suppressing Ribs in Chest Radiographs by Means of Massive-Training Artificial Neural Network (MTANN)

When lung nodules overlap with ribs or clavicles in chest radiographs, it can be difficult for radiologists as well as computer-aided diagnostic (CAD) schemes to detect these nodules.  In this study, we developed an image-processing technique for suppressing the contrast of ribs and clavicles in chest radiographs by means of a multi-resolution massive-training artificial neural network (MTANN).  An MTANN is a nonlinear filter that can be trained by use of input chest radiographs and the corresponding “teaching�Eimages.  We employed “bone�Eimages obtained with a dual-energy subtraction radiography system as the teaching images.  For effective suppression of ribs having various spatial frequencies, we developed a multi-resolution MTANN consisting of multi-resolution decomposition/composition techniques and three MTANNs for three different-resolution images.  After training with input chest radiographs and the corresponding dual-energy bone images, the multi-resolution MTANN was able to provide "bone-image-like" images which were similar to the teaching dual-energy bone images.  By subtracting the bone-image-like images from the corresponding chest radiographs, we were able to produce "soft-tissue-image-like" images where ribs and clavicles were substantially suppressed.  The major advantages of our virtual dual-energy radiography compared to "gold standard" duel-energy subtraction radiography are: (1) no additional radiation dose to patients is required, and (2) no specialized equipment for generating dual-energy x-rays is required. Thus, this technique is applicable to any chest radiographs acquired with a standard system for producing soft-tissue images and bone images.  Our image-processing technique for rib suppression by means of a multi-resolution MTANN would be useful for radiologists as well as for CAD schemes in the detection of lung nodules on chest radiographs.

Basic principle of “virtual dual-energy radiography�E(VDER) for separating soft tissues from ribs in chest radiographs, consisting of MTANNs and multi-resolution image decomposition/composition.  In the training phase, VDER is trained with input chest radiographs and the corresponding “teaching�Ebone images acquired with a commercial dual-energy radiography system (upper figure).  VDER can derive “soft-tissue�Eand “bone�Eimages from other standard radiographs based on this training (lower figure). Our virtual dual-energy radiography produces a soft-tissue image (middle image in the upper row) and a bone image (lower left image) from the original chest radiograph (upper left image) which are similar to the corresponding “gold-standard�E dual-energy radiographs (upper right image and lower right image).

Improving the Conspicuity of Nodules in Chest Radiographs by Use of Virtual Dual-Energy Radiography

We evaluated our virtual dual-energy radiography technique based on a massive-training artificial neural network (MTANN) in the improvement of the conspicuity of nodules in chest radiographs.  To do this, we used a validation test database consisting of 118 chest radiographs with pulmonary nodules and an independent test database consisting of 136 digitized screen-film chest radiographs with 136 solitary pulmonary nodules collected from 14 medical institutions.  When our virtual dual-energy radiography technique was applied to non-training chest radiographs, ribs and clavicles in the chest radiographs were suppressed substantially while the visibility of nodules and lung vessels was maintained.  Thus, our virtual dual-energy radiography technique by means of the MTANN could potentially improve radiologists' detection accuracy of lung nodules on chest radiographs.

Comparison of the original chest radiograph (left), our virtual dual-energy soft-tissue image (middle), and the “gold-standard�Edual-energy soft-tissue image (right). Comparison of the conspicuity of a lung cancer in the original chest radiograph (left), our virtual dual-energy soft-tissue image (middle), and the corresponding “gold-standard�E dual-energy soft-tissue image (right).

                                                                                                           ↑ Go to top

Cardiac

Computer-Aided Diagnostic System for Detection and Estimation of Coronary Artery Stenosis by Use of a Linear-Output Artificial Neural Network

We developed a new computer-aided diagnostic (CAD) system for coronary artery stenosis, which can learn physicians' diagnosis.  To realize such a system, we developed a linear-output artificial neural network (LOANN) which is capable of learning of experts' judgment.  Our CAD system consisted of (a) an automated method for tracing vessels, (b) a robust method for determining the edges of the traced vessels, and (c) the LOANN for estimating stenosis based on the information obtained from the segmented vessels.  To evaluate the performance of our CAD system, we carried out an experiment with vessel phantoms that had simulated stenoses.  After the acquisition of the vessel phantom images with a digital subtraction angiography (DSA) system, we measured actual transverse areas of the vessel phantoms to determine stenoses.  In a leave-one-out cross-validation test, our CAD system accurately estimated the stenoses determined based on the actual measurement of the vessel phantoms.   To evaluate the performance of our CAD system on clinical cases, we used angiograms acquired with a DSA system.  The stenoses estimated by our CAD system agreed well with those diagnosed by an experienced cardiologist.  Therefore, our CAD system which has the capability to learn physicians' diagnosis would be useful for assisting their diagnosis.

Original digital subtraction angiographic image (upper left image), the original image overlaid with the edges of the arteries extracted by our method (upper right image), stenosis diagnosed by an experienced cardiologist (lower left image), and stenosis detected and estimated by our CAD system (lower right image).

Reduction of Quantum Noise and Radiation Dose in Coronary Angiography by Means of a Neural Filter

Neural Edge Enhancer for Supervised Edge Enhancement from Noisy Images

We propose a new edge enhancer based on a modified multilayer neural network, which is called a "neural edge enhancer" (NEE), for enhancing the desired edges clearly from noisy images.  The NEE is a supervised edge enhancer: through training with a set of input noisy images and teaching edges, the NEE acquires the function of a desired edge enhancer.  The input images are synthesized from noiseless images by addition of noise.  The teaching edge images are made from the noiseless images by performing the desired edge enhancer.  To investigate the performance, we carried out experiments to enhance edges from noisy artificial and natural images.  By comparison with conventional edge enhancers, the following was demonstrated: the NEE was robust against noise, was able to enhance continuous edges from noisy images, and was superior to the conventional edge enhancers in terms of the similarity to the desired edges.  Furthermore, we have proposed a method for edge localization for the NEE.  We compared the NEE, together with the proposed edge localization method, with a leading edge detector.  The NEE was proved to be useful for enhancing edges from noisy images.

Noisy input image (left) and an ideal edge image (right). Sobel edge enhancer (left) and Marr-Hildreth edge enhancer (right). Huckel edge enhancer (left) and neural edge enhancer (right). Comparison of the performance of the trained neural edge enhancer for a non-training image with that of conventional edge enhancers.

Analysis of the Neural Edge Enhancer Trained for Edge Enhancement in Noisy Images

A Method for Designing the Optimal Structure of a Neural Filter

We developed a method for designing the structure of a neural filter and a neural edge enhancer.  Because the optimal structure of a neural filter cannot be determined through training, the structure is usually determined empirically.  Thus, redundant units generally exist in a trained neural filter, which results in over-fitting, i.e., the network in a neural filter fits only the specific training data.  This yields a low performance for non-training data.  In addition, redundant units result in a high computational cost.  Therefore, it is important to remove as many redundant units as possible to obtain a high performance for non-training data.  Our method is a sensitivity-based pruning method, i.e., the sensitivity to the training error is calculated when a certain unit is removed experimentally, and the unit with the smallest training error is removed.  Removing the redundant units and retraining to recover any potential performance loss are performed repeatedly, resulting in a reduced network structure after the redundant units are removed.  With our method, redundant units were removed from a trained neural filter without any loss in performance.  The number of units in the neural filter was reduced by 71.6% (126/176), whereas the performance (image quality) for non-training images was maintained.  Thus, our method for designing a neural filter was effective for reducing the units in a trained neural filter.

Error maps showing the generalization ability of the original neural filter (structure: the number of input units �Ethe number of hidden units �Ethe number of output units are 121-50-1, respectively) and that of the neural filter (structure: 5-5-1) obtained by use of our proposed method. Noisy input image (left) and an ideal image without noise (right). Output image of the original neural filter (left) and the output image of the neural filter obtained by use of the proposed method (right). Comparison between the image quality of the output image of the original neural filter (structure: 125-50-1) and that of the neural filter (structure: 40-9-1) obtained by use of the proposed method.

Efficient Approximation of Neural Filters for Removing Quantum Noise from Images

We have developed efficient filters that approximate neural filters that are trained to remove quantum noise from images.  We have proposed novel analysis method for making clear the characteristics of the trained neural filter.  In the proposed analysis method, an unknown nonlinear deterministic system with plural inputs such as a trained neural filter can be analyzed by use of its outputs when the specific input signals are input to it.  Experiments on the neural filters trained to remove quantum noise from medical and natural images were performed.  The results demonstrated that the approximate filters, which were obtained by use of the results of the analysis, were sufficient for approximation of the trained neural filters and efficient computationally cost.

Architecture of a spatiotemporal neural filter (left) and its approximate filter (right) obtained by use of our proposed method.   Comparison of the performance of the neural filter with that of the approximate filter.

Determining the Receptive Field of a Neural Filter

A method for determining the receptive field and the structure of hidden layers of a neural filter (NF) was developed and evaluated.  With the proposed method, redundant units are removed from input and hidden layers in a neural filter based on the influence of the removal of units on the error between output and teaching images.  By performing the removal of units and retraining for recovery of the loss of the removal repeatedly, the receptive field and a reduced structure of hidden layers are determined.  Experiments with neural filters were performed for acquiring the function of a known filter, for the reduction of noise in natural images, and for the reduction of noise in medical image sequences.  By use of the proposed method, redundant units were able to be removed from the neural filters while the performance of the neural filters was maintained.  Experimental results suggested that, with the proposed method, a reasonable receptive field for a given image-processing task could be determined, i.e., the receptive field of the neural filter trained to acquire the function of a Laplacian filter corresponded to the exact kernel of the filter, and the neurons in receptive field of the neural filter for noise reduction gathered around the center pixel in the input region of the neural filter.

The receptive field obtained by use of the proposed method for the neural filter trained to acquire the function of a Laplacian filter. Comparison of the receptive field obtained by use of the conventional method with that obtained by use of the proposed method for the neural filter for the reduction of noise in images.

↑ Go to top

Pattern Recognition

Linear-Time Connected-Component Labeling Based on Sequential Local Operations

Labeling of connected components in binary images is one of the most fundamental operations in pattern recognition.  Labeling is indispensable whenever a computer needs to recognize objects (connected components) in an image as independent objects.  The major problem of labeling algorithms is that they are time-consuming.  A labeling algorithm is generally more time-consuming than any other fundamental image-processing and pattern-recognition operations.  It often prevents a pattern-recognition system from application to real-time processing.  This is called a "bottleneck" problem of labeling in the field of pattern recognition.  The time required for labeling depends on the order of the square of the number of pixels in an image; thus, the time problem is more serious when images are larger.  To overcome this problem, we developed a linear-time algorithm for labeling connected components in binary images based on sequential local operations.  A one-dimensional table, which memorizes label equivalences, is used for uniting equivalent labels successively during the operations in forward and backward raster directions.  The proposed algorithm has a desirable characteristic: the execution time is directly proportional to the number of pixels in connected components in an image.  By comparative evaluation, it was shown that the efficiency of the proposed algorithm was superior to that of the conventional labeling algorithms.

Comparison of the execution time against the number of pixels in connected components. Comparison of the execution time against the number of pixels in an image.

 ↑ Go to top