|Year : 2020 | Volume
| Issue : 4 | Page : 780-787
Cause determination of missed lung nodules and impact of reader training and education: Simulation study with nodule insertion software
Subba R Digumarthy1, Roberto Lo Gullo2, Marie-Helene Levesque3, Karl Sayegh4, Sishir Rao1, Scott B Raymond1, Alexi Otrakji1, Mannudeep K Kalra1
1 Department of Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
2 Department of Radiology, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA; Department of Radiology, School of Medicine, University of Milano, Milan, Italy
3 Institut Universitaire de cardiologie et de Pneumologie de Quebec, Canada
4 Department of Radiology, McGill University, Montreal, Canada
|Date of Submission||30-Mar-2017|
|Date of Acceptance||21-May-2018|
|Date of Web Publication||30-Oct-2018|
Subba R Digumarthy
Department of Radiology, Massachusetts General Hospital, 75 Blossom Court, Unit#230, Boston, MA 02114
Source of Support: None, Conflict of Interest: None
Background: There are “blind spots” on chest computed tomography (CT) where pulmonary nodules can easily be overlooked. The number of missed pulmonary nodules can be minimized by instituting a training program with particular focus on the depiction of nodules at blind spots.
Purpose: The purpose of this study was to assess the variation in lung nodule detection in chest CT based on location, attenuation characteristics, and reader experience.
Materials and Methods: We selected 18 noncalcified lung nodules (6–8 mm) suspicious of primary and metastatic lung cancer with solid (n = 7), pure ground-glass (6), and part-solid ground-glass (5) attenuation from 12 chest CT scans. These nodules were randomly inserted in chest CT of 34 patients in lung hila, 1st costochondral junction, branching vessels, paramediastinal lungs, lung apices, juxta-diaphragm, and middle and outer thirds of the lungs. Two residents and two chest imaging clinical fellows evaluated the CT images twice, over a 4-month interval. Before the second reading session, the readers were trained and made aware of the potential blind spots. Chi-square test was used to assess statistical significance.
Results: Pretraining session: Fellows detected significantly more part-solid ground-glass nodules compared to residents (P = 0.008). A substantial number of nodules adjacent to branching vessels and posterior mediastinum were missed. Posttraining session: There was a significant increase in detectability independent of attenuation and location of nodules for all readers (P < 0.0008).
Conclusion: Dedicated chest CT training improves detection of lung nodules, especially the part-solid ground-glass nodules. Detection of nodules adjacent to branching vessels and the posterior mediastinal lungs is difficult even for fellowship-trained radiologists.
Keywords: Blind spots, chest, computed tomography, nodules, training
|How to cite this article:|
Digumarthy SR, Gullo RL, Levesque MH, Sayegh K, Rao S, Raymond SB, Otrakji A, Kalra MK. Cause determination of missed lung nodules and impact of reader training and education: Simulation study with nodule insertion software. J Can Res Ther 2020;16:780-7
|How to cite this URL:|
Digumarthy SR, Gullo RL, Levesque MH, Sayegh K, Rao S, Raymond SB, Otrakji A, Kalra MK. Cause determination of missed lung nodules and impact of reader training and education: Simulation study with nodule insertion software. J Can Res Ther [serial online] 2020 [cited 2020 Sep 23];16:780-7. Available from: http://www.cancerjournal.net/text.asp?2020/16/4/780/244451
| > Introduction|| |
Computed tomography (CT) is the imaging modality of choice for detection and follow-up of lung nodules. Lung nodules identified on CT are an important clinical concern because they may prompt follow-up imaging or interventions such as biopsy or surgery. In patients with extrapulmonary malignancy, detection of lung nodules on CT may also result in change of therapeutic options and prognosis. With the current adoption of lung cancer screening programs, using low-radiation dose chest CT, accurate detection of lung nodules is even more relevant. The nodule density and size are important determinants of management in LungRADS™ categories.
Despite the critical relevance of lung nodules, they can be overlooked on chest radiograph,,, as well as routine or low-dose chest CT.,,, With the implementation of low-dose CT for lung cancer screening, the number of errors in detection of lung nodules as well as lung cancer can potentially increase. Due to high morbidity and mortality, missed lung cancers are among one of the common causes for malpractice claims in radiology.,
Errors in CT diagnosis of lung nodule and cancer fall under the following three categories: (1) False-positive diagnosis: misinterpretation of a normal structure as a lung nodule, often due to partial volume averaging or lack of awareness of normal anatomic structures. An example is misperception of the lowermost portion of the first costochondral junction as a pulmonary nodule. (2) False-negative (FN) diagnosis: Failure to detect an abnormality that is readily visible in retrospect. This can occur from lack of experience, loss of attention, inadequate training, or because of the challenging location of a lesion; for example, when a nodule is located at the branching site of pulmonary vessels. A nodule may also be less conspicuous due to the lack of contrast between the nodule and background and small size, such as small pure ground-glass or part-solid ground-glass nodule. Another cause for missed lung cancer is related to “satisfaction of search:” when detection of an abnormality diverts the attention of the radiologist from other potentially malignant lesion. (3) Decision-making error: A common example is misinterpretation of lung cancer as a benign process. It is also well established that some lung nodules are more likely to be malignant based on their imaging features. For example, part-solid ground-glass nodules and nodules >1 cm have higher potential for malignancy. These errors may be unique to a reader, but are more commonly universal. Of all the causes listed above, FN diagnosis due to missed nodule is the most worrisome and perhaps easy to fix by focused training. Self-awareness of these limitations and their causes is the first step in improving the search pattern and optimizing the performance of the radiologist. Therefore, it is imperative to understand the reasons behind a missed pulmonary nodule, before corrective measures can be applied.
It has been well established that interpretation by specialists is more accurate and results in fewer errors. However, it may be impractical for only experts to interpret all chest CT scans. The purpose of our study was to assess the influence of nodule location, nodule density, and training of the readers in the detection of lung nodules.
The interpretation or perception error is important to identify which avoids the pitfalls in the detection of pulmonary nodules and potential lung cancers. Image perception studies have been difficult to perform using clinical images because of the problems associated with obtaining proven abnormalities. We used interactive simulation software that allows seamless removal, archiving, and insertion of abnormal areas from CT images to assess variations in the detection of pulmonary nodules on chest CT based on their location, attenuation characteristics, and reader experience.
| > Materials and Methods|| |
The institutional review board approved the study. The study was compliant with the Health Insurance Portability and Accountability Act (HIPAA). This is a retrospective simulation study with fabricated findings for which informed consent of patients was waived. The authors have no financial disclosure for this study.
Software used for the study
We used Interactive Data Language-based software which allows lesion removal, storage, and re-insertion of lesion in the same or different CT examinations [Figure 1].
|Figure 1: Lesion insertion method: The operator selects a pulmonary nodule from the abnormality library. The lesion can then be added in any desired location of any chest CT study; in this case, the pulmonary nodule was inserted next to the left ventricle|
Click here to view
The software creates a round or oval-shaped mask around the selected target lesion. This mask incorporates the lesion as well as part of normal surrounding tissue in both transverse and sagittal directions. The mask enables “lesion removal” and archiving in an “abnormality library.” These archived lesions can then be inserted into any CT image or examination from the same patient or from a different patient. The software matches the attenuation density of the lung parenchyma surrounding the lesion of the original scan to that of the scan in which the lesion is transplanted. The user can control position, size, and orientation of the transplanted lesion.
Our study included chest CT scans of 46 patients, divided into two groups.
Group 1, with real pulmonary nodules
Two radiologists identified 18 nodules (7 solid, 6 pure ground glass, and 5 part-solid ground glass) in chest CT scans of 12 patients (3 males and 9 females; average age: 64.6 years; range: 32–83 years). The scans were selected from radiology reports using an online, intranet-based, radiology report archive and search engine and then viewed on a Picture Archiving and Communication Systems workstation to determine their suitability.
The inclusion criteria for the lung nodules were as follows: nodule size between 6 and 8 mm, oval or round shape, different attenuation patterns (pure ground glass, part-solid ground glass, and solid), absence of calcifications, and presence of surrounding aerated lung parenchyma. All selected nodules were suspicious of primary and metastatic lung cancer based on their growth on subsequent follow-up imaging studies. These nodules were removed using our software and archived in the “abnormality library.” The nodules were extracted along with a small portion of the surrounding aerated lung. We excluded nodules <6 mm, calcified nodules, and nodules situated near areas with motion-related artifacts. Chest CT examinations with artifacts (motion or streak) were excluded. All the 12 patients were scanned using routine chest CT protocol of our institution. The scanning reconstruction protocols were 100–120 kV, automatic exposure control technique, 0.9:1 pitch, rotation time of 0.4 s, and 2.5-mm reconstructed section thickness with detailed soft-tissue reconstruction kernel. DICOM® image datasets (2.5 mm) of these CT examinations were exported offline for postprocessing with the software.
Group 2, with inserted nodules
This group comprised chest CT from 34 consecutive adult patients (12 males and 22 females; average age: 56.8 years; range: 23–83 years) without any pulmonary nodules on their native/original CT examination.
The inclusion criteria included lack of pulmonary and mediastinal abnormalities and lack of any image artifacts including motion-related artifacts. The scan and reconstruction parameters for these CT examinations were identical to those described for patient Group 1. All DICOM image series (2.5 mm) were anonymized and exported to an offline workstation for lesion insertion.
A total of 47 pulmonary nodules were inserted in 25 scans: One nodule was inserted in 11 CT scans, two nodules were inserted in 6 CT, and three nodules were inserted in 8 CT scans. No nodules were inserted in 9 chest CT scans.
Lung nodules from the “abnormality library” were inserted into normal CT scans carefully matching the background of lung parenchyma. Pulmonary nodules from the abnormality library were inserted multiple times (2–6 times) in different chest CT examinations. The same nodule was never inserted more than once in the same chest CT.
The nodules were placed in the following locations: Adjacent to the lung hila; behind the first costochondral junction, in proximity to the branching point of the vessels and airways; and near mediastinal organs such as esophagus, azygous vein, azygous arch, superior vena cava, descending aorta/aortic arch, and the heart [Figure 2]. Nodules were also inserted in the middle and outer thirds of the lung parenchyma and in regions that are commonly affected by motion (lung bases in proximity to the diaphragm) or beam-hardening artifacts (lung apices) [Figure 3]. All patient identifiers were removed conforming to HIPAA regulations. The images were viewed on a dedicated viewing station, in lung soft-tissue and bone windows. These image datasets were not part of the patient image archives.
|Figure 2: Examples of inserted pulmonary nodules in proximity to the diaphragm (arrow in a); in the left apex (arrow in b); adjacent to the first costochondral junction (arrow in c); randomly inserted in the peripheral portion of the lung (arrow in d)|
Click here to view
|Figure 3: Examples of inserted pulmonary nodules close to the branching point of pulmonary vasculature (arrow in a); close to the mediastinum at the azygo-esophageal recess (arrow in b); adjacent to the left heart (arrow in c); in the perihilar region (arrow in d)|
Click here to view
Four radiologists, with different levels of training, were recruited as readers. These included two chest imaging clinical fellows in postresidency training positions and two junior radiology residents during their chest imaging rotation.
All image interpretations were performed on a DICOM-compliant image processing workstation. Readers reviewed a total of 34 cases including 25 CT scans with inserted pulmonary nodules and 9 negative chest CT (without nodules or lung opacities), which were randomized in order to avoid the readers from getting accustomed to finding pulmonary nodules in every scan. Readers were blinded to the purpose of the study and were unaware of the nodule removal and insertion software. Interpretation of CT scans was carried out in a calm environment, free of interruption or time pressure. Each radiologist performed independent and blinded evaluation of all the 34 chest CT examinations twice, over a 4-month interval.
Each reader was trained with a demonstration case (not part of the study group) to familiarize with the viewing station. In the first reading session, the participants were asked to review both the soft-tissue window and the lung tissue window for every CT scan. When a pulmonary nodule was identified, the readers were asked to comment on the location and attenuation characteristics of the nodule. No details pertaining to clinical history, physical examination, or radiological findings were shared at the time of image interpretation. The interpretation was performed on different days for each reader and the readers were not aware of the other participants. A second review session was conducted about 4 months after the end of the first session. In the second reading session, the participants were made aware of the purpose of the study and results of the first reading session. In particular, they were aware of the potential blind spots in reference to location and nodule density. In addition, few examples that were selected from the original dataset were also shown. After the brief training session, each radiologist re-evaluated the same CT examinations from the first reading session, in the same randomized order, to detect lung nodules. Only the lung window was reviewed for each examination during the second reading session. The second review session was conducted about 4 months after the end of the first session.
Results from the pre- and the post-training reading sessions were analyzed separately. For each reading session and each reader, we reported the number of true-positive (TP) cases (correctly identified nodules) and FN cases (missed nodules). Sensitivity (TP/[TP + FN]) of pulmonary nodule detection was calculated separately for each session and radiologists. Chi-square test was performed to assess statistical differences between the two reading sessions and the radiologists. P < 0.05 was considered statistically significant.
| > Results|| |
Of the 47 nodules inserted in 25 chest CT, 10 nodules (4 solid, 3 pure ground glass, and 3 part-solid ground glass) were randomly placed in the central areas of the lungs. The other 10 nodules (3 solid, 3 pure ground glass, and 4 part-solid ground glass) were placed in proximity to the branching vessels or airways; 7 nodules (3 solid, 2 pure ground-glass, and 2 part-solid ground glass) were placed anteriorly close to the azygous arch, the superior vena cava, or the heart; 7 nodules (2 solid, 3 pure ground glass, and 2 part-solid ground glass) were placed posteriorly, adjacent to the descending aorta, behind the aortic arch, esophagus, or azygous vein; 13 nodules were transplanted in the periphery of the lung which comprised 4 nodules (1 solid, 2 pure ground glass, and 1 part-solid ground glass) close to the diaphragm, 5 nodules (2 solid, 2 pure ground glass, and 1 part-solid ground glass) behind or immediately inferior to the first costochondral junction, and 4 nodules (2 solid, 1 pure ground glass, and 1 part-solid ground glass) in the lung apices.
Summary of results for individual readers is summarized in [Table 1]. Following training session, there was a significant increase in overall detectability independent of attenuation and location of pulmonary nodules for all readers (P < 0.0008).
|Table 1: Nodule detection and detection error for all readers on the first (pretraining) and second (posttraining) reading sessions|
Click here to view
Detectability of lung nodules based on their attenuation characteristics is summarized in [Table 2]. In general, all readers missed a higher proportion of pure ground-glass nodules as compared to solid and part-solid ground-glass lung nodules [Figure 4] and [Figure 5]. Training resulted in more significant improvement in the detection of part-solid ground-glass attenuation nodules (pretraining sensitivity range: 36%–71% and posttraining sensitivity range: 93%–100%). There was a similar improvement in the detectability of solid (pretraining sensitivity range: 41%–65% and posttraining sensitivity range: 82–88%) and pure ground-glass nodules (pretraining sensitivity range: 12%–38% and posttraining sensitivity range: 81%–88%).
|Table 2: Detectability of lung nodules based on their attenuation characteristics|
Click here to view
|Figure 4: Examples of inserted pulmonary nodules missed by all the readers during the first reading session only|
Click here to view
|Figure 5: Examples of inserted pulmonary nodules missed by residents but not by fellows during the first reading session|
Click here to view
Detection of pulmonary nodules based on their location is summarized in [Table 3]. The most commonly missed nodules were located adjacent to the branching vessels or bronchi as well as posteriorly, near the descending aorta, behind aortic arch, esophagus, and azygous vein [Figure 6]. Nodules randomly distributed outside of these blind spots were least likely to be missed [Table 3]. Following training, there was a significant improvement in the detection of all nodules although detectability of lung nodules at the branching structures remained the lowest for all the readers.
|Figure 6: Examples of inserted pulmonary nodules missed by most readers during both reading sessions|
Click here to view
| > Discussion|| |
CT is the most commonly performed cross-sectional imaging modality in chest imaging. Recent introduction of multidetector row CT (MDCT) has further increased its stature and use in a plethora of clinical indications including detection and follow-up of lung nodules. Low radiation dose techniques available on the modern MDCT now enable acquisition of extremely low radiation dose chest CT for lung cancer screening and follow-up of lung nodules.
Diagnoses missed on imaging are a common problem, representing close to 70% of all medical malpractice lawsuits filed against medical imaging professionals. Missed lung cancer raises an important medicolegal issue and contributes to one of the most common causes for malpractice actions against radiologists. Perceptual errors occur when abnormalities that are readily detectable in retrospect are not detected on the initial interpretation. These errors represent the most common reason for missed diagnosis and account for about 60% of medicolegal cases., Increased use of CT brings an increased risk of missing potentially important lung nodules on chest CT. This is particularly relevant given the fact that modern MDCT commonly generates more than 200 transverse section images of the lungs, along with additional sagittal, coronal, and maximum intensity projection (MIP) images. The information overload from such large chest CT datasets is challenging and increases the risk of missing lesions. Consensus or double readings can help to reduce the number of missed lesions but are labor intensive and impractical in busy radiology practices.,
In a study of diagnostic error conducted by Schiff et al., missed lung cancer was the third-most frequently missed diagnosis, accounting for about 4% of missed lesions among 583 physician-reported cases of diagnostic error (missed pulmonary embolism and failure to report drug reaction or overdose were listed in the first and second places, respectively). CT-related missed lung cancer can occur in either routine chest CT or in the course of a low-dose CT screening.,,,, Missed lung cancer on chest CT represents a mixture of central and peripheral lesions., Smaller pulmonary nodules are more likely to be missed as compared to the larger ones. Missed lung cancers in association with chest CT screening protocols have been reported predominantly in peripheral lungs and in the early stage of the disease.
We found that a significant number of nodules can be missed regardless of the experience of the interpreting radiologists. Although trained radiologists missed fewer nodules, all radiologists improved their performance following a short training session. Prior studies have also reported a high incidence of missed lung nodules. For example, Swensen et al. have reported an incidence of 26% of missed lung nodules at the baseline scan of patients screened for lung cancer with low-dose CT.
Our study emphasizes the role of a training program for detection of lung nodules, with particular focus on depiction of nodules at the conventional “blind spots” on chest CT, which are located in proximity to the branching pulmonary vasculature as well as in the posterior para-mediastinal region of the lungs adjacent to descending aorta, aortic arch, esophagus, and azygous vein. Such program can substantially decrease the incidence of missed lung nodules that can otherwise prove costly for the patients and the interpreting radiologist. We believe that such comprehensive training should address key locations such as the branching sites of pulmonary vessels and airways which have the highest rate of missed diagnosis in the pre- and post-training reading sessions. Regional variations in detectability of lung nodules have also been explained in prior studies.,
We found that pure ground-glass nodules are more likely to be missed as compared to solid and part-solid ground-glass nodules. Li et al. have also reported that 91% of missed nodules in their study were predominantly pure ground glass in attenuation. According to the National Comprehensive Cancer Network practice guidelines, the prevalence of malignancy is 59% for pure ground-glass nodules, 48% for part-solid ground-glass nodules, and 11% for solid nodules. Given the higher prevalence of malignancy in pure ground-glass nodules, a training program should specifically include challenging examples of pure ground-glass nodules. Nodule insertion software such as the one used in our study can be helpful to create training modules of cases. We believe that such training program should become mandatory for radiologists, particularly for those involved in interpreting lung cancer screening CT in high-risk patients.
Naidich et al. have also used a similar experimental model for simulating lung nodules ranging from 1 to 7 mm. They have reported a higher prevalence of missed diagnosis for small lung nodules (1.5 and 3 mm) as compared to the larger nodules (4, 5, and 7 mm).
Other known ways to reduce diagnostic detection error include the use of computer-aided diagnosis (CAD) and MIP images. The use of CAD has been successfully applied to lung nodule detection.,, The consistency of a CAD system combined with the knowledge and experience of the human observer can have a positive impact on detection accuracy. Likewise, MIP images (8–10 mm thickness) can also increase the detectability of small (<10 mm) lung nodules, particularly those in the inner third of the lung.
Limitations of our study include small sample size, although we used multiple readers to compensate for this limitation. We focused our attention exclusively on reducing perception error and did not consider classification error. However, even though errors in classification and appropriate patient management are important, they are not relevant if a lesion is missed in the first place. Furthermore, during the second reading session, each reader was aware of the purpose of the study and of their misses in the prior reading session. The readers focused on the areas that were not included in their initial search pattern.
We inserted more nodules in difficult locations, which may not reflect the true distribution of lung cancers in clinical practice. This may explain the high number of missed nodules in our study. This was deliberately done to assess the impact of training, as part of our study design.
The recipient chest CT scans were normal, except for inserted nodules, to limit confounding factors such as satisfaction of search. This was done to simulate the reader experience while interpreting lung cancer screening scans in asymptomatic patients.
Our study focused exclusively on pulmonary nodules, between 6 and 8 mm in size, these nodules require a closer follow-up screening CT scan, according to the LungRADS™ categories. We have not selected nodules that are not significant according to the Fleischner Society recommendations.
Detection of part-solid ground-glass and pure ground-glass nodules was better for the chest imaging fellows compared to the residents (P = 0.008); this implies that dedicated chest CT training is helpful in identifying more concerning lung nodules; however, junior residents also significantly improved the detection of pulmonary nodules after education, proving that there are benefits related to focused training and self-awareness.
| > Conclusion|| |
It is important to be aware of the pitfalls of lung cancer diagnosis on chest CT and their medicolegal ramifications. A variety of strategies can be directed toward minimizing the incidence of missed lung cancer. Errors, especially related to lack of experience, can be reduced by instituting a training program for detection of lung nodules with particular focus on the depiction of nodules at conventional “blind spots.” This training program could be valuable, especially for radiology residents and practicing radiologists, particularly those involved in the interpretation of lung cancer screening examinations.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| > References|| |
Shah PK, Austin JH, White CS, Patel P, Haramati LB, Pearson GD, et al.
Missed non-small cell lung cancer: Radiographic findings of potentially resectable lesions evident only in retrospect. Radiology 2003;226:235-41.
Woodring JH. Pitfalls in the radiologic diagnosis of lung cancer. AJR Am J Roentgenol 1990;154:1165-75.
White CS, Salis AI, Meyer CA. Missed lung cancer on chest radiography and computed tomography: Imaging and medicolegal issues. J Thorac Imaging 1999;14:63-8.
Fardanesh M, White C. Missed lung cancer on chest radiography and computed tomography. Semin Ultrasound CT MR 2012;33:280-7.
Li F, Sone S, Abe H, MacMahon H, Armato SG 3rd
, Doi K, et al.
Lung cancers missed at low-dose helical CT screening in a general population: Comparison of clinical, histopathologic, and imaging findings. Radiology 2002;225:673-83.
White CS. Pitfalls in lung cancer assessment with CT. J Thorac Imaging 2004;19:32-4.
Berlin L. Malpractice issues in radiology. Perceptual errors. AJR Am J Roentgenol 1996;167:587-90.
Berbaum KS, Schartz KM, Caldwell RT, Madsen MT, Thompson BH, Mullan BF, et al.
Satisfaction of search from detection of pulmonary nodules in computed tomography of the chest. Acad Radiol 2013;20:194-201.
McWilliams A, Tammemagi MC, Mayo JR, Roberts H, Liu G, Soghrati K, et al.
Probability of cancer in pulmonary nodules detected on first screening CT. N
Engl J Med 2013;369:910-9.
European Society of Gastrointestinal and Abdominal Radiology CT Colonography Group Investigators. Effect of directed training on reader performance for CT colonography: Multicenter study. Radiology 2007;242:152-61.
Madsen MT, Berbaum KS, Ellingson AN, Thompson BH, Mullan BF, Caldwell RT, et al.
Anew software tool for removing, storing, and adding abnormalities to medical images for perception research studies. Acad Radiol 2006;13:305-12.
Berlin L, Berlin JW. Malpractice and radiologists in cook county, IL: Trends in 20 years of litigation. AJR Am J Roentgenol 1995;165:781-8.
Renfrew DL, Franken EA Jr., Berbaum KS, Weigelt FH, Abu-Yousef MM. Error in radiology: Classification and lessons in 182 cases presented at a problem case conference. Radiology 1992;183:145-50.
Brown J, Bryan S, Warren R. Mammography screening: An incremental cost effectiveness analysis of double versus single reading of mammograms. BMJ 1996;312:809-12.
Gollub MJ, Panicek DM, Bach AM, Penalver A, Castellino RA. Clinical importance of reinterpretation of body CT scans obtained elsewhere in patients referred for care at a tertiary cancer center. Radiology 1999;210:109-12.
Schiff GD, Hasan O, Kim S, Abrams R, Cosby K, Lambert BL, et al.
Diagnostic error in medicine: Analysis of 583 physician-reported errors. Arch Intern Med 2009;169:1881-7.
Gurney JW. Missed lung cancer at CT: Imaging findings in nine patients. Radiology 1996;199:117-22.
White CS, Romney BM, Mason AC, Austin JH, Miller BH, Protopapas Z, et al.
Primary carcinoma of the lung overlooked at CT: Analysis of findings in 14 patients. Radiology 1996;199:109-15.
Kakinuma R, Ohmatsu H, Kaneko M, Eguchi K, Naruke T, Nagai K, et al.
Detection failures in spiral CT screening for lung cancer: Analysis of CT findings. Radiology 1999;212:61-6.
Mancini J, Feragalli B, Ciccotosto C, Storto ML, Guidotti A. Possibility of false negative results in the diagnosis of lung neoplasms with computerized tomography. Radiol Med 1999;97:479-85.
Swensen SJ, Jett JR, Sloan JA, Midthun DE, Hartman TE, Sykes AM, et al.
Screening for lung cancer with low-dose spiral computed tomography. Am J Respir Crit Care Med 2002;165:508-13.
Naidich DP, Rusinek H, McGuinness G, Leitman B, McCauley DI, Henschke CI, et al.
Variables affecting pulmonary nodule detection with computed tomography: Evaluation with three-dimensional computer simulation. J Thorac Imaging 1993;8:291-9.
MacMahon H, Engelmann R, Behlen FM, Hoffmann KR, Ishida T, Roe C, et al.
Computer-aided diagnosis of pulmonary nodules: Results of a large-scale observer test. Radiology 1999;213:723-6.
Armato SG 3rd
, Li F, Giger ML, MacMahon H, Sone S, Doi K, et al.
Lung cancer: Performance of automated lung nodule detection applied to cancers missed in a CT screening program. Radiology 2002;225:685-92.
Abe H, MacMahon H, Engelmann R, Li Q, Shiraishi J, Katsuragawa S, et al.
Computer-aided diagnosis in chest radiography: Results of large-scale observer tests at the 1996-2001 RSNA scientific assemblies. Radiographics 2003;23:255-65.
Gruden JF, Ouanounou S, Tigges S, Norris SD, Klausner TS. Incremental benefit of maximum-intensity-projection images on observer detection of small pulmonary nodules revealed by multidetector CT. AJR Am J Roentgenol 2002;179:149-57.
[Figure 1], [Figure 2], [Figure 3], [Figure 4], [Figure 5], [Figure 6]
[Table 1], [Table 2], [Table 3]