Open Access
Add to BibSonomyAbstract
Two hundred eighty three uniaxial ellipsoids with sizes from 4 mm to 11 mm were measured with a coordinate measuring matching (CMM) and also scanned using a medical computed tomography (CT) machine. Their volumes were determined by counting voxels over a threshold, as well as using equivalent volumes from the length given by the RECIST 1.1 criterion (Response Evaluation Criteria in Solid Tumors). The volumetric measurements yield an order of magnitude reduction in residuals compared to the CMM measurements than the residuals of the RECIST measurements also compared to the CMM measurements.
©2010 Optical Society of America
Introduction
In the diagnosis of cancer, it is a very common medical practice to perform a computed tomography (CT) scan at a given time and a second one perhaps six months later. If a nodule appears larger in the second scan, it is possibly cancerous, and frequently it will be biopsied. The principal reason for waiting six months is to make sure that the nodule has grown sufficiently to exceed the variation of measurement in the system. It would be beneficial to patents, doctors, and pharmaceutical companies alike if this time could be reduced in a reliable way.
There are standard methods for determining the change of size of nodules. Perhaps the most important of these is the Response Evaluation Criteria in Solid Tumors (RECIST). Guidelines for RECIST 1.1 have been issued recently [1]. The key feature of RECIST is that the size of the nodule is given as the largest distance contained within any slice (i.e., a two-dimensional image) of the nodule. The criterion is very practical in the long-standing practice in which a radiologist examines the reconstruction slicewise, as was done with film. As more data are acquired at the CT, and computing technology is more able to handle large amounts of data, it becomes a pressing practical question as to whether the standard practice should remain so.
The RECIST 1.1 guidelines recommend only applying it for nodules of at least 10 mm. There has been increasing interest in using volumetric methods to quantify nodules half that size. Issues in lung volumetry have recently been reviewed [2].
In this paper, we explore the implications of using the RECIST criterion and volumetry on well-characterized reference objects down to about half of the linear size recommended by the RECIST 1.1 guidelines. The size range is of particular interest to the Early Lung Cancer Action Project [3–5] (ELCAP) which follows nodules of at least 5 mm diameter [6], and to the National Lung Screening Trial [7] (NLST) and the NELSON study [8] which follow nodules of at least 4 mm in diameter. Similarly, the Fleischner Society classified nodules into four categories, with different treatment recommendations depending on whether the nodule size exceeded 4 mm, 6 mm, or 8 mm in the average of their length and width [9].
Methods and Materials
Samples were prepared from digital representations of uniaxial ellipsoids with nine values of the volume and nine values for the ratio of the unique diameter to the diameter of the circular cross section at right angles to the unique axis, for a total of 81 design classes. The ratio of the unique diameter to the circular diameter is called the “elliptic ratio” in this paper. To change volume with a constant elliptic ratio, the design lengths were each multiplied by a factor of 1.05. To change the elliptic ratio at a constant volume, the circular diameter was varied by a factor of 1.05, and the unique diameter was varied by a factor of 1.05−2. The central design class was a sphere with 6.350 mm diameter. The 9 spheres ranged in diameter from 5.224 mm to 7.718 mm. The elliptic ratio ranged from 0.56 (oblate) through 1.00 (spherical) to 1.80 (prolate). The minimum design dimension was 4.298 mm and the maximum dimension was 11.404 mm. The samples were produced using a Z Corporation model 510 3D printer (Burlington, MA, USA) at the National Institute of Standards and Technology (NIST). (Commercial products or services are not recommended or endorsed by the authors or their institutions. They may not be the best available for the purpose.) The material is a proprietary mixture of resins and silicates. Binder was applied under program control, leading to a hard shell and a somewhat less dense central region. After production by the printer, unbound powder was blown away. Then the ellipsoids were dipped in epoxy and shaken on a screen for fifteen minutes during the early phase of drying to ensure a uniform coat. From a total of 324 ellipsoids (four for each of the 81 design classes) 314 were deemed to have a good shape, based on a visual inspection. These 314 ellipsoids were placed in the seams of blocks of four polyurethane foam cubes then placed in three compartmented plastic boxes. (The foam cubes are part 3157T21 and the boxes are part 4629T34; both are from McMaster-Carr, Robbinsville, NJ, USA.) The ellipsoids and the foam cubes were held by the pressure of the foam which had to be squeezed slightly to be placed into the compartments. The boxes were taped into a stack, yielding a 6x3x6 array of compartments, and scanned together.
The samples were brought to a Sensation 64 CT (Siemens, Malvern, PA, USA) operated at the Veterans Administration Hospital in Baltimore, MD, USA. The CT was usually used for patients. The measurements were made on February 24, 2009 after the daily calibration, but before any patients were scanned that day. The boxes were taped into a stack, yielding a 6x3x6 array of compartments, and scanned together.
We chose operating parameters which are typical of lung cancer studies, including reconstruction with the sharp lung kernel, designated by the manufacturer as B80f within the syngo CT 2006A software. In the base case, we had an in-plane pixel size of 0.631 mm, with 512 x 512 pixels in each slice, a slice thickness of 0.6 mm, with 313 slices excluding empty regions, a primary electron energy of 120 keV (i.e., 120 kVp), a dosage due to a current-time product of 80 mAs from cathode to anode, a pitch of 0.9, and gantry rotation in the clockwise direction. Only a single parameter was changed from these values for a given run during this study.
The reconstructions were segmented using simple thresholding using a custom code written in IDL 6.4. The reconstruction was not filtered. A single threshold value of 76 HU (Hounsfield Units [10]) was applied to each reconstruction. The value chosen was nearly the maximum which permitted all of the interior regions to be included, yet made it relatively easy to exclude background regions such as the compartments of the plastic boxes. The volume of any given background region was at least a factor of two smaller than the smallest ellipsoid. The ellipsoids typically had peak radiodensities of 2000 HU; the foam background was typically below −900 HU. The segmented image was automatically labeled using the intrinsic LABEL_REGION routine in the IDL software language (ITT, White Plains, NY, USA). This strategy is similar to that employed recently in a study of the volumes of spheres in CT [11]. Volumes were reported by counting pixels within each region. Similarly, the RECIST criterion was implemented slicewise on these segmented regions. The position of the centroid of each region was determined, which played a key role in identifying the individual ellipsoids. The six second moments (x2, y2, z2, xy, yz, xz) about the centroid were found for each ellipsoid. The region volumes and moments were found efficiently in single sweep through the data by incrementing the region of a given voxel with the factor appropriate for its position. From these moments, for each region a tensor similar to the moment-of-inertia tensor was found, and diagonalized. The three diagonal elements were reduced to two by taking the harmonic mean of the two closest values as the circular value and the other value as the unique value. Before the elliptic ratio was found, the square root was taken to obtain quantities proportional to length.
Reference volumes were found as follows: Each (nominal) ellipsoid was measured individually on a coordinate measuring machine (CMM) at NIST. A minimum of 81 points per ellipsoid were probed. Each set of points was then mathematically fit to a uniaxial ellipsoid (in a least-squares sense) to determine axes lengths. These calculated axes lengths were then used to calculate a reference volume for each ellipsoid. Fitting to a general ellipsoid, as was done for several ellipsoids as a test, did not change the results within uncertainties.
Uncertainty evaluation included analyzing the form of the ellipsoids to determine the systematic departure from a nominal uniaxial ellipsoid. Simulations were also performed based on the number of points and the sampling strategy used. The expanded uncertainty (k = 2) for reference volumes was evaluated to be 2%, meaning that the reference volumes are accurate to within 2% at a 95% confidence level for 283 ellipsoids. (Regrettably, the other 31 measurements were not usable due to a failure in recording our data). The expanded uncertainty is also 2% for the elliptic ratios.
Results
First, we consider how well the widely used RECIST criterion does in determining the sizes of these objects. Technically, under the rules of RECIST 1.1 objects must be at least 10 mm in size, but we ignore this for the present. We implemented the RECIST criterion by finding the maximum distance between any two pixels within the same slice of a given region. These values are compared to those of the CMM, which are accurate to within 2%. As may be seen in Fig. 1 , the RECIST values are only loosely correlated with the CMM values. We provide Table 1 as a reference for readers who are more conversant with diameters than volumes as a measure of size. The CMM dimensions for each ellipsoid are given in Media 1. Here, column A is an ellipsoid ID, common to all the files of Media 1 through , column B is the circular diameter of the ellipsoid (mm), column C is the unique diameter of the ellipsoid (mm), and column D is the volume of the ellipsoid (mm3).
Fig. 1 Size of ellipsoids a medical CT using the RECIST criterion vs. volume measured with a coordinate-measuring machine for the “base case” with parameters given in the text. A linear fit to the logarithm is shown, along with lines with the same slope with 2.5% of the residuals above the upper line and another 2.5% of the residuals below the lower line. A power law is found with an exponent of 0.31. The upper line is above the central fit by a factor of 1.252 and the lower line is below the central fit by a factor of 0.786.
Table 1. Volumes V of spheres as a function of diameter d, according to V = (π/6)d3.
Because the RECIST criterion is a linear measure and volume is proportional to a length cubed, we present a log-log plot of their relation. The exponent of the best fit is 0.31 which is close to the value of 1/3 expected for a linear measure given as a function of volume. The other lines shown on the plot are a factor of 1.252 and 0.786 of the central fit and contain 95% of the data.
Next, the volume obtained from the base case is plotted against the volume obtained by the CMM in Fig. 2 . The fit is much better than that shown in Fig. 1, with a coefficient of correlation of 0.989 compared to 0.716 for the RECIST data of Fig. 1.
Fig. 2 Volume measured in a medical CT vs. volume measured with a coordinate-measuring machine for the “base case” with parameters given in the text. A linear fit is shown, along with lines with the same slope with 2.5% of the residuals above the upper line and another 2.5% of the residuals below the lower line. Slope and standard deviation are reported in Table 2.
The elliptic ratio, defined here as the ratio of the unique diameter to the circular diameter for the uniaxial ellipsoid, is shown in Fig. 3 . The shape parameter is reasonably well accounted for, as seen by the relatively small residuals in the figure, and also given in Table 2 .
Fig. 3 Elliptic ratio measured in a medical CT vs. elliptic ratio measured with a coordinate-measuring machine for the “base case” with parameters given in the text. Confidence intervals as in Fig. 1. A linear fit is shown, along with lines with the same slope with 2.5% of the residuals above the upper line and another 2.5% of the residuals below the lower line. Slope and standard deviation are reported in Table 2.
Table 2. Slope of volume measured in a medical CT vs. volume measured with a coordinate-measuring machine (CMM), slope of elliptic ratio in a medical CT vs. elliptic ratio measured with a CMM, and the standard deviation of the residuals for both results. Rotation 1 and rotation 2 are two distinct 90° rotations relative to the base case and relative to each other. Volume measurements are used except as noted. View 1 is an annotated screenshot of the ellipsoid sample. Links to the original scans are presented as View 2 through View 11. Media 2 through Media 11 contain an ellipsoid ID in column A and the centroids of the ellipsoids (x, y, z, in mm) in columns B, C, and D.
The sample was rescanned to give a sense of the intrinsic variation of the measured parameters. We also varied the primary beam energy from 120 keV to 80 keV (i.e., 120 kVp to 80 kVp), the slice thickness, the dosage, and we rotated the orientation of the three boxes of samples. We found the volumes of the ellipsoids with 90 degree rotations about two different axes reproduced the base scan as well as simply rescanning without reorienting the sample. This comparison was performed with the near-isotropic resolution 0.63 mm x 0.63 mm x 0.6 mm.
The most significant trend which emerges from these studies is that with a larger slice thickness, it becomes increasingly difficult to obtain reliable shapes for the ellipsoids, even if the volume is still reasonably well accounted for. The 1.5 mm slice thickness does show a small, but significant, increase in the standard deviation of the residuals compared to the base case of 0.6 mm. The results for the 1.5 mm scan are seen in Fig. 4 for the volume and in Fig. 5 for the elliptic ratio, both compared to the CMM measurements. While the fit to the CMM volume is only somewhat worse, the elliptic ratio is seen to be much worse, particularly for the prolate ellipsoids.
Fig. 4 Volume measured in a medical CT vs. volume measured with a coordinate-measuring machine for the case of 1.5 mm slice thickness with parameters given in the text. A linear fit is shown, along with lines with the same slope with 2.5% of the residuals above the upper line and another 2.5% of the residuals below the lower line. Slope and standard deviation are reported in Table 2
Fig. 5 Elliptic ratio measured in a medical CT vs. elliptic ratio measured with a coordinate-measuring machine for the case of 1.5 mm slice thickness with parameters given in the text. Confidence intervals as in Fig. 1. A linear fit is shown, along with lines with the same slope with 2.5% of the residuals above the upper line and another 2.5% of the residuals below the lower line. Slope and standard deviation are reported in Table 2.
The 1.0 mm slice thickness case is within the range of repeated scans with 0.6 mm and therefore is not a significant change. Another sign of the deterioration of performance with increasing slice thickness is that the number of ellipsoids detected fell from 314 of 314 with an 0.6 mm slice thickness, to 312 (99%) with a 1.0 mm slice thickness, and again to 308 (97%) with a 1.5 mm slice thickness.
Discussion
Volumetric CT measurements can distinguish at the 95% confidence level uniaxial ellipsoids with volumes in the range of 80 mm3 to 300 mm3 with a high-contrast to their backgrounds whose volumes differ by about 30 mm3. The RECIST criterion, which is not formally applicable to such small objects, has little ability to distinguish objects in the whole size range.
The RECIST guidelines state that a size increase of 20% or a size decrease of 30% are significant. These factors are close to the values of a factor of 1.25 (25% increase) or 0.79 (21% decrease) reported as our confidence limits in Fig. 1. Assuming the size changes recommended by RECIST for a minimally significant change ( + 20% or −30%) are related to a scaling of the volume, the corresponding volume changes are −65% and + 73%. Using volumetric methods, taking the standard deviation be the average of the 8 standard deviations given in Table 2 corresponding to the 0.6 mm slice thickness, namely 8.12 mm3, volume changes of −65% and + 73% correspond to changes of −19.3 and + 21.7 standard deviations.
Earlier, it had been determined that volumetric CT measurements were superior to examining the maximum in-plane cross section in a study of nylon or acrylic balls 1 mm to 9.5 mm in diameter [11]. As slice thickness increases, the ability to estimate the aspect ratio of the ellipsoids deteriorates faster than does the ability to estimate volume.
Although we have created an idealized setting for measurement, it is likely that the results presented here represent a lower bound to accuracies which may be achieved in clinical practice. The deterioration in the ability to obtain the quantitative shape of an ellipsoid could, in clinical practice, translate into a misclassification of the type of lesion, e.g., whether it is speculated, which, like the elliptic ratio, is a property of the shape of the lesion. Thin slices, e.g., 0.6 mm but not 1.5 mm, are required to obtain faithful shapes in this size range, although 1.5 mm slices are adequate for an estimate of the volume.
The voxels in the base case also give a good account of the shape of the ellipsoids, as shown by the elliptic ratios, plotted in Fig. 3. The slope of the elliptic ratio is slightly less than one which implies that the ellipsoids appear to be more spherical than they are. This phenomenon is probably due to the convolution of the point spread function with the objects. As a model, imagine the objects were 3D uniaxial Gaussians with length scales given by a. and b with a associated with the unique axis, i.e., the elliptic ratio is a/b. If the point spread function is given by a Gaussian with a length scale of c, the imaged Gaussians will have length scales of (a 2 + c 2)1/2 and (b 2 + c 2)1/2 and the imaged elliptic ratio is (a 2 + c 2)1/2/ (b 2 + c 2)1/2 which is closer to 1 than a/b regardless of whether the Gaussian is prolate with a>b or oblate with a<b.
Volumetric CT can distinguish the volumes of objects with diameters of about 5 mm to 10 mm dramatically better than the RECIST criterion. Some form of volumetric analysis will be necessary to evaluate changes in the size of lesions with diameters of 5 mm to 10 mm.
Acknowledgments
We are grateful for discussions with and technical assistance from Uwe Arp, Charles Fenimore, Steven Grantham, Jeff Gunn, Lisa Karam, Anthony Reeves, Daniel Sawyer IV, David Yankelevitz, and Terry Yoo.
References and links
1. E. A. Eisenhauer, P. Therasse, J. Bogaerts, L. H. Schwartz, D. Sargent, R. Ford, J. Dancey, S. Arbuck, S. Gwyther, M. Mooney, L. Rubinstein, L. Shankar, L. Dodd, R. Kaplan, D. Lacombe, J. Verweij, and J. Verweij, “New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1),” Eur. J. Cancer 45(2), 228–247 (2009). [CrossRef]
2. M. A. Gavrielides, L. M. Kinnard, K. J. Myers, and N. Petrick, “Noncalcified lung nodules: volumetric assessment with thoracic CT,” Radiology 251(1), 26–37 (2009). [CrossRef] [PubMed]
3. C. I. Henschke, D. I. McCauley, D. F. Yankelevitz, D. P. Naidich, G. McGuinness, O. S. Miettinen, D. M. Libby, M. W. Pasmantier, J. Koizumi, N. K. Altorki, and J. P. Smith, “Early Lung Cancer Action Project: overall design and findings from baseline screening,” Lancet 354(9173), 99–105 (1999). [CrossRef] [PubMed]
4. C. I. Henschke and D. F. Yankelevitz, “CT screening for lung cancer: update 2007,” Oncologist 13(1), 65–78 (2008).<jrn[REMOVED IF= FIELD]></jrn> [CrossRef] [PubMed]
5. C. I. Henschke and D. F. Yankelevitz, “Erratum for ‘CT screening for lung cancer: Update 2007,’,” Oncologist 13(5), 619 (2008) (</jrn>).<jrn[REMOVED IF= FIELD]> [CrossRef]
6. A. P. Reeves, A. B. Chan, D. F. Yankelevitz, C. I. Henschke, B. Kressler, and W. J. Kostis, “On measuring the change in size of pulmonary nodules,” IEEE Trans. Med. Imaging 25(4), 435–450 (2006). [CrossRef] [PubMed]
7. K. W. Clark, D. S. Gierada, S. M. Moore, D. R. Maffitt, P. Koppel, S. R. Phillips, and F. W. Prior, “Creation of a CT Image Library for the Lung Screening Study of the National Lung Screening Trial,” J. Digit. Imaging 20(1), 23–31 (2007). [CrossRef]
8. D. M. Xu, H. J. van der Zaag-Loonen, M. Oudkerk, Y. Wang, R. Vliegenthart, E. T. Scholten, J. Verschakelen, M. Prokop, H. J. de Koning, and R. J. van Klaveren, “Smooth or attached solid indeterminate nodules detected at baseline CT screening in the NELSON study: cancer risk during 1 year of follow-up,” Radiology 250(1), 264–272 (2009). [CrossRef]
9. H. MacMahon, J. H. M. Austin, G. Gamsu, C. J. Herold, J. R. Jett, D. P. Naidich, E. F. Patz Jr, and S. J. Swensen, “Guidelines for management of small pulmonary nodules detected on CT scans: a statement from the Fleischner Society,” Radiology 237(2), 395–400 (2005). [CrossRef] [PubMed]
10. G. N. Hounsfield, “Nobel Award address: Computed medical imaging,” Med. Phys. 7(4), 283–290 (1980). [CrossRef] [PubMed]
11. N. D. Prionas, S. Ray, and J. M. Boone, “Volume Assessment Accuracy in Computed Tomography: A Phantom Study,” unpublished.
