Two step convolutional neural network for automatic glottis localization and segmentation in stroboscopic videos

Varun Belagali; Achuth Rao M V; Pebbili Gopikishore; Rahul Krishnamurthy; Prasanta Kumar Ghosh

doi:10.1364/BOE.396252

Biomedical Optics Express
Vol. 11,
Issue 8,
pp. 4695-4713
(2020)
•https://doi.org/10.1364/BOE.396252

Two step convolutional neural network for automatic glottis localization and segmentation in stroboscopic videos

Varun Belagali, Achuth Rao M V, Pebbili Gopikishore, Rahul Krishnamurthy, and Prasanta Kumar Ghosh

Open Access

Get PDF
Email
Share
Get Citation
Copy Citation Text
Varun Belagali, Achuth Rao M V, Pebbili Gopikishore, Rahul Krishnamurthy, and Prasanta Kumar Ghosh, "Two step convolutional neural network for automatic glottis localization and segmentation in stroboscopic videos," Biomed. Opt. Express 11, 4695-4713 (2020)

Export Citation
- BibTex
- Endnote (RIS)
- HTML
- Plain Text
Citation alert
Save article

Check for updates

Abstract

Precise analysis of the vocal fold vibratory pattern in a stroboscopic video plays a key role in the evaluation of voice disorders. Automatic glottis segmentation is one of the preliminary steps in such analysis. In this work, it is divided into two subproblems namely, glottis localization and glottis segmentation. A two step convolutional neural network (CNN) approach is proposed for the automatic glottis segmentation. Data augmentation is carried out using two techniques : 1) Blind rotation (WB), 2) Rotation with respect to glottis orientation (WO). The dataset used in this study contains stroboscopic videos of 18 subjects with Sulcus vocalis, in which the glottis region is annotated by three speech language pathologists (SLPs). The proposed two step CNN approach achieves an average localization accuracy of 90.08% and a mean dice score of 0.65.

Full Article | PDF Article

More Like This

Effect of patch size and network architecture on a convolutional neural network approach for automatic segmentation of OCT retinal layers

Jared Hamwood, David Alonso-Caneiro, Scott A. Read, Stephen J. Vincent, and Michael J. Collins
Biomed. Opt. Express 9(7) 3049-3066 (2018)

FlyNet 2.0: drosophila heart 3D (2D + time) segmentation in optical coherence microscopy images using a convolutional long short-term memory neural network

Zhao Dong, Jing Men, Zhiwen Yang, Jason Jerwick, Airong Li, Rudolph E. Tanzi, and Chao Zhou
Biomed. Opt. Express 11(3) 1568-1579 (2020)

Segmentation of mouse skin layers in optical coherence tomography image data using deep convolutional neural networks

Timo Kepp, Christine Droigk, Malte Casper, Michael Evers, Gereon Hüttmann, Nunciada Salma, Dieter Manstein, Mattias P. Heinrich, and Heinz Handels
Biomed. Opt. Express 10(7) 3484-3496 (2019)

Previous Article Next Article

References

View by:
Article Order
Year
Author
Publication

I. R. Titze and F. Alipour, The myoelastic aerodynamic theory of phonation (National Center for Voice and Speech, 2006).
O. Gloger, B. Lehnert, A. Schrade, and H. Völzke, “Fully automated glottis segmentation in endoscopic videos using local color and shape features of glottal regions,” IEEE Trans. Biomed. Eng. 62(3), 795–806 (2015).
[Crossref]
J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).
T. Nawka and U. Konerding, “The interrater reliability of stroboscopy evaluations,” J. Voice 26(6), 812.E1–812.E10 (2012).
[Crossref]
L. Rudmik, Evidence-based Clinical Practice in Otolaryngology (Elsevier Health Sciences, 2018).
A. Rao MV, R. Krishnamurthy, P. Gopikishore, V. Priyadharshini, and P. K. Ghosh, “Automatic glottis localization and segmentation in stroboscopic videos using deep neural network,” in Proc. Interspeech 2018, (2018), pp. 3007–3011.
J. J. Cerrolaza, V. Osma-Ruiz, N. Sáenz-Lechón, A. Villanueva, J. M. Gutiérrez-Arriola, J. I. Godino-Llorente, and R. Cabeza, “Fully-automatic glottis segmentation with active shape models,” in MAVEBA, (2011), pp. 35–38.
J. Lin, E. S. Walsted, V. Backer, J. H. Hull, and D. S. Elson, “Quantification and analysis of laryngeal closure from endoscopic videos,” IEEE Trans. Biomed. Eng. 66(4), 1127–1136 (2019).
[Crossref]
J. Lohscheller, H. Toy, F. Rosanowski, U. Eysholdt, and M. Döllinger, “Clinically evaluated procedure for the reconstruction of vocal fold vibrations from endoscopic digital high-speed videos,” Med. Image Anal. 11(4), 400–413 (2007).
[Crossref]
M.-H. Laves, J. Bicker, L. A. Kahrs, and T. Ortmaier, “A dataset of laryngeal endoscopic images with comparative study on convolution neural network-based semantic segmentation,” Int. J. CARS 14(3), 483–492 (2019).
[Crossref]
M. K. Fehling, F. Grosch, M. E. Schuster, B. Schick, and J. Lohscheller, “Fully automatic segmentation of glottis and vocal folds in endoscopic laryngeal high-speed videos using a deep convolutional lstm network,” PLoS One 15(2), e0227791 (2020).
[Crossref]
J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), pp. 3431–3440.
V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017).
[Crossref]
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, (Springer, 2015), pp. 234–241.
R. Hemelings, B. Elen, I. Stalmans, K. Van Keer, P. De Boever, and M. B. Blaschko, “Artery–vein segmentation in fundus images using a fully convolutional network,” Comput. Med. Imag. Grap. 76, 101636 (2019).
[Crossref]
F. H. Araújo, R. R. Silva, D. M. Ushizima, M. T. Rezende, C. M. Carneiro, A. G. C. Bianchi, and F. N. Medeiros, “Deep learning for cell image segmentation and ranking,” Comput. Med. Imag. Grap. 72, 13–21 (2019).
[Crossref]
Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]
D. Owen, “The power of student’s t-test,” J. Am. Stat. Assoc. 60(309), 320–333 (1965).
[Crossref]
L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology 26(3), 297–302 (1945).
[Crossref]
D. Abdullah, F. Fajriana, M. Maryana, L. Rosnita, A. P. U. Siahaan, R. Rahim, P. Harliana, H. Harmayani, Z. Ginting, and C. I. Erliana et al., “Application of interpolation image by using bi-cubic algorithm,” in Journal of Physics: Conference Series, vol. 1114 (IOP Publishing, 2018), p. 012066.
K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).
O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, and M. Bernstein, “Imagenet large scale visual recognition challenge,” Int. J. Comput. Vis. 115(3), 211–252 (2015).
[Crossref]
D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).
R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, vol. 1 (Addison-Wesley Reading, 1992).
L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010, (Springer, 2010), pp. 177–186.
C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” >J. Big Data 6(1), 60 (2019).
[Crossref]
S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]
F. Chollet et al., “Keras,” (2015).
T. T. D. Team, R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer, and A. Belikov et al., “Theano: A python framework for fast computation of mathematical expressions,” arXiv preprint arXiv:1605.02688 (2016).
W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]
D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]
A. Sadovski, “Algorithm as 74: L1-norm fit of a straight line,” J. Royal Stat. Soc. Ser. C (Applied Stat.) 23(2), 244–248 (1974).
[Crossref]

2020 (1)

M. K. Fehling, F. Grosch, M. E. Schuster, B. Schick, and J. Lohscheller, “Fully automatic segmentation of glottis and vocal folds in endoscopic laryngeal high-speed videos using a deep convolutional lstm network,” PLoS One 15(2), e0227791 (2020).
[Crossref]

2019 (5)

M.-H. Laves, J. Bicker, L. A. Kahrs, and T. Ortmaier, “A dataset of laryngeal endoscopic images with comparative study on convolution neural network-based semantic segmentation,” Int. J. CARS 14(3), 483–492 (2019).
[Crossref]

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” >J. Big Data 6(1), 60 (2019).
[Crossref]

J. Lin, E. S. Walsted, V. Backer, J. H. Hull, and D. S. Elson, “Quantification and analysis of laryngeal closure from endoscopic videos,” IEEE Trans. Biomed. Eng. 66(4), 1127–1136 (2019).
[Crossref]

R. Hemelings, B. Elen, I. Stalmans, K. Van Keer, P. De Boever, and M. B. Blaschko, “Artery–vein segmentation in fundus images using a fully convolutional network,” Comput. Med. Imag. Grap. 76, 101636 (2019).
[Crossref]

F. H. Araújo, R. R. Silva, D. M. Ushizima, M. T. Rezende, C. M. Carneiro, A. G. C. Bianchi, and F. N. Medeiros, “Deep learning for cell image segmentation and ranking,” Comput. Med. Imag. Grap. 72, 13–21 (2019).
[Crossref]

2018 (1)

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

2017 (1)

V. Badrinarayanan, A. Kendall, and R. Cipolla, “Segnet: A deep convolutional encoder-decoder architecture for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2481–2495 (2017).
[Crossref]

2015 (2)

O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, and M. Bernstein, “Imagenet large scale visual recognition challenge,” Int. J. Comput. Vis. 115(3), 211–252 (2015).
[Crossref]

O. Gloger, B. Lehnert, A. Schrade, and H. Völzke, “Fully automated glottis segmentation in endoscopic videos using local color and shape features of glottal regions,” IEEE Trans. Biomed. Eng. 62(3), 795–806 (2015).
[Crossref]

2012 (1)

T. Nawka and U. Konerding, “The interrater reliability of stroboscopy evaluations,” J. Voice 26(6), 812.E1–812.E10 (2012).
[Crossref]

2007 (1)

J. Lohscheller, H. Toy, F. Rosanowski, U. Eysholdt, and M. Döllinger, “Clinically evaluated procedure for the reconstruction of vocal fold vibrations from endoscopic digital high-speed videos,” Med. Image Anal. 11(4), 400–413 (2007).
[Crossref]

2006 (1)

W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]

2001 (1)

S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]

1989 (1)

D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]

1974 (1)

A. Sadovski, “Algorithm as 74: L1-norm fit of a straight line,” J. Royal Stat. Soc. Ser. C (Applied Stat.) 23(2), 244–248 (1974).
[Crossref]

1965 (1)

D. Owen, “The power of student’s t-test,” J. Am. Stat. Assoc. 60(309), 320–333 (1965).
[Crossref]

1945 (1)

L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology 26(3), 297–302 (1945).
[Crossref]

Abdullah, D.

D. Abdullah, F. Fajriana, M. Maryana, L. Rosnita, A. P. U. Siahaan, R. Rahim, P. Harliana, H. Harmayani, Z. Ginting, and C. I. Erliana et al., “Application of interpolation image by using bi-cubic algorithm,” in Journal of Physics: Conference Series, vol. 1114 (IOP Publishing, 2018), p. 012066.

Ahn, S. J.

S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]

Alain, G.

T. T. D. Team, R. Al-Rfou, G. Alain, A. Almahairi, C. Angermueller, D. Bahdanau, N. Ballas, F. Bastien, J. Bayer, and A. Belikov et al., “Theano: A python framework for fast computation of mathematical expressions,” arXiv preprint arXiv:1605.02688 (2016).

Alipour, F.

I. R. Titze and F. Alipour, The myoelastic aerodynamic theory of phonation (National Center for Voice and Speech, 2006).

Almahairi, A.

Al-Rfou, R.

Angermueller, C.

Araújo, F. H.

Ba, J.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

Backer, V.

Badrinarayanan, V.

Bahdanau, D.

Ballas, N.

Bastien, F.

Bayer, J.

Belikov, A.

Bernstein, M.

Bianchi, A. G. C.

Bicker, J.

Blaschko, M. B.

Bottou, L.

L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010, (Springer, 2010), pp. 177–186.

Brox, T.

O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in International Conference on Medical image computing and computer-assisted intervention, (Springer, 2015), pp. 234–241.

Cabeza, R.

J. J. Cerrolaza, V. Osma-Ruiz, N. Sáenz-Lechón, A. Villanueva, J. M. Gutiérrez-Arriola, J. I. Godino-Llorente, and R. Cabeza, “Fully-automatic glottis segmentation with active shape models,” in MAVEBA, (2011), pp. 35–38.

Camara, O.

W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]

Carneiro, C. M.

Cerrolaza, J. J.

Chollet, F.

F. Chollet et al., “Keras,” (2015).

Cipolla, R.

Crum, W. R.

W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]

Darrell, T.

J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, (2015), pp. 3431–3440.

De Boever, P.

Demeyer, J.

J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).

Deng, J.

Dice, L. R.

L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology 26(3), 297–302 (1945).
[Crossref]

Döllinger, M.

Dubuisson, T.

J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).

Elen, B.

Elson, D. S.

Erliana, C. I.

Eysholdt, U.

Fajriana, F.

Fehling, M. K.

Fischer, P.

Ghosh, P. K.

A. Rao MV, R. Krishnamurthy, P. Gopikishore, V. Priyadharshini, and P. K. Ghosh, “Automatic glottis localization and segmentation in stroboscopic videos using deep neural network,” in Proc. Interspeech 2018, (2018), pp. 3007–3011.

Ginting, Z.

Gloger, O.

Godino-Llorente, J. I.

Gopikishore, P.

Gosselin, B.

J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).

Grosch, F.

Gutiérrez-Arriola, J. M.

Haralick, R. M.

R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, vol. 1 (Addison-Wesley Reading, 1992).

Harliana, P.

Harmayani, H.

Hemelings, R.

Hill, D. L.

W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]

Huang, Z.

Hull, J. H.

Jiang, Z.

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

Kahrs, L. A.

Karpathy, A.

Kendall, A.

Kendrick, J. S.

D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]

Khoshgoftaar, T. M.

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” >J. Big Data 6(1), 60 (2019).
[Crossref]

Khosla, A.

Kingma, D. P.

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

Ko, S.-B.

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

Konerding, U.

T. Nawka and U. Konerding, “The interrater reliability of stroboscopy evaluations,” J. Voice 26(6), 812.E1–812.E10 (2012).
[Crossref]

Krause, J.

Krishnamurthy, R.

Laves, M.-H.

Lehnert, B.

Lin, J.

Lohscheller, J.

Long, J.

Ma, S.

Maryana, M.

Medeiros, F. N.

Nawka, T.

T. Nawka and U. Konerding, “The interrater reliability of stroboscopy evaluations,” J. Voice 26(6), 812.E1–812.E10 (2012).
[Crossref]

Ortmaier, T.

Osma-Ruiz, V.

Owen, D.

D. Owen, “The power of student’s t-test,” J. Am. Stat. Assoc. 60(309), 320–333 (1965).
[Crossref]

Parker, R. A.

D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]

Priyadharshini, V.

Rahim, R.

Rao MV, A.

Rauh, W.

S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]

Remacle, M.

J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).

Rezende, M. T.

Ronneberger, O.

Rosanowski, F.

Rosnita, L.

Rudmik, L.

L. Rudmik, Evidence-based Clinical Practice in Otolaryngology (Elsevier Health Sciences, 2018).

Russakovsky, O.

Sadovski, A.

A. Sadovski, “Algorithm as 74: L1-norm fit of a straight line,” J. Royal Stat. Soc. Ser. C (Applied Stat.) 23(2), 244–248 (1974).
[Crossref]

Sáenz-Lechón, N.

Satheesh, S.

Schick, B.

Schrade, A.

Schuster, M. E.

Shapiro, L. G.

R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, vol. 1 (Addison-Wesley Reading, 1992).

Shelhamer, E.

Shorten, C.

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” >J. Big Data 6(1), 60 (2019).
[Crossref]

Siahaan, A. P. U.

Silva, R. R.

Simonyan, K.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).

Stalmans, I.

Su, H.

Team, T. T. D.

Titze, I. R.

I. R. Titze and F. Alipour, The myoelastic aerodynamic theory of phonation (National Center for Voice and Speech, 2006).

Toy, H.

Ushizima, D. M.

Van Keer, K.

Villanueva, A.

Völzke, H.

Walsted, E. S.

Wang, Y.

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

Warnecke, H.-J.

S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]

Williamson, D. F.

D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]

Zhang, H.

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

Zisserman, A.

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).

>J. Big Data (1)

C. Shorten and T. M. Khoshgoftaar, “A survey on image data augmentation for deep learning,” >J. Big Data 6(1), 60 (2019).
[Crossref]

Ann. Intern. Med. (1)

D. F. Williamson, R. A. Parker, and J. S. Kendrick, “The box plot: a simple visual method to interpret data,” Ann. Intern. Med. 110(11), 916–921 (1989).
[Crossref]

Comput. Med. Imag. Grap. (3)

Z. Jiang, H. Zhang, Y. Wang, and S.-B. Ko, “Retinal blood vessel segmentation using fully convolutional network with transfer learning,” Comput. Med. Imag. Grap. 68, 1–15 (2018).
[Crossref]

Ecology (1)

L. R. Dice, “Measures of the amount of ecologic association between species,” Ecology 26(3), 297–302 (1945).
[Crossref]

IEEE Trans. Biomed. Eng. (2)

IEEE Trans. Med. Imaging (1)

W. R. Crum, O. Camara, and D. L. Hill, “Generalized overlap measures for evaluation and validation in medical image analysis,” IEEE Trans. Med. Imaging 25(11), 1451–1461 (2006).
[Crossref]

IEEE Trans. Pattern Anal. Mach. Intell. (1)

Int. J. CARS (1)

Int. J. Comput. Vis. (1)

J. Am. Stat. Assoc. (1)

D. Owen, “The power of student’s t-test,” J. Am. Stat. Assoc. 60(309), 320–333 (1965).
[Crossref]

J. Royal Stat. Soc. Ser. C (Applied Stat.) (1)

A. Sadovski, “Algorithm as 74: L1-norm fit of a straight line,” J. Royal Stat. Soc. Ser. C (Applied Stat.) 23(2), 244–248 (1974).
[Crossref]

J. Voice (1)

T. Nawka and U. Konerding, “The interrater reliability of stroboscopy evaluations,” J. Voice 26(6), 812.E1–812.E10 (2012).
[Crossref]

Med. Image Anal. (1)

Pattern Recognit. (1)

S. J. Ahn, W. Rauh, and H.-J. Warnecke, “Least-squares orthogonal distances fitting of circle, sphere, ellipse, hyperbola, and parabola,” Pattern Recognit. 34(12), 2283–2303 (2001).
[Crossref]

PLoS One (1)

Other (14)

J. Demeyer, T. Dubuisson, B. Gosselin, and M. Remacle, “Glottis segmentation with a high-speed glottography: a fully automatic method,” in 3rd Adv. Voice Funct. Assess. Int. Workshop, (2009).

L. Rudmik, Evidence-based Clinical Practice in Otolaryngology (Elsevier Health Sciences, 2018).

F. Chollet et al., “Keras,” (2015).

D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” arXiv preprint arXiv:1412.6980 (2014).

R. M. Haralick and L. G. Shapiro, Computer and Robot Vision, vol. 1 (Addison-Wesley Reading, 1992).

L. Bottou, “Large-scale machine learning with stochastic gradient descent,” in Proceedings of COMPSTAT’2010, (Springer, 2010), pp. 177–186.

I. R. Titze and F. Alipour, The myoelastic aerodynamic theory of phonation (National Center for Voice and Speech, 2006).

K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” arXiv preprint arXiv:1409.1556 (2014).

Cited By

Optica participates in Crossref's Cited-By Linking service. Citing articles from Optica Publishing Group journals and other participating publishers are listed here.

Alert me when this article is cited.

Click here to see a list of articles that cite this paper

Fig. 1. Sample videostroboscopy images from 18 subjects with SV showing the variation in terms of shape of the glottis, illumination and position of the camera while recording the data.

Download Full Size | PPT Slide | PDF

Fig. 2. Frames where the supraglottic structures block the glottis.

Download Full Size | PPT Slide | PDF

Fig. 3. Screenshot of Matlab based GUI (top left) used for annotation by SLPs. Histogram of the dice score (bottom left) calculated for all pairs of annotators to show the inter-annotator agreement and three exemplary images each annotated by three SLPs

$\mathbf {(a_{1},a_{2},a_{3})}$

and the corresponding mean dice score obtained by averaging the dice scores obtained from all three pairs of annotators, (

$\mathbf {(a_{1},a_{2})}$

$\mathbf {(a_{2},a_{3})}$

$\mathbf {(a_{1},a_{3})}$

), shown in the right side of each row.

Download Full Size | PPT Slide | PDF

Fig. 4. Block diagram of the proposed glottis segmentation approach where an image is passed through two steps: 1) localization step which uses CNN1 architecture, 2) segmentation step which uses CNN2 architecture. BI denotes bicubic interpolation.

Download Full Size | PPT Slide | PDF

Fig. 5. Orientation estimation involved in WO based augmentation process.

Download Full Size | PPT Slide | PDF

Fig. 6. Histogram of orientation angle (

$\phi$

) between the major axis of glottis and the horizontal axis measured in degrees in NO, WB and WO based augmentation.

Download Full Size | PPT Slide | PDF

Fig. 7. Boxplot of dice score achieved by the 11 combinations of network architecture on validation data.

$M_{1100}$

achieves the highest median dice score indicated by dashed horizontal line. The labels on the boxplot indicate quartiles and medians.

Download Full Size | PPT Slide | PDF

Fig. 8.

$B_{a}$

using CNN1 of the two step CNN approach with three data augmentation schemes, NO, WB and WO evaluated on three SLPs, namely a1, a2, a3. It is clear that the WO data augmentation technique achieves the best

$B_{a}$

Download Full Size | PPT Slide | PDF

Fig. 9. Boxplot of dice score obtained using CNN_C and two step CNN approaches without augmentation (NO), with WB based augmentation and WO based augmentation. The dashed horizontal line indicates the average of three median dice scores obtained using the two step CNN WO evaluated on three SLPs.

Download Full Size | PPT Slide | PDF

Fig. 10. Bar graph indicating localization accuracy using various approaches evaluated across three annotators.

Download Full Size | PPT Slide | PDF

Fig. 11. Illustration of the glottis segmentation on various sample images, one from each of the 18 subjects with SV. Column (a)

$\colon$

Subject number with corresponding dice score achieved by the proposed approach (blue) and average inter annotator agreement (red) on the sample image obtained by averaging the dice scores from every pair of annotators. Column (b)

$\colon$

720

$\times$

576 original image with bounding box obtained in the localization step. Column (c)

$\colon$

cropped image obtained from the localization step. Column (d)

$\colon$

The predicted segment by the two step CNN approach. Column (e)

$\colon$

Groundtruth segment corresponding to the cropped image labeled by annotator a1.

Download Full Size | PPT Slide | PDF

Fig. 12. Foldwise normalized count (Nc) histogram of mean dice scores from every fold in the test set evaluated on three SLPs and three histogram curves corresponding to evaluation on three annotators separately.

Download Full Size | PPT Slide | PDF

Fig. 13. Mean dice score averaged across three SLPs vs the mean dice score obtained using the proposed algorithm evaluated on three SLPs and the

$l_{1}$

line fit to plot (red).

Download Full Size | PPT Slide | PDF

Fig. 14. Mean dice score using the proposed method evaluated across three SLPs vs the normalized glottis opening area with respect to the image size.

Download Full Size | PPT Slide | PDF

Tables (2)

Table 1. p-values of the Student’s t-test performed on dice scores between all pairs of segmentation methods. A bold entry in a cell indicates that the method in the corresponding column has higher mean dice score than that in the corresponding row. Blue entry in a cell indicates that the methods in the corresponding row and column do not yield significantly different average dice score

View Table | View all tables in this article

Table 2. Localization accuracy and corresponding dice score achieved by the baseline and the proposed approach

View Table | View all tables in this article

Equations (2)

Equations on this page are rendered with MathJax. Learn more.

P_{i, j} = {\begin{cases} 1 & P r_{i, j} > m a x_{i, j} (P r_{i, j}) - ϵ \\ 0 & otherwise \end{cases}

D (P, G) = \frac{2 \times N (P \cap G)}{N (P) + N (G)}

	CNN_C	Two-step CNN	CNN_C WB	Two-step CNN WB	CNN_C WO	Two-step CNN WO
CNN_C	*	< $10^{- 7}$	0.2506	< $10^{- 83}$	< $10^{- 5}$	< $10^{- 183}$
Two-step CNN	< $10^{- 7}$	*	< $10^{- 10}$	< $10^{- 30}$	< $10^{- 16}$	< $10^{- 77}$
CNN_C WB	0.2506	< $10^{- 10}$	*	< $10^{- 89}$	0.1323	< $10^{- 186}$
Two-step CNN WB	< $10^{- 83}$	< $10^{- 30}$	< $10^{- 89}$	*	< $10^{- 109}$	< $10^{- 13}$
CNN_C WO	< $10^{- 5}$	< $10^{- 16}$	0.1323	< $10^{- 109}$	*	< $10^{- 216}$
Two-step CNN WO	< $10^{- 183}$	< $10^{- 77}$	< $10^{- 186}$	< $10^{- 13}$	$10^{- 216}$	*

	Localization accuracy (a)		Dice score (b)
	Baseline	Proposed	Baseline	Proposed
fold1	89.9,78,79.1	97.75,98.09,98.07	0.56,0.54,0.54	0.66,0.62,0.62
fold2	38.0,69.0,40.0	93.92,92.82,92.82	0.28,0.31,0.28	0.60,0.54,0.59
fold3	72.2,68.6,72.8	80.56,81.67,81.68	0.38,0.42,0.38	0.79,0.78,0.77
fold4	39.9,76.2,41.4	88.18,87.27,88.18	0.32,0.32,0.32	0.65,0.63,0.63
average	60.0,73.1,63.2	90.10,89.96,90.18	0.39,0.40,0.38	0.68,0.64,0.65

	CNN_C	Two-step CNN	CNN_C WB	Two-step CNN WB	CNN_C WO	Two-step CNN WO
CNN_C	*	< $10^{- 7}$	0.2506	< $10^{- 83}$	< $10^{- 5}$	< $10^{- 183}$
Two-step CNN	< $10^{- 7}$	*	< $10^{- 10}$	< $10^{- 30}$	< $10^{- 16}$	< $10^{- 77}$
CNN_C WB	0.2506	< $10^{- 10}$	*	< $10^{- 89}$	0.1323	< $10^{- 186}$
Two-step CNN WB	< $10^{- 83}$	< $10^{- 30}$	< $10^{- 89}$	*	< $10^{- 109}$	< $10^{- 13}$
CNN_C WO	< $10^{- 5}$	< $10^{- 16}$	0.1323	< $10^{- 109}$	*	< $10^{- 216}$
Two-step CNN WO	< $10^{- 183}$	< $10^{- 77}$	< $10^{- 186}$	< $10^{- 13}$	$10^{- 216}$	*

	Localization accuracy (a)		Dice score (b)
	Baseline	Proposed	Baseline	Proposed
fold1	89.9,78,79.1	97.75,98.09,98.07	0.56,0.54,0.54	0.66,0.62,0.62
fold2	38.0,69.0,40.0	93.92,92.82,92.82	0.28,0.31,0.28	0.60,0.54,0.59
fold3	72.2,68.6,72.8	80.56,81.67,81.68	0.38,0.42,0.38	0.79,0.78,0.77
fold4	39.9,76.2,41.4	88.18,87.27,88.18	0.32,0.32,0.32	0.65,0.63,0.63
average	60.0,73.1,63.2	90.10,89.96,90.18	0.39,0.40,0.38	0.68,0.64,0.65