Seung Hee Yang

Prof. Dr. Seung Hee Yang

Professor (W1) for Artificial Intelligence in Speech and Language Processing

Department Artificial Intelligence in Biomedical Engineering (AIBE)
Juniorprofessur für Artificial Intelligence in Biomedical Speech Processing (Stiftungsprofessur)

Room: Room 01.333
Department AIBE / Henkestr. 91
91052 Erlangen

Office hours

Please contact me via e-mail to schedule an appointment.

Prof. Dr. Seung Hee Yang is currently leading the Spoken Language Lab in AIBE (AI in Biomedical Engineering) in FAU. Before joining FAU, she acquired PhD in Engineering from Seoul National University in South Korea in 2020. She was also a graduate student at Columbia University in the City of New York. She has a strong background in speech and natural language processing. Currently, her research interest includes two Siemen’s funded projects, “Biomarkers for Stroke Diagnosis and Therapy” and “Smart Operating Room Speech Assistant”, by working closely with the University Hospital in Erlangen. She applies speech and language processing and machine learning technology in biomedical engineering field.

Academic CV

Seoul National University, Seoul: Ph.D. in Engineering degree received February, 2020
– Seoul National University, Seoul: M.Sc. in Engineering degree received in February, 2016
– Columbia University in the City of New York: M.A. in Philosophy degree received in June, 2011
– Ewha Women’s University, Seoul: B.A. in Philosophy degree received in February, 2009
– Daewon Foreign Language High School, Seoul: Department of French, 2005
– Tiffin Girls’ School, London: Secondary Education (GCSE)’s, 2003
– Grade 8 in Violin, Associated Board of Royal School of Music, U.K., 2003 (permission to teach)

Projects

Speech Biomarkers for Stroke Diagnosis and Therapy, supported by Medical Valley e.V. and Siemens Healthineers (2021-present)
– Smart Operating Room Speech Assistant, supported by Medical Valley e.V. and Siemens Healthineers (2021-present)
– Human Resource Development for the Biomedical Unstructured Big Data Analysis, supported by Korea Ministry of Science and ICT, and hosted by Medical Big Data Research Center, sized $120,000 (2016-2020).
– Emergency Medical Services using Speech Recognition Technology, supported by Korea Evaluation Institute of Industrial Technology, sized $120,000 (2019).
– Accessible and Intelligent Solution Technology Development for Communication Disabilities, supported by Korea Creative Content Agency, sized $300,000 (2019).
– QoLT (Quality of Life Technology), supported by Korea Ministry of Trade, Industry and Energy, sized $ 2 million (2011–2014).
– Video Turing Test (VTT), supported by Korea Ministry of Science and ICT, sized $100,000 (2016-2020).
– Dialected Korean ASR System Development, supported by Lotte Data Communication Company, sized $ 50,000 (2019).
– Core Technology development for Natural Speech Dialogue Speech Recognition in Free-running Embedded Robots, supported by Korea Ministry of Science and ICT, sized $100,000 (2016-2020).
– Automatic Speech Assessment System Development for Foreign Language Learners of Korean, supported by Naver, sized $ 30,000 (2016).

Publications

Conferences & Journals

[23] Oppelt, M. P., Foltyn, A., Deuschel, J., Lang, N. R., Holzer, N., Eskofier, B. M., & Yang, S. H. “ADABase: A Multimodal Dataset for Cognitive Load Estimation”. Sensors, 23(1), 340. (2023).

[22] Abner Hernandez, Paula Andrea Perez-Toro, Elmar Noeth, Juan Rafael Orozco-Arroyave, Andreas Maier, Seung Hee Yang. “Cross-lingual Self-Supervised Speech Representations for Improved Dysarthric Speech Recognition.” Proceedings of INTERSPEECH (2022).

[20] Abner Hernandez, et al. “Self-Supervised Speech Representations Preserve Speech Characteristics while Anonymizing Voices.” arXiv preprint arXiv:2204.01677 (2022).

[19] Tobias Weise, Andreas Maier, Elmar Noeth, Bjoern Heismann, Maria Schuster, Seung Hee Yang. “Disentangled Latent Speech Representation for Automatic Pathological Intelligibility Assessment.” Proceedings of INTERSPEECH (2022).

[18] Maier, Andreas, et al. “Known operator learning and hybrid machine learning in medical imaging—a review of the past, the present, and the future.” Progress in Biomedical Engineering (2022).

[17] Hernandez, Abner, and Seung Hee Yang. “Multimodal Corpus Analysis of Autoblog 2020: Lecture Videos in Machine Learning.” International Conference on Speech and Computer. Springer (2021).

[16] Maier, Andreas, et al. “Does Proprietary Software Still Offer Protection of Intellectual Property in the Age of Machine Learning? A Case Study using Dual Energy CT Data.” arXiv preprint arXiv:2112.03678 (2021).

[15] Seung Hee Yang & Minhwa Chung. “Speech-to-Speech Conversion for Applications in Pronunciation Training and Impaired Speech Aid.” Journal of the Korean Society of Phonetic Science and Speech Technology (2020).

[14] Seung Hee Yang & Minhwa Chung. “Improving Dysarthric Speech Intelligibility Using Cycle-consistent Adversarial Training.” BioSIGNALS, Malta (2020).

[13] Seung Hee Yang & Minhwa Chung. “Including Linguistic Knowledge in an Auxiliary Classifier CycleGAN for Corrective Feedback Generation in Korean Speech.” LT4ALL, Paris, France (2020).

[12] Sihyeon Jo, Seungryong Yoo, Sangwon Im, Seung Hee Yang, Tong Zuo, Hee-Eun Kim, SangWook Han, Seong-Woo Kim. “A Scalable Chatbot Platform Leveraging Online Community Posts: A Proof-of-Concept Study.” Human-Computer Interaction, Korea (2020).

[11] Seung Hee Yang. “Automatic Speech Recognition-enabled Language Learning System Development for Second Language Speakers of Korean.” Doctoral Consortium, INTERSPEECH, Graz, Austria (2019).

[10] Seung Hee Yang & Minhwa Chung. “Self-imitating Feedback Generation Using GAN for Computer-Assisted Pronunciation Training.” Proceedings of INTERSPEECH, Graz, Austria (2019).

[9] Taehyeong Kim, Seung Hee Yang, Hyunwoong Ko, Sungjae Cho, Jun-Young Lee, Byoung-Tak Zhang. Emotion Recognition from “Facial Expression Images Produced by Non-Actors. Real-World Recognition from Low-Quality Images and Videos.” Satellite workshop on International Conference on Computer Vision (ICCV), Seoul, Korea (2019).

[8] Taehyeong Kim, Minji Kwak, Seung Hee Yang, Jaeseo Lim, Byoung-Tak Zhang. “WithDorm: A Dormitory Solution for Linking Roomates.” MobileHCI, Taipei, Taiwan (2019).

[7] Seung Hee Yang & Minhwa Chung. “Speech Assessment using Generative Adversarial Network.” Proceedings of Machine Learning in Speech and Language Processing – Interspeech Satellite workshop, Hyderabad, India (2018).

[6] Seung Hee Yang, Sangwoo Park, Taemyung Yang, Ilhyung Jin, Wooil Kim, Chingwei Liu, Seong-Woo Kim and Juhyun Eune. “Introducing Smart Pillow using Actuator Mechanism, Pressure Sensors, and Deep Learning-based ASR.” ACM Augmented Human International Conference. Seoul, Korea (2018).

[5] Seung Hee Yang & Minhwa Chung. “Correlation analysis of linguistic factors in non-native Korean speech and proficiency evaluation.” Phonetics and Speech Sciences (2017).

[4] Crego, J., Kim, J., Klein, G., Rebollo, A., Yang, S., Senellart, J., Enoue, S. (2016). “Systran’s pure neural machine translation systems.” arXiv preprint arXiv:1610.05540.

[3] Seung Hee Yang & Minhwa Chung. “A Corpus-based Analysis of Korean Segments Produced by Chinese Learners.” Proceedings of Asia-Pacific Signal and Information Processing Association (APSIPA), Hong Kong Polytechnic University, Hong Kong (2015).

[2] Seung Hee Yang & Minhwa Chung. “Automatic Classification of Retroflex Segmental Variations for Korean Produced by Chinese.” Proceedings of International Conference on Speech Science, Seoul, Korea (2015).

[1] Seung Hee Yang, Minsu Na & Minhwa Chung. “Modeling Pronunciation Variations for Non-native Speech Recognition of Korean Produced by Chinese Learners.” Proceedings of SLaTE – Interspeech Satellite workshop on Speech and Language Technology in Education, Dresden, Germany (2015).

Patents

[2] Seung Hee Yang & Jae Chang Yang. “Unsupervised learning-based detection method and driver profile-based vehicle theft detection device and method using same”. European Patent Application No. 19943297, February, 2023.

[1] Yang, Seung Hee. “Unsupervised learning-based detection method and driver profile-based vehicle”. U.S. Patent No. 11,498,575. 15 Nov. 2022.

Career and Achievements

FAU, Erlangen, Germany, May 2021-present

Junior Professor, Department of AI in Biomedical Engineering

Secondary Member, Department of Computer Science

Lectures in the courses “Spoken Language Processing”, “Pathological Speech Processing”, and “Analysis of Neural and Muscle Signals”

– Published 2 journal papers, and 6 international conference papers

 

Seoul National University, Seoul, South Korea, 2016 – 2020
Ph.D. Student, Spoken Language Processing Lab in Seoul National University under supervision of Prof. Minhwa Chung
– Published 11 international conference papers, 1 domestic journal, and 7 domestic conference papers
– Published a Ph.D. thesis titled, “Pronunciation Variation Analysis and CycleGAN-based Feedback Generation for CAPT in Korean”
– Developed a speech recognition enabled Computer-Assisted Language Learning technology
– Lead an interdisciplinary research group in biomedical engineering with computer science and medical Ph.D.
students

Seoul National University, Seoul, South Korea Feb., 2014 – Feb., 2016
Ms.c. Student & Student head, Spoken Language Processing Lab in Seoul National University
– Supervision of 13 Master theses
– Grant Writing
– Managed and participated in 10 government and industry projects

OTHER PROFESSIONAL ACTIVITIES

Reviewer for Scientific Conferences & Grants:
– INTERSPEECH
– ICASSP (International Conference on Acoustics, Speech)
– DFG
– RGC (Research Grants Council)
– LREC (Language Resources and Evaluation Conference)
– Oriental COCOSDA (International Committee for the Co-ordination and Standardisation of Speech Databases and Assessment Techniques)

AWARDS

– Sponsored Student for “Outstanding Ph.D. Thesis Candidate,” Seoul National University, 2019
– “Best Session Paper Award,” Conference on Korea Information and Communications and Information Sciences (KICS), 2019
– 1st place, “Creative Challenge,” Conference on Human Computer Interaction (HCI), Seoul, Korea, 2018
– 2nd place, “Korea Robotics Engineering & Design Show (KROS),” for Smart Pillow with Speech Recognition, 2018
– Scholarship for M.A. and Ph.D. degree, Seoul National University, Spring 2015 – Fall 2018
– 1st place, 100m Short Distance Seoul Women’s Race, 1997

Theses

Christopher Popp (Ms.C): Domain-Specific ASR for German Automotive Domain, 2023.
– Matteo Torcoli (Dr.-Ing.): Dialogue Enhancement and Personalisation – Contributions to Quality Assessment and Control, 2023.
– Nina Brolich (Bs.C): Automatic Pathological Speech Intelligibility Assessment, 2023.
– Sai Varun Varanasi (Ms.C): STApp: Stroke Therapy Application Development using Speech Disentanglement Analysis, 2023.
– Nadine Rucker (Dr.-Ing.): Log File Processing – Changes, Challenges, and Chances, 2022.
– Celine Pohl (Bs.C): Self-supervised learning for pathology classification, 2022.
– Maximilian Riehl (Bs.C): Development and Evaluation of Transformer-based Deep Learning Model for 12-lead ECG Classification, 2022.
– Luis Schmid (Bs.C): Localisation of Ischemic Heart Disease using Machine Learning Methods with Magnetocardiography Data, 2022.
– Sungjae Cho (Ms.C): Simulating Problem Difficulty in Arithmetic Cognition Through Dynamic Connectionist Models, 2019, whose work resulted in international conference publication, and started as a researcher in KAIST (Korea Advanced Institute of Science and Technology).
– Yu Seoha (Ms.C): Longitudinal study of cochlear implantation infants’ development of articulation, 2018, and started a PhD in SNU, 2018.
– Jooyoung Lee (Ph.D): Age Classification from Speech using LSTM, whose work resulted in international conference publication, and started a PhD in SNU, 2018.
– Jongin Kim (Ph.D): Automatic classification of classical music based on musical contents, and started a PhD in SNU, 2018.
– Sabaleuski Matsvei (Ms.C): Vowel Duration and Fundamental Frequency Prediction for Automatic Transplantation of Native English Prosody onto Korean-accented Speech, and started a career as a computational linguist in Megputer in Russia, 2018.
– Evgenia Nedelko (Ms.C): Automatic phrase break and sentence stress prediction in English with RNN, 2018.