David Seunghyun Yoon

Academic Activities

• Service:
  Program Committee, NAACL (since 2019), ACL (since 2020), EMNLP (since 2019), AACL (since 2020), EACL (since 2021), COLING (since 2022), LREC (since 2023), ARR (since 2022)
  Program Committee, AAAI (since 2020), WWW (since 2021), INTERSPEECH (2019), ICLR (since 2023)
  Journal Reviewer, Information Processing and Management, 2020
  Journal Reviewer, IEEE Signal Processing Letters, 2020
  
  
• Invited Talks: Vision and Language Representation with LLM for Multimodal Understanding, Chung-Ang University., Sep. 2024
  Robust Textual Representation for Text and Multimodal Understanding, Dongguk University., Mar. 2024
  Learning Text Representation for NLP Application, Seoul National Univ., Aug. 2023
  Pretrained Language Model and Semantic Textual Understanding, SKKU, Sep. 2022
  Semantic Textual Understanding for Information Retrieval, Seoul National Univ., Aug. 2022
  Mutimodal Evaluation Metric and Image Captioning Model, Korea Univ., Dec. 2021
  Recent Advancements in NLP for QA, LM, and Evaluation Metric, Dongguk Univ., Sep. 2020
  Understanding Long Texts for Question Answering System Using DNN, KAIST/IBS, Jul. 2020
  Question Answering System for Long Text, Adobe Research (San Jose, CA, US), Dec. 2019
  Question Answering System and Multimodal Speech Emotion Recognition, DEEPEST, Aug. 2019
  Research in Natural Language Processing, NVIDIA AI Conference, Jul. 2019
  Question Answering for Short Answer, Adobe Research (San Jose, CA, US), Dec. 2018
  QA-pair ranking algorithm and its applications, NAVER, Aug. 2018
  Learning to Rank Question-Answer Pairs, PyTorch KR, Jun. 2018
  Advancement of the Neural Dialogue Model, Fast campus, Jul. 2018
  
  
• Teaching Assistant:
  Programming Methodology, Seoul National University, Spring 2018
  Machine Learning, Seoul National University, Fall 2015
  Lab. Sentiment Analysis, BigCamp (Big Data Academy), Big Data Institute, 2016-2019
  
  

Professional Experiences

• NLP Research Scientist: Adobe Research (San Jose, CA, US), 2020-present
• Staff Engineer: Samsung Research (Seoul, KR), 2006-2017
• Representative of employees: Samsung Electronics (Seoul, KR), 2012-2014
• Trainer of Global New Employee Course: Samsung Electronics (Seoul, KR), Spring 2011

Publications

[arXiv]

1. VLind-Bench: Measuring Language Priors in Large Vision-Language Models [pdf]
   Kang-il Lee, Minbeom Kim, Seunghyun Yoon, Minsung Kim, Dongryeol Lee, Hyukhun Koh, Kyomin Jung

[2024]

2. The Herd of Open LLMs for Verifying Real-World Claims
   (2nd place / 1st place among open source)
   Yejun Yoon, Jaeyoon Jung, Seunghyun Yoon, Kunwoo Park
   EMNLP 2024 Workshop FEVER

3. PDFTriage: Question Answering over Long, Structured Documents [pdf]
   Jon Saad-Falcon, Joe Barrow, Alexa Siu, Ani Nenkova, Seunghyun Yoon, Ryan A. Rossi, Franck Dernoncourt
   EMNLP 2024 Industry

4. FIZZ: Factual Inconsistency Detection by Zoom-in Summary and Zoom-out Document [pdf]
   Joonho Yang, Seunghyun Yoon, Byeongjeong Kim, Hwanhee Lee
   EMNLP 2024

5. Towards Enhancing Coherence in Extractive Summarization: Dataset and Experiments with LLMs [pdf]
   Mihir Parmar, Hanieh Deilamsalehy, Franck Dernoncourt, Seunghyun Yoon, Ryan A. Rossi, Trung Bui
   EMNLP 2024

6. MVMR: A New Framework for Evaluating Faithfulness of Video Moment Retrieval against Multiple Distractors [pdf]
   Nakyeong Yang, Minsung Kim, Seunghyun Yoon, Joongbo Shin, Kyomin Jung
   CIKM 2024

7. Identifying Speakers in Dialogue Transcripts: A Text-based Approach Using Pretrained Language Models [pdf]
   Van Minh Nguyen, Franck Dernoncourt, Seunghyun Yoon, Hanieh Deilamsalehy, Hao Tan, Ryan Rossi, Quan Hung Tran, Trung Bui, Thien Nguyen
   INTERSPEECH 2024

8. Understanding News Thumbnail Representativeness by Counterfactual Text-Guided Contrastive Language-Image Pretraining [pdf]
   Yejun Yoon, Seunghyun Yoon, Kunwoo Park
   ACL 2024 Findings

9. Multi-hop Database Reasoning with Virtual Knowledge Graph [pdf]
   Juhee Son, Yeon Seonwoo, Alice Oh, James Thorne, Seunghyun Yoon
   ACL 2024 Workshop KaLLM

10. KaPQA: Knowledge-Augmented Product Question-Answering [pdf]
    Swetha Eppalapally, Daksh Dangi, Chaithra Bhat, Ankita Gupta, Ruiyi Zhang, Karishma Bagga, Seunghyun Yoon, Nedim Lipka, Ryan A. Rossi, Franck Dernoncourt
    ACL 2024 Workshop KnowledgeNLP

11. Multilingual Sentence-Level Semantic Search using Meta-Distillation Learning [pdf]
    Meryem M'hamdi, Jonathan May, Franck Dernoncourt, Trung Bui, Seunghyun Yoon
    SIGIR 2024

12. PEEB: Part-based Bird Classifiers with an Explainable and Editable Language Bottleneck [pdf]
    Thang M. Pham, Peijie Chen, Tin Nguyen, Seunghyun Yoon, Trung Bui, Anh Nguyen
    NAACL 2024 Findings

13. Scaling Up Video Summarization Pretraining with Large Language Models [pdf]
    Dawit Mureja Argaw, Seunghyun Yoon, Fabian Caba Heilbron, Hanieh Deilamsalehy, Trung Bui, Zhaowen Wang, Franck Dernoncourt, Joon Son Chung
    CVPR 2024

14. Fine-tuning CLIP Text Encoders with Two-step Paraphrasing [pdf]
    Hyunjae Kim, Seunghyun Yoon, Trung Bui, Handong Zhao, Quan Tran, Franck Dernoncourt, Jaewoo Kang
    EACL 2024 Findings

15. Retrieval Augmented Generation for Domain-specific Question [pdf]
    Sanat Sharma, Seunghyun Yoon, Franck Dernoncourt, Dewang Sultania, Karishma Bagga, Mengjiao Zhang, Trung Bui, Varun Kotte
    AAAI 2024 Workshop SDU

16. Multi-Modal Video Topic Segmentation with Dual-Contrastive Domain Adaptation [pdf]
    Linzi Xing, Quan Tran, Fabian Caba Heilbron, Franck Dernoncourt, Seunghyun Yoon, Zhaowen Wang, Trung Bui, Giuseppe Carenini
    Multimedia Modelling 2024

[2023]

17. Aspect-based Meeting Transcript Summarization: A Two-Stage Approach with Weak Supervision on Sentence Classification [pdf]
    Zhongfen Deng, Seunghyun Yoon, Trung Bui, Franck Dernoncourt, Quan Tran, Shuaiqi Liu, Wenting Zhao, Tao Zhang, Yibo Wang, Philip Yu
    IEEE BigData 2023

18. Perturbation Robust Metric for Multi-Lingual Image Captioning [pdf]
    Yongil Kim, Yerin Hwang, Hyeongu Yun, Seunghyun Yoon, Trung Bui, Kyomin Jung
    EMNLP 2023 Findings

19. Moment Detection in Long Tutorial Videos [pdf] [code]
    Ioana Croitoru, Simion-Vlad Bogolin, Samuel Albanie, Yang Liu, Zhaowen Wang, Seunghyun Yoon, Franck Dernoncourt, Hailin Jin, Trung Bui
    ICCV 2023

20. Boosting Punctuation Restoration with Data Generation and Reinforcement Learning [pdf]
    Viet Lai, Abel Salinas, Hao Tan, Trung Bui, Quan Tran, Seunghyun Yoon, Hanieh Deilamsalehy, Franck Dernoncourt, Thien Nguyen
    INTERSPEECH 2023

21. Automatic Creation of Named Entity Recognition Datasets by Querying Phrase Representations [pdf]
    Hhynjae Kim, Jaehyo Yoo, Seunghyun Yoon, Jaewoo Kang
    ACL 2023

22. MEETINGQA: Extractive Question-Answering on Meeting Transcripts [pdf]
    Archiki Prasad, Trung Bui, Seunghyun Yoon, Hanieh Deilamsalehy, Franck Dernoncourt, Mohit Bansal
    ACL 2023

23. PiC: A Phrase-in-Context Dataset for Phrase Understanding and Semantic Search [pdf] [page]
    Thang M. Pham, Seunghyun Yoon, Trung Bu, Anh Nguyeng
    EACL 2023

[2022]

24. Factual Error Correction for Abstractive Summaries Using Entity Retrieval [pdf]
    Hwanhee Lee, Cheoneum Park, Seunghyun Yoon, Trung Bu, Franck Dernoncourt, Juae Kim, Kyomin Jung
    EMNLP 2022 Workshop GEM

25. Improving cross-modal attention via object detection [pdf]
    Yongil Kim, Yerin Hwang, Seunghyun Yoon, Hyeongu Yun, Kyomin Jung
    NeurIPS 2022 Workshop All Things Attention

26. Simple Questions Generate Named Entity Recognition Datasets [pdf] [code]
    Hyunjae Kim, Jaehyo Yoo, Seunghyun Yoon, Jinhyuk Lee, Jaewoo Kang
    EMNLP 2022

27. Virtual Knowledge Graph Construction for Zero-Shot Domain-Specific Document Retrieval [pdf]
    Yeon Seonwoo, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Alice Oh
    COLING 2022

28. Medical Question Understanding and Answering with Knowledge Grounding and Semantic Self-Supervision [pdf]
    Khalil Mrini, Harpreet Singh, Franck Dernoncourt, Seunghyun Yoon, Trung Bui, Walter W. Chang, Emilia Farcas, Ndapa Nakashole
    COLING 2022

29. Offensive Content Detection Via Synthetic Code-Switched Text [pdf]
    Cesa Salaam, Franck Dernoncourt, Trung Bui, Seunghyun Yoon
    COLING 2022

30. Keyphrase Prediction from Video Transcripts: New Dataset and Directions [pdf]
    Amir Pouran Ben Veyseh, Quan Tran, Seunghyun Yoon, Varun Manjunatha, Hanieh Deilamsalehy, Rajiv Jain, Trung Bui, Walter W. Chang, Franck Dernoncourt, Thien Huu Nguyen
    COLING 2022

31. MACRONYM: A Large-Scale Dataset for Multilingual and Multi-Domain Acronym Extraction [pdf]
    Amir Pouran Ben Veyseh, Nicole Meister, Seunghyun Yoon, Rajiv Jain, Franck Dernoncourt, Thien Huu Nguyen
    COLING 2022

32. Fine-grained Image Captioning with CLIP Reward [pdf] [code] [demo]
    Jaemin Cho, Seunghyun Yoon, Ajinkya Kale, Franck Dernoncourt, Trung Bui, M Bansal
    NAACL 2022 Findings

33. Multimodal Intent Discovery from Livestream Videos [pdf] [code]
    Adyasha Maharana, Quan Tran, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Walter Chang, M Bansal
    NAACL Findings 2022

34. How does fake news use a thumbnail? CLIP-based Multimodal Detection on the Unrepresentative News Image [pdf]
    Hyewon Choi, Yejun Yoon, Seunghyun Yoon, Kunwoo Park
    ACL CONSTRAINT 2022

35. CAISE: Conversational Agent for Image Search and Editing [pdf] [code]
    Hyounghun Kim, Doo Soon Kim, Seunghyun Yoon, Franck Dernoncourt, Trung Bui, Mohit Bansal
    AAAI 2022

[2021]

36. Few-Shot Intent Detection via Contrastive Pre-Training and Fine-Tuning [pdf]
    J Zhang, T Bui, S Yoon, X Chen, Z Liu, C Xia, QH Tran, W Chang, P Yue
    EMNLP 2021

37. QACE: Asking Questions to Evaluate an Image Caption [pdf] [code]
    H Lee, T Scialom, S Yoon, F Dernoncourt, K Jung
    EMNLP 2021 Findings

38. A Gradually Soft Multi-Task and Data-Augmented Approach to Medical Question Understanding [pdf]
    K Mrini, F Dernoncourt, S Yoon, T Bui, W Chang, E Farcas, N Nakashole
    ACL 2021

39. UMIC: An Unreferenced Metric for Image Captioning via Contrastive Learning [pdf] [code]
    H Lee, S Yoon, F Dernoncourt, T Bui, K Jung
    ACL 2021

40. UCSD-Adobe at MEDIQA 2021: Transfer Learning and Answer Sentence Selection for Medical Summarization [pdf]
    K Mrini, F Dernoncourt, S Yoon, T Bui, W Chang, E Farcas, N Nakashole
    NAACL 2021 Workshop BioNLP

41. KPQA: A Metric for Generative Question Answering Using Keyphrase Weights [pdf] [code]
    H Lee, S Yoon, F Dernoncourt, DS Kim, T Bui, J Shin, K Jung
    NAACL 2021

42. Learning to Detect Incongruence in News Headline and Body Text via a Graph Neural Network [pdf] [code]
    (SCI, IF=3.745)
    S Yoon*, K Park*, M Lee, T Kim, M Cha, K Jung
    IEEE Access 2021

[2020]

43. Collaborative Training of GANs in Continuous and Discrete Spaces for Text Generation [pdf]
    (SCI, IF=3.745)
    Y Kim, S Won, S Yoon, K Jung
    IEEE Access 2020

44. ViLBERTScore: Evaluating Image Caption Using Vision-and-Language BERT [pdf] [code]
    H Lee, S Yoon, F Dernoncourt, DS Kim, T Bui, K Jung
    EMNLP 2020 Workshop Eval4NLP

45. Multimodal Speech Emotion Recognition using Cross Attention with Aligned Audio and Text [pdf]
    Y Lee, S Yoon, K Jung
    INTERSPEECH 2020

46. Fast and Accurate Deep Bidirectional Language Representations for Unsupervised Learning [pdf]
    J Shin, Y Lee, S Yoon, K Jung
    ACL 2020

47. Propagate-Selector: Detecting Supporting Sentences for Question Answering via Graph Neural Networks [pdf] [code]
    S Yoon, F Dernoncourt, DS Kim, T Bui, K Jung
    LREC 2020

48. Drug-disease Graph: Predicting Adverse Drug Reaction Signals via Graph Neural Network with Clinical Data [pdf] [slide]
    (oral presentation)
    H Kwak, M Lee, S Yoon, J Chang, S Park, K Jung
    PAKDD 2020

49. DSTC8-AVSD: Multimodal Semantic Transformer Network with Retrieval Style Word Generator [pdf]
    H Lee, S Yoon, F Dernoncourt, DS Kim, T Bui, K Jung
    AAAI 2020 Workshop DSTC8

50. Comparative Studies on Machine Learning for Paralinguistic Signal Compression and Classification [pdf]
    (SCI, IF=2.157)
    S Byun*, S Yoon*, K Jung
    Journal of Supercomputing 2020

51. Attentive Modality Hopping Mechanism for Speech Emotion Recognition [pdf] [code] [slide]
    (oral presentation)
    S Yoon, S Dey, H Lee, K Jung
    IEEE ICASSP 2020

52. BaitWatcher: A lightweight web interface for the detection of incongruent news headlines [pdf] [book]
    K Park, T Kim, S Yoon, M Cha, K Jung
    Disinformation, Misinformation, and Fake News in Social Media-Emerging Research Challenges and Opportunities, Springer 2020

[2019]

53. A Compare-Aggregate Model with Latent Clustering for Answer Selection [pdf] [slide] [poster]
    (oral presentation)
    S Yoon, F Dernoncourt, DS Kim, T Bui, K Jung
    CIKM 2019

54. Surf at MEDIQA 2019: Improving Performance of Natural Language Inference in the Clinical Domain by Adopting Pre-trained Language Model [pdf]
    J Nam, S Yoon, K Jung
    ACL 2019 Workshop BioNLP

55. Speech Emotion Recognition Using Multi-hop Attention Mechanism [pdf] [slide]
    (oral presentation)
    S Yoon, S Byun, S Dey, K Jung
    IEEE ICASSP 2019

56. Neural Networks for Compressing and Classifying Speaker-Independent Paralinguistic Signals [pdf]
    S Byun, S Yoon, K Jung
    IEEE BigComp 2019

57. Detecting Incongruity Between News Headline and Body Text via a Deep Hierarchical Encoder [pdf] [code]
    (oral presentation)
    S Yoon*, K Park*, J Shin, H Lim, S Won, M Cha, K Jung
    AAAI 2019

[2018 and earlier]

58. Multimodal Speech Emotion Recognition using Audio and Text [pdf] [code]
    S Yoon, S Byun, K Jung
    IEEE SLT 2018

59. Comparative Studies of Detecting Abusive Language on Twitter [pdf] [code]
    Y Lee*, S Yoon*, K Jung
    EMNLP ALW 2018

60. Learning to Rank Question-Answer Pairs using Hierarchical Recurrent Encoder with Latent Topic Clustering [pdf] [code]
    S Yoon, J Shin, K Jung
    NAACL 2018

61. Contextual-CNN: A Novel Architecture Capturing Unified Meaning for Sentence Classification [pdf]
    J Shin, Y Kim, S Yoon, K Jung
    IEEE BigComp 2018

62. Synonym Discovery with Etymology-based Word Embeddings [pdf]
    S Yoon, P Estrada, K Jung
    IEEE SSCI 2017

63. Efficient Transfer Learning Schemes for Personalized Language Modeling using Recurrent Neural Network [pdf]
    S Yoon, H Yun, Y Kim, G Park, K Jung
    AAAI 2017 (Workshop)

64. Automatic Question Answering System for Consumer Product [pdf]
    S Yoon, M Sundar, A Gupta, K Jung
    IntelliSys 2016

65. Mining the Minds of Customers from Online Chat Logs [pdf]
    K Park, J Kim, J Park, M Cha, J Nam, S Yoon, E Rhim
    CIKM 2015

66. Domain Question Answering System [pdf]
    S Yoon, E Rhim, D Kim
    KIISE Transactions on Computing Practices 2015

67. Media clips: Implementation of an intuitive media linker [pdf]
    S Yoon, K Lee, H Shin
    IEEE BMSB 2011

Patents

1. [issued] Using neural networks to detect incongruence between headlines and body text of documents [link]
   Seunghyun Yoon
   US 12,038,960, 16-Jul-24

2. [issued] Utilizing a graph neural network to identify supporting text phrases and generate digital query responses [link]
   S Yoon, F Dernoncourt, DS Kim, T Bui
   US 11,271,876, 8-Mar-22

3. [issued] Utilizing bi-directional recurrent encoders with multi-hop attention for speech emotion recognition [link]
   T Bui, S Dey, S Yoon
   US 11,205,444, 21-Dec-21

4. [issued] Answer selection using a compare-aggregate model with language model and condensed similarity information from latent clustering [link]
   S Yoon, F Dernoncourt, T Bui, DS Kim, CI Dockhorn, Y Gong
   US 11,113,323, 7-Sep-21

5. [issued] Terminal apparatus, server and method of controlling the same [link]
   Y Kim, O Kwon, S Kim, H Oh, S Yoon, S Cha, J Lee
   US 10,084,850, CN 201410085759, EP20140154718, 25-Sep-18

6. Method and device for analyzing user's emotion [link]
   E Rhim, J Kim, J Nam, S Yoon, K Park, J Park, M Cha
   WO2016182393, 13-May-16

7. [issued] Method of recommending application, mobile terminal using the method, and communication system using the method [link]
   J Nam, M Lee, M Koo, S Yoon
   US 9,247,376, 26-Jan-16

8. [issued] Method and apparatus for displaying photo on screen having any shape [link]
   S Yoon, M Lee
   US 9,049,383, 2-Jun-15

9. [issued] Method and apparatus for providing information and computer readable storage medium having a program recorded thereon for executing the method [link]
   S Yoon, M Lee, M Koo, J Nam
   US 8,958,824, 17-Feb-15

10. [issued] Apparatus and method for clipping and sharing content at a portable terminal [link]
    S Yoon, M Lee, M Koo, J Nam
    US 13/629,394, CN103827913A, EP20120837007, PCT/KR1020110097578, 28-May-14

11. [issued] Method and apparatus for fast tracking position by using global positioning system [link]
    S Yoon, S Kim
    US 8,094,070, 10-Jan-12

1. [issued] Apparatus and method for evaluating sentense by using bidirectional language model
   K Jung, J Shin, S Yoon
   KR 10-2436900, 23-Aug-22

2. [issued] Method and apparatus for emotion recognition based on cross attentionmodel
   K Jung, Y Lee, S Yoon
   KR 10-2365433, 16-Feb-22

3. [issued] Artificial intelligence based dialog system and response control method thereof
   K Jung, S Yoon, J Shin, H Kwak, S Byun
   KR 10-2059015, 18-Dec-19

4. [issued] Terminal apparatus, server and method of controlling the same
   Y Kim, O Kwon, S Kim, H Oh, S Yoon, S Cha, J Lee
   KR 10-1832394, 20-Feb-18

5. [issued] Apparatus and method for collecting information of destination in portable terminal
   S Yoon, J Nam, M Koo, M Lee
   KR 10-1914632, 29-Oct-18

6. [issued] Method and apparatus for providing information, and computer readable storage medium
   S Yoon, M Lee, M Koo, J Nam
   KR 10-1773167, 24-Aug-17

7. [issued] Method for recommendation of application, mobile terminal thereof and communication system thereof
   J Nam, M Lee, M Koo, S Yoon
   KR 10-1747303, 8-Jun-17

8. Device and method for analyzing user emotion
   E Rhim, J Kim, J Nam, S Yoon, K Park, J Park, M Cha
   KR 1020160058782, 13-May-16

9. [issued] Method and apparatus for fast positioning using global positioning system
   S Yoon, S Kim
   KR 10-1564938, 27-Oct-15