The search functionality is under construction.
The search functionality is under construction.

Rephrasing Visual Questions by Specifying the Entropy of the Answer Distribution

Kento TERAO, Toru TAMAKI, Bisser RAYTCHEV, Kazufumi KANEDA, Shin'ichi SATOH

  • Full Text Views

    0

  • Cite this

Summary :

Visual question answering (VQA) is a task of answering a visual question that is a pair of question and image. Some visual questions are ambiguous and some are clear, and it may be appropriate to change the ambiguity of questions from situation to situation. However, this issue has not been addressed by any prior work. We propose a novel task, rephrasing the questions by controlling the ambiguity of the questions. The ambiguity of a visual question is defined by the use of the entropy of the answer distribution predicted by a VQA model. The proposed model rephrases a source question given with an image so that the rephrased question has the ambiguity (or entropy) specified by users. We propose two learning strategies to train the proposed model with the VQA v2 dataset, which has no ambiguity information. We demonstrate the advantage of our approach that can control the ambiguity of the rephrased questions, and an interesting observation that it is harder to increase than to reduce ambiguity.

Publication
IEICE TRANSACTIONS on Information Vol.E103-D No.11 pp.2362-2370
Publication Date
2020/11/01
Publicized
2020/08/20
Online ISSN
1745-1361
DOI
10.1587/transinf.2020EDP7089
Type of Manuscript
PAPER
Category
Image Recognition, Computer Vision

Authors

Kento TERAO
  Hiroshima University
Toru TAMAKI
  Hiroshima University
Bisser RAYTCHEV
  Hiroshima University
Kazufumi KANEDA
  Hiroshima University
Shin'ichi SATOH
  National Institute of Informatics

Keyword