Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning

Takumi EGE; Keiji YANAI

doi:10.1587/transinf.2018CEP0004

Open Access
Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning

Takumi EGE, Keiji YANAI

Full Text Views

76

Cite this

Free PDF (2.6MB)

Summary :

In recent years, a rise in healthy eating has led to various food management applications which have image recognition function to record everyday meals automatically. However, most of the image recognition functions in the existing applications are not directly useful for multiple-dish food photos and cannot automatically estimate food calories. Meanwhile, methodologies on image recognition have advanced greatly because of the advent of Convolutional Neural Network (CNN). CNN has improved accuracies of various kinds of image recognition tasks such as classification and object detection. Therefore, we propose CNN-based food calorie estimation for multiple-dish food photos. Our method estimates dish locations and food calories simultaneously by multi-task learning of food dish detection and food calorie estimation with a single CNN. It is expected to achieve high speed and small network size by simultaneous estimation in a single network. Because currently there is no dataset of multiple-dish food photos annotated with both bounding boxes and food calories, in this work we use two types of datasets alternately for training a single CNN. For the two types of datasets, we use multiple-dish food photos annotated with bounding boxes and single-dish food photos with food calories. Our results showed that our multi-task method achieved higher accuracy, higher speed and smaller network size than a sequential model of food detection and food calorie estimation.

Publication: IEICE TRANSACTIONS on Information Vol.E102-D No.7 pp.1240-1246

Publication Date: 2019/07/01

Publicized: 2019/04/25

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2018CEP0004

Type of Manuscript: Special Section PAPER (Special Section on Multimedia for Cooking and Eating Activities)

Category

Authors

Takumi EGE
The University of Electro-Communications
Keiji YANAI
The University of Electro-Communications

Keyword

food calorie estimation, food dish detection, multi-task learning

Cite this

Copy

Takumi EGE, Keiji YANAI, "Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning" in IEICE TRANSACTIONS on Information, vol. E102-D, no. 7, pp. 1240-1246, July 2019, doi: 10.1587/transinf.2018CEP0004.
Abstract: In recent years, a rise in healthy eating has led to various food management applications which have image recognition function to record everyday meals automatically. However, most of the image recognition functions in the existing applications are not directly useful for multiple-dish food photos and cannot automatically estimate food calories. Meanwhile, methodologies on image recognition have advanced greatly because of the advent of Convolutional Neural Network (CNN). CNN has improved accuracies of various kinds of image recognition tasks such as classification and object detection. Therefore, we propose CNN-based food calorie estimation for multiple-dish food photos. Our method estimates dish locations and food calories simultaneously by multi-task learning of food dish detection and food calorie estimation with a single CNN. It is expected to achieve high speed and small network size by simultaneous estimation in a single network. Because currently there is no dataset of multiple-dish food photos annotated with both bounding boxes and food calories, in this work we use two types of datasets alternately for training a single CNN. For the two types of datasets, we use multiple-dish food photos annotated with bounding boxes and single-dish food photos with food calories. Our results showed that our multi-task method achieved higher accuracy, higher speed and smaller network size than a sequential model of food detection and food calorie estimation.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2018CEP0004/_p

Copy

@ARTICLE{e102-d_7_1240,
author={Takumi EGE, Keiji YANAI, },
journal={IEICE TRANSACTIONS on Information},
title={Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning},
year={2019},
volume={E102-D},
number={7},
pages={1240-1246},
abstract={In recent years, a rise in healthy eating has led to various food management applications which have image recognition function to record everyday meals automatically. However, most of the image recognition functions in the existing applications are not directly useful for multiple-dish food photos and cannot automatically estimate food calories. Meanwhile, methodologies on image recognition have advanced greatly because of the advent of Convolutional Neural Network (CNN). CNN has improved accuracies of various kinds of image recognition tasks such as classification and object detection. Therefore, we propose CNN-based food calorie estimation for multiple-dish food photos. Our method estimates dish locations and food calories simultaneously by multi-task learning of food dish detection and food calorie estimation with a single CNN. It is expected to achieve high speed and small network size by simultaneous estimation in a single network. Because currently there is no dataset of multiple-dish food photos annotated with both bounding boxes and food calories, in this work we use two types of datasets alternately for training a single CNN. For the two types of datasets, we use multiple-dish food photos annotated with bounding boxes and single-dish food photos with food calories. Our results showed that our multi-task method achieved higher accuracy, higher speed and smaller network size than a sequential model of food detection and food calorie estimation.},
keywords={},
doi={10.1587/transinf.2018CEP0004},
ISSN={1745-1361},
month={July},}

Copy

TY - JOUR
TI - Simultaneous Estimation of Dish Locations and Calories with Multi-Task Learning
T2 - IEICE TRANSACTIONS on Information
SP - 1240
EP - 1246
AU - Takumi EGE
AU - Keiji YANAI
PY - 2019
DO - 10.1587/transinf.2018CEP0004
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E102-D
IS - 7
JA - IEICE TRANSACTIONS on Information
Y1 - July 2019
AB - In recent years, a rise in healthy eating has led to various food management applications which have image recognition function to record everyday meals automatically. However, most of the image recognition functions in the existing applications are not directly useful for multiple-dish food photos and cannot automatically estimate food calories. Meanwhile, methodologies on image recognition have advanced greatly because of the advent of Convolutional Neural Network (CNN). CNN has improved accuracies of various kinds of image recognition tasks such as classification and object detection. Therefore, we propose CNN-based food calorie estimation for multiple-dish food photos. Our method estimates dish locations and food calories simultaneously by multi-task learning of food dish detection and food calorie estimation with a single CNN. It is expected to achieve high speed and small network size by simultaneous estimation in a single network. Because currently there is no dataset of multiple-dish food photos annotated with both bounding boxes and food calories, in this work we use two types of datasets alternately for training a single CNN. For the two types of datasets, we use multiple-dish food photos annotated with bounding boxes and single-dish food photos with food calories. Our results showed that our multi-task method achieved higher accuracy, higher speed and smaller network size than a sequential model of food detection and food calorie estimation.
ER -