IEICE TRANSACTIONS on Information
Unbiased Pseudo-Labeling for Learning with Noisy Labels
Summary :
Noisy labels in training data can significantly harm the performance of deep neural networks (DNNs). Recent research on learning with noisy labels uses a property of DNNs called the memorization effect to divide the training data into a set of data with reliable labels and a set of data with unreliable labels. Methods introducing semi-supervised learning strategies discard the unreliable labels and assign pseudo-labels generated from the confident predictions of the model. So far, this semi-supervised strategy has yielded the best results in this field. However, we observe that even when models are trained on balanced data, the distribution of the pseudo-labels can still exhibit an imbalance that is driven by data similarity. Additionally, a data bias is seen that originates from the division of the training data using the semi-supervised method. If we address both types of bias that arise from pseudo-labels, we can avoid the decrease in generalization performance caused by biased noisy pseudo-labels. We propose a learning method with noisy labels that introduces unbiased pseudo-labeling based on causal inference. The proposed method achieves significant accuracy gains in experiments at high noise rates on the standard benchmarks CIFAR-10 and CIFAR-100.
- Publication
- IEICE TRANSACTIONS on Information Vol.E107-D No.1 pp.44-48
- Publication Date
- 2024/01/01
- Publicized
- 2023/09/19
- Online ISSN
- 1745-1361
- DOI
- 10.1587/transinf.2023MUL0002
- Type of Manuscript
- Special Section LETTER (Special Section on Enriched Multimedia — Media technologies opening up the future —)
- Category
Authors
Ryota HIGASHIMOTO
Kansai University
Soh YOSHIDA
Kansai University
Takashi HORIHATA
Kansai University
Mitsuji MUNEYASU
Kansai University
Keyword
Latest Issue
Copyrights notice of machine-translated contents
The copyright of the original papers published on this site belongs to IEICE. Unauthorized use of the original or translated papers is prohibited. See IEICE Provisions on Copyright for details.
Cite this
Copy
Ryota HIGASHIMOTO, Soh YOSHIDA, Takashi HORIHATA, Mitsuji MUNEYASU, "Unbiased Pseudo-Labeling for Learning with Noisy Labels" in IEICE TRANSACTIONS on Information,
vol. E107-D, no. 1, pp. 44-48, January 2024, doi: 10.1587/transinf.2023MUL0002.
Abstract: Noisy labels in training data can significantly harm the performance of deep neural networks (DNNs). Recent research on learning with noisy labels uses a property of DNNs called the memorization effect to divide the training data into a set of data with reliable labels and a set of data with unreliable labels. Methods introducing semi-supervised learning strategies discard the unreliable labels and assign pseudo-labels generated from the confident predictions of the model. So far, this semi-supervised strategy has yielded the best results in this field. However, we observe that even when models are trained on balanced data, the distribution of the pseudo-labels can still exhibit an imbalance that is driven by data similarity. Additionally, a data bias is seen that originates from the division of the training data using the semi-supervised method. If we address both types of bias that arise from pseudo-labels, we can avoid the decrease in generalization performance caused by biased noisy pseudo-labels. We propose a learning method with noisy labels that introduces unbiased pseudo-labeling based on causal inference. The proposed method achieves significant accuracy gains in experiments at high noise rates on the standard benchmarks CIFAR-10 and CIFAR-100.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2023MUL0002/_p
Copy
@ARTICLE{e107-d_1_44,
author={Ryota HIGASHIMOTO, Soh YOSHIDA, Takashi HORIHATA, Mitsuji MUNEYASU, },
journal={IEICE TRANSACTIONS on Information},
title={Unbiased Pseudo-Labeling for Learning with Noisy Labels},
year={2024},
volume={E107-D},
number={1},
pages={44-48},
abstract={Noisy labels in training data can significantly harm the performance of deep neural networks (DNNs). Recent research on learning with noisy labels uses a property of DNNs called the memorization effect to divide the training data into a set of data with reliable labels and a set of data with unreliable labels. Methods introducing semi-supervised learning strategies discard the unreliable labels and assign pseudo-labels generated from the confident predictions of the model. So far, this semi-supervised strategy has yielded the best results in this field. However, we observe that even when models are trained on balanced data, the distribution of the pseudo-labels can still exhibit an imbalance that is driven by data similarity. Additionally, a data bias is seen that originates from the division of the training data using the semi-supervised method. If we address both types of bias that arise from pseudo-labels, we can avoid the decrease in generalization performance caused by biased noisy pseudo-labels. We propose a learning method with noisy labels that introduces unbiased pseudo-labeling based on causal inference. The proposed method achieves significant accuracy gains in experiments at high noise rates on the standard benchmarks CIFAR-10 and CIFAR-100.},
keywords={},
doi={10.1587/transinf.2023MUL0002},
ISSN={1745-1361},
month={January},}
Copy
TY - JOUR
TI - Unbiased Pseudo-Labeling for Learning with Noisy Labels
T2 - IEICE TRANSACTIONS on Information
SP - 44
EP - 48
AU - Ryota HIGASHIMOTO
AU - Soh YOSHIDA
AU - Takashi HORIHATA
AU - Mitsuji MUNEYASU
PY - 2024
DO - 10.1587/transinf.2023MUL0002
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E107-D
IS - 1
JA - IEICE TRANSACTIONS on Information
Y1 - January 2024
AB - Noisy labels in training data can significantly harm the performance of deep neural networks (DNNs). Recent research on learning with noisy labels uses a property of DNNs called the memorization effect to divide the training data into a set of data with reliable labels and a set of data with unreliable labels. Methods introducing semi-supervised learning strategies discard the unreliable labels and assign pseudo-labels generated from the confident predictions of the model. So far, this semi-supervised strategy has yielded the best results in this field. However, we observe that even when models are trained on balanced data, the distribution of the pseudo-labels can still exhibit an imbalance that is driven by data similarity. Additionally, a data bias is seen that originates from the division of the training data using the semi-supervised method. If we address both types of bias that arise from pseudo-labels, we can avoid the decrease in generalization performance caused by biased noisy pseudo-labels. We propose a learning method with noisy labels that introduces unbiased pseudo-labeling based on causal inference. The proposed method achieves significant accuracy gains in experiments at high noise rates on the standard benchmarks CIFAR-10 and CIFAR-100.
ER -
English
