Similarity Search in InterPlanetary File System with the Aid of Locality Sensitive Hash

Satoshi FUJITA

doi:10.1587/transinf.2020EDP7198

Similarity Search in InterPlanetary File System with the Aid of Locality Sensitive Hash

Satoshi FUJITA

Full Text Views

0

Cite this

Summary :

To realize an information-centric networking, IPFS (InterPlanetary File System) generates a unique ContentID for each content by applying a cryptographic hash to the content itself. Although it could improve the security against attacks such as falsification, it makes difficult to realize a similarity search in the framework of IPFS, since the similarity of contents is not reflected in the proximity of ContentIDs. To overcome this issue, we propose a method to apply a locality sensitive hash (LSH) to feature vectors extracted from contents as the key of indexes stored in IPFS. By conducting experiments with 10,000 random points corresponding to stored contents, we found that more than half of randomly given queries return a non-empty result for the similarity search, and yield an accurate result which is outside the σ confidence interval of an ordinary flooding-based method. Note that such a collection of random points corresponds to the worst case scenario for the proposed scheme since the performance of similarity search could improve when points and queries follow an uneven distribution.

Publication: IEICE TRANSACTIONS on Information Vol.E104-D No.10 pp.1616-1623

Publication Date: 2021/10/01

Publicized: 2021/07/08

Online ISSN: 1745-1361

DOI: 10.1587/transinf.2020EDP7198

Type of Manuscript: PAPER

Category: Information Network

Authors

Satoshi FUJITA
Hiroshima University

Keyword

Cite this

Copy

Satoshi FUJITA, "Similarity Search in InterPlanetary File System with the Aid of Locality Sensitive Hash" in IEICE TRANSACTIONS on Information, vol. E104-D, no. 10, pp. 1616-1623, October 2021, doi: 10.1587/transinf.2020EDP7198.
Abstract: To realize an information-centric networking, IPFS (InterPlanetary File System) generates a unique ContentID for each content by applying a cryptographic hash to the content itself. Although it could improve the security against attacks such as falsification, it makes difficult to realize a similarity search in the framework of IPFS, since the similarity of contents is not reflected in the proximity of ContentIDs. To overcome this issue, we propose a method to apply a locality sensitive hash (LSH) to feature vectors extracted from contents as the key of indexes stored in IPFS. By conducting experiments with 10,000 random points corresponding to stored contents, we found that more than half of randomly given queries return a non-empty result for the similarity search, and yield an accurate result which is outside the σ confidence interval of an ordinary flooding-based method. Note that such a collection of random points corresponds to the worst case scenario for the proposed scheme since the performance of similarity search could improve when points and queries follow an uneven distribution.
URL: https://global.ieice.org/en_transactions/information/10.1587/transinf.2020EDP7198/_p

Copy

@ARTICLE{e104-d_10_1616,
author={Satoshi FUJITA, },
journal={IEICE TRANSACTIONS on Information},
title={Similarity Search in InterPlanetary File System with the Aid of Locality Sensitive Hash},
year={2021},
volume={E104-D},
number={10},
pages={1616-1623},
abstract={To realize an information-centric networking, IPFS (InterPlanetary File System) generates a unique ContentID for each content by applying a cryptographic hash to the content itself. Although it could improve the security against attacks such as falsification, it makes difficult to realize a similarity search in the framework of IPFS, since the similarity of contents is not reflected in the proximity of ContentIDs. To overcome this issue, we propose a method to apply a locality sensitive hash (LSH) to feature vectors extracted from contents as the key of indexes stored in IPFS. By conducting experiments with 10,000 random points corresponding to stored contents, we found that more than half of randomly given queries return a non-empty result for the similarity search, and yield an accurate result which is outside the σ confidence interval of an ordinary flooding-based method. Note that such a collection of random points corresponds to the worst case scenario for the proposed scheme since the performance of similarity search could improve when points and queries follow an uneven distribution.},
keywords={},
doi={10.1587/transinf.2020EDP7198},
ISSN={1745-1361},
month={October},}

Copy

TY - JOUR
TI - Similarity Search in InterPlanetary File System with the Aid of Locality Sensitive Hash
T2 - IEICE TRANSACTIONS on Information
SP - 1616
EP - 1623
AU - Satoshi FUJITA
PY - 2021
DO - 10.1587/transinf.2020EDP7198
JO - IEICE TRANSACTIONS on Information
SN - 1745-1361
VL - E104-D
IS - 10
JA - IEICE TRANSACTIONS on Information
Y1 - October 2021
AB - To realize an information-centric networking, IPFS (InterPlanetary File System) generates a unique ContentID for each content by applying a cryptographic hash to the content itself. Although it could improve the security against attacks such as falsification, it makes difficult to realize a similarity search in the framework of IPFS, since the similarity of contents is not reflected in the proximity of ContentIDs. To overcome this issue, we propose a method to apply a locality sensitive hash (LSH) to feature vectors extracted from contents as the key of indexes stored in IPFS. By conducting experiments with 10,000 random points corresponding to stored contents, we found that more than half of randomly given queries return a non-empty result for the similarity search, and yield an accurate result which is outside the σ confidence interval of an ordinary flooding-based method. Note that such a collection of random points corresponds to the worst case scenario for the proposed scheme since the performance of similarity search could improve when points and queries follow an uneven distribution.
ER -