To realize an information-centric networking, IPFS (InterPlanetary File System) generates a unique ContentID for each content by applying a cryptographic hash to the content itself. Although it could improve the security against attacks such as falsification, it makes difficult to realize a similarity search in the framework of IPFS, since the similarity of contents is not reflected in the proximity of ContentIDs. To overcome this issue, we propose a method to apply a locality sensitive hash (LSH) to feature vectors extracted from contents as the key of indexes stored in IPFS. By conducting experiments with 10,000 random points corresponding to stored contents, we found that more than half of randomly given queries return a non-empty result for the similarity search, and yield an accurate result which is outside the σ confidence interval of an ordinary flooding-based method. Note that such a collection of random points corresponds to the worst case scenario for the proposed scheme since the performance of similarity search could improve when points and queries follow an uneven distribution.

DHT routing algorithms can provide efficient mechanisms for resource placement and lookup for distributed file sharing systems. However, we must still deal with irregular and frequent join/leave of nodes and the problem of load unbalancing between nodes in DHT-based file sharing systems. This paper presents an efficient file backup scheme based on dynamic DHT key space clustering in order to guarantee data availability and support load balancing. The main idea of our method is to dynamically divide the DHT network into a number of clusters, each of which locally stores and maintains data chunks of data files to guarantee the data availability of user data files even when node churn occurs. Further, high-capacity nodes in clusters are selected as backup nodes to achieve adequate load balancing. Simulation results demonstrate the superior effectiveness of the proposed scheme over other file replication schemes.

At present, vast numbers of information resources are available on the Internet. However, one emerging issue is how to search and exploit these information resources in an efficient and flexible manner with high scalability. In this study, we focused our attention on the design of a distributed hash table (DHT)-based search system that supports efficient scalable multi-attribute queries of information resources in a distributed manner. Our proposed system, named D-AVTree, is built on top of a ring-based DHT, which partitions a one-dimensional key space across nodes in the system. It utilizes a descriptive naming scheme, which names each resource using an attribute-value (AV) tree, and the resource names are mapped to d-bit keys in order to distribute the resource information to responsible nodes based on a DHT routing algorithm. Our mapping scheme maps each AV branch of a resource name to a d-bit key where AV branches that share a subsequence of AV pairs are mapped to a continuous portion of the key space. Therefore, our mapping scheme ensures that the number of resources distributed to a node is small and it facilitates efficient multi-attribute queries by querying only a small number of nodes. Further, the scheme has good compatibility with key-based load balancing algorithms for DHT-based networks. Our system can achieve both efficiency and a good degree of load balancing even when the distribution of AV pairs in the resource names is skewed. Our simulation results demonstrated the efficiency of our solution in terms of the distribution cost, query hit ratio, and the degree of load balancing compared with conventional approaches.

The rendering mechanism plays an indispensable role in browser-based Web application. It generates active webpages dynamically and provides human-readable layout through template engines, which are used as a standard programming model to separate the business logic and data computations from the webpage presentation. The client-side rendering mechanism, owing to the advances of rich application technologies, has been widely adopted. The adoption of client side rendering brings not only various merits but also new problems. In this paper, we propose and construct “pagelet”, a segment-based template engine for developing flexible and extensible Web applications. By presenting principles, practice and usage experience of pagelet, we conduct a comprehensive analysis of possible advantages and disadvantages brought by client-side rendering mechanism from the viewpoints of both developers and end-users.

The number of networked devices of sensors and actuators continues to increase. We are developing a data-sharing mechanism called uTupleSpace as middleware for storing and retrieving ubiquitous data that are input or output by such devices. uTupleSpace enables flexible retrieval of sensor data and flexible control of actuator devices, and it simplifies the development of various applications. Though uTupleSpace requires scalability against increasing amounts of ubiquitous data, traditional load-distribution methods using a distributed hash table (DHT) are unsuitable for our case because of the ununiformity of the data. Data are nonuniformly generated at some particular times, in some particular positions, and by some particular devices, and their hash values focus on some particular values. This feature makes it difficult for the traditional methods to sufficiently distribute the load by using the hash values. Therefore, we propose a new load-distribution method using a DHT called the dynamic-help method. The proposed method enables one or more peers to handle loads related to the same hash value redundantly. This makes it possible to handle the large load related to one hash value by distributing the load among peers. Moreover, the proposed method reduces the load caused by dynamic load-redistribution. Evaluation experiments showed that the proposed method achieved sufficient load-distribution even when the load was concentrated on one hash value with low overhead. We also confirmed that the proposed method enabled uTupleSpace to accommodate the increasing load with simple operational rules stably and with economic efficiency.

In DHT network, a node can get/put a requested data by only log N look-up steps. However, conventional DHT network only supports single query look-up to search data. From the reason, each node in a DHT network must execute look-up process for each query even if a large number of put and get operations are executed. Therefore, this results in high network load in massive data management such as MapReduce, sensor network, and web information. To address the problem, we propose multiple queries look-up architecture using range information feedback (MARIF). MARIF extends the conventional KBR protocol to supports range information that is a scope of ID space a node keeps. When a source node receives range information from a destination node, the source node checks all queries in the range information and forwards queries matching the range information to the destination node directly. This effectively reduces the number of look-up queries and the network load for the IP network. In addition, MARIF can be implemented into conventional DHT networks and can easily be combined to effective DHT routing algorithms such as Chord, Kademlia, Pastry, and one-hop DHT. In evaluation, we implement MARIF into three DHT networks and compare its performance with that of conventional query bundling mechanisms based on the KBR protocol. The results show that MARIF reduces by up to 40% the total number of forwarding queries to put data compared with other mechanisms. In addition, MARIF saves the number of forwarding queries per look-up process by up to 85% compared to other mechanisms with low bundling overhead.

With the wide-spread use of high-speed network connections and high performance mobile/sensor terminals available, new interactive services based on real-time contents have become available over the Internet. In these services, end-nodes (e.g, smart phone, sensors), which are dispersed over the Internet, generates the real-time contents (e.g, live video, sensor data about human activity), and those contents are utilized to support many kinds of human activities seen in the real world. For the services, a new decentralized contents distribution system which can accommodate a large number of content distributions and which can minimize the end-to-end streaming delay between the content publisher and the subscribers is proposed. In order to satisfy the requirements, the proposed content distribution system is equipped with utilizing two distributed resource selection methods. The first method, distributed hash table (DHT)-based contents management, makes it possible for the system to efficiently decide and locate the server managing content distributions in completely decentralized manner. And, the second one, location-aware server selection, is utilized to quickly select the appropriate servers that distribute the streamed contents to all subscribers in real time. This paper considers the performance of the proposed resource selection methods using a realistic computer simulation and shows that the system with the proposed methods has scalability for a large-scale distributed system that attracts a very large number of users, and achieves real-time locating of the contents without degrading end-to-end streaming delay of content.

Distributed hash tables (DHTs) enable companies or campuses to use voice over IP (VoIP) systems in private networks at low cost. However, DHTs incur latency stretch caused by a topology mismatch between the application layer and the physical layer. Reducing this latency stretch is critical when implementing real-time applications that use DHTs, such as VoIP systems. This letter proposes a new method for reducing the latency stretch by creating node IDs, which form structured overlays that consider node proximity. Simulation results show that the proposed method is superior to Chord in terms of the distance between nodes, look-up path length, network burden, and look-up latency.

The Internet has grown extremely fast in the last two decades. The number of routes to be supported by the routers has become very large. Moreover, the number of messages exchanged to distribute the routes has increased even faster. In this paper, we propose SpliTable, a scalable way to support the Internet routes in a Service Provider network. In our proposal, BGP route selection is done by distributed servers on behalf of the routers. They are called route selection servers. The selected routes are then stored in distributed routing tables. Each router maintains only its share of Internet routes, not the routes for each Internet prefix as it is the case today. We adapted the concept of Distributed Hash Tables (DHT) for that purpose. We show analytically that our proposal is more scalable in the number of routes supported in each router than current iBGP route distribution solutions. Moreover, the number of control messages exchanged with our proposal is bounded contrary to current sparse iBGP route distribution solutions which may never converge. We confirm these findings in an evaluation of a prototype implementation.

The basic requirements of the distributed Web crawling systems are: short download time, low communication overhead and balanced load which largely depends on the systems' Web partition strategies. In this paper, we propose a DHT-based distributed Web crawling system and several DHT-based Web partition methods. First, a new system model based on a DHT method called the Content Addressable Network (CAN) is proposed. Second, based on this model, a network-distance-based Web partition is implemented to reduce the crawler-crawlee network distance in a fully distributed manner. Third, by utilizing the locality on the link space, we propose the concept of link-based Web partition to reduce the communication overhead of the system. This method not only reduces the number of inter-links to be exchanged among the crawlers but also reduces the cost of routing on the DHT overlay. In order to combine the benefits of the above two Web partition methods, we then propose 2 distributed multi-objective Web partition methods. Finally, all the methods we propose in this paper are compared with existing system models in the simulated experiments under different datasets and different system scales. In most cases, the new methods show their superiority.

Internet computing is proposed to exploit personal computing resources across the Internet in order to build large-scale Web applications at lower cost. In this paper, a DHT-based distributed Web crawling model based on the concept of Internet computing is proposed. Also, we propose two optimizations to reduce the download time and waiting time of the Web crawling tasks in order to increase the system's throughput and update rate. Based on our contributor-friendly download scheme, the improvement on the download time is achieved by shortening the crawler-crawlee RTTs. In order to accurately estimate the RTTs, a network coordinate system is combined with the underlying DHT. The improvement on the waiting time is achieved by redirecting the incoming crawling tasks to light-loaded crawlers in order to keep the queue on each crawler equally sized. We also propose a simple Web site partition method to split a large Web site into smaller pieces in order to reduce the task granularity. All the methods proposed are evaluated through real Internet tests and simulations showing satisfactory results.

In this paper, a novel orthogonal frequency-division multiplexing (OFDM) modulator/demodulator based on real-valued discrete Hartley transform (DHT) is presented and implemented for the IEEE 802.11a/g wireless local area network (LAN). Instead of the conventional complex-valued fast Fourier transform (FFT) for OFDM systems, the proposed architecture employs two real-valued fast DHT (FHT) kernels and one post processing unit. By taking advantage of the real-valued operation of FHT, this approach reduces the number of multiplications compared with the radix-2 FFT. The proposed DHT-based modulator/demodulator was designed and fabricated in 0.18-µm CMOS technology with a core area of 928935 µm2. The average power consumption is about 20.16 mW at 20 MHz and 1.8 V supply voltage. Measurement results of the integrated circuit illustrate its superior chip area and power consumption.

This paper presents an FPGA implementation of highly modular universal discrete transforms. The implementation relies upon the unified discrete Fourier Hartley transform (UDFHT), based on which essential sinusoidal transforms including discrete Fourier transform (DFT), discrete Hartley transform (DHT), discrete cosine transform (DCT) and discrete sine transform (DST) can be realized. It employs a reconfigurable, scalable and modular architecture that consists of a memory-based FFT processor equipped with pre- and post-processing units. Besides, a pipelining technique is exploited to seamlessly harmonize the operation between each sub-module. Experimental results based on Xilinx Virtex-II Pro are given to examine the performance of the proposed UDFHT implementation. Two practical applications are also shown to demonstrate the flexibility and modularity of the proposed work.

Mobile Ad Hoc Networks (MANETs) are temporal and infrastructure-independent wireless networks that consist of mobile nodes. For instance, a MANET can be used as an emergent network for communication among people when a disaster occurred. Since there is no central server in the network, each node has to find out its desired information (objects) by itself. Constructing a mobile Peer-to-Peer (P2P) network over the MANET can support the object search. Some researchers proposed construction schemes of mobile P2P networks, such as Ekta and MADPastry. They integrated DHT-based application-layer routing and network-layer routing to increase search efficiency. Furthermore, MADPastry proposed a clustering method which groups the overlay nodes according to their physical distance. However, it has also been pointed out that the search efficiency deteriorates in highly dynamic environments where nodes quickly move around. In this paper, we focus on route disappearances in the network layer which cause the deterioration of search efficiency. We describe the detail of this problem and evaluate quantitatively it through simulation experiments. We extend MADPastry by introducing a method sharing objects among nodes in a cluster. Through simulation experiments, we show that the proposed method can achieve up to 2.5 times larger success rate of object search than MADPastry.

P2P DHT (Peer-to-Peer Distributed Hash Table) is one of typical techniques for realizing an efficient management of shared resources distributed over a network and a keyword search over such networks in a fully distributed manner. In this paper, we propose a new method for supporting conjunctive queries in P2P DHT. The basic idea of the proposed technique is to share a global information on past trials by conducting a local caching of search results for conjunctive queries and by registering the fact to the global DHT. Such a result caching is expected to significantly reduce the amount of transmitted data compared with conventional schemes. The effect of the proposed method is experimentally evaluated by simulation. The result of experiments indicates that by using the proposed method, the amount of returned data is reduced by 60% compared with conventional P2P DHT which does not support conjunctive queries.

DHT based p2p systems appear to provide scalable storage services with idle resource from many unreliable clients. If a DHT is used in storage intensive applications where data loss must be minimized, quick replication is especially important to replace lost redundancy on other nodes in reaction to failures. To achieve this easily, a simple replication method directly uses a consistent set, such as a leaf set and a successor list. However, this set is tightly coupled to the current state of nodes and the traffic needed to support this replication can be high and bursty under churn. This paper explores efficient replication methods that only glimpse a consistent set to select a new replica. Replicas are loosely coupled to a consistent set and we can eliminate the compulsory replication under churn. Because of a complication of the new replication methods, the careful data management is needed under churn for the correct and efficient data lookup. Results from a simulation study suggest that our methods can reduce network traffic enormously for high data durability.

Several structured peer-to-peer networks have been created to solve the scalability problem of previous peer-to-peer systems such as Gnutella and Napster. These peer-to-peer networks which support distributed hash table functionality construct a sort of structured overlay network, which can cause a topology mismatch between the overlay and the underlying physical network. To solve this mismatch problem, we propose a topology-aware hierarchical overlay framework for DHTs. The hierarchical approach for the overlay is based on the concept that the underlying global Internet is also a hierarchical architecture, that is, a network of networks. This hierarchical approach for the overlay puts forth two benefits: finding data in a physically near place with a high probability, and smaller lookup time. Our hierarchical overlay framework is different from other hierarchical architecture systems in a sense that it provides a specific self-organizing grouping algorithm. Our additional optimization schemes complete the basic algorithm which constructs a hierarchical structure without any central control.

This paper describes an architecture for deploying virtual IP networks with P2P-like dynamic topology and routing management. Existing virtual IP network deployment mechanisms do not allow for dynamic topology adaptation and fault-tolerance because provisioning of IP tunnels is performed only once--when a virtual network is deployed. We propose a P2P-XBone, in which a P2P protocol such as DHT drives the topology and the routing table of a virtual IP network consistent with its neighbor node state. We describe how to extend both the existing X-Bone system and P2P mechanisms to achieve interworking between them. The P2P-XBone not only provides P2P's characteristics such as self-organization, fault-tolerance and content-based routing to virtual IP networks but also provides higher forwarding performance and simpler implementation to P2P systems due to the support for the use of existing network services. We also show several results on the evaluation of overhead of P2P-driven provisioning and on forwarding performance.

We have designed a scalable and expressive naming system called SENS, capable of retrieving information of computing and content resources distributed widely across the Internet through exact queries and multi-attribute range queries over resource names. Our system utilizes a descriptive naming scheme to name resources and a multi-dimensional resource ID space for message routing through an overlay network of name servers (NSs). The resource ID space is constructed on the overlay network based on CAN routing algorithm. Our novel mapping scheme between resource names and resource IDs preserves resource ID locality while still achieving good load balancing regarding resource information distribution. We also propose a multicast routing algorithm to deliver resource information and a broadcast routing algorithm to route query messages to corresponding NSs with small cost of message transmission. Our simulation results show that our system can achieve good routing performance and load balancing.

Many P2P lookup services based on distributed hash tables (DHT) have appeared recently. These schemes are built upon overlay networks and ignore distance to the target resources. As a result, P2P lookups often suffer from unnecessarily long routes in the underlay network, which we call overlay dilation. This paper proposes a new scheme for resource routing, called hybrid hierarchical overlay routing, dubbed Hyho. We introduce distance-weighted Bloom filters (dwBFs) as a concise representation of routing information for scattered resources in overlay networks. To further reduce the size of Bloom filters, so that they are linear in the number of distinct resources, Hyho splits overlay networks in accordance with DHT, where each subnetwork has a smaller set of resources and spans the entire network thinly. As a result, Hyho constructs a hierarchical overlay network and routes requests accordingly. Simulation results show that Hyho can reduce overlay dilation to one half that yielded by the Chord lookup service.