Parallel processing is a typical approach to answer analytical queries on large database. As the size of the database increases, we often try to increase the parallelism by incorporating more processing nodes. However, this approach increases the possibility of node failure as well. According to the conventional practice, if a failure occurs during query processing, the database system restarts the query processing from the beginning. Such temporal cost may be unacceptable to the user. This paper proposes a fault-tolerant query processing mechanism, named PhoeniQ, for analytical parallel database systems. PhoeniQ continuously takes a checkpoint for every operator pipeline and replicates the output of each stateful operator among different processing nodes. If a single processing node fails during query processing, another can promptly take over the processing. Hence, PhoneniQ allows the database system to efficiently resume query processing after a partial failure event. This paper presents a key design of PhoeniQ and prototype-based experiments to demonstrate that PhoeniQ imposes negligible performance overhead and efficiently continues query processing in the face of node failure.

In this paper, we exploit MapReduce framework and other optimizations to improve the performance of hash join algorithms on multi-core CPUs, including No partition hash join and partition hash join. We first implement hash join algorithms with a shared-memory MapReduce model on multi-core CPUs, including partition phase, build phase, and probe phase. Then we design an improved cuckoo hash table for our hash join, which consists of a cuckoo hash table and a chained hash table. Based on our implementation, we also propose two optimizations, one for the usage of SIMD instructions, and the other for partition phase. Through experimental result and analysis, we finally find that the partition hash join often outperforms the No partition hash join, and our hash join algorithm is faster than previous work by an average of 30%.

In recent years, society has experienced several changes in its ways and methods of consuming. Nowadays, the diversity and the customization of products and services have provoked that the consumer needs continuously change. Hence, the database systems support e-business processes are required to be timeliness and adaptable to the changing preferences. Autonomous Decentralized Database System (ADDS), has been proposed in order to satisfy the enhanced requirements of current on-line e-business applications. Autonomy and decentralization of subsystems help to achieve short response times in highly competitive situations and an autonomous Coordination Mobile Agent (CMA) has been proposed to achieve flexibility in a highly dynamic environment. However, a problem in ADDS is as the number of sites increases, the distribution and harmonization of product information among the sites are turning difficult. Therefore, many users cannot be satisfied quickly. As a result, system timeliness is inadequate. To solve this problem, a self configuration technology is proposed. This technology can configure the system to the evolving situation dynamically for achieving high response. A simulation shows the effectiveness of the proposed technology in a large-scale system. Finally, an implementation of this technology is presented.

Information Security Management Systems (ISMSs) play important roles in helping organizations to manage their information securely. However, establishing, managing, and maintaining ISMSs is not an easy task for most organizations because an ISMS has many participants and tasks, and requires many kinds of documents. Therefore, organizations with ISMSs demand tools that can support them to perform all tasks in ISMS lifecycle processes consistently and continuously. To realize such support tools, a database system that manages ISO/IEC 27000 series, which are international standards for ISMSs, and ISMS documents, which are the products of tasks in ISMS lifecycle processes, is indispensable. The database system should manage data of the standards and documents for all available versions and translations, relationship among the standards and documents, authorization to access the standards and documents, and metadata of the standards and documents. No such database system has existed until now. This paper presents an information security management database system (ISMDS) that manages ISO/IEC 27000 series and ISMS documents. ISMDS is a meta-database system that manages several databases of standards and documents. ISMDS is used by participants in ISMS as well as tools supporting the participants to perform tasks in ISMS lifecycle processes. The users or tools can retrieve data from all versions and translations of the standards and documents. The paper also presents some use cases to show the effectiveness of ISMDS.

To better support data-intensive workflows which are typically built out of various independently developed executables, this paper proposes extensions to parallel database systems called User-Defined eXecutables (UDX) and collective queries. UDX facilitates the description of workflows by enabling seamless integrations of external executables into SQL statements without any efforts to write programs confirming to strict specifications of databases. A collective query is an SQL query whose results are distributed to multiple clients and then processed by them in parallel, using arbitrary UDX. It provides efficient parallelization of executables through the data transfer optimization algorithms that distribute query results to multiple clients, taking both communication cost and computational loads into account. We implement this concept in a system called ParaLite, a parallel database system based on a popular lightweight database SQLite. Our experiments show that ParaLite has several times higher performance over Hive for typical SQL tasks and has 10x speedup compared to a commercial DBMS for executables. In addition, this paper studies a real-world text processing workflow and builds it on top of ParaLite, Hadoop, Hive and general files. Our experiences indicate that ParaLite outperforms other systems in both productivity and performance for the workflow.

The turn of the century is witnessing radical changes in the way information services are spreading due to the progress of IT and the constantly increase in the number of users of the WWW. Therefore, the business market is changing its strategy for a modern online business environment. Autonomous Decentralized Database System (ADDS), based on autonomous coordinating subsystems, has been proposed as a system architecture in order to meet the innovative e-business requirements for consistency and high response among distributed database systems. Autonomy and decentralization of subsystems help achieving high response time in highly competitive situation and autonomous Mobile Agent based coordination has been proposed to achieve flexibility in a highly dynamic environment. In this paper, it is analyzed the case in which the system size increases; and a multi agent coordination, the same number of mobile agents and sites coexist in the system, is proposed for achieving the timeliness property. The response time in the system is conformed by those transactions that require coordination and those that can be satisfied immediately. In accordance, the distribution of the data in the system for coordination is a medullar issue for the improvement of the response time. A trade-off exits between these two kind of transactions depending on the coordination of the Mobile Agents, the capacity of allocating data among the sites, and as well as the distribution of the data and user requests in the system. In this sense, since the system requires high response time, a data allocation technology in which each mobile agent autonomously determine its own capacity for adjusting data among the sites is proposed. Thus, the system will adapt itself to the dynamic environment. The effectiveness of the proposed architecture and technologies are evaluated by simulation.

Secret sharing is a method for distributing a secret among a party of participants. Each of them is allocated a share of the secret, and the secret can only be reconstructed when the shares are combined together. We have been proposing a secret sharing distributed database system (SSDDB) that uses a secret sharing scheme to improve confidentiality and robustness of distributed database systems. This paper proposes a vertical partitioning algorithm for the SSDDB, and evaluates the algorithm by computational experiments.

Distributed database systems require a commit process to preserve the ACID property of transactions executed on a number of system sites. With the appearance of main memory database system, the database processing time has been reduced in the order of magnitude, since the database access does not incur any disk access at all. However, when it comes to distributed main memory database systems, the distributed commit process is still very slow since the disk logging at several sites has to precede the transaction commit. In this paper, we re-evaluate various distributed commit protocols and come up with a causal commit protocol suitable for distributed main memory database systems. To evaluate the performance of the proposed commit protocol, extensive simulation study has been performed. The simulation results confirm that the new protocol greatly reduces the time to commit the distributed transactions without any consistency problem.

We suggest a new probe message structure and an efficient probe-based deadlock detection and recovery algorithm that can be used in distributed database systems. We determine the characteristics of the probe messages and suggest an algorithm that can reduce the communication cost required for deadlock detection and recovery.

In this paper, we consider optimal mirror allocation problems for the purpose of load balancing in network servers. We focus on constructing high-reliability networks and propose the optimal mirror allocation model such that the system reliability is maximized subject to costs and delays, in view of the trade-off between the reliability and cost. This optimization model is capable of dealing with various kinds of network topologies, although for simplicity, we assume the read-only situation. We formulate this optimization problem into a 0-1 integer programming model, and we use an approximate method for numerical analysis in order to analyze more large-scale systems. Our objective is to find the optimal mirror allocation by solving this model, and to show quantitatively the general characteristics of the load balancing and the improvement of the system reliability by the distributed mirror allocation.

In this paper, an effective index structure for dynamic main memory database systems, which we call the T2-tree, is presented. A notion of a thread pointer is introduced to overcome some of the limitations of the T-tree and the T*-tree. There are several advantages to this structure. First, the T2-tree reduces the number of rotate operations and the overhead required for balancing the tree by restraining new node creation and deletion. Second, the T2-tree shows good performance for sequential search of range queries as these requests can be effectively handled using the successor pointer. Finally, the T2-tree allows for higher space utilization amplicating the aforementioned benefits. These advantages are obtained with minimal changes to the existing T-tree structure. Experimental studies showing evidence of the benefits of the T2-tree are also presented.

In distributed network systems, it is one of the most important problems how to assign the files to servers in view of cost and delay. It is obvious that there is a trading-off relationship between costs and delays in these systems. In order to evaluate the optimization that the total cost is minimized subject to the total delay, we have presented the Optimal File Allocation Model as 0-1 integer programming, and have investigated the general characteristics in distributed systems. In this model, we have introduced many cost and delay parameters to evaluate the total cost and delay in the system more exactly. In constructing practical systems, it is necessary to investigate the weight and the contribution of each parameter to the total cost. It is very useful to show how to estimate cost and delay parameters on the basis of this analysis. In this paper, we analyze the sensitivity of these parameters and make clear the influence between principal parameters.

This paper surveys recent research activities on three major areas of digital media information base, namely, video database systems as a typical example of temporal application, database systems for mixed reality as an instance of spatial application, and kansei management for digital media retrieval as a case of humanistic feelings application. Current research results by the project Advanced Database Systems for Integration of Media and User Environments are reported.

In this paper, we study the optimal allocation of multimedia files in distributed network systems. In these systems, the files are shared by users connected with different servers geographically separated, and each file must be stored in at least one of servers. Users can access any files stored in any servers connected with high-speed communication networks. Copies of the files accessed frequently are to be stored in several servers that have databases. So, it is one of the most important problems how to assign the files to servers in view of costs and delays. Considering these problems in heterogeneous network environments, we present a new system model that covers wide range of multimedia network applications like VOD, CALS, and so on. In these systems, it is obvious that there is trading-off relationship between costs and delays. Our objective is to find the optimal file allocation such that the total cost is minimized subject to the total delay. We introduce a 0-1 integer programming formulation for the optimization problem, and find the optimal file allocation by solving these formulae.

We present a new scheduling policy named Value-based Processor Allocation (VPA-k) for scheduling value-based transactions in a multiprocessor real-time database system. The value of a transaction represents the profit the transaction contributes to the system if it is completed before its deadline. Using VPA-k policy, the transactions with higher values are given higher priorities to execute first, while at most k percentage of the total processors are allocated to the urgent transactions dynamically. Through simulation experiments, VPA-k policy is shown to outperform other scheduling policies substantially in both maximizing the totally obtained values and minimizing the number of missed transactions.

An advanced intelligent network (IN) provides service management along with telecommunication services, and has a two-layer architecture, i.e., a transmission layer and an intelligent layer. An advanced IN's programmability is achieved by a service-independent platform of nodes in the intelligent layer, and service-dependent software called logic programs. In contrast to telecommunication services, models for service management have not yet been established. This paper presents both execution and specification models for service management. The execution model is composed of three hierarchies that apply to various kinds of management operation. The specification model has the capability to define the details of data items. The specification language for service management is also proposed. Simulation on dynamic SQL based DBMS solved that; (1) Logic programs for service management can be made small size on the model, and (2) To provide efficient database operation, programmability must be enhanced if service management includes table with variable number of field operation.

Our objective is to resolve three types of heterogeneity - data model,database system, and semantic - in heterogeneous databases. The basic framework which we propose for this objective is realized in an autonomous decentralized database system (i.e., an interoperable database system), called Jasmine/M. Users describe their relational or object-oriented data models and schemas locally using the model primitives which Jasmine/M provides as a scripting language. Description using such primitives or scripts constitutes viewports, which have a role to resolve heterogeneity in data models and database systems at local sites. At relational viewports, both relational and object-oriented schemas defined at other sites are translated via scripts and are viewed as relational schemas. Similarly at object-oriented viewports, any schema defined at other sites is viewed object-oriented schemas. Relational and object-oriented views are used to resolve semantic heterogeneity within viewports. This paper describes a step wise approach to resolving the three types of heterogeneity, using scripts, viewports, and views, and its implementation using active objects.

In order to make the hyperling generation mechanism not only dynamic but also systematic, we will mainly describe the use of the querying and abstraction facilities in hypertext database systems. Our work focuses on a dynamic, abstract and systematic link generation mechanism. The prototype system named TextLink/Gem is implemented by taking an object-oriented approach, and the notions of anchor objects, query-pair links, polymorphic links and inheritable links, on which the system is based, and their implementations are described in detail.

Recent advances of processing speed and window systems in computers, especially workstations, accelerate multi-media data processing (MMDP). Then, a variety of data such as numerics, characters, voice, video, animation and so on, are processed concurrently in a workstation. In data processings, concurrent execution of transactions is a key to improve through-puts. However, concurrent execution without concurrency control may cause inconsistent results. Thus, the concurrency control must be introduced in such systems. However, in MMDP it is ineffective to adopt previous concurrency control methods for ordinal databases since multi-media data are huge and possess a real-time property. This paper discusses concurrency control for MMDP. We propose some new concepts for MMDP, and define a new serializability class called Permissible Serializability which provides high concurrency in MMDP compared with ordinal classes. Then, we propose a concurrency control algorithm TYPE for the Permissible Serializability, and show some simulation results.

The security problems of object-oriented database system are investigated and security level assignment constraints and an access control mechanism based on the multilevel access control security policy are proposed. The proposed mechanism uses the Trusted Computing Base. A unique feature of the mechanism is that security levels are assigned not only to data items (objects), but also to methods and methods are not shown to the users whose security level is lower than that of the methods. And we distinguish between the security level of a variable in a class and that in an instance and distinguish between the level of an object when it is taken by itself and it is taken as a variable or an element of another complex object. All of this realizes the policy of multilevel access control.