The multi-category support vector machine (MC-SVM) is one of the most popular machine learning algorithms. There are numerous MC-SVM variants, although different optimization algorithms were developed for diverse learning machines. In this study, we developed a new optimization algorithm that can be applied to several MC-SVM variants. The algorithm is based on the Frank-Wolfe framework that requires two subproblems, direction-finding and line search, in each iteration. The contribution of this study is the discovery that both subproblems have a closed form solution if the Frank-Wolfe framework is applied to the dual problem. Additionally, the closed form solutions on both the direction-finding and line search exist even for the Moreau envelopes of the loss functions. We used several large datasets to demonstrate that the proposed optimization algorithm rapidly converges and thereby improves the pattern recognition performance.

The recursive transfer method (RTM) is a numerical technique that was developed to analyze scattering phenomena and its formulation is constructed with a difference equation derived from a differential equation by Numerov's discretization method. However, the differential equation to which Numerov's method is applicable is restricted and therefore the application range of RTM is also limited. In this paper, we provide a new discretization scheme to extend RTM formulation using the weak form theory framework. The effectiveness of the proposed formulation is confirmed by microwave scattering induced by a metallic pillar placed asymmetrically in the waveguide. A notable feature of RTM is that it can extract a localized wave from scattering waves even if the tail of the localized wave reaches to the ends of analyzing region. The discrepancy between the experimental and theoretical data is suppressed with in an upper bound determined by the standing wave ratio of the waveguide.

Domain knowledge is useful to improve the generalization performance of learning machines. Sign constraints are a handy representation to combine domain knowledge with learning machine. In this paper, we consider constraining the signs of the weight coefficients in learning the linear support vector machine, and develop an optimization algorithm for minimizing the empirical risk under the sign constraints. The algorithm is based on the Frank-Wolfe method that also converges sublinearly and possesses a clear termination criterion. We show that each iteration of the Frank-Wolfe also requires O(nd+d2) computational cost. Furthermore, we derive the explicit expression for the minimal iteration number to ensure an ε-accurate solution by analyzing the curvature of the objective function. Finally, we empirically demonstrate that the sign constraints are a promising technique when similarities to the training examples compose the feature vector.

It has been reported repeatedly that discriminative learning of distance metric boosts the pattern recognition performance. Although the ITML (Information Theoretic Metric Learning)-based methods enjoy an advantage that the Bregman projection framework can be applied for optimization of distance metric, a weak point of ITML-based methods is that the distance threshold for similarity/dissimilarity constraints must be determined manually, onto which the generalization performance is sensitive. In this paper, we present a new formulation of metric learning algorithm in which the distance threshold is optimized together. Since the optimization is still in the Bregman projection framework, the Dykstra algorithm can be applied for optimization. A nonlinear equation has to be solved to project the solution onto a half-space in each iteration. We have developed an efficient technique for projection onto a half-space. We empirically show that although the distance threshold is automatically tuned for the proposed metric learning algorithm, the accuracy of pattern recognition for the proposed algorithm is comparable, if not better, to the existing metric learning methods.

In recent years, covariance descriptors have received considerable attention as a strong representation of a set of points. In this research, we propose a new metric learning algorithm for covariance descriptors based on the Dykstra algorithm, in which the current solution is projected onto a half-space at each iteration, and which runs in O(n3) time. We empirically demonstrate that randomizing the order of half-spaces in the proposed Dykstra-based algorithm significantly accelerates convergence to the optimal solution. Furthermore, we show that the proposed approach yields promising experimental results for pattern recognition tasks.

Automatic detection of immunoreactive areas in fluorescence microscopic images is becoming a key technique in the field of biology including neuroscience, although it is still challenging because of several reasons such as low signal-to-noise ratio and contrast variation within an image. In this study, we developed a new algorithm that exhaustively detects co-localized areas in multi-channel fluorescence images, where shapes of target objects may differ among channels. Different adaptive binarization thresholds for different local regions in different channels are introduced and the condition of each segment is assessed to recognize the target objects. The proposed method was applied to detect immunoreactive spots that labeled membrane receptors on dendritic spines of mouse cerebellar Purkinje cells. Our method achieved the best detection performance over five pre-existing methods.

Contamination of water resources with pathogenic microorganisms excreted in human feces is a worldwide public health concern. Surveillance of fecal contamination is commonly performed by routine monitoring for a single type or a few types of microorganism(s). To design a feasible routine for periodic monitoring and to control risks of exposure to pathogens, reliable statistical algorithms for inferring correlations between concentrations of microorganisms in water need to be established. Moreover, because pathogens are often present in low concentrations, some contaminations are likely to be under a detection limit. This yields a pairwise left-censored dataset and complicates computation of correlation coefficients. Errors of correlation estimation can be smaller if undetected values are imputed better. To obtain better imputations, we utilize side information and develop a new technique, the asymmetric Tobit model which is an extension of the Tobit model so that domain knowledge can be exploited effectively when fitting the model to a censored dataset. The empirical results demonstrate that imputation with domain knowledge is effective for this task.

Flexural waves on a thin elastic plate are governed by the fourth-order differential equation, which is attractive not only from a harmonic analysis viewpoint but also useful for an efficient numerical method in the elastdynamics. In this paper, we proposed two novel ideas: (1) use of the tensor bases to describe flexural waves on inhomogeneous elastic plates, (2) weak form discretization to derive the second-order difference equation from the fourth-order differential equation. The discretization method proposed in this study is of preliminary consideration about the recursive transfer method (RTM) to analyse the scattering problem of flexural waves. More importantly, the proposed discretization method can be applied to any system which can be formulated by the weak form theory. The accuracy of the difference equation derived by the proposed discretization method is confirmed by comparing the analytical and numerical solutions of waveguide modes. As a typical problem to confirm the validity of the resultant governing equation, the influence of the spatially modulated elastic constant in waveguide modes is discussed.

Classification tasks in computer vision and brain-computer interface research have presented several applications such as biometrics and cognitive training. However, like in any other discipline, determining suitable representation of data has been challenging, and recent approaches have deviated from the familiar form of one vector for each data sample. This paper considers a kernel between vector sets, the mean polynomial kernel, motivated by recent studies where data are approximated by linear subspaces, in particular, methods that were formulated on Grassmann manifolds. This kernel takes a more general approach given that it can also support input data that can be modeled as a vector sequence, and not necessarily requiring it to be a linear subspace. We discuss how the kernel can be associated with the Projection kernel, a Grassmann kernel. Experimental results using face image sequences and physiological signal data show that the mean polynomial kernel surpasses existing subspace-based methods on Grassmann manifolds in terms of predictive performance and efficiency.

Anomaly detection has several practical applications in different areas, including intrusion detection, image processing, and behavior analysis among others. Several approaches have been developed for this task such as detection by classification, nearest neighbor approach, and clustering. This paper proposes alternative clustering algorithms for the task of anomaly detection. By employing a weighted kernel extension of the least squares fitting of linear manifolds, we develop fuzzy clustering algorithms for kernel manifolds. Experimental results show that the proposed algorithms achieve promising performances compared to hard clustering techniques.

Although many 3D structures have been solved for proteins to date, functions of some proteins remain unknown. To predict protein functions, comparison of local structures of proteins with pre-defined model structures, whose functions have been elucidated, is widely performed. For the comparison, the root mean square deviation (RMSD) has been used as a conventional index. In this work, adaptive deviation was incorporated, along with Bregmann Divergence Regularized Machine, in order to detect analogous local structures with such model structures more effectively than the conventional index.

Recently, we proposed a weak-form discretization scheme to derive second-order difference equations from the governing equation of the scattering problem. In this paper, under the scope of the proposed scheme, numerical expressions for the waveguide boundary conditions are derived as perfectly absorbing conditions for input and output ports. The waveguide boundary conditions play an important role in extracting the quasi-localized wave as an eigenstate with a complex eigenvalue. The wave-number dependence of the resonance curve in Fano resonance is reproduced by using a semi-analytic model that is developed on the basis of the phase change relevant to the S-matrix. The reproduction confirms that the eigenstate with a complex eigenvalue does cause the observed Fano resonance.

Currently, the top-k error ratio is one of the primary methods to measure the accuracy of multi-category classification. Top-k multiclass SVM was designed to minimize the empirical risk based on the top-k error ratio. Two SDCA-based algorithms exist for learning the top-k SVM, both of which have several desirable properties for achieving optimization. However, both algorithms suffer from a serious disadvantage, that is, they cannot attain the optimal convergence in most cases owing to their theoretical imperfections. As demonstrated through numerical simulations, if the modified SDCA algorithm is employed, optimal convergence is always achieved, in contrast to the failure of the two existing SDCA-based algorithms. Finally, our analytical results are presented to clarify the significance of these existing algorithms.

Resonant scattering of flexural waves in acoustic waveguide is analysed by using the recursive transfer method (RTM). Because flexural waves are governed by a fourth-order differential equation, a localized wave tends to be induced around the scattering region and dampening wave tails from the localized wave may reach the ends of a simulation domain. A notable feature of RTM is its ability to extract the localized wave even if the dampening tail reaches the end of the simulation domain. Using RTM, the enhanced reflection caused by a localized wave is predicted and the shape of the localized wave is explored at its resonance with the incident wave.

Recent studies utilize multiple kernel learning to deal with incomplete-data problem. In this study, we introduce new methods that do not only complete multiple incomplete kernel matrices simultaneously, but also allow control of the flexibility of the model by parameterizing the model matrix. By imposing restrictions on the model covariance, overfitting of the data is avoided. A limitation of kernel matrix estimations done via optimization of an objective function is that the positive definiteness of the result is not guaranteed. In view of this limitation, our proposed methods employ the LogDet divergence, which ensures the positive definiteness of the resulting inferred kernel matrix. We empirically show that our proposed restricted covariance models, employed with LogDet divergence, yield significant improvements in the generalization performance of previous completion methods.

With the huge influx of various data nowadays, extracting knowledge from them has become an interesting but tedious task among data scientists, particularly when the data come in heterogeneous form and have missing information. Many data completion techniques had been introduced, especially in the advent of kernel methods — a way in which one can represent heterogeneous data sets into a single form: as kernel matrices. However, among the many data completion techniques available in the literature, studies about mutually completing several incomplete kernel matrices have not been given much attention yet. In this paper, we present a new method, called Mutual Kernel Matrix Completion (MKMC) algorithm, that tackles this problem of mutually inferring the missing entries of multiple kernel matrices by combining the notions of data fusion and kernel matrix completion, applied on biological data sets to be used for classification task. We first introduced an objective function that will be minimized by exploiting the EM algorithm, which in turn results to an estimate of the missing entries of the kernel matrices involved. The completed kernel matrices are then combined to produce a model matrix that can be used to further improve the obtained estimates. An interesting result of our study is that the E-step and the M-step are given in closed form, which makes our algorithm efficient in terms of time and memory. After completion, the (completed) kernel matrices are then used to train an SVM classifier to test how well the relationships among the entries are preserved. Our empirical results show that the proposed algorithm bested the traditional completion techniques in preserving the relationships among the data points, and in accurately recovering the missing kernel matrix entries. By far, MKMC offers a promising solution to the problem of mutual estimation of a number of relevant incomplete kernel matrices.

Kernel methods such as the support vector machine are one of the most successful algorithms in modern machine learning. Their advantage is that linear algorithms are extended to non-linear scenarios in a straightforward way by the use of the kernel trick. However, naive use of kernel methods is computationally expensive since the computational complexity typically scales cubically with respect to the number of training samples. In this article, we review recent advances in the kernel methods, with emphasis on scalability for massive problems.