As an efficient classical machine learning classifier, the Softmax regression uses cross-entropy as the loss function. Therefore, it has high accuracy in classification. However, when there is inconsistency between the distribution of training samples and test samples, the performance of traditional Softmax regression models will degrade. A transfer discriminant Softmax regression model called Transfer Discriminant Softmax Regression with Weighted MMD (TDS-WMMD) is proposed in this paper. With this method, the Weighted Maximum Mean Divergence (WMMD) is introduced into the objective function to reduce the marginal distribution and conditional distribution between domains both locally and globally, realizing the cross domain transfer of knowledge. In addition, to further improve the classification performance of the model, Linear Discriminant Analysis (LDA) is added to the label iteration refinement process to improve the class separability of the designed method by keeping the same kind of samples together and the different kinds of samples repeling each other. Finally, after conducting classification experiments on several commonly used public transfer learning datasets, the results verify that the designed method can enhance the knowledge transfer ability of the Softmax regression model, and deliver higher classification performance compared with other current transfer learning classifiers.

Unsupervised Feature selection is an important dimensionality reduction technique to cope with high-dimensional data. It does not require prior label information, and has recently attracted much attention. However, it cannot fully utilize the discriminative information of samples, which may affect the feature selection performance. To tackle this problem, in this letter, we propose a novel discriminative virtual label regression method (DVLR) for unsupervised feature selection. In DVLR, we develop a virtual label regression function to guide the subspace learning based feature selection, which can select more discriminative features. Moreover, a linear discriminant analysis (LDA) term is used to make the model be more discriminative. To further make the model be more robust and select more representative features, we impose the ℓ2,1-norm on the regression and feature selection terms. Finally, extensive experiments are carried out on several public datasets, and the results demonstrate that our proposed DVLR achieves better performance than several state-of-the-art unsupervised feature selection methods.

In this paper, we propose a novel algorithm called multi-projection ensemble discriminant clustering (MPEDC) for JPEG steganalysis. The scheme makes use of the optimal projection of linear discriminant analysis (LDA) algorithm to get more projection vectors by using the micro-rotation method. These vectors are similar to the optimal vector. MPEDC combines unsupervised K-means algorithm to make a comprehensive decision classification adaptively. The power of the proposed method is demonstrated on three steganographic methods with three feature extraction methods. Experimental results show that the accuracy can be improved using iterative discriminant classification.

As an effective method, exponential discriminant analysis (EDA) has been proposed and widely used to solve the so-called small-sample-size (SSS) problem. In this paper, a simple and effective generalization of EDA is presented and named as GEDA. In GEDA, a general exponential function, where the base of exponential function is larger than the Euler number, is used. Due to the property of general exponential function, the distance between samples belonging to different classes is larger than that of EDA, and then the discrimination property is largely emphasized. The experiment results on the Extended Yale and CMU-PIE face databases show that, GEDA gets more advantageous recognition performance compared to EDA.

A novel dimensionality reduction method, Fisher Discriminant Locality Preserving Canonical Correlation Analysis (FDLP-CCA), for visualizing Web images is presented in this paper. FDLP-CCA can integrate two modalities and discriminate target items in terms of their semantics by considering unique characteristics of the two modalities. In this paper, we focus on Web images with text uploaded on Social Networking Services for these two modalities. Specifically, text features have high discriminate power in terms of semantics. On the other hand, visual features of images give their perceptual relationships. In order to consider both of the above unique characteristics of these two modalities, FDLP-CCA estimates the correlation between the text and visual features with consideration of the cluster structure based on the text features and the local structures based on the visual features. Thus, FDLP-CCA can integrate the different modalities and provide separated manifolds to organize enhanced compactness within each natural cluster.

Kernel discriminant analysis (KDA) is the mainstream approach of nonlinear discriminant analysis (NDA). Since it uses the kernel trick, KDA does not consider its nonlinear discriminant mapping explicitly. In this paper, another NDA approach where the nonlinear discriminant mapping is analytically given is developed. This study is based on the theory of optimal nonlinear discriminant analysis (ONDA) of which the nonlinear mapping is exactly expressed by using the Bayesian posterior probability. This theory indicates that various NDA can be derived by estimating the Bayesian posterior probability in ONDA with various estimation methods. Also, ONDA brings an insight about novel kernel functions, called discriminant kernel (DK), which is defined by also using the posterior probabilities. In this paper, several NDA and DK derived from ONDA with several posterior probability estimators are developed and evaluated. Given fine estimation methods of the Bayesian posterior probability, they give good discriminant spaces for visualization or classification.

In this Letter, a new iris recognition approach based on local Gabor orientation feature is proposed. On one hand, the iris feature extraction method using the traditional Gabor filters can cause time-consuming and high-feature dimension. On the other hand, we can find that the changes of original iris texture in angle and radial directions are more obvious than the other directions by observing the iris images. These changes in the preprocessed iris images are mainly reflected in vertical and horizontal directions. Therefore, the local directional Gabor filters are constructed to extract the horizontal and vertical texture characteristics of iris. First, the iris images are preprocessed by iris and eyelash location, iris segmentation, normalization and zooming. After analyzing the variety of iris texture and 2D-Gabor filters, we construct the local directional Gabor filters to extract the local Gabor features of iris. Then, the Gabor & Fisher features are obtained by Linear Discriminant Analysis (LDA). Finally, the nearest neighbor method is used to recognize the iris. Experimental results show that the proposed method has better iris recognition performance with less feature dimension and calculation time.

In Still-to-Video (S2V) face recognition, only a few high resolution images are registered for each subject, while the probe is video clips of complex variations. As faces present distinct characteristics under different scenarios, recognition in the original space is obviously inefficient. Thus, in this paper, we propose a novel discriminant analysis method to learn separate mappings for different scenario patterns (still, video), and further pursue a common discriminant space based on these mappings. Concretely, by modeling each video as a manifold and each image as point data, we form the scenario-oriented mapping learning as a Point-Manifold Discriminant Analysis (PMDA) framework. The learning objective is formulated by incorporating the intra-class compactness and inter-class separability for good discrimination. Experiments on the COX-S2V dataset demonstrate the effectiveness of the proposed method.

Text sentiment classification aims to automatically classify subjective documents into different sentiment-oriented categories (e.g. positive/negative). Given the high dimensionality of features describing documents, how to effectively select the most useful ones, referred to as sentiment-bearing features, with a lack of sentiment class labels is crucial for improving the classification performance. This paper proposes an unsupervised sentiment-bearing feature selection method (USFS), which incorporates sentiment discriminant analysis (SDA) into sentiment strength calculation (SSC). SDA applies traditional linear discriminant analysis (LDA) in an unsupervised manner without losing local sentiment information between documents. We use SSC to calculate the overall sentiment strength for each single feature based on its affinities with some sentiment priors. Experiments, performed using benchmark movie reviews, demonstrated the superior performance of USFS.

The topic of retrieving videos containing a desired person from a dataset just using the content of faces without any help of textual information has many interesting applications like video surveillance, social network, video mining, etc. However, traditional face matching against a huge number of detected faces leads to an unacceptable response time and may also reduce the accuracy due to the large variations in facial expressions, poses, lighting, etc. Therefore, in this paper we propose a novel method to generate discriminative “signatures” for efficiently retrieving the videos containing the same person with a query. In this research, the signature is defined as a compact, discriminative and reduced dimensionality representation, which is generated from a set of high-dimensional feature vectors of an individual. The desired videos are retrieved based on the similarities between the signature of the query and those of individuals in the database. In particular, we make the following contributions. Firstly, we give an algorithm of two directional linear discriminant analysis with maximum correntropy criterion (2DLDA-MCC) as an extension to our recently proposed maximum correntropy criterion based linear discriminant analysis (LDA-MCC). Both algorithms are robust to outliers and noise. Secondly, we present an approach for transferring a set of exemplars to a fixed-length signature using LDA-MCC and 2DLDA-MCC, resulting in two kinds of signatures that are called 1D signature and 2D signature. Finally, a novel video retrieval scheme is given based on the signatures, which has low storage requirement and can achieve a fast search. Evaluations on a large dataset of videos show reliable measurement of similarities by using the proposed signatures to represent the identities generated from videos. Experimental results also demonstrate that the proposed video retrieval scheme has the potential to substantially reduce the response time and slightly increase the mean average precision of retrieval.

Linear Discriminant Analysis (LDA) is a well-known feature extraction method for supervised subspace learning in statistical pattern recognition. In this paper, a novel method of LDA based on a new L1-norm optimization technique and its variances are proposed. The conventional LDA, which is based on L2-norm, is sensitivity to the presence of outliers, since it used the L2-norm to measure the between-class and within-class distances. In addition, the conventional LDA often suffers from the so-called small sample size (3S) problem since the number of samples is always smaller than the dimension of the feature space in many applications, such as face recognition. Based on L1-norm, the proposed methods have several advantages, first they are robust to outliers because they utilize the L1-norm, which is less sensitive to outliers. Second, they have no 3S problem. Third, they are invariant to rotations as well. The proposed methods are capable of reducing the influence of outliers substantially, resulting in a robust classification. Performance assessment in face application shows that the proposed approaches are more effectiveness to address outliers issue than traditional ones.

We extend the Nonparametric Discriminant Analysis (NDA) algorithm to a semi-supervised dimensionality reduction technique, called Semi-supervised Nonparametric Discriminant Analysis (SNDA). SNDA preserves the inherent advantages of NDA, that is, relaxing the Gaussian assumption required for the traditional LDA-based methods. SNDA takes advantage of both the discriminating power provided by the NDA method and the locality-preserving power provided by the manifold learning. Specifically, the labeled data points are used to maximize the separability between different classes and both the labeled and unlabeled data points are used to build a graph incorporating neighborhood information of the data set. Experiments on synthetic as well as real datasets demonstrate the effectiveness of the proposed approach.

In this paper we consider the two-class classification problem with high-dimensional data. It is important to find a class of distributions such that we cannot expect good performance in classification for any classifier. In this paper, when two population variance-covariance matrices are different, we give a reasonable sufficient condition for distributions such that the misclassification rate converges to the worst value as the dimension of data tends to infinity for any classifier. Our results can give guidelines to decide whether or not an experiment is worth performing in many fields such as bioinformatics.

Linear Discriminant Analysis (LDA) is one of the most popular dimensionality reduction techniques in existing handwritten Chinese character (HCC) recognition systems. However, when used for unconstrained handwritten Chinese character recognition, the traditional LDA algorithm is prone to two problems, namely, the class separation problem and multimodal sample distributions. To deal with these problems,we propose a new locally linear discriminant analysis (LLDA) method for handwritten Chinese character recognition.Our algorithm operates as follows. (1) Using the clustering algorithm, find clusters for the samples of each class. (2) Find the nearest neighboring clusters from the remaining classes for each cluster of one class. Then, use the corresponding cluster means to compute the between-class scatter matrix in LDA while keeping the within-class scatter matrix unchanged. (3) Finally, apply feature vector normalization to further improve the class separation problem. A series of experiments on both the HCL2000 and CASIA Chinese character handwriting databases show that our method can effectively improve recognition performance, with a reduction in error rate of 28.7% (HCL2000) and 16.7% (CASIA) compared with the traditional LDA method.Our algorithm also outperforms DLA (Discriminative Locality Alignment,one of the representative manifold learning-based dimensionality reduction algorithms proposed recently). Large-set handwritten Chinese character recognition experiments also verified the effectiveness of our proposed approach.

In the past few years, discriminant analysis and manifold learning have been widely used in feature extraction. Recently, the sparse representation technique has advanced the development of pattern recognition. In this paper, we combine both discriminant analysis and manifold learning with sparse representation technique and propose a novel feature extraction approach named sparsity preserving embedding with manifold learning and discriminant analysis. It seeks an embedded space, where not only the sparse reconstructive relations among original samples are preserved, but also the manifold and discriminant information of both original sample set and the corresponding reconstructed sample set is maintained. Experimental results on the public AR and FERET face databases show that our approach outperforms relevant methods in recognition performance.

Kernel biased discriminant analysis (KBDA), as a subspace learning algorithm, has been an attractive approach for the relevance feedback in content-based image retrieval. Its performance, however, still suffers from the “small sample learning” problem and “kernel learning” problem. Aiming to solve these problems, in this paper, we present a new semi-supervised scheme of KBDA (S-KBDA), in which the projection learning and the “kernel learning” are interweaved into a constrained optimization framework. Specifically, S-KBDA learns a subspace that preserves both the biased discriminant structure among the labeled samples, and the geometric structure among all training samples. In kernel optimization, we directly optimize the kernel matrix, rather than a kernel function, which makes the kernel learning more flexible and appropriate for the retrieval task. To solve the constrained optimization problem, a fast algorithm based on gradient ascent is developed. The image retrieval experiments are given to show the effectiveness of the S-KBDA scheme in comparison with the original KBDA, and the other two state-of-the-art algorithms.

Linear discriminant analysis (LDA) is one of the well-known schemes for feature extraction and dimensionality reduction of labeled data. Recently, two-dimensional LDA (2DLDA) for matrices such as images has been reformulated into symmetric 2DLDA (S2DLDA), which is solved by an iterative algorithm. In this paper, we propose a non-iterative S2DLDA and experimentally show that the proposed method achieves comparable classification accuracy with the conventional S2DLDA, while the proposed method is computationally more efficient than the conventional S2DLDA.

Generic image recognition techniques are widely studied for automatic image indexing. However, many of these methods are computationally too heavy for a practically large setup. Thus, for realizing scalability, it is important to properly balance the trade-off between performance and computational cost. In recent years, methods based on a bag-of-keypoints approach have been successful and widely used. However, the preprocessing cost for building visual words becomes immense in large-scale datasets. On the other hand, methods based on global image features have been used for a long time. Because global image features can be extracted rapidly, it is relatively easy to use them with large datasets. However, the performance of global feature methods is usually poor compared to the bag-of-keypoints methods. This paper proposes a simple but powerful scheme of boosting the performance of global image features by densely sampling low-level statistical moments of local features. Also, we use a scalable learning and classification method which is substantially lighter than a SVM. Our method achieved performance comparable to state-of-the-art methods despite its remarkable simplicity.

The optimization of polarimetric contrast enhancement (OPCE) is a widely used method for maximizing the received power ratio of a desired target versus an undesired target (clutter). In this letter, a new model of the OPCE is proposed based on the Fisher criterion. By introducing the well known two-class problem of linear discriminant analysis (LDA), the proposed model is to enlarge the normalized distance of mean value between the target and the clutter. In addition, a cross-iterative numerical method is proposed for solving the optimization with a quadratic constraint. Experimental results with the polarimetric SAR (POLSAR) data demonstrate the effectiveness of the proposed method.

In this paper, we propose a novel image projection technique for face recognition applications based on Fisher Linear Discriminant Analysis (LDA). The projection is performed through a couple subspace analysis for overcoming the "small sample size" problem. Also, weighted pairwise discriminant hyperplanes are used in order to provide a more accurate discriminant decision than that produced by the conventional LDA. The proposed technique has been successfully tested on three face databases. Experimental results indicate that the proposed algorithm outperforms the conventional algorithms.