1 millisecond (1-ms) vision systems are gaining increasing attention in diverse fields like factory automation and robotics, as the ultra-low delay ensures seamless and timely responses. Superpixel segmentation is a pivotal preprocessing to reduce the number of image primitives for subsequent processing. Recently, there has been a growing emphasis on leveraging deep network-based algorithms to pursue superior performance and better integration into other deep network tasks. Superpixel Sampling Network (SSN) employs a deep network for feature generation and employs differentiable SLIC for superpixel generation. SSN achieves high performance with a small number of parameters. However, implementing SSN on FPGAs for ultra-low delay faces challenges due to the final layer’s aggregation of intermediate results. To address this limitation, this paper proposes an aggregated to pipelined structure for FPGA implementation. The final layer is decomposed into individual final layers for each intermediate result. This architectural adjustment eliminates the need for memory to store intermediate results. Concurrently, the proposed structure leverages decomposed layers to facilitate a pipelined structure with pixel streaming input to achieve ultra-low latency. To cooperate with the pipelined structure, layer-partitioned memory architecture is proposed. Each final layer has dedicated memory for storing superpixel center information, allowing values to be read and calculated from memory without conflicts. Calculation results of each final layer are accumulated, and the result of each pixel is obtained as the stream reaches the last layer. Evaluation results demonstrate that boundary recall and under-segmentation error remain comparable to SSN, with an average label consistency improvement of 0.035 over SSN. From a hardware performance perspective, the proposed system processes 1000 FPS images with a delay of 0.947 ms/frame.

A 1 millisecond (1-ms) vision system, which processes videos at 1000 frames per second (FPS) within 1 ms/frame delay, plays an increasingly important role in fields such as robotics and factory automation. Superpixel as one of the most extensively employed image oversegmentation methods is a crucial pre-processing step for reducing computations in various computer vision applications. Among the different superpixel methods, simple linear iterative clustering (SLIC) has gained widespread adoption due to its simplicity, effectiveness, and computational efficiency. However, the iterative assignment and update steps in SLIC make it challenging to achieve high processing speed. To address this limitation and develop a SLIC superpixel segmentation system with a 1 ms delay, this paper proposes grid sample based temporal iteration. By leveraging the high frame rate of the input video, the proposed method distributes the iterations into the temporal domain, ensuring that the system's delay keeps within one frame. Additionally, grid sample information is added as initialization information to the obtained superpixel centers for enhancing the stability of superpixels. Furthermore, a selective label propagation based pipeline architecture is proposed for parallel computation of all the possibilities of label propagation. This eliminates data dependency between adjacent pixels and enables a fully pipelined system. The evaluation results demonstrate that the proposed superpixel segmentation system achieves boundary recall and under-segmentation error comparable to the original SLIC algorithm. When considering label consistency, the proposed system surpasses the performance of state-of-the-art superpixel segmentation methods. Moreover, in terms of hardware performance, the proposed system processes 1000 FPS images with 0.985 ms/frame delay.

In this letter, we propose a hierarchical segmentation (HS) method for color images, which can not only maintain the segmentation accuracy, but also ensure a good speed. In our method, HS adopts the fuzzy simple linear iterative clustering (Fuzzy SLIC) to obtain an over-segmentation result. Then, HS uses the fast fuzzy C-means clustering (FFCM) to produce the rough segmentation result based on superpixels. Finally, HS takes the non-iterative K-means clustering using priority queue (KPQ) to refine the segmentation result. In the validation experiments, we tested our method and compared it with state-of-the-art image segmentation methods on the Berkeley (BSD500) benchmark under different types of noise. The experiment results show that our method outperforms state-of-the-art techniques in terms of accuracy, speed and robustness.

Finding the correspondence between two images of the same object or scene is an active research field in computer vision. This paper develops a rapid and effective Content-based Superpixel Image matching and Stitching (CSIS) scheme, which utilizes the content of superpixel through multi-features fusion technique. Unlike popular keypoint-based matching method, our approach proposes a superpixel internal feature-based scheme to implement image matching. In the beginning, we make use of a novel superpixel generation algorithm based on content-based feature representation, named Content-based Superpixel Segmentation (CSS) algorithm. Superpixels are generated in terms of a new distance metric using color, spatial, and gradient feature information. It is developed to balance the compactness and the boundary adherence of resulted superpixels. Then, we calculate the entropy of each superpixel for separating some superpixels with significant characteristics. Next, for each selected superpixel, its multi-features descriptor is generated by extracting and fusing local features of the selected superpixel itself. Finally, we compare the matching features of candidate superpixels and their own neighborhoods to estimate the correspondence between two images. We evaluated superpixel matching and image stitching on complex and deformable surfaces using our superpixel region descriptors, and the results show that new method is effective in matching accuracy and execution speed.

This letter proposes a new superpixel segmentation algorithm based on global similarity and contour region transformation. The basic idea is that pixels surrounded by the same contour are more likely to belong to the same object region, which could be easily clustered into the same superpixel. To this end, we use contour scanning to estimate the global similarity between pixels and corresponded centers. In addition, we introduce pixel's gradient information of contour transform map to enhance the pixel's global similarity to overcome the missing contours in blurred region. Benefited from our global similarity, the proposed method could adherent with blurred and low contrast boundaries. A large number of experiments on BSDS500 and VOC2012 datasets show that the proposed algorithm performs better than traditional SLIC.

A novel superpixel segmentation approach driven by uniform mixture model with spatially constrained (UMMS) is proposed. Under this algorithm, each observation, i.e. pixel is first represented as a five-dimensional vector which consists of colour in CLELAB space and position information. And then, we define a new uniform distribution through adding pixel position, so that this distribution can describe each pixel in input image. Applied weighted 1-Norm to difference between pixels and mean to control the compactness of superpixel. In addition, an effective parameter estimation scheme is introduced to reduce computational complexity. Specifically, the invariant prior probability and parameter range restrict the locality of superpixels, and the robust mean optimization technique ensures the accuracy of superpixel boundaries. Finally, each defined uniform distribution is associated with a superpixel and the proposed UMMS successfully implements superpixel segmentation. The experiments on BSDS500 dataset verify that UMMS outperforms most of the state-of-the-art approaches in terms of segmentation accuracy, regularity, and rapidity.

This paper proposes an enhanced affinity graph (EA-graph) for image segmentation. Firstly, the original image is over-segmented to obtain several sets of superpixels with different scales, and the color and texture features of the superpixels are extracted. Then, the similarity relationship between neighborhood superpixels is used to construct the local affinity graph. Meanwhile, the global affinity graph is obtained by sparse reconstruction among all superpixels. The local affinity graph and global affinity graph are superimposed to obtain an enhanced affinity graph for eliminating the influences of noise and isolated regions in the image. Finally, a bipartite graph is introduced to express the affiliation between pixels and superpixels, and segmentation is performed using a spectral clustering algorithm. Experimental results on the Berkeley segmentation database demonstrate that our method achieves significantly better performance compared to state-of-the-art algorithms.

This letter proposes a fast superpixel segmentation method based on boundary sampling and interpolation. The basic idea is as follow: instead of labeling local region pixels, we estimate superpixel boundary by interpolating candidate boundary pixel from a down-sampling image segmentation. On the one hand, there exists high spatial redundancy within each local region, which could be discarded. On the other hand, we estimate the labels of candidate boundary pixels via sampling superpixel boundary within corresponding neighbour. Benefiting from the reduction of candidate pixel distance calculation, the proposed method significantly accelerates superpixel segmentation. Experiments on BSD500 benchmark demonstrate that our method needs half the time compared with the state-of-the-arts while almost no accuracy reduction.

The 3D measurement is widely required in modern industries. In this letter, a method based on the RGBD saliency detection with depth range adjusting (RGBD-DRA) is proposed for 3D measurement. By using superpixels and prior maps, RGBD saliency detection is utilized to detect and measure the target object automatically Meanwhile, the proposed depth range adjusting is processing while measuring to prompt the measuring accuracy further. The experimental results demonstrate the proposed method automatic and accurate, with 3 mm and 3.77% maximum deviation value and rate, respectively.

In this paper, we propose a boundary-aware superpixel segmentation method, which could quickly and exactly extract superpixel with a non-iteration framework. The basic idea is to construct a minimum spanning tree (MST) based on structure edge to measure the local similarity among pixels, and then label each pixel as the index with shortest path seeds. Intuitively, we first construct MST on the original pixels with boundary feature to calculate the similarity of adjacent pixels. Then the geodesic distance between pixels can be exactly obtained based on two-round tree recursions. We determinate pixel label as the shortest path seed index. Experimental results on BSD500 segmentation benchmark demonstrate the proposed method obtains best performance compared with seven state-of-the-art methods. Especially for the low density situation, our method can obtain the boundary-aware oversegmentation region.

Synthetic aperture radar (SAR) image classification is a popular yet challenging research topic in the field of SAR image interpretation. This paper presents a new classification method based on extreme learning machine (ELM) and the superpixel-guided composite kernels (SGCK). By introducing the generalized likelihood ratio (GLR) similarity, a modified simple linear iterative clustering (SLIC) algorithm is firstly developed to generate superpixel for SAR image. Instead of using a fixed-size region, the shape-adaptive superpixel is used to exploit the spatial information, which is effective to classify the pixels in the detailed and near-edge regions. Following the framework of composite kernels, the SGCK is constructed base on the spatial information and backscatter intensity information. Finally, the SGCK is incorporated an ELM classifier. Experimental results on both simulated SAR image and real SAR image demonstrate that the proposed framework is superior to some traditional classification methods.

In this paper, an algorithm is proposed for license plate recognition (LPR) in video traffic surveillance applications. In an LPR system, the primary steps are license plate detection and character segmentation. However, in practice, false alarms often occur due to images of vehicle parts that are similar in appearance to a license plate or detection rate degradation due to local illumination changes. To alleviate these difficulties, the proposed license plate segmentation employs an adaptive binarization using a superpixel-based local contrast measurement. From the binarization, we apply a set of rules to a sequence of characters in a sub-image region to determine whether it is part of a license plate. This process is effective in reducing false alarms and improving detection rates. Our experimental results demonstrate a significant improvement over conventional methods.

This paper proposes a robust superpixel-based tracker via multiple-instance learning, which exploits the importance of instances and mid-level features captured by superpixels for object tracking. We first present a superpixels-based appearance model, which is able to compute the confidences of the object and background. Most importantly, we introduce the sample importance into multiple-instance learning (MIL) procedure to improve the performance of tracking. The importance for each instance in the positive bag is defined by accumulating the confidence of all the pixels within the corresponding instance. Furthermore, our tracker can help recover the object from the drifting scene using the appearance model based on superpixels when the drift occurs. We retain the first (k-1) frames' information during the updating process to alleviate drift to some extent. To evaluate the effectiveness of the proposed tracker, six video sequences of different challenging situations are tested. The comparison results demonstrate that the proposed tracker has more robust and accurate performance than six ones representing the state-of-the-art.

In this paper, we propose a superpixel based depth map generation scheme for the application to monoscopic to stereoscopic video conversion. The proposed algorithm employs four main processes to generate depth maps for all frames in the video sequences. First, the depth maps of the key frames in the input sequence are generated by superpixel merging and some user interactions. Second, the frames in the input sequences are over-segmented by Simple Linear Iterative Clustering (SLIC) or depth aided SLIC method depending on whether or not they have the depth maps. Third, each superpixel in current frame is used to match the corresponding superpixel in its previous frame. Finally, depth map is propagated with a joint bilateral filter based on the estimated matching vector of each superpixel. We show an improved performance of the proposed algorithm through experimental results.

This paper presents a novel algorithm to generate homogeneous superpixels from Markov random walks. We exploit Markov clustering (MCL) as the methodology, a generic graph clustering method based on stochastic flow circulation. In particular, we introduce a graph pruning strategy called compact pruning in order to capture intrinsic local image structure. The resulting superpixels are homogeneous, i.e. uniform in size and compact in shape. The original MCL algorithm does not scale well to a graph of an image due to the square computation of the Markov matrix which is necessary for circulating the flow. The proposed pruning scheme has the advantages of faster computation, smaller memory footprint, and straightforward parallel implementation. Through comparisons with other recent techniques, we show that the proposed algorithm achieves state-of-the-art performance.