Currently, the most advanced knowledge distillation models use a metric learning approach based on probability distributions. However, the correlation between supervised probability distributions is typically geometric and implicit, causing inefficiency and an inability to capture structural feature representations among different tasks. To overcome this problem, we propose a knowledge distillation loss using the robust sliced Wasserstein distance with geometric median (GMSW) to estimate the differences between the teacher and student representations. Due to the intuitive geometric properties of GMSW, the student model can effectively learn to align its produced hidden states from the teacher model, thereby establishing a robust correlation among implicit features. In experiment, our method outperforms state-of-the-art models in both high-resource and low-resource settings.

The prediction of peak power load is a critical factor directly impacting the stability of power supply, characterized significantly by its time series nature and intricate ties to the seasonal patterns in electricity usage. Despite its crucial importance, the current landscape of power peak load forecasting remains a multifaceted challenge in the field. This study aims to contribute to this domain by proposing a method that leverages a combination of three primary models - the GRU model, self-attention mechanism, and Transformer mechanism - to forecast peak power load. To contextualize this research within the ongoing discourse, it’s essential to consider the evolving methodologies and advancements in power peak load forecasting. By delving into additional references addressing the complexities and current state of the power peak load forecasting problem, this study aims to build upon the existing knowledge base and offer insights into contemporary challenges and strategies adopted within the field. Data preprocessing in this study involves comprehensive cleaning, standardization, and the design of relevant functions to ensure robustness in the predictive modeling process. Additionally, recognizing the necessity to capture temporal changes effectively, this research incorporates features such as “Weekly Moving Average” and “Monthly Moving Average” into the dataset. To evaluate the proposed methodologies comprehensively, this study conducts comparative analyses with established models such as LSTM, Self-attention network, Transformer, ARIMA, and SVR. The outcomes reveal that the models proposed in this study exhibit superior predictive performance compared to these established models, showcasing their effectiveness in accurately forecasting electricity consumption. The significance of this research lies in two primary contributions. Firstly, it introduces an innovative prediction method combining the GRU model, self-attention mechanism, and Transformer mechanism, aligning with the contemporary evolution of predictive modeling techniques in the field. Secondly, it introduces and emphasizes the utility of “Weekly Moving Average” and “Monthly Moving Average” methodologies, crucial in effectively capturing and interpreting seasonal variations within the dataset. By incorporating these features, this study enhances the model’s ability to account for seasonal influencing factors, thereby significantly improving the accuracy of peak power load forecasting. This contribution aligns with the ongoing efforts to refine forecasting methodologies and addresses the pertinent challenges within power peak load forecasting.

In this study, we introduce a reservoir-based one-dimensional (1D) convolutional neural network that processes time-series data at a low computational cost, and investigate its performance and training time. Experimental results show that the proposed network consumes lower training computational costs and that it outperforms the conventional reservoir computing in a sound-classification task.

To mitigate the interference caused by frequency reuse between inter-layer and intra-layer users for Non-Orthogonal Multiple Access (NOMA) based device-to-device (D2D) communication underlaying cellular systems, this paper proposes a joint optimization strategy that combines user grouping and resource allocation. Specifically, the optimization problem is formulated to maximize the sum rate while ensuring the minimum rate of cellular users, considering three optimization parameters: user grouping, sub channel allocation and power allocation. However, this problem is a mixed integer nonlinear programming (MINLP) problem and is hard to solve directly. To address this issue, we divide the problem into two sub-problems: user grouping and resource allocation. First, we classify D2D users into D2D pairs or D2D NOMA groups based on the greedy algorithm. Then, in terms of resource allocation, we allocate the sub-channel to D2D users by swap matching algorithm to reduce the co-channel interference, and optimize the transmission power of D2D by the local search algorithm. Simulation results show that, compared to other schemes, the proposed algorithm significantly improves the system sum rate and spectral utilization.

Graph dissimilarities provide a powerful and ubiquitous approach for applying machine learning algorithms to edge-attributed graphs. However, conventional optimal transport-based dissimilarities cannot handle edge-attributes. In this paper, we propose an optimal transport-based dissimilarity between graphs with edge-attributes. The proposed method, multi-dimensional fused Gromov-Wasserstein discrepancy (MFGW), naturally incorporates the mismatch of edge-attributes into the optimal transport theory. Unlike conventional optimal transport-based dissimilarities, MFGW can directly handle edge-attributes in addition to structural information of graphs. Furthermore, we propose an iterative algorithm, which can be computed on GPUs, to solve non-convex quadratic programming problems involved in MFGW.  Experimentally, we demonstrate that MFGW outperforms the conventional optimal transport-based dissimilarity in several machine learning applications including supervised classification, subgraph matching, and graph barycenter calculation.

Forecasting time-series data is useful in many fields, such as stock price predicting system, autonomous driving system, weather forecast, etc. Many existing forecasting models tend to work well when forecasting short-sequence time series. However, when working with long sequence time series, the performance suffers significantly. Recently, there has been more intense research in this direction, and Informer is currently the most efficient predicting model. Informer’s main drawback is that it does not allow for incremental learning. In this paper, we propose a Fast Informer called Finformer, which addresses the above bottleneck by reducing the training/predicting time of Informer. Finformer can efficiently compute the positional/temporal/value embedding and Query/Key/Value of the self-attention incrementally. Theoretically, Finformer can improve the speed of both training and predicting over the state-of-the-art model Informer. Extensive experiments show that Finformer is about 26% faster than Informer for both short and long sequence time series prediction. In addition, Finformer is about 20% faster than InTrans for the general Conv1d, which is one of our previous works and is the predecessor of Finformer.

Model-based sensorless control of permanent magnet synchronous motor (PMSM) is promising for high-speed operation to estimate motor state, which is the speed and the position of the rotor, via electric signals of the stator, beside the inevitable fact that estimation accuracy is degraded by electromagnet interference (EMI) from switching devices of the converter. In this paper, the simulation system based on Luenberger observer and phase-locked loop (PLL) has been established, analyzing impacts of EMI on motor state estimations theoretically, exploring influences of EMI with different cutoff frequency, rated speeds, frequencies and amplitudes. The results show that Luenberger observer and PLL have strong immunity, which enable PMSM can still operate stably even under certain degrees of interference. EMI produces sideband harmonics that enlarge pulsation errors of speed and position estimations. Additionally, estimation errors are positively correlated with cutoff frequency of low-pass filter and the amplitude of EMI, and negatively correlated with rated speed of the motor and the frequency of EMI.  When the frequency is too high, its effects on motor state estimations are negligible. This work contributes to the comprehensive understanding of how EMI affects motor state estimations, which further enhances practical application of sensorless PMSM.

With the recent advances in IoT, there is a growing interest in multi-surface computing, where a mobile app can cooperatively utilize multiple devices' surfaces. We propose a novel framework that seamlessly augments mobile apps with multi-surface computing capabilities. It enables various apps to employ multiple surfaces with acceptable performance.

Despite the increasing use of conversational artificial intelligence (AI) in online discussion environments, few studies explore the application of AI as a facilitator in forming problem-solving debates and influencing opinions in cross-venue scenarios, particularly in diverse and war-ravaged countries. This study aims to investigate the impact of AI on enhancing participant engagement and collaborative problem-solving in online-mediated discussion environments, especially in diverse and heterogeneous discussion settings, such as the five cities in Afghanistan. We seek to assess the extent to which AI participation in online conversations succeeds by examining the depth of discussions and participants' contributions, comparing discussions facilitated by AI with those not facilitated by AI across different venues. The results are discussed with respect to forming and changing opinions with and without AI-mediated communication. The findings indicate that the number of opinions generated in AI-facilitated discussions significantly differs from discussions without AI support. Additionally, statistical analyses reveal quantitative disparities in online discourse sentiments when conversational AI is present compared to when it is absent. These findings contribute to a better understanding of the role of AI-mediated discussions and offer several practical and social implications, paving the way for future developments and improvements.

The combination of multiple-input multiple-output (MIMO) technology and sparse code multiple access (SCMA) can significantly enhance the spectral efficiency of future wireless communication networks. However, the receiver design for downlink MIMO-SCMA systems faces challenges in developing multi-user detection (MUD) schemes that achieve both low latency and low bit error rate (BER). The separated detection scheme in the MIMO-SCMA system involves performing MIMO detection first to obtain estimated signals, followed by SCMA decoding. We propose an enhanced separated detection scheme based on lightweight graph neural networks (GNNs). In this scheme, we raise the concept of coordinate point relay and full-category training, which allow for the substitution of the conventional message passing algorithm (MPA) in SCMA decoding with image classification techniques based on deep learning (DL). The features of the images used for training encompass crucial information such as the amplitude and phase of estimated signals, as well as channel characteristics they have encountered. Furthermore, various types of images demonstrate distinct directional trends, contributing additional features that enhance the precision of classification by GNNs. Simulation results demonstrate that the enhanced separated detection scheme outperforms existing separated and joint detection schemes in terms of computational complexity, while having a better BER performance than the joint detection schemes at high Eb/N0 (energy per bit to noise power spectral density ratio) values.

In this paper, we investigated the overfitting characteristics of nonlinear equalizers based on an artificial neural network (ANN) and the Volterra series transfer function (VSTF), which were designed to compensate for optical nonlinear waveform distortion in optical fiber communication systems. Linear waveform distortion caused by, e.g., chromatic dispersion (CD) is commonly compensated by linear equalizers using digital signal processing (DSP) in digital coherent receivers. However, mitigation of nonlinear waveform distortion is considered to be one of the next important issues. An ANN-based nonlinear equalizer is one possible candidate for solving this problem. However, the risk of overfitting of ANNs is one obstacle in using the technology in practical applications. We evaluated and compared the overfitting of ANN- and conventional VSTF-based nonlinear equalizers used to compensate for optical nonlinear distortion. The equalizers were trained on repeated random bit sequences (RRBSs), while varying the length of the bit sequences. When the number of hidden-layer units of the ANN was as large as 100 or 1000, the overfitting characteristics were comparable to those of the VSTF. However, when the number of hidden-layer units was 10, which is usually enough to compensate for optical nonlinear distortion, the overfitting was weaker than that of the VSTF. Furthermore, we confirmed that even commonly used finite impulse response (FIR) filters showed overfitting to the RRBS when the length of the RRBS was equal to or shorter than the length of the tapped delay line of the filters. Conversely, when the RRBS used for the training was sufficiently longer than the tapped delay line, the overfitting could be suppressed, even when using an ANN-based nonlinear equalizer with 10 hidden-layer units.

We considered the problem of forecasting the degradation recovery process of civil structures for prognosis and health management. In this process, structural health degrades over time but recovers when a maintenance intervention is performed. Maintenance interventions are typically recorded in terms of date and type. Such records can be represented as binary time series. Using binary maintenance intervention records, we forecast the process by using Long Short-Term Memory (LSTM). In this study, we experimentally examined how to feed binary time series data into LSTM. To this end, we compared the concatenation and reinitialization methods. The former is used to concatenate maintenance intervention records and health data and feed them into LSTM. The latter is used to reinitialize the LSTM internal memory when maintenance intervention is performed. The experimental results with the synthetic data revealed that the concatenation method outperformed the reinitialization method.

Multivariate public key cryptosystems (MPKC) are constructed based on the problem of solving multivariate quadratic equations (MQ problem). Among various multivariate schemes, UOV is an important signature scheme since it is underlying some signature schemes such as MAYO, QR-UOV, and Rainbow which was a finalist of NIST PQC standardization project. To analyze the security of a multivariate scheme, it is necessary to analyze the first fall degree or solving degree for the system of polynomial equations used in specific attacks. It is known that the first fall degree or solving degree often relates to the Hilbert series of the ideal generated by the system. In this paper, we study the Hilbert series of the UOV scheme, and more specifically, we study the Hilbert series of ideals generated by quadratic polynomials used in the central map of UOV. In particular, we derive a prediction formula of the Hilbert series by using some experimental results. Moreover, we apply it to the analysis of the reconciliation attack for MAYO.

Recently, multivariate time-series data has been generated in various environments, such as sensor networks and IoT, making anomaly detection in time-series data an essential research topic. Unsupervised learning anomaly detectors identify anomalies by training a model on normal data and producing high residuals for abnormal observations. However, a fundamental issue arises as anomalies do not consistently result in high residuals, necessitating a focus on the time-series patterns of residuals rather than individual residual sizes. In this paper, we present a novel framework comprising two serialized anomaly detectors: the first model calculates residuals as usual, while the second one evaluates the time-series pattern of the computed residuals to determine whether they are normal or abnormal. Experiments conducted on real-world time-series data demonstrate the effectiveness of our proposed framework.

This paper proposes a network design model, considering data consistency for a delay-sensitive distributed processing system. The data consistency is determined by collating the own state and the states of slave servers. If the state is mismatched with other servers, the rollback process is initiated to modify the state to guarantee data consistency. In the proposed model, the selected servers and the master-slave server pairs are determined to minimize the end-to-end delay and the delay for data consistency. We formulate the proposed model as an integer linear programming problem. We evaluate the delay performance and computation time. We evaluate the proposed model in two network models with two, three, and four slave servers. The proposed model reduces the delay for data consistency by up to 31 percent compared to that of a typical model that collates the status of all servers at one master server. The computation time is a few seconds, which is an acceptable time for network design before service launch. These results indicate that the proposed model is effective for delay-sensitive applications.

Representation learning is a crucial and complex task for multivariate time series data analysis, with a wide range of applications including trend analysis, time series data search, and forecasting. In practice, unsupervised learning is strongly preferred owing to sparse labeling. However, most existing studies focus on the representation of individual subseries without considering relationships between different subseries. In certain scenarios, this may lead to downstream task failures. Here, an unsupervised representation learning model is proposed for multivariate time series that considers the semantic relationship among subseries of time series. Specifically, the covariance calculated by the Gaussian process (GP) is introduced to the self-attention mechanism, capturing relationship features of the subseries. Additionally, a novel unsupervised method is designed to learn the representation of multivariate time series. To address the challenges of variable lengths of input subseries, a temporal pyramid pooling (TPP) method is applied to construct input vectors with equal length. The experimental results show that our model has substantial advantages compared with other representation learning models. We conducted experiments on the proposed algorithm and baseline algorithms in two downstream tasks: classification and retrieval. In classification task, the proposed model demonstrated the best performance on seven of ten datasets, achieving an average accuracy of 76%. In retrieval task, the proposed algorithm achieved the best performance under different datasets and hidden sizes. The result of ablation study also demonstrates significance of semantic relationship in multivariate time series representation learning.

Negation scope resolution is the process of detecting the negated part of a sentence. Unlike the syntax-based approach employed in previous researches, state-of-the-art methods performed better without the explicit use of syntactic structure. This work revisits the syntax-based approach and re-evaluates the effectiveness of syntactic structure in negation scope resolution. We replace the parser utilized in the prior works with state-of-the-art parsers and modify the syntax-based heuristic rules. The experimental results demonstrate that the simple modifications enhance the performance of the prior syntax-based method to the same level as state-of-the-art end-to-end neural-based methods.

In this paper, we explore privacy-preserving techniques in federated learning, including those can be used with both neural networks and decision trees. We begin by identifying how information can be leaked in federated learning, after which we present methods to address this issue by introducing two privacy-preserving frameworks that encompass many existing privacy-preserving federated learning (PPFL) systems. Through experiments with publicly available financial, medical, and Internet of Things datasets, we demonstrate the effectiveness of privacy-preserving federated learning and its potential to develop highly accurate, secure, and privacy-preserving machine learning systems in real-world scenarios. The findings highlight the importance of considering privacy in the design and implementation of federated learning systems and suggest that privacy-preserving techniques are essential in enabling the development of effective and practical machine learning systems.

In this paper, we investigate optimal design of millimeter-wave (mmWave) multiuser line-of-sight multiple-input-multiple-output (LOS MIMO) systems using hybrid arrays of subarrays based on hybrid block diagonalization (BD) precoding and combining scheme. By introducing a general 3D geometric channel model, the optimal subarray separation products of the transmitter and receiver for maximizing sum-rate is designed in terms of two regular configurations of adjacent subarrays and interleaved subarrays for different users, respectively. We analyze the sensitivity of the optimal design parameters on performance in terms of a deviation factor, and derive expressions for the eigenvalues of the multiuser equivalent LOS MIMO channel matrix, which are also valid for non-optimal design. Simulation results show that the interleaved subarrays can support longer distance communication than the adjacent subarrays given the appropriate fixed subarray deployment.

To establish a network service in network functions virtualization (NFV) networks, the orchestrator addresses the challenge of service chaining and virtual network function placement (SC-VNFP) by mapping virtual network functions (VNFs) and virtual links onto physical nodes and links. Unlike traditional networks, network operators in NFV networks must contend with both hardware and software failures in order to ensure resilient network services, as NFV networks consist of physical nodes and software-based VNFs. To guarantee network service quality in NFV networks, the existing work has proposed an approach for the SC-VNFP problem that considers VNF diversity and redundancy. VNF diversity splits a single VNF into multiple lightweight replica instances that possess the same functionality as the original VNF, which are then executed in a distributed manner. VNF redundancy, on the other hand, deploys backup instances with standby mode on physical nodes to prepare for potential VNF failures. However, the existing approach does not adequately consider the tradeoff between resource efficiency and service availability in the context of VNF diversity and redundancy. In this paper, we formulate the SC-VNFP problem with VNF diversity and redundancy as a two-step integer linear program (ILP) that adjusts the balance between service availability and resource efficiency. Through numerical experiments, we demonstrate the fundamental characteristics of the proposed ILP, including the tradeoff between resource efficiency and service availability.