Deep Domain Confusion: Maximizing for Domain Invariance

Abstract: Recent reports suggest that a generic supervised deep CNN model trained on a large-scale dataset reduces, but does not remove, dataset bias on a standard benchmark. Fine-tuning deep models in a new domain can require a significant amount of data, which for many applications is simply not available. We propose a new CNN architecture which introduces an adaptation layer and an additional domain confusion loss, to learn a representation that is both semantically meaningful and domain invariant. We additionally show that a domain confusion metric can be used for model selection to determine the dimension of an adaptation layer and the best position for the layer in the CNN architecture. Our proposed adaptation method offers empirical performance which exceeds previously published results on a standard benchmark visual domain adaptation task.

Deep Transfer Learning with Joint Adaptation Networks

Abstract: Deep networks have been successfully applied to learn transferable features for adapting models from a source domain to a different target domain. In this paper, we present joint adaptation networks (JAN), which learn a transfer network by aligning the joint distributions of multiple domain-specific layers across domains based on a joint maximum mean discrepancy (JMMD) criterion. Adversarial training strategy is adopted to maximize JMMD such that the distributions of the source and target domains are made more distinguishable. Learning can be performed by stochastic gradient descent with the gradients computed by back-propagation in linear-time. Experiments testify that our model yields state of the art results on standard datasets.

Unsupervised Domain Adaptation with Residual Transfer Networks

Abstract: The recent success of deep neural networks relies on massive amounts of labeled data. For a target task where labeled data is unavailable, domain adaptation can transfer a learner from a different source domain. In this paper, we propose a new approach to domain adaptation in deep networks that can jointly learn adaptive classifiers and transferable features from labeled data in the source domain and unlabeled data in the target domain. We relax a shared-classifier assumption made by previous methods and assume that the source classifier and target classifier differ by a residual function. We enable classifier adaptation by plugging several layers into deep network to explicitly learn the residual function with reference to the target classifier. We fuse features of multiple layers with tensor product and embed them into reproducing kernel Hilbert spaces to match distributions for feature adaptation. The adaptation can be achieved in most feed-forward models by extending them with new residual layers and loss functions, which can be trained efficiently via back-propagation. Empirical evidence shows that the new approach outperforms state of the art methods on standard domain adaptation benchmarks.

Mind the Class Weight Bias: Weighted Maximum Mean Discrepancy for Unsupervised Domain Adaptation

Abstract: In domain adaptation, maximum mean discrepancy (MMD) has been widely adopted as a discrepancy metric between the distributions of source and target domains. However, existing MMD-based domain adaptation methods generally ignore the changes of class prior distributions, i.e., class weight bias across domains. This remains an open problem but ubiquitous for domain adaptation, which can be caused by changes in sample selection criteria and application scenarios. We show that MMD cannot account for class weight bias and results in degraded domain adaptation performance. To address this issue, a weighted MMD model is proposed in this paper. Specifically, we introduce class-specific auxiliary weights into the original MMD for exploiting the class prior probability on source and target domains, whose challenge lies in the fact that the class label in target domain is unavailable. To account for it, our proposed weighted MMD model is defined by introducing an auxiliary weight for each class in the source domain, and a classification EM algorithm is suggested by alternating between assigning the pseudo-labels, estimating auxiliary weights and updating model parameters. Extensive experiments demonstrate the superiority of our weighted MMD over conventional MMD for domain adaptation.

Central Moment Discrepancy (CMD) for Domain-Invariant Representation Learning

Abstract: The learning of domain-invariant representations in the context of domain adaptation with neural networks is considered. We propose a new regularization method that minimizes the discrepancy between domain-specific latent feature representations directly in the hidden activation space. Although some standard distribution matching approaches exist that can be interpreted as the matching of weighted sums of moments, e.g. Maximum Mean Discrepancy (MMD), an explicit order-wise matching of higher order moments has not been considered before. We propose to match the higher order central moments of probability distributions by means of order-wise moment differences. Our model does not require computationally expensive distance and kernel matrix computations. We utilize the equivalent representation of probability distributions by moment sequences to define a new distance function, called Central Moment Discrepancy (CMD). We prove that CMD is a metric on the set of probability distributions on a compact interval. We further prove that convergence of probability distributions on compact intervals w.r.t. the new metric implies convergence in distribution of the respective random variables. We test our approach on two different benchmark data sets for object recognition (Office) and sentiment analysis of product reviews (Amazon reviews). CMD achieves a new state-of-the-art performance on most domain adaptation tasks of Office and outperforms networks trained with MMD, Variational Fair Autoencoders and Domain Adversarial Neural Networks on Amazon reviews. In addition, a post-hoc parameter sensitivity analysis shows that the new approach is stable w.r.t. parameter changes in a certain interval. The source code of the experiments is publicly available.

Unsupervised Domain Adaptation by Backpropagation

Abstract:
Top-performing deep architectures are trained on massive amounts of labeled data. In the absence of labeled data for a certain task, domain adaptation often provides an attractive option given that labeled data of similar nature but from a different domain (e.g. synthetic images) are available. Here, we propose a new approach to domain adaptation in deep architectures that can be trained on large amount of labeled data from the source domain and large amount of unlabeled data from the target domain (no labeled target-domain data is necessary).
As the training progresses, the approach promotes the emergence of "deep" features that are (i) discriminative for the main learning task on the source domain and (ii) invariant with respect to the shift between the domains. We show that this adaptation behaviour can be achieved in almost any feed-forward model by augmenting it with few standard layers and a simple new gradient reversal layer. The resulting augmented architecture can be trained using standard backpropagation.
Overall, the approach can be implemented with little effort using any of the deep-learning packages. The method performs very well in a series of image classification experiments, achieving adaptation effect in the presence of big domain shifts and outperforming previous state-of-the-art on Office datasets.

Domain-Symmetric Networks for Adversarial Domain Adaptation

Abstract: Unsupervised domain adaptation aims to learn a model of classifier for unlabeled samples on the target domain, given training data of labeled samples on the source domain. Impressive progress is made recently by learning invariant features via domain-adversarial training of deep networks. In spite of the recent progress, domain adaptation is still limited in achieving the invariance of feature distributions at a finer category level. To this end, we propose in this paper a new domain adaptation method called Domain-Symmetric Networks (SymNets). The proposed SymNet is based on a symmetric design of source and target task classifiers, based on which we also construct an additional classifier that shares with them its layer neurons. To train the SymNet, we propose a novel adversarial learning objective whose key design is based on a two-level domain confusion scheme, where the category-level confusion loss improves over the domain-level one by driving the learning of intermediate network features to be invariant at the corresponding categories of the two domains. Both domain discrimination and domain confusion are implemented based on the constructed additional classifier. Since target samples are unlabeled, we also propose a scheme of cross-domain training to help learn the target classifier. Careful ablation studies show the efficacy of our proposed method. In particular, based on commonly used base networks, our SymNets achieve the new state of the art on three benchmark domain adaptation datasets.

Unsupervised Domain Adaptation via Regularized Conditional Alignment

Abstract: We propose a method for unsupervised domain adaptation that trains a shared embedding to align the joint distributions of inputs (domain) and outputs (classes), making any classifier agnostic to the domain. Joint alignment ensures that not only the marginal distributions of the domains are aligned, but the labels as well. We propose a novel objective function that encourages the class-conditional distributions to have disjoint support in feature space. We further exploit adversarial regularization to improve the performance of the classifier on the domain for which no annotated data is available.

Asymmetric Tri-training for Unsupervised Domain Adaptation

Abstract: It is important to apply models trained on a large number of labeled samples to different domains because collecting many labeled samples in various domains is expensive. To learn discriminative representations for the target domain, we assume that artificially labeling the target samples can result in a good representation. Tri-training leverages three classifiers equally to provide pseudo-labels to unlabeled samples; however, the method does not assume labeling samples generated from a different domain. In this paper, we propose the use of an asymmetric tri-training method for unsupervised domain adaptation, where we assign pseudo-labels to unlabeled samples and train the neural networks as if they are true labels. In our work, we use three networks asymmetrically, and by asymmetric, we mean that two networks are used to label unlabeled target samples, and one network is trained by the pseudo-labeled samples to obtain target-discriminative representations. Our proposed method was shown to achieve a state-of-the-art performance on the benchmark digit recognition datasets for domain adaptation.

Progressive Feature Alignment for Unsupervised Domain Adaptation

Abstract: Unsupervised domain adaptation (UDA) transfers knowledge from a label-rich source domain to a fully-unlabeled target domain. To tackle this task, recent approaches resort to discriminative domain transfer in virtue of pseudo-labels to enforce the class-level distribution alignment across the source and target domains. These methods, however, are vulnerable to the error accumulation and thus incapable of preserving cross-domain category consistency, as the pseudo-labeling accuracy is not guaranteed explicitly. In this paper, we propose the Progressive Feature Alignment Network (PFAN) to align the discriminative features across domains progressively and effectively, via exploiting the intra-class variation in the target domain. To be specific, we first develop an Easy-to-Hard Transfer Strategy (EHTS) and an Adaptive Prototype Alignment (APA) step to train our model iteratively and alternatively. Moreover, upon observing that a good domain adaptation usually requires a non-saturated source classifier, we consider a simple yet efficient way to retard the convergence speed of the source classification loss by further involving a temperature variate into the soft-max function. The extensive experimental results reveal that the proposed PFAN exceeds the state-of-the-art performance on three UDA datasets.

Comparison of existing unsupervised domain adaptation methods. The average accuracies tested on Office-31 dataset are reported.

There is always a distribution change or domain gap between training and testing sets, which would degrade the performance. Yosinski et al. {{cite:6e67ca3}} comprehensively explored the transferability of deep neural networks and finetuned the network with sufficient target labeled data to improve performance on target domain. However, in practical scenario, labeled target data is usually limited or unacquirable. To address this issue, many UDA approaches {{cite:1561f09}} are proposed, as shown in Table {{table:888a6c04-e5af-4bda-90b5-a5665f9501e9}} .


LSCP: Locally Selective Combination in Parallel Outlier Ensembles

Abstract: In unsupervised outlier ensembles, the absence of ground truth makes the combination of base outlier detectors a challenging task. Specifically, existing parallel outlier ensembles lack a reliable way of selecting competent base detectors, affecting accuracy and stability, during model combination. In this paper, we propose a framework---called Locally Selective Combination in Parallel Outlier Ensembles (LSCP)---which addresses the issue by defining a local region around a test instance using the consensus of its nearest neighbors in randomly selected feature subspaces. The top-performing base detectors in this local region are selected and combined as the model's final output. Four variants of the LSCP framework are compared with seven widely used parallel frameworks. Experimental results demonstrate that one of these variants, LSCP_AOM, consistently outperforms baselines on the majority of twenty real-world datasets.

Variational Graph Auto-Encoders

Abstract: We introduce the variational graph auto-encoder (VGAE), a framework for unsupervised learning on graph-structured data based on the variational auto-encoder (VAE). This model makes use of latent variables and is capable of learning interpretable latent representations for undirected graphs. We demonstrate this model using a graph convolutional network (GCN) encoder and a simple inner product decoder. Our model achieves competitive results on a link prediction task in citation networks. In contrast to most existing models for unsupervised learning on graph-structured data and link prediction, our model can naturally incorporate node features, which significantly improves predictive performance on a number of benchmark datasets.

Anomalydae: Dual Autoencoder for Anomaly Detection on Attributed Networks

Abstract: Anomaly detection on attributed networks aims at finding nodes whose patterns deviate significantly from the majority of reference nodes, which is pervasive in many applications such as network intrusion detection and social spammer detection. However, most existing methods neglect the complex cross-modality interactions between network structure and node attribute. In this paper, we propose a deep joint representation learning framework for anomaly detection through a dual autoencoder (AnomalyDAE), which captures the complex interactions between network structure and node attribute for high-quality embeddings. Specifically, Anoma-lyDAE consists of a structure autoencoder and an attribute autoencoder to learn both node embedding and attribute embedding jointly in latent space. Moreover, attention mechanism is employed in structure encoder to learn the importance between a node and its neighbors for an effective capturing of structure pattern, which is important to anomaly detection. Besides, by taking both the node embedding and attribute embedding as inputs of attribute decoder, the cross-modality interactions between network structure and node attribute are learned during the reconstruction of node attribute. Finally, anomalies can be detected by measuring the reconstruction errors of nodes from both the structure and attribute perspectives. Extensive experiments on real-world datasets demonstrate the effectiveness of the proposed method.

Selected algorithms for anomaly detection.The input {{formula:f47b0520-489b-4dc4-8451-4dfc67f9d601}}  represents using features only, while {{formula:ab69a5c9-6ca9-472a-a1f2-48c693d5c310}}  represents using both structure and features.

Unsupervised anomaly detection models do not rely on labeled data and instead look for patterns or data points that are significantly different from the rest of the data. These methods are useful when it is difficult or impossible to obtain labeled data, which is typically the case in computational workflows. Exampled methods like:
clustering (e.g., k-means {{cite:8868e67}})
, density-based methods (e.g., k-nearest neighbors {{cite:26857b8}})
, generative models (e.g., auto-encoder {{cite:446bb36}})
, etc.
In this work, we adopt the methods from PyOD {{cite:7240b28}}, where the data are considered as tabular data, and PyGOD {{cite:fa0cfca}}, where the data are considered as structural data, as benchmarks, reporting a set of metrics, and provide insightful analysis of the data we collected. Table {{table:fcf96b5f-8621-405b-9937-d29f7905ef8f}}  in appendix  provides a set of selected algorithms with their details.

Furthermore, it is observed that deep learning methods, aided by an additional input from the structure, denoted as {{formula:64bb3ad7-b90a-4a99-b19c-0ac844f986b5}}  in Table {{table:fcf96b5f-8621-405b-9937-d29f7905ef8f}} , exhibit improved results relative to traditional statistical methods. For instance, the GCNAE algorithm for the 1000 Genome workflow, and Radar for Predict Future Sales workflow demonstrate better performance. Moreover, despite better performance, the precision achieved by these approaches is unsatisfactory. However, to effectively apply deep learning methods for anomaly detection, it is crucial to carefully design algorithms that can learn good representations of the data and overcome OOM and TLE errors and moreover, improve the accuracy.

Table {{table:fcf96b5f-8621-405b-9937-d29f7905ef8f}}  provide the selected benchmark algorithms and their references.
The input {{formula:4198a43c-015b-4a76-933b-2e3a70ace247}}  represents using features only, while {{formula:1f388aac-7434-487a-8666-efb691ac0b31}}  represents using both structure and features.

Detailed benchmark settings for algorithms

Following in Table {{table:fcf96b5f-8621-405b-9937-d29f7905ef8f}} , we provide the detailed settings for each algorithm used in the benchmark. Notice that we use the default settings for all the algorithms, and we do not tune the hyperparameters for each algorithm. For the hyperparameters that are not listed below, we use the default settings in the implementation of each algorithm from PyOD and PyGOD.


Neural Graph Collaborative Filtering

Abstract: Learning vector representations (aka. embeddings) of users and items lies at the core of modern recommender systems. Ranging from early matrix factorization to recently emerged deep learning based methods, existing efforts typically obtain a user's (or an item's) embedding by mapping from pre-existing features that describe the user (or the item), such as ID and attributes. We argue that an inherent drawback of such methods is that, the collaborative signal, which is latent in user-item interactions, is not encoded in the embedding process. As such, the resultant embeddings may not be sufficient to capture the collaborative filtering effect. In this work, we propose to integrate the user-item interactions - more specifically the bipartite graph structure - into the embedding process. We develop a new recommendation framework Neural Graph Collaborative Filtering (NGCF), which exploits the user-item graph structure by propagating embeddings on it. This leads to the expressive modeling of high-order connectivity in user-item graph, effectively injecting the collaborative signal into the embedding process in an explicit manner. We conduct extensive experiments on three public benchmarks, demonstrating significant improvements over several state-of-the-art models like HOP-Rec [39] and Collaborative Memory Network [5]. Further analysis verifies the importance of embedding propagation for learning better user and item representations, justifying the rationality and effectiveness of NGCF. Codes are available at https://github.com/xiangwang1223/neural_graph_collaborative_filtering.

LightGCN: Simplifying and Powering Graph Convolution Network for Recommendation

Abstract: Graph Convolution Network (GCN) has become new state-of-the-art for collaborative filtering. Nevertheless, the reasons of its effectiveness for recommendation are not well understood. Existing work that adapts GCN to recommendation lacks thorough ablation analyses on GCN, which is originally designed for graph classification tasks and equipped with many neural network operations. However, we empirically find that the two most common designs in GCNs -- feature transformation and nonlinear activation -- contribute little to the performance of collaborative filtering. Even worse, including them adds to the difficulty of training and degrades recommendation performance. In this work, we aim to simplify the design of GCN to make it more concise and appropriate for recommendation. We propose a new model named LightGCN, including only the most essential component in GCN -- neighborhood aggregation -- for collaborative filtering. Specifically, LightGCN learns user and item embeddings by linearly propagating them on the user-item interaction graph, and uses the weighted sum of the embeddings learned at all layers as the final embedding. Such simple, linear, and neat model is much easier to implement and train, exhibiting substantial improvements (about 16.0% relative improvement on average) over Neural Graph Collaborative Filtering (NGCF) -- a state-of-the-art GCN-based recommender model -- under exactly the same experimental setting. Further analyses are provided towards the rationality of the simple LightGCN from both analytical and empirical perspectives.

KGAT: Knowledge Graph Attention Network for Recommendation

Abstract: To provide more accurate, diverse, and explainable recommendation, it is compulsory to go beyond modeling user-item interactions and take side information into account. Traditional methods like factorization machine (FM) cast it as a supervised learning problem, which assumes each interaction as an independent instance with side information encoded. Due to the overlook of the relations among instances or items (e.g., the director of a movie is also an actor of another movie), these methods are insufficient to distill the collaborative signal from the collective behaviors of users. In this work, we investigate the utility of knowledge graph (KG), which breaks down the independent interaction assumption by linking items with their attributes. We argue that in such a hybrid structure of KG and user-item graph, high-order relations --- which connect two items with one or multiple linked attributes --- are an essential factor for successful recommendation. We propose a new method named Knowledge Graph Attention Network (KGAT) which explicitly models the high-order connectivities in KG in an end-to-end fashion. It recursively propagates the embeddings from a node's neighbors (which can be users, items, or attributes) to refine the node's embedding, and employs an attention mechanism to discriminate the importance of the neighbors. Our KGAT is conceptually advantageous to existing KG-based recommendation methods, which either exploit high-order relations by extracting paths or implicitly modeling them with regularization. Empirical results on three public benchmarks show that KGAT significantly outperforms state-of-the-art methods like Neural FM and RippleNet. Further studies verify the efficacy of embedding propagation for high-order relation modeling and the interpretability benefits brought by the attention mechanism. We release the codes and datasets at https://github.com/xiangwang1223/knowledge_graph_attention_network.

Neural Collaborative Filtering

Abstract: In recent years, deep neural networks have yielded immense success on speech recognition, computer vision and natural language processing. However, the exploration of deep neural networks on recommender systems has received relatively less scrutiny. In this work, we strive to develop techniques based on neural networks to tackle the key problem in recommendation --- collaborative filtering --- on the basis of implicit feedback. Although some recent work has employed deep learning for recommendation, they primarily used it to model auxiliary information, such as textual descriptions of items and acoustic features of musics. When it comes to model the key factor in collaborative filtering --- the interaction between user and item features, they still resorted to matrix factorization and applied an inner product on the latent features of users and items. By replacing the inner product with a neural architecture that can learn an arbitrary function from data, we present a general framework named NCF, short for Neural network-based Collaborative Filtering. NCF is generic and can express and generalize matrix factorization under its framework. To supercharge NCF modelling with non-linearities, we propose to leverage a multi-layer perceptron to learn the user-item interaction function. Extensive experiments on two real-world datasets show significant improvements of our proposed NCF framework over the state-of-the-art methods. Empirical evidence shows that using deeper layers of neural networks offers better recommendation performance.

Neural Collaborative Filtering

DeepCF: A Unified Framework of Representation Learning and Matching Function Learning in Recommender System

Abstract: In general, recommendation can be viewed as a matching problem, i.e., match proper items for proper users. However, due to the huge semantic gap between users and items, it’s almost impossible to directly match users and items in their initial representation spaces. To solve this problem, many methods have been studied, which can be generally categorized into two types, i.e., representation learning-based CF methods and matching function learning-based CF methods. Representation learning-based CF methods try to map users and items into a common representation space. In this case, the higher similarity between a user and an item in that space implies they match better. Matching function learning-based CF methods try to directly learn the complex matching function that maps user-item pairs to matching scores. Although both methods are well developed, they suffer from two fundamental flaws, i.e., the limited expressiveness of dot product and the weakness in capturing low-rank relations respectively. To this end, we propose a general framework named DeepCF, short for Deep Collaborative Filtering, to combine the strengths of the two types of methods and overcome such flaws. Extensive experiments on four publicly available datasets demonstrate the effectiveness of the proposed DeepCF framework.

A summarization of the properties to the most related works.

In order to better illustrate the differences between the most related works with ours, we provide a summarization table to list their properties, as shown in Table {{table:ecbf24d7-238c-473e-b470-7dc108223393}} .


Multiple Access technologies (TDMA), WPT

A2G MU SISO downlink transmission assisted by two RISs

A2G MU SISO downlink transmission assisted by multiple TRISs

A2G MU SISO downlink transmission served by multiple ABSs

Distributional Reinforcement Learning for mmWave Communications with Intelligent Reflectors on a UAV

Abstract: In this paper, a novel communication framework that uses an unmanned aerial vehicle (UAV)-carried intelligent reflector (IR) is proposed to enhance multi-user downlink transmissions over millimeter wave (mmWave) frequencies. In order to maximize the downlink sum-rate, the optimal precoding matrix (at the base station) and reflection coefficient (at the IR) are jointly derived. Next, to address the uncertainty of mmWave channels and maintain line-of-sight links in a real-time manner, a distributional reinforcement learning approach, based on quantile regression optimization, is proposed to learn the propagation environment of mmWave communications, and, then, optimize the location of the UAV-IR so as to maximize the long-term downlink communication capacity. Simulation results show that the proposed learning-based deployment of the UAV-IR yields a significant advantage, compared to a non-learning UAV-IR, a static IR, and a direct transmission schemes, in terms of the average data rate and the achievable line-of-sight probability of downlink mmWave communications.

Data Downlink System in the Vast IOT Node Condition Assisted by UAV, Large Intelligent Surface, and Power and Data Beacon

Abstract: The future development of communication systems will create a great demand for the internet of things (IOT), where the overall control of all IOT nodes will become an important problem. Considering the essential issues of miniaturization and energy conservation, in this study, a new data downlink system is designed in which all IOT nodes harvest energy first and then receive data. To avoid the unsolvable problem of pre-locating all positions of vast IOT nodes, a device called the power and data beacon (PDB) is proposed. This acts as a relay station for energy and data. In addition, we model future scenes in which a communication system is assisted by unmanned aerial vehicles (UAVs), large intelligent surfaces (LISs), and PDBs. In this paper, we propose and solve the problem of determining the optimal flight trajectory to reach the minimum energy consumption or minimum time consumption. Four future feasible scenes are analyzed and then the optimization problems are solved based on numerical algorithms. Simulation results show that there are significant performance improvements in energy/time with the deployment of LISs and reasonable UAV trajectory planning.

Sum-Rate Maximization for IRS-Assisted UAV OFDMA Communication Systems

Abstract: In this paper, we propose the use of intelligent reflecting surface (IRS) in unmanned aerial vehicle (UAV)- based orthogonal frequency division multiple access (OFDMA) communication systems. The proposed scheme exploits both the rich beamforming gain brought by the IRS and the high mobility of UAV for improving the system sum-rate. The joint design of UAV’s trajectory, IRS scheduling, and communication resource allocation for the proposed system is formulated as a non-convex optimization problem to maximize the system sum-rate. The existence of an IRS introduces both frequencyselectivity and spatial-selectivity in the fading of the composite channel from the UAV to ground users. To facilitate the design, we first derive the expression of the composite channel gain and propose a parametric approximation approach to establish a lower bound for the formulated problem. An alternating optimization algorithm is devised to handle the lower bound optimization problem. Simulation results unveil the promising sum-rate gain achieved by the deployment of an IRS in UAVbased communication systems.

Intelligent Reflecting Surface Enhanced Multi-UAV NOMA Networks

Abstract: Intelligent reflecting surface (IRS) enhanced multi-unmanned aerial vehicle (UAV) non-orthogonal multiple access (NOMA) networks are investigated. A new transmission framework is proposed, where multiple UAV-mounted base stations employ NOMA to serve multiple groups of ground users with the aid of an IRS. The three-dimensional (3D) placement and transmit power of UAVs, the reflection matrix of the IRS, and the NOMA decoding orders among users are jointly optimized for maximization of the sum rate of considered networks. To tackle the formulated mixed-integer non-convex optimization problem with coupled variables, a block coordinate descent (BCD)-based iterative algorithm is developed. Specifically, the original problem is decomposed into three subproblems, which are alternatingly solved by exploiting the penalty method and the successive convex approximation technique. The proposed BCD-based algorithm is demonstrated to be able to obtain a stationary point of the original problem with polynomial time complexity. Numerical results show that: 1) the proposed NOMA-IRS scheme for multi-UAV networks achieves a higher sum rate compared to the benchmark schemes, i.e., orthogonal multiple access (OMA)-IRS and NOMA without IRS; 2) the use of IRS is capable of providing performance gain for multi-UAV networks by both enhancing channel qualities of UAVs to their served users and mitigating the inter-UAV interference; and 3) optimizing the UAV placement can make the sum rate gain brought by NOMA more distinct due to the flexible decoding order design.

Joint Trajectory and Passive Beamforming Design for Intelligent Reflecting Surface-Aided UAV Communications: A Deep Reinforcement Learning Approach

Abstract: In this paper, the intelligent reflecting surface (IRS)-aided unmanned aerial vehicle (UAV) communication system is studied, where the UAV is deployed to serve the user equipment (UE) with the assistance of multiple IRSs mounted on several buildings to enhance the communication quality between UAV and UE. We aim to maximize the energy efficiency of the system, including the data rate of UE and the energy consumption of UAV via jointly optimizing the UAV's trajectory and the phase shifts of reflecting elements of IRS, when the UE moves and the selection of IRSs is considered for the energy saving purpose. Since the system is complex and the environment is dynamic, it is challenging to derive low-complexity algorithms by using conventional optimization methods. To address this issue, we first propose a deep Q-network (DQN)-based algorithm by discretizing the trajectory, which has the advantage of training time. Furthermore, we propose a deep deterministic policy gradient (DDPG)-based algorithm to tackle the case with continuous trajectory for achieving better performance. The experimental results show that the proposed algorithms achieve considerable performance compared to other traditional solutions.

Optimal Resource Allocation for Multi-UAV Assisted Visible Light Communication

Abstract: In this paper, the optimization of deploying unmanned aerial vehicles (UAVs) over a reconfigurable intelligent surfaces (RISs)-assisted visible light communication (VLC) system is studied. In the considered model, UAVs are required to simultaneously provide wireless services as well as illumination for ground users. To meet the traffic and illumination demands of the ground users while minimizing the energy consumption of the UAVs, one must optimize UAV deployment, phase shift of RISs, user association and RIS association. This problem is formulated as an optimization problem whose goal is to minimize the transmit power of UAVs via adjusting UAV deployment, phase shift of RISs, user association and RIS association. To solve this problem, the original optimization problem is divided into four subproblems and an alternating algorithm is proposed. Specifically, phases alignment method and semidefinite program (SDP) algorithm are proposed to optimize the phase shift of RISs. Then, the UAV deployment optimization is solved by the successive convex approximation (SCA) algorithm. Since the problems of user association and RIS association are integer programming, the fraction relaxation method is adopted before using dual method to find the optimal solution. For simplicity, a greedy algorithm is proposed as an alternative to optimize RIS association. The proposed two schemes demonstrate the superior performance of 34:85% and 32:11% energy consumption reduction over the case without RIS, respectively, through extensive numerical study.

UAV-Assisted and Intelligent Reflecting Surfaces-Supported Terahertz Communications

Abstract: In this paper, unmanned aerial vehicles (UAVs) and intelligent reflective surface (IRS) are utilized to support terahertz (THz) communications. To this end, the joint optimization of UAV's trajectory, the phase shift of IRS, the allocation of THz sub-bands, and the power control is investigated to maximize the minimum average achievable rate of all the users. An iteration algorithm based on successive Convex Approximation with the Rate constraint penalty (CAR) is developed to obtain UAV's trajectory, and the IRS phase shift is formulated as a closed-form expression with introduced pricing factors. Simulation results show that the proposed scheme significantly enhances the rate performance of the whole system.

Deep Learning for THz Drones with Flying Intelligent Surfaces: Beam and Handoff Prediction

Abstract: We consider the problem of proactive handoff and beam selection in Terahertz (THz) drone communication networks assisted with reconfigurable intelligent surfaces (RIS). Drones have emerged as critical assets for next-generation wireless networks to provide seamless connectivity and extend the coverage, and can largely benefit from operating in the THz band to achieve high data rates (such as considered for 6G). However, THz communications are highly susceptible to channel impairments and blockage effects that become extra challenging when accounting for drone mobility. RISs offer flexibility to extend coverage by adapting to channel dynamics. To integrate RISs into THz drone communications, we propose a novel deep learning solution based on a recurrent neural network, namely the Gated Recurrent Unit (GRU), that proactively predicts the serving base station/RIS and the serving beam for each drone based on the prior observations of drone location/beam trajectories. This solution has the potential to extend the coverage of drones and enhance the reliability of next-generation wireless communications. Predicting future beams based on the drone beam/position trajectory significantly reduces the beam training overhead and its associated latency, and thus emerges as a viable solution to serve time-critical applications. Numerical results based on realistic 3D ray-tracing simulations show that the proposed deep learning solution is promising for future RIS-assisted THz networks by achieving near-optimal proactive hand-off performance and more than 90% accuracy for beam prediction.

Machine Learning Empowered Trajectory and Passive Beamforming Design in UAV-RIS Wireless Networks

Abstract: A novel framework is proposed for integrating reconfigurable intelligent surfaces (RIS) in unmanned aerial vehicle (UAV) enabled wireless networks, where an RIS is deployed for enhancing the service quality of the UAV. Non-orthogonal multiple access (NOMA) technique is invoked to further improve the spectrum efficiency of the network, while mobile users (MUs) are considered as roaming continuously. The energy consumption minimizing problem is formulated by jointly designing the movement of the UAV, phase shifts of the RIS, power allocation policy from the UAV to MUs, as well as determining the dynamic decoding order. A decaying deep Q-network (D-DQN) based algorithm is proposed for tackling this pertinent problem. In the proposed D-DQN based algorithm, the central controller is selected as an agent for periodically observing the state of UAV-enabled wireless network and for carrying out actions to adapt to the dynamic environment. In contrast to the conventional DQN algorithm, the decaying learning rate is leveraged in the proposed D-DQN based algorithm for attaining a tradeoff between accelerating training speed and converging to the local optimal. Numerical results demonstrate that: 1) In contrast to the conventional Q-learning algorithm, which cannot converge when being adopted for solving the formulated problem, the proposed D-DQN based algorithm is capable of converging with minor constraints; 2) The energy dissipation of the UAV can be significantly reduced by integrating RISs in UAV-enabled wireless networks; 3) By designing the dynamic decoding order and power allocation policy, the RIS-NOMA case consumes 11.7% less energy than the RIS-OMA case.

Summary of existing literature about the integration of other promising technologies into RIS-assisted NTNs

Summary: This section opens up the vast possibility of using various promising technologies in RANTNs. The optical wireless communication enabled by free space optics and visible light communication, mmWave, and THz can be adopted to solve the spectrum scarcity problem. The wireless power transfer technology contribute towards extending the operation duration of aerial platforms, RISs, even IoTDs, whereas the multiple access schemes and spectral sharing strategies open opportunities to support massive users with limited channel resources. The summary of existing literature can be seen in Table {{table:1bf2b75c-5730-4ba7-9128-53426db0643e}} .


Nopol: Automatic Repair of Conditional Statement Bugs in Java Programs

Abstract: We propose Nopol, an approach to automatic repair of buggy conditional statements (i.e.,  if-then-else statements). This approach takes a buggy program as well as a test suite as input and generates a patch with a conditional expression as output. The test suite is required to contain passing test cases to model the expected behavior of the program and at least one failing test case that reveals the bug to be repaired. The process of Nopol consists of three major phases. First, Nopol employs angelic fix localization to identify expected values of a condition during the test execution. Second, runtime trace collection is used to collect variables and their actual values, including primitive data types and objected-oriented features (e.g., nullness checks), to serve as building blocks for patch generation. Third, Nopol encodes these collected data into an instance of a Satisfiability Modulo Theory (SMT) problem; then a feasible solution to the SMT instance is translated back into a code patch. We evaluate Nopol on 22 real-world bugs (16 bugs with buggy if conditions and six bugs with missing preconditions) on two large open-source projects, namely Apache Commons Math and Apache Commons Lang. Empirical analysis on these bugs shows that our approach can effectively fix bugs with buggy if conditions and missing preconditions. We illustrate the capabilities and limitations of Nopol using case studies of real bug fixes.

Precise Condition Synthesis for Program Repair

Abstract: Due to the difficulty of repairing defect, many research efforts have been devoted into automatic defect repair. Given a buggy program that fails some test cases, a typical automatic repair technique tries to modify the program to make all tests pass. However, since the test suites in real world projects are usually insufficient, aiming at passing the test suites often leads to incorrect patches. This problem is known as weak test suites or overfitting. In this paper we aim to produce precise patches, that is, any patch we produce has a relatively high probability to be correct. More concretely, we focus on condition synthesis, which was shown to be able to repair more than half of the defects in existing approaches. Our key insight is threefold. First, it is important to know what variables in a local context should be used in an "if" condition, and we propose a sorting method based on the dependency relations between variables. Second, we observe that the API document can be used to guide the repair process, and propose document analysis technique to further filter the variables. Third, it is important to know what predicates should be performed on the set of variables, and we propose to mine a set of frequently used predicates in similar contexts from existing projects. Based on the insight, we develop a novel program repair system, ACS, that could generate precise conditions at faulty locations. Furthermore, given the generated conditions are very precise, we can perform a repair operation that is previously deemed to be too overfitting: directly returning the test oracle to repair the defect. Using our approach, we successfully repaired 18 defects on four projects of Defects4J, which is the largest number of fully automatically repaired defects reported on the dataset so far. More importantly, the precision of our approach in the evaluation is 78.3%, which is significantly higher than previous approaches, which are usually less than 40%.

Number of bugs reported having been fixed by different APR tools. APR systems are ordered by year of publication.

Similarly, selecting a FL technique could be another issue since it can
make the performanc assessment biased.
Our investigations will use Table {{table:7c34f774-2a5e-4d22-8838-04c438c48024}}  as a starting point to highlight the problem of fault localization bias.
This table shows the number of fixed bugs out of the bugs in the Defects4J {{cite:2069292}} benchmark, which are reported by the authors of the current state-of-the-art APR tools in the literature.
The results of jGenProg, jKali and Nopol are extracted from the experimental data reported by Martinez et al. {{cite:bea2ce0}}. The results of other tools are collected from data reported by papers' authors in the literature.

Impact of Effective Ranking in Fault Localization

Given the bias that can be introduced by unlocalizable bugs being fixed by specific tweaking,
which are not clearly outlined by the authors,
we propose to count the numbers of bugs that are fixed by
APR systems among those bugs that are known to be localizable.
Table {{table:9d4f4cff-7785-499d-bf5c-1ac50b7d02e0}}  thus represents an updated version of
Table {{table:7c34f774-2a5e-4d22-8838-04c438c48024}}  where performance can be compared on the same basis.
To illustrate the differences between the two comparison tables, we compute three scores: (1) NPFB: number of plausibly-fixed bugs, (2) NCFB: number of correctly-fixed bugs, and (3) P{{formula:fda7aa9f-38c6-4b56-ac86-a7e0cad02e5f}} C: probability of plausible patch correctness.


Analyses the security threats, challenges, and mechanisms in all edge paradigms.

Explores dependability and deployment challenges in edge computing. Considers dependability in a wider meaning that includes security.

Surveys the security and privacy issues in the context of
edge-computing-assisted IoT.

Surveys security challenges, issues, and countermeasures in edge and fog computing.

Discusses MEC from the perspectives of security, dependability, and performance but keeps the discussion decoupled from the architecture.

Presents new challenges in physical security and scalable authentication, considering both centralized and decentralized security mechanisms.

Defines security and privacy in the context of the edge-computing-assisted IoT, gives some classifications of attacks, and discusses countermeasures.

Discusses security and privacy aspects in fog and edge computing.

Mobile Edge Computing, Fog et al.: A Survey and Analysis of Security Threats and Challenges

Abstract: For various reasons, the cloud computing paradigm is unable to meet certain requirements (e.g. low latency and jitter, context awareness, mobility support) that are crucial for several applications (e.g. vehicular networks, augmented reality). To fulfil these requirements, various paradigms, such as fog computing, mobile edge computing, and mobile cloud computing, have emerged in recent years. While these edge paradigms share several features, most of the existing research is compartmentalised; no synergies have been explored. This is especially true in the field of security, where most analyses focus only on one edge paradigm, while ignoring the others. The main goal of this study is to holistically analyse the security threats, challenges, and mechanisms inherent in all edge paradigms, while highlighting potential synergies and venues of collaboration. In our results, we will show that all edge paradigms should consider the advances in other paradigms.

Dependability in edge computing

Abstract: Edge computing holds great promise, and almost as many challenges in deployment.

A Survey on Security and Privacy Issues in Edge Computing-Assisted Internet of Things

Abstract: Internet of Things (IoT) is an innovative paradigm envisioned to provide massive applications that are now part of our daily lives. Millions of smart devices are deployed within complex networks to provide vibrant functionalities including communications, monitoring, and controlling of critical infrastructures. However, this massive growth of IoT devices and the corresponding huge data traffic generated at the edge of the network created additional burdens on the state-of-the-art centralized cloud computing paradigm due to the bandwidth and resources scarcity. Hence, edge computing (EC) is emerging as an innovative strategy that brings data processing and storage near to the end users, leading to what is called EC-assisted IoT. Although this paradigm provides unique features and enhanced quality of service (QoS), it also introduces huge risks in data security and privacy aspects. This paper conducts a comprehensive survey on security and privacy issues in the context of EC-assisted IoT. In particular, we first present an overview of EC-assisted IoT including definitions, applications, architecture, advantages, and challenges. Second, we define security and privacy in the context of EC-assisted IoT. Then, we extensively discuss the major classifications of attacks in EC-assisted IoT and provide possible solutions and countermeasures along with the related research efforts. After that, we further classify some security and privacy issues as discussed in the literature based on security services and based on security objectives and functions. Finally, several open challenges and future research directions for secure EC-assisted IoT paradigm are also extensively provided.

An Overview of Fog Computing and Edge Computing Security and Privacy Issues

Abstract: With the advancement of different technologies such as 5G networks and IoT the use of different cloud computing technologies became essential. Cloud computing allowed intensive data processing and warehousing solution. Two different new cloud technologies that inherit some of the traditional cloud computing paradigm are fog computing and edge computing that is aims to simplify some of the complexity of cloud computing and leverage the computing capabilities within the local network in order to preform computation tasks rather than carrying it to the cloud. This makes this technology fits with the properties of IoT systems. However, using such technology introduces several new security and privacy challenges that could be huge obstacle against implementing these technologies. In this paper, we survey some of the main security and privacy challenges that faces fog and edge computing illustrating how these security issues could affect the work and implementation of edge and fog computing. Moreover, we present several countermeasures to mitigate the effect of these security issues.

Academic Surveys Related to MEC Security

Table {{table:a915157f-99c0-468e-92b1-dfcbd2a09737}}  lists recent surveys on security and privacy for MEC. Papers that refer to general MEC security aspects are also considered. The number of publications considering MEC security and privacy is large, so the table does not intend to be exhaustive. Not many papers are fully dedicated to MEC but consider it together with other technologies such as cloud or fog (sometimes only marginally such as in e.g., {{cite:827fbb9}}). As mentioned in some of these (e.g., {{cite:b573a93}}, {{cite:0e86653}}), some specialized work on the security for MEC has been performed. However, the number of papers referring to particular aspects regarding general edge technologies (and, to some extent, applicable to MEC too) or general security issues that are not MEC specific is large and thus out of the goal of this paper. For example, a large number of papers are dedicated to defenses against (D)DoS in MEC (e.g., {{cite:955eba0}}, {{cite:26e2dd8}}, {{cite:5d1d20e}}, {{cite:aba663b}}) or usage of ML for MEC (e.g., {{cite:293c95f}}, {{cite:730767a}}, {{cite:83db2cf}}).
Security in 5G network slicing has been analyzed in {{cite:78963dd}}, and some aspects are relevant for isolation in MEC too.
Nevertheless, papers that survey aspects of MEC privacy and security do exist, e.g., {{cite:7e5c5d7}}, a comprehensive study performed in parallel and independent of our work, or {{cite:9b82b3b}} a MEC security analysis for each of the twelve considered vertical industries: (1) manufacturing industry, (2) financial sector, (3) healthcare, (4) education, (5) telecommunication, (6) authorities, (7) media and entertainment, (8) smart city, (9) agriculture and food industry, (10) logistics, (11) education, culture, and science, and critical infrastructure sectors.


A Robust Learning Approach to Domain Adaptive Object Detection

Abstract: Domain shift is unavoidable in real-world applications of object detection. For example, in self-driving cars, the target domain consists of unconstrained road environments which cannot all possibly be observed in training data. Similarly, in surveillance applications sufficiently representative training data may be lacking due to privacy regulations. In this paper, we address the domain adaptation problem from the perspective of robust learning and show that the problem may be formulated as training with noisy labels. We propose a robust object detection framework that is resilient to noise in bounding box class labels, locations and size annotations. To adapt to the domain shift, the model is trained on the target domain using a set of noisy object bounding boxes that are obtained by a detection model trained only in the source domain. We evaluate the accuracy of our approach in various source/target domain pairs and demonstrate that the model significantly improves the state-of-the-art on multiple domain adaptation scenarios on the SIM10K, Cityscapes and KITTI datasets.

Domain Adaptive Faster R-CNN for Object Detection in the Wild

Abstract: Object detection typically assumes that training and test data are drawn from an identical distribution, which, however, does not always hold in practice. Such a distribution mismatch will lead to a significant performance drop. In this work, we aim to improve the cross-domain robustness of object detection. We tackle the domain shift on two levels: 1) the image-level shift, such as image style, illumination, etc, and 2) the instance-level shift, such as object appearance, size, etc. We build our approach based on the recent state-of-the-art Faster R-CNN model, and design two domain adaptation components, on image level and instance level, to reduce the domain discrepancy. The two domain adaptation components are based on H-divergence theory, and are implemented by learning a domain classifier in adversarial training manner. The domain classifiers on different levels are further reinforced with a consistency regularization to learn a domain-invariant region proposal network (RPN) in the Faster R-CNN model. We evaluate our newly proposed approach using multiple datasets including Cityscapes, KITTI, SIM10K, etc. The results demonstrate the effectiveness of our proposed approach for robust object detection in various domain shift scenarios.

Multi-Adversarial Faster-RCNN for Unrestricted Object Detection

Abstract: Conventional object detection methods essentially suppose that the training and testing data are collected from a restricted target domain with expensive labeling cost. For alleviating the problem of domain dependency and cumbersome labeling, this paper proposes to detect objects in unrestricted environment by leveraging domain knowledge trained from an auxiliary source domain with sufficient labels. Specifically, we propose a multi-adversarial Faster-RCNN (MAF) framework for unrestricted object detection, which inherently addresses domain disparity minimization for domain adaptation in feature representation. The paper merits are in three-fold: 1) With the idea that object detectors often becomes domain incompatible when image distribution resulted domain disparity appears, we propose a hierarchical domain feature alignment module, in which multiple adversarial domain classifier submodules for layer-wise domain feature confusion are designed; 2) An information invariant scale reduction module (SRM) for hierarchical feature map resizing is proposed for promoting the training efficiency of adversarial domain adaptation; 3) In order to improve the domain adaptability, the aggregated proposal features with detection results are feed into a proposed weighted gradient reversal layer (WGRL) for characterizing hard confused domain samples. We evaluate our MAF on unrestricted tasks including Cityscapes, KITTI, Sim10k, etc. and the experiments show the state-of-the-art performance over the existing detectors.

Diversify and Match: A Domain Adaptive Representation Learning Paradigm for Object Detection

Abstract: We introduce a novel unsupervised domain adaptation approach for object detection. We aim to alleviate the imperfect translation problem of pixel-level adaptations, and the source-biased discriminativity problem of feature-level adaptations simultaneously. Our approach is composed of two stages, i.e., Domain Diversification (DD) and Multi-domain-invariant Representation Learning (MRL). At the DD stage, we diversify the distribution of the labeled data by generating various distinctive shifted domains from the source domain. At the MRL stage, we apply adversarial learning with a multi-domain discriminator to encourage feature to be indistinguishable among the domains. DD addresses the source-biased discriminativity, while MRL mitigates the imperfect image translation. We construct a structured domain adaptation framework for our learning paradigm and introduce a practical way of DD for implementation. Our method outperforms the state-of-the-art methods by a large margin of 3%~11% in terms of mean average precision (mAP) on various datasets.

Multi-Level Domain Adaptive Learning for Cross-Domain Detection

Abstract: In recent years, object detection has shown impressive results using supervised deep learning, but it remains challenging in a cross-domain environment. The variations of illumination, style, scale, and appearance in different domains can seriously affect the performance of detection models. Previous works use adversarial training to align global features across the domain shift and to achieve image information transfer. However, such methods do not effectively match the distribution of local features, resulting in limited improvement in cross-domain object detection. To solve this problem, we propose a multi-level domain adaptive model to simultaneously align the distributions of local-level features and global-level features. We evaluate our method with multiple experiments, including adverse weather adaptation, synthetic data adaptation, and cross camera adaptation. In most object categories, the proposed method achieves superior performance against state-of-the-art techniques, which demonstrates the effectiveness and robustness of our method.

Progressive Domain Adaptation for Object Detection

Abstract: Recent deep learning methods for object detection rely on a large amount of bounding box annotations. Collecting these annotations is laborious and costly, yet supervised models do not generalize well when testing on images from a different distribution. Domain adaptation provides a solution by adapting existing labels to the target testing data. However, a large gap between domains could make adaptation a challenging task, which leads to unstable training processes and sub-optimal results. In this paper, we propose to bridge the domain gap with an intermediate domain and progressively solve easier adaptation subtasks. This intermediate domain is constructed by translating the source images to mimic the ones in the target domain. To tackle the domain-shift problem, we adopt adversarial learning to align distributions at the feature level. In addition, a weighted task loss is applied to deal with unbalanced image quality in the intermediate domain. Experimental results show that our method performs favorably against the state-of-the-art method in terms of the performance on the target domain.

Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation

Abstract: Recent years have witnessed great progress in deep learning based object detection. However, due to the domain shift problem, applying off-the-shelf detectors to an unseen domain leads to significant performance drop. To address such an issue, this paper proposes a novel coarse-to-fine feature adaptation approach to cross-domain object detection. At the coarse-grained stage, different from the rough image-level or instance-level feature alignment used in the literature, foreground regions are extracted by adopting the attention mechanism, and aligned according to their marginal distributions via multi-layer adversarial learning in the common feature space. At the fine-grained stage, we conduct conditional distribution alignment of foregrounds by minimizing the distance of global prototypes with the same category but from different domains. Thanks to this coarse-to-fine feature adaptation, domain knowledge in foreground regions can be effectively transferred. Extensive experiments are carried out in various cross-domain detection scenarios. The results are state-of-the-art, which demonstrate the broad applicability and effectiveness of the proposed approach.

Performance comparison (in %) of different methods from Cityscapes to FoggyCityscapes for object detection. The 4th to the 11st columns indicate the Average Precision (AP) for the 8 different classes.

All the experiment result comparisons are shown in Table {{table:816f14d5-5a3e-467a-b244-047cad594e72}} , {{table:12183196-612e-44f8-9510-254e06d0e515}} , {{table:a8b91040-694c-48e8-b077-aa3ecc38c0a5}} , {{table:1d2faad5-5348-43c4-9b10-45b7e2c343c2}}  (image classification); Table {{table:ec89894f-14cb-4b96-8040-09be1b8df687}} , {{table:47b50a18-86d6-4305-9c53-56c16a1dc26e}}  (object detection); and Table {{table:6d808aed-8ebc-44e5-9b64-da50aebeeee1}} , {{table:13c1396e-7ed9-46b8-9bdb-f56f00a4feb8}}  (semantic segmentation). For each backbone, the methods are sorted by average classification accuracy, mAP and mIoU.


Conditional Generative Adversarial Networks

Recurrent Neural Networks and Convolutional Networks

Recurrent Neural Networks and Language Models

Graph Neural Networks and Language Models

Generating Digital Twins with Multiple Sclerosis Using Probabilistic Neural Networks

Abstract: Multiple Sclerosis (MS) is a neurodegenerative disorder characterized by a complex set of clinical assessments. We use an unsupervised machine learning model called a Conditional Restricted Boltzmann Machine (CRBM) to learn the relationships between covariates commonly used to characterize subjects and their disease progression in MS clinical trials. A CRBM is capable of generating digital twins, which are simulated subjects having the same baseline data as actual subjects. Digital twins allow for subject-level statistical analyses of disease progression. The CRBM is trained using data from 2395 subjects enrolled in the placebo arms of clinical trials across the three primary subtypes of MS. We discuss how CRBMs are trained and show that digital twins generated by the model are statistically indistinguishable from their actual subject counterparts along a number of measures.

Modeling Disease Progression in Mild Cognitive Impairment and Alzheimer's Disease with Digital Twins

Abstract: Alzheimer's Disease (AD) is a neurodegenerative disease that affects subjects in a broad range of severity and is assessed in clinical trials with multiple cognitive and functional instruments. As clinical trials in AD increasingly focus on earlier stages of the disease, especially Mild Cognitive Impairment (MCI), the ability to model subject outcomes across the disease spectrum is extremely important. We use unsupervised machine learning models called Conditional Restricted Boltzmann Machines (CRBMs) to create Digital Twins of AD subjects. Digital Twins are simulated clinical records that share baseline data with actual subjects and comprehensively model their outcomes under standard-of-care. The CRBMs are trained on a large set of records from subjects in observational studies and the placebo arms of clinical trials across the AD spectrum. These data exhibit a challenging, but common, patchwork of measured and missing observations across subjects in the dataset, and we present a novel model architecture designed to learn effectively from it. We evaluate performance against a held-out test dataset and show how Digital Twins simultaneously capture the progression of a number of key endpoints in clinical trials across a broad spectrum of disease severity, including MCI and mild-to-moderate AD.

Generating Multi-label Discrete Patient Records using Generative Adversarial Networks

Abstract: Access to electronic health record (EHR) data has motivated computational advances in medical research. However, various concerns, particularly over privacy, can limit access to and collaborative use of EHR data. Sharing synthetic EHR data could mitigate risk. In this paper, we propose a new approach, medical Generative Adversarial Network (medGAN), to generate realistic synthetic patient records. Based on input real patient records, medGAN can generate high-dimensional discrete variables (e.g., binary and count features) via a combination of an autoencoder and generative adversarial networks. We also propose minibatch averaging to efficiently avoid mode collapse, and increase the learning efficiency with batch normalization and shortcut connections. To demonstrate feasibility, we showed that medGAN generates synthetic patient records that achieve comparable performance to real data on many experiments including distribution statistics, predictive modeling tasks and a medical expert review. We also empirically observe a limited privacy risk in both identity and attribute disclosure using medGAN.

Generation of Synthetic Electronic Medical Record Text

Abstract: Machine learning (ML) and Natural Language Processing (NLP) have achieved remarkable success in many fields and have brought new opportunities and high expectation in the analyses of medical data. The most common type of medical data is the massive free-text electronic medical records (EMR). It is widely regarded that mining such massive data can bring up important information for improving medical practices as well as for possible new discoveries on complex diseases. However, the free EMR texts are lacking consistent standards, rich of private information, and limited in availability. Also, as they are accumulated from everyday practices, it is often hard to have a balanced number of samples for the types of diseases under study. These problems hinder the development of ML and NLP methods for EMR data analysis. To tackle these problems, we developed a model to generate synthetic text of EMRs called Medical Text Generative Adversarial Network or mtGAN. It is based on the GAN framework and is trained by the REINFORCE algorithm. It takes disease features as inputs and generates synthetic texts as EMRs for the corresponding diseases. We evaluate the model from micro-level, macro-level and application-level on a Chinese EMR text dataset. The results show that the method has a good capacity to fit real data and can generate realistic and diverse EMR samples. This provides a novel way to avoid potential leakage of patient privacy while still supply sufficient well-controlled cohort data for developing downstream ML and NLP methods. It can also be used as a data augmentation method to assist studies based on real EMR data.

CONAN: Complementary Pattern Augmentation for Rare Disease Detection

Abstract: Rare diseases affect hundreds of millions of people worldwide but are hard to detect since they have extremely low prevalence rates (varying from 1/1,000 to 1/200,000 patients) and are massively underdiagnosed. How do we reliably detect rare diseases with such low prevalence rates? How to further leverage patients with possibly uncertain diagnosis to improve detection? In this paper, we propose a Complementary pattern Augmentation (CONAN) framework for rare disease detection. CONAN combines ideas from both adversarial training and max-margin classification. It first learns self-attentive and hierarchical embedding for patient pattern characterization. Then, we develop a complementary generative adversarial networks (GAN) model to generate candidate positive and negative samples from the uncertain patients by encouraging a max-margin between classes. In addition, CONAN has a disease detector that serves as the discriminator during the adversarial training for identifying rare diseases. We evaluated CONAN on two disease detection tasks. For low prevalence inflammatory bowel disease (IBD) detection, CONAN achieved .96 precision recall area under the curve (PR-AUC) and 50.1% relative improvement over the best baseline. For rare disease idiopathic pulmonary fibrosis (IPF) detection, CONAN achieves .22 PR-AUC with 41.3% relative improvement over the best baseline.

EVA: Generating Longitudinal Electronic Health Records Using Conditional Variational Autoencoders

Abstract: Researchers require timely access to real-world longitudinal electronic health records (EHR) to develop, test, validate, and implement machine learning solutions that improve the quality and efficiency of healthcare. In contrast, health systems value deeply patient privacy and data security. De-identified EHRs do not adequately address the needs of health systems, as de-identified data are susceptible to re-identification and its volume is also limited. Synthetic EHRs offer a potential solution. In this paper, we propose EHR Variational Autoencoder (EVA) for synthesizing sequences of discrete EHR encounters (e.g., clinical visits) and encounter features (e.g., diagnoses, medications, procedures). We illustrate that EVA can produce realistic EHR sequences, account for individual differences among patients, and can be conditioned on specific disease conditions, thus enabling disease-specific studies. We design efficient, accurate inference algorithms by combining stochastic gradient Markov Chain Monte Carlo with amortized variational inference. We assess the utility of the methods on large real-world EHR repositories containing over 250, 000 patients. Our experiments, which include user studies with knowledgeable clinicians, indicate the generated EHR sequences are realistic. We confirmed the performance of predictive models trained on the synthetic data are similar with those trained on real EHRs. Additionally, our findings indicate that augmenting real data with synthetic EHRs results in the best predictive performance - improving the best baseline by as much as 8% in top-20 recall.

Modeling Tabular data using Conditional GAN

Abstract: Modeling the probability distribution of rows in tabular data and generating realistic synthetic data is a non-trivial task. Tabular data usually contains a mix of discrete and continuous columns. Continuous columns may have multiple modes whereas discrete columns are sometimes imbalanced making the modeling difficult. Existing statistical and deep neural network models fail to properly model this type of data. We design TGAN, which uses a conditional generative adversarial network to address these challenges. To aid in a fair and thorough comparison, we design a benchmark with 7 simulated and 8 real datasets and several Bayesian network baselines. TGAN outperforms Bayesian methods on most of the real datasets whereas other deep learning methods could not.

SurvTRACE: transformers for survival analysis with competing events

Abstract: In medicine, survival analysis studies the time duration to events of interest such as mortality. One major challenge is how to deal with multiple competing events (e.g., multiple disease diagnoses). In this work, we propose a transformer-based model that does not make the assumption for the underlying survival distribution and is capable of handling competing events, namely SurvTRACE. We account for the implicit confounders in the observational setting in multi-events scenarios, which causes selection bias as the predicted survival probability is influenced by irrelevant factors. To sufficiently utilize the survival data to train transformers from scratch, multiple auxiliary tasks are designed for multi-task learning. The model hence learns a strong shared representation from all these tasks and in turn serves for better survival analysis. We further demonstrate how to inspect the covariate relevance and importance through interpretable attention mechanisms of SurvTRACE, which suffices to great potential in enhancing clinical trial design and new treatment development. Experiments on METABRIC, SUPPORT, and SEER data with 470k patients validate the all-around superiority of our method. Software is available at https://github.com/RyanWangZf/SurvTRACE.

TransTab: Learning Transferable Tabular Transformers Across Tables

Abstract: Tabular data (or tables) are the most widely used data format in machine learning (ML). However, ML models often assume the table structure keeps fixed in training and testing. Before ML modeling, heavy data cleaning is required to merge disparate tables with different columns. This preprocessing often incurs significant data waste (e.g., removing unmatched columns and samples). How to learn ML models from multiple tables with partially overlapping columns? How to incrementally update ML models as more columns become available over time? Can we leverage model pretraining on multiple distinct tables? How to train an ML model which can predict on an unseen table? To answer all those questions, we propose to relax fixed table structures by introducing a Transferable Tabular Transformer (TransTab) for tables. The goal of TransTab is to convert each sample (a row in the table) to a generalizable embedding vector, and then apply stacked transformers for feature encoding. One methodology insight is combining column description and table cells as the raw input to a gated transformer model. The other insight is to introduce supervised and self-supervised pretraining to improve model performance. We compare TransTab with multiple baseline methods on diverse benchmark datasets and five oncology clinical trial datasets. Overall, TransTab ranks 1.00, 1.00, 1.78 out of 12 methods in supervised learning, feature incremental learning, and transfer learning scenarios, respectively; and the proposed pretraining leads to 2.3% AUC lift on average over the supervised learning.

Multi-Label Classification of Patient Notes a Case Study on ICD Code Assignment

Abstract: In the context of the Electronic Health Record, automated diagnosis coding of patient notes is a useful task, but a challenging one due to the large number of codes and the length of patient notes. We investigate four models for assigning multiple ICD codes to discharge summaries taken from both MIMIC II and III. We present Hierarchical Attention-GRU (HA-GRU), a hierarchical approach to tag a document by identifying the sentences relevant for each label. HA-GRU achieves state-of-the art results. Furthermore, the learned sentence-level attention layer highlights the model decision process, allows easier error analysis, and suggests future directions for improvement.

MiME: Multilevel Medical Embedding of Electronic Health Records for Predictive Healthcare

Abstract: Deep learning models exhibit state-of-the-art performance for many predictive healthcare tasks using electronic health records (EHR) data, but these models typically require training data volume that exceeds the capacity of most healthcare systems. External resources such as medical ontologies are used to bridge the data volume constraint, but this approach is often not directly applicable or useful because of inconsistencies with terminology. To solve the data insufficiency challenge, we leverage the inherent multilevel structure of EHR data and, in particular, the encoded relationships among medical codes. We propose Multilevel Medical Embedding (MiME) which learns the multilevel embedding of EHR data while jointly performing auxiliary prediction tasks that rely on this inherent EHR structure without the need for external labels. We conducted two prediction tasks, heart failure prediction and sequential disease prediction, where MiME outperformed baseline methods in diverse evaluation settings. In particular, MiME consistently outperformed all baselines when predicting heart failure on datasets of different volumes, especially demonstrating the greatest performance improvement (15% relative gain in PR-AUC over the best baseline) on the smallest dataset, demonstrating its ability to effectively model the multilevel structure of EHR data.

Scalable and accurate deep learning for electronic health records

Abstract: Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient's record. We propose a representation of patients' entire, raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two U.S. academic medical centers with 216,221 adult patients hospitalized for at least 24 hours. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting in-hospital mortality (AUROC across sites 0.93-0.94), 30-day unplanned readmission (AUROC 0.75-0.76), prolonged length of stay (AUROC 0.85-0.86), and all of a patient's final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed state-of-the-art traditional predictive models in all cases. We also present a case-study of a neural-network attribution system, which illustrates how clinicians can gain some transparency into the predictions. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios, complete with explanations that directly highlight evidence in the patient's chart.

RAIM: Recurrent Attentive and Intensive Model of Multimodal Patient Monitoring Data

Abstract: With the improvement of medical data capturing, vast amount of continuous patient monitoring data, e.g., electrocardiogram (ECG), real-time vital signs and medications, become available for clinical decision support at intensive care units (ICUs). However, it becomes increasingly challenging to model such data, due to high density of the monitoring data, heterogeneous data types and the requirement for interpretable models. Integration of these high-density monitoring data with the discrete clinical events (including diagnosis, medications, labs) is challenging but potentially rewarding since richness and granularity in such multimodal data increase the possibilities for accurate detection of complex problems and predicting outcomes (e.g., length of stay and mortality). We propose Recurrent Attentive and Intensive Model (RAIM) for jointly analyzing continuous monitoring data and discrete clinical events. RAIM introduces an efficient attention mechanism for continuous monitoring data (e.g., ECG), which is guided by discrete clinical events (e.g, medication usage). We apply RAIM in predicting physiological decompensation and length of stay in those critically ill patients at ICU. With evaluations on MIMIC- III Waveform Database Matched Subset, we obtain an AUC-ROC score of 90.18% for predicting decompensation and an accuracy of 86.82% for forecasting length of stay with our final model, which outperforms our six baseline models.

StageNet: Stage-Aware Neural Networks for Health Risk Prediction

Abstract: Deep learning has demonstrated success in health risk prediction especially for patients with chronic and progressing conditions. Most existing works focus on learning disease Network (StageNet) model to extract disease stage information from patient data and integrate it into risk prediction. StageNet is enabled by (1) a stage-aware long short-term memory (LSTM) module that extracts health stage variations unsupervisedly; (2) a stage-adaptive convolutional module that incorporates stage-related progression patterns into risk prediction. We evaluate StageNet on two real-world datasets and show that StageNet outperforms state-of-the-art models in risk prediction task and patient subtyping task. Compared to the best baseline model, StageNet achieves up to 12% higher AUPRC for risk prediction task on two real-world patient datasets. StageNet also achieves over 58% higher Calinski-Harabasz score (a cluster quality metric) for a patient subtyping task.

Online Disease Diagnosis with Inductive Heterogeneous Graph Convolutional Networks

Abstract: We propose a Healthcare Graph Convolutional Network (HealGCN) to offer disease self-diagnosis service for online users based on Electronic Healthcare Records (EHRs). Two main challenges are focused in this paper for online disease diagnosis: (1) serving cold-start users via graph convolutional networks and (2) handling scarce clinical description via a symptom retrieval system. To this end, we first organize the EHR data into a heterogeneous graph that is capable of modeling complex interactions among users, symptoms and diseases, and tailor the graph representation learning towards disease diagnosis with an inductive learning paradigm. Then, we build a disease self-diagnosis system with a corresponding EHR Graph-based Symptom Retrieval System (GraphRet) that can search and provide a list of relevant alternative symptoms by tracing the predefined meta-paths. GraphRet helps enrich the seed symptom set through the EHR graph when confronting users with scarce descriptions, hence yield better diagnosis accuracy. At last, we validate the superiority of our model on a large-scale EHR dataset.

Doctor AI: Predicting Clinical Events via Recurrent Neural Networks

Abstract: Leveraging large historical data in electronic health record (EHR), we developed Doctor AI, a generic predictive model that covers observed medical conditions and medication uses. Doctor AI is a temporal model using recurrent neural networks (RNN) and was developed and applied to longitudinal time stamped EHR data from 260K patients over 8 years. Encounter records (e.g. diagnosis codes, medication codes or procedure codes) were input to RNN to predict (all) the diagnosis and medication categories for a subsequent visit. Doctor AI assesses the history of patients to make multilabel predictions (one label for each diagnosis or medication category). Based on separate blind test set evaluation, Doctor AI can perform differential diagnosis with up to 79% recall@30, significantly higher than several baselines. Moreover, we demonstrate great generalizability of Doctor AI by adapting the resulting models from one institution to another without losing substantial accuracy.

Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks

Abstract: Predicting the future health information of patients from the historical Electronic Health Records (EHR) is a core research task in the development of personalized healthcare. Patient EHR data consist of sequences of visits over time, where each visit contains multiple medical codes, including diagnosis, medication, and procedure codes. The most important challenges for this task are to model the temporality and high dimensionality of sequential EHR data and to interpret the prediction results. Existing work solves this problem by employing recurrent neural networks (RNNs) to model EHR data and utilizing simple attention mechanism to interpret the results. However, RNN-based approaches suffer from the problem that the performance of RNNs drops when the length of sequences is large, and the relationships between subsequent visits are ignored by current RNN-based approaches. To address these issues, we propose {\sf Dipole}, an end-to-end, simple and robust model for predicting patients' future health information. Dipole employs bidirectional recurrent neural networks to remember all the information of both the past visits and the future visits, and it introduces three attention mechanisms to measure the relationships of different visits for the prediction. With the attention mechanisms, Dipole can interpret the prediction results effectively. Dipole also allows us to interpret the learned medical code representations which are confirmed positively by medical experts. Experimental results on two real world EHR datasets show that the proposed Dipole can significantly improve the prediction accuracy compared with the state-of-the-art diagnosis prediction approaches and provide clinically meaningful interpretation.

ConCare: Personalized Clinical Feature Embedding via Capturing the Healthcare Context

Abstract: Predicting the patient's clinical outcome from the historical electronic medical records (EMR) is a fundamental research problem in medical informatics. Most deep learning-based solutions for EMR analysis concentrate on learning the clinical visit embedding and exploring the relations between visits. Although those works have shown superior performances in healthcare prediction, they fail to explore the personal characteristics during the clinical visits thoroughly. Moreover, existing works usually assume that the more recent record weights more in the prediction, but this assumption is not suitable for all conditions. In this paper, we propose ConCare to handle the irregular EMR data and extract feature interrelationship to perform individualized healthcare prediction. Our solution can embed the feature sequences separately by modeling the time-aware distribution. ConCare further improves the multi-head self-attention via the cross-head decorrelation, so that the inter-dependencies among dynamic features and static baseline information can be effectively captured to form the personal health context. Experimental results on two real-world EMR datasets demonstrate the effectiveness of ConCare. The medical findings extracted by ConCare are also empirically confirmed by human experts and medical literature.

CATNet: Cross-event Attention-based Time-aware Network for Medical Event Prediction

Abstract: Medical event prediction (MEP) is a fundamental task in the healthcare domain, which needs to predict medical events, including medications, diagnosis codes, laboratory tests, procedures, outcomes, and so on, according to historical medical records of patients. Many researchers have tried to build MEP models to overcome the challenges caused by the heterogeneous and irregular temporal characteristics of EHR data. However, most of them consider the heterogenous and temporal medical events separately and ignore the correlations among different types of medical events, especially relations between heterogeneous historical medical events and target medical events. In this paper, we propose a novel neural network based on attention mechanism called Cross-event Attention-based Time-aware Network (CATNet) for MEP. It is a time-aware, event-aware and task-adaptive method with the following advantages: 1) modeling heterogeneous information and temporal information in a unified way and considering irregular temporal characteristics locally and globally respectively, 2) taking full advantage of correlations among different types of events via cross-event attention. Experiments on two public datasets (MIMIC-III and eICU) show CATNet outperforms other state-of-the-art methods on various MEP tasks. The source code of CATNet is released at https://github.com/sherry6247/CATNet.git.

Inferring Which Medical Treatments Work from Reports of Clinical Trials

Abstract: How do we know if a particular medical treatment actually works? Ideally one would consult all available evidence from relevant clinical trials. Unfortunately, such results are primarily disseminated in natural language scientific articles, imposing substantial burden on those trying to make sense of them. In this paper, we present a new task and corpus for making this unstructured evidence actionable. The task entails inferring reported findings from a full-text article describing a randomized controlled trial (RCT) with respect to a given intervention, comparator, and outcome of interest, e.g., inferring if an article provides evidence supporting the use of aspirin to reduce risk of stroke, as compared to placebo. We present a new corpus for this task comprising 10,000+ prompts coupled with full-text articles describing RCTs. Results using a suite of models --- ranging from heuristic (rule-based) approaches to attentive neural architectures --- demonstrate the difficulty of the task, which we believe largely owes to the lengthy, technical input texts. To facilitate further work on this important, challenging problem we make the corpus, documentation, a website and leaderboard, and code for baselines and evaluation available at http://evidence-inference.ebm-nlp.com/.

Trialstreamer: Mapping and Browsing Medical Evidence in Real-Time

Abstract: We introduce Trialstreamer, a living database of clinical trial reports. Here we mainly describe the evidence extraction component; this extracts from biomedical abstracts key pieces of information that clinicians need when appraising the literature, and also the relations between these. Specifically, the system extracts descriptions of trial participants, the treatments compared in each arm (the interventions), and which outcomes were measured. The system then attempts to infer which interventions were reported to work best by determining their relationship with identified trial outcome measures. In addition to summarizing individual trials, these extracted data elements allow automatic synthesis of results across many trials on the same topic. We apply the system at scale to all reports of randomized controlled trials indexed in MEDLINE, powering the automatic generation of evidence maps, which provide a global view of the efficacy of different interventions combining data from all relevant clinical trials on a topic. We make all code and models freely available alongside a demonstration of the web interface.

Trial2Vec: Zero-Shot Clinical Trial Document Similarity Search using Self-Supervision

Abstract: Clinical trials are essential for drug development but are extremely expensive and time-consuming to conduct. It is beneficial to study similar historical trials when designing a clinical trial. However, lengthy trial documents and lack of labeled data make trial similarity search difficult. We propose a zero-shot clinical trial retrieval method, Trial2Vec, which learns through self-supervision without annotating similar clinical trials. Specifically, the meta-structure of trial documents (e.g., title, eligibility criteria, target disease) along with clinical knowledge (e.g., UMLS knowledge base https://www.nlm.nih.gov/research/umls/index.html) are leveraged to automatically generate contrastive samples. Besides, Trial2Vec encodes trial documents considering meta-structure thus producing compact embeddings aggregating multi-aspect information from the whole document. We show that our method yields medically interpretable embeddings by visualization and it gets a 15% average improvement over the best baselines on precision/recall for trial retrieval, which is evaluated on our labeled 1600 trial pairs. In addition, we prove the pre-trained embeddings benefit the downstream trial outcome prediction task over 240k trials. Software ias available at https://github.com/RyanWangZf/Trial2Vec.

HINT: Hierarchical Interaction Network for Trial Outcome Prediction Leveraging Web Data

Abstract: Clinical trials are crucial for drug development but are time consuming, expensive, and often burdensome on patients. More importantly, clinical trials face uncertain outcomes due to issues with efficacy, safety, or problems with patient recruitment. If we were better at predicting the results of clinical trials, we could avoid having to run trials that will inevitably fail more resources could be devoted to trials that are likely to succeed. In this paper, we propose Hierarchical INteraction Network (HINT) for more general, clinical trial outcome predictions for all diseases based on a comprehensive and diverse set of web data including molecule information of the drugs, target disease information, trial protocol and biomedical knowledge. HINT first encode these multi-modal data into latent embeddings, where an imputation module is designed to handle missing data. Next, these embeddings will be fed into the knowledge embedding module to generate knowledge embeddings that are pretrained using external knowledge on pharmaco-kinetic properties and trial risk from the web. Then the interaction graph module will connect all the embedding via domain knowledge to fully capture various trial components and their complex relations as well as their influences on trial outcomes. Finally, HINT learns a dynamic attentive graph neural network to predict trial outcome. Comprehensive experimental results show that HINT achieves strong predictive performance, obtaining 0.772, 0.607, 0.623, 0.703 on PR-AUC for Phase I, II, III, and indication outcome prediction, respectively. It also consistently outperforms the best baseline method by up to 12.4\% on PR-AUC.

A list of papers related to AI and in silico clinical trials.

We list the main technical papers mentioned in this article in Table {{table:9d6b027c-ca70-443f-ac3e-8b5a12eda09a}} , which will be discussed in later sections. In addition, we also present the machine learning formulations and available data sources for each task. Since this review focuses on in silico trials, we will not cover how machine learning is used for other tasks in clinical trials, e.g., participant recruitment {{cite:68772f6}}, decentralized trials {{cite:a129df3}}, and trial site selection {{cite:25d1c63}}.