Improving Optimization Bounds using Machine Learning: Decision Diagrams meet Deep Reinforcement Learning

Abstract: Finding tight bounds on the optimal solution is a critical element of practical solution methods for discrete optimization problems. In the last decade, decision diagrams (DDs) have brought a new perspective on obtaining upper and lower bounds that can be significantly better than classical bounding mechanisms, such as linear relaxations. It is well known that the quality of the bounds achieved through this flexible bounding method is highly reliant on the ordering of variables chosen for building the diagram, and finding an ordering that optimizes standard metrics is an NP-hard problem. In this paper, we propose an innovative and generic approach based on deep reinforcement learning for obtaining an ordering for tightening the bounds obtained with relaxed and restricted DDs. We apply the approach to both the Maximum Independent Set Problem and the Maximum Cut Problem. Experimental results on synthetic instances show that the deep reinforcement learning approach, by achieving tighter objective function bounds, generally outperforms ordering methods commonly used in the literature when the distribution of instances is known. To the best knowledge of the authors, this is the first paper to apply machine learning to directly improve relaxation bounds obtained by general-purpose bounding mechanisms for combinatorial optimization problems.

Co-training for Policy Learning

Abstract: We study the problem of learning sequential decision-making policies in settings with multiple state-action representations. Such settings naturally arise in many domains, such as planning (e.g., multiple integer programming formulations) and various combinatorial optimization problems (e.g., those with both integer programming and graph-based formulations). Inspired by the classical co-training framework for classification, we study the problem of co-training for policy learning. We present sufficient conditions under which learning from two views can improve upon learning from a single view alone. Motivated by these theoretical insights, we present a meta-algorithm for co-training for sequential decision making. Our framework is compatible with both reinforcement learning and imitation learning. We validate the effectiveness of our approach across a wide range of tasks, including discrete/continuous control and combinatorial optimization.

Attention, Learn to Solve Routing Problems!

Abstract: The recently presented idea to learn heuristics for combinatorial optimization problems is promising as it can save costly development. However, to push this idea towards practical implementation, we need better models and better ways of training. We contribute in both directions: we propose a model based on attention layers with benefits over the Pointer Network and we show how to train this model using REINFORCE with a simple baseline based on a deterministic greedy rollout, which we find is more efficient than using a value function. We significantly improve over recent learned heuristics for the Travelling Salesman Problem (TSP), getting close to optimal results for problems up to 100 nodes. With the same hyperparameters, we learn strong heuristics for two variants of the Vehicle Routing Problem (VRP), the Orienteering Problem (OP) and (a stochastic variant of) the Prize Collecting TSP (PCTSP), outperforming a wide range of baselines and getting results close to highly optimized and specialized algorithms.

Neural Combinatorial Optimization with Reinforcement Learning

Abstract: This paper presents a framework to tackle combinatorial optimization problems using neural networks and reinforcement learning. We focus on the traveling salesman problem (TSP) and train a recurrent network that, given a set of city coordinates, predicts a distribution over different city permutations. Using negative tour length as the reward signal, we optimize the parameters of the recurrent network using a policy gradient method. We compare learning the network parameters on a set of training graphs against learning them on individual test graphs. Despite the computational expense, without much engineering and heuristic designing, Neural Combinatorial Optimization achieves close to optimal results on 2D Euclidean graphs with up to 100 nodes. Applied to the KnapSack, another NP-hard problem, the same method obtains optimal solutions for instances with up to 200 items.

Reinforcement Learning for Solving the Vehicle Routing Problem

Abstract: We present an end-to-end framework for solving the Vehicle Routing Problem (VRP) using reinforcement learning. In this approach, we train a single model that finds near-optimal solutions for problem instances sampled from a given distribution, only by observing the reward signals and following feasibility rules. Our model represents a parameterized stochastic policy, and by applying a policy gradient algorithm to optimize its parameters, the trained model produces the solution as a sequence of consecutive actions in real time, without the need to re-train for every new problem instance. On capacitated VRP, our approach outperforms classical heuristics and Google's OR-Tools on medium-sized instances in solution quality with comparable computation time (after training). We demonstrate how our approach can handle problems with split delivery and explore the effect of such deliveries on the solution quality. Our proposed framework can be applied to other variants of the VRP such as the stochastic VRP, and has the potential to be applied more generally to combinatorial optimization problems.

Solving a New 3D Bin Packing Problem with Deep Reinforcement Learning Method

Abstract: In this paper, a new type of 3D bin packing problem (BPP) is proposed, in which a number of cuboid-shaped items must be put into a bin one by one orthogonally. The objective is to find a way to place these items that can minimize the surface area of the bin. This problem is based on the fact that there is no fixed-sized bin in many real business scenarios and the cost of a bin is proportional to its surface area. Our research shows that this problem is NP-hard. Based on previous research on 3D BPP, the surface area is determined by the sequence, spatial locations and orientations of items. Among these factors, the sequence of items plays a key role in minimizing the surface area. Inspired by recent achievements of deep reinforcement learning (DRL) techniques, especially Pointer Network, on combinatorial optimization problems such as TSP, a DRL-based method is applied to optimize the sequence of items to be packed into the bin. Numerical results show that the method proposed in this paper achieve about 5% improvement than heuristic method.

Ranked Reward: Enabling Self-Play Reinforcement Learning for Combinatorial Optimization

Abstract: Adversarial self-play in two-player games has delivered impressive results when used with reinforcement learning algorithms that combine deep neural networks and tree search. Algorithms like AlphaZero and Expert Iteration learn tabula-rasa, producing highly informative training data on the fly. However, the self-play training strategy is not directly applicable to single-player games. Recently, several practically important combinatorial optimisation problems, such as the travelling salesman problem and the bin packing problem, have been reformulated as reinforcement learning problems, increasing the importance of enabling the benefits of self-play beyond two-player games. We present the Ranked Reward (R2) algorithm which accomplishes this by ranking the rewards obtained by a single agent over multiple games to create a relative performance metric. Results from applying the R2 algorithm to instances of a two-dimensional and three-dimensional bin packing problems show that it outperforms generic Monte Carlo tree search, heuristic algorithms and integer programming solvers. We also present an analysis of the ranked reward mechanism, in particular, the effects of problem instances with varying difficulty and different ranking thresholds.

Coloring Big Graphs with AlphaGoZero

Abstract: We show that recent innovations in deep reinforcement learning can effectively color very large graphs -- a well-known NP-hard problem with clear commercial applications. Because the Monte Carlo Tree Search with Upper Confidence Bound algorithm used in AlphaGoZero can improve the performance of a given heuristic, our approach allows deep neural networks trained using high performance computing (HPC) technologies to transform computation into improved heuristics with zero prior knowledge. Key to our approach is the introduction of a novel deep neural network architecture (FastColorNet) that has access to the full graph context and requires $O(V)$ time and space to color a graph with $V$ vertices, which enables scaling to very large graphs that arise in real applications like parallel computing, compilers, numerical solvers, and design automation, among others. As a result, we are able to learn new state of the art heuristics for graph coloring.

Categorization of the main approaches (Value-based, Policy-Based, MCTS) used for solving CO problems with RL. Problems include Maximum Independent Set (MIS), Maximum Coverage (MC), Maximum Common Subgraph (MCS), Minimum Vertex Cover (MVC), Traveling Salesman Problem (TSP), Knapsack Problem (KP), Vehicle Routing Problem (VRP), 3D Bin Packing Problem (3DBP), Graph Coloring (GC). For brevity we omit some of the problems and works, which we describe in detail in the remainder of the paper.

The survey is split into three parts - each covers a particular type of RL methods applied to solve CO problems. Among different approaches, we distinguish value-based, policy-based, and Monte Carlo Tree Search methods, which are shown in Table {{table:a85b2acc-8c6b-4a3c-a7b9-c8b355adce1e}} . For each method, we first provide a theoretical explanation that describes the main definitions and ideas, followed by the applications of each method to the specific CO problems. We conclude the survey by providing some future directions in Section .


Analysis on planned v/s unplanned mobility

LTE-A (same band for in and out of vehicle links)

Offloading traffic hotspots using moving small cells

Abstract: In this paper, the concept of moving small cells in mobile networks is presented and evaluated taking into account the dynamics of the system. We consider a small cell moving according to a Manhattan mobility model which is the case when the small cell is deployed on the top of a bus following a predefined trajectory in areas which are generally crowded. Taking into account the distribution of user locations, we study the dynamic level considering a queuing model composed of multi-class Processor Sharing queues. Macro and small cells are assumed to be operating in the same bandwidth. Consequently, they are coupled due to the mutual interferences generated by each cell to the other. Our results show that deploying moving small cells could be an efficient solution to offload traffic hotspots.

Shared Spectrum for Mobile-Cells Backhaul and Access Link

Abstract: Offloading cellular hotspot regions to small-cells has been the main theme for the fifth generation of cellular network. One such hotspot is the public transport which carries a large number of cellular users who frequently receive low quality of service (QoS) due to vehicular penetration effect (VPE). Hence installation of mobile-cell (MC) within public transport is seen as a potential enabler to enhance QoS for commuting users. However, unlike fixed cells, MC requires wireless backhaul (BH) connectivity along with in-vehicle Access-Link (AL) communication. These additional wireless links for MC communication will pose an excessive burden on an already scarce frequency spectrum. Hence, in this research, we exploit VPE and line-of-sight (LOS) communication to allow the downlink backhaul (DL-BH) sub-channels to be shared by in-vehicle downlink access-link (DL-AL) transmission. Our analysis and simulations show that using the above-mentioned technique, both links maintain high success probability, especially in regions with low signal to interference ratios.

Summary for existing solutions dealing with interference management.

A Cucker-Smale flocking model based transmission power control scheme was designed in {{cite:db557c7}} for fair bandwidth allocation in mobile relays. In {{cite:73ea60d}}, the authors proposed interference-weighted clustering algorithm to extract the interference patterns of mobile cells and proposed the dynamic resource allocation algorithm for heterogeneous networks with mobile cells. On the other hand, cross-tier interference for mobile cells using statistical channel estimation with mobility consideration was presented in {{cite:54fe7d8}}. In Table {{table:8fd1f65d-0fee-4573-8001-6af0af62934a}} , we summarize the existing works that discussed interference management issues in moving networks.


Machine learning: selecting and refining the beam pairs with two-layer online learning framework

Online Learning for Position-Aided Millimeter Wave Beam Training

Abstract: Accurate beam alignment is essential for beam-based millimeter wave communications. Conventional beam sweeping solutions often have large overhead, which is unacceptable for mobile applications like vehicle-to-everything. Learning-based solutions that leverage sensor data like position to identify good beam directions are one approach to reduce the overhead. Most existing solutions, though, are supervised-learning where the training data is collected beforehand. In this paper, we use a multi-armed bandit framework to develop online learning algorithms for beam pair selection and refinement. The beam pair selection algorithm learns coarse beam directions in some predefined beam codebook, e.g., in discrete angles separated by the 3dB beamwidths. The beam refinement fine-tunes the identified directions to match the peak of the power angular spectrum at that position. The beam pair selection uses the upper confidence bound (UCB) with a newly proposed risk-aware feature, while the beam refinement uses a modified optimistic optimization algorithm. The proposed algorithms learn to recommend good beam pairs quickly. When using 16x16 arrays at both the transmitter and receiver, it can achieve on average 1dB gain over the exhaustive search (over 271x271 beam pairs) on the unrefined codebook within 100 time-steps with a training budget of only 30 beam pairs.

High-Resolution Angle Tracking for Mobile Wideband Millimeter-Wave Systems With Antenna Array Calibration

Abstract: Millimeter-wave (mmWave) systems use directional beams to support high-rate data communications. Small misalignment between the transmit and receive beams (e.g., due to the mobility) can result in a significant drop of the received signal quality, especially in line-of-sight communication channels. In this paper, we propose and evaluate high-resolution angle tracking strategies for wideband mmWave systems with mobility. We custom design pairs of auxiliary beams as the tracking beams, and use them to capture the angle variations, toward which the steering directions of the data beams are adjusted. Different from conventional beam tracking designs, the proposed framework neither depends on the angle variation model nor requires an on-grid assumption. For practical implementation of the proposed methods, we examine the impact of the array calibration errors on the auxiliary beam pair design. Numerical results reveal that by employing the proposed methods, good angle tracking performance can be achieved under various antenna array configurations, channel models, and mobility conditions.

The Evaluation for The Existing BM Schemes

Extensibility of The Existing BM Mechanisms

In this subsection, we discuss the extensibility of existing BM schemes of conventional one-hop mmWave networks towards the IRS-assisted mmWave networks. For clarity, different BM mechanisms adopted at conventional mmWave networks are evaluated as Table {{table:d7063d35-7c00-487d-bcfc-b1894e6de444}} . Notably, the majority of these studies focus on beam-space searching based schemes. Specifically, to achieve initial access for the idle users, the authors in {{cite:62ec29a}} proposed a sequential beam training protocol, where BS performs narrow-beam exhaustive searching within the full space. However, the training overhead of the sequential beam training protocol is already prohibitive for the one-hop mmWave network, and the overhead will grow exponentially when extended to the IRS-assisted two-hop networks. Aiming to improve the efficiency of beam training, hierarchical beam training protocol is further designed by combining wide-beam and narrow-beam searching {{cite:927ff76}}. Utilizing the position statistic of the user, the authors in {{cite:b926d34}} present a fast beam training protocol by narrowing the searching range dynamically. It is noted that these ideas can be the essential guidance to the design of BM for IRS assisted mmWave networks. To maintain the mmWave links, extended Kalman filter {{cite:9d43651}} is utilized for beam tracking, whose performance highly relies on the precise modeling of user motion. Then, the authors in {{cite:e6051aa}} proposed an auxiliary beams pairs assisted beam tracking scheme, while high overhead and limited performance will be generated when extend to the IRS-assisted mmWave networks due to its awkwardness in handover. Recently, ML is emerging as a promising technology to achieve efficient BM for conventional mmWave networks. Specifically, the authors in {{cite:44d3466}}, {{cite:67a5a04}} proposed the ML based BM mechanisms by using the position information of users, which significantly improve the efficiency of BM compared to their counterparts. However, it is still impossible to fully address the challenges of BM for IRS assisted mmWave networks.


Compress and decompress the input samples

Aggregating the decision of several networks

Adding auxiliary noise modules in the network

Adversarial Examples for Semantic Segmentation and Object Detection

Abstract: It has been well demonstrated that adversarial examples, i.e., natural images with visually imperceptible perturbations added, generally exist for deep networks to fail on image classification. In this paper, we extend adversarial examples to semantic segmentation and object detection which are much more difficult. Our observation is that both segmentation and detection are based on classifying multiple targets on an image (e.g., the basic target is a pixel or a receptive field in segmentation, and an object proposal in detection), which inspires us to optimize a loss function over a set of pixels/proposals for generating adversarial perturbations. Based on this idea, we propose a novel algorithm named Dense Adversary Generation (DAG), which generates a large family of adversarial examples, and applies to a wide range of state-of-the-art deep networks for segmentation and detection. We also find that the adversarial perturbations can be transferred across networks with different training data, based on different architectures, and even for different recognition tasks. In particular, the transferability across networks with the same architecture is more significant than in other cases. Besides, summing up heterogeneous perturbations often leads to better transfer performance, which provides an effective method of black-box adversarial attack.

Explaining and Harnessing Adversarial Examples

Abstract: Several machine learning models, including neural networks, consistently misclassify adversarial examples---inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed input results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Using this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.

Countering Adversarial Images using Input Transformations

Abstract: This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier. Our experiments on ImageNet show that total variance minimization and image quilting are very effective defenses in practice, in particular, when the network is trained on transformed images. The strength of those defenses lies in their non-differentiable nature and their inherent randomness, which makes it difficult for an adversary to circumvent the defenses. Our best defense eliminates 60% of strong gray-box and 90% of strong black-box attacks by a variety of major attack methods

Adversarially Robust Distillation

Abstract: Knowledge distillation is effective for producing small, high-performance neural networks for classification, but these small networks are vulnerable to adversarial attacks. This paper studies how adversarial robustness transfers from teacher to student during knowledge distillation. We find that a large amount of robustness may be inherited by the student even when distilled on only clean images. Second, we introduce Adversarially Robust Distillation (ARD) for distilling robustness onto student networks. In addition to producing small models with high test accuracy like conventional distillation, ARD also passes the superior robustness of large networks onto the student. In our experiments, we find that ARD student models decisively outperform adversarially trained networks of identical architecture in terms of robust accuracy, surpassing state-of-the-art methods on standard robustness benchmarks. Finally, we adapt recent fast adversarial training methods to ARD for accelerated robust distillation.

Improving Adversarial Robustness of Ensembles with Diversity Training

Abstract: Deep Neural Networks are vulnerable to adversarial attacks even in settings where the attacker has no direct access to the model being attacked. Such attacks usually rely on the principle of transferability, whereby an attack crafted on a surrogate model tends to transfer to the target model. We show that an ensemble of models with misaligned loss gradients can provide an effective defense against transfer-based attacks. Our key insight is that an adversarial example is less likely to fool multiple models in the ensemble if their loss functions do not increase in a correlated fashion. To this end, we propose Diversity Training, a novel method to train an ensemble of models with uncorrelated loss functions. We show that our method significantly improves the adversarial robustness of ensembles and can also be combined with existing methods to create a stronger defense.

Learn2Perturb: an End-to-end Feature Perturbation Learning to Improve Adversarial Robustness

Abstract: While deep neural networks have been achieving state-of-the-art performance across a wide variety of applications, their vulnerability to adversarial attacks limits their widespread deployment for safety-critical applications. Alongside other adversarial defense approaches being investigated, there has been a very recent interest in improving adversarial robustness in deep neural networks through the introduction of perturbations during the training process. However, such methods leverage fixed, pre-defined perturbations and require significant hyper-parameter tuning that makes them very difficult to leverage in a general fashion. In this study, we introduce Learn2Perturb, an end-to-end feature perturbation learning approach for improving the adversarial robustness of deep neural networks. More specifically, we introduce novel perturbation-injection modules that are incorporated at each layer to perturb the feature space and increase uncertainty in the network. This feature perturbation is performed at both the training and the inference stages. Furthermore, inspired by the Expectation-Maximization, an alternating back-propagation training algorithm is introduced to train the network and noise parameters consecutively. Experimental results on CIFAR-10 and CIFAR-100 datasets show that the proposed Learn2Perturb method can result in deep neural networks which are $4-7\%$ more robust on $l_{\infty}$ FGSM and PDG adversarial attacks and significantly outperforms the state-of-the-art against $l_2$ $C\&W$ attack and a wide range of well-known black-box attacks.

Different defense mechanisms to improve the robustness of deep neural network models against adversarial attacks. These approaches can be performed in pre-processing, during training, post-processing, or even by changing the network architecture.

It is also possible to take advantage of post-processing techniques to improve the network robustness. Utilizing an ensemble of deep neural networks {{cite:8805dcf}} allows decision-making to be aggregated across several models, improving robustness against adversarial attacks. This strategy can be combined with averaging {{cite:cc4c212}} or noise perturbation {{cite:f5bc42e}} to make adversarial attacks more difficult. Table {{table:ca793502-6240-4067-b898-421c26dd7eb0}}  summarizes the main techniques to improve robustness of deep neural networks against adversarial attacks.

Adversarial Attacks & Defence Techniques

Several methods have been proposed to address the issue of deep neural networks facing adversarial attacks. These methods can be applied in different stages of networks training, from pre-processing to post-processing or even changing the architecture of the model. Table {{table:ca793502-6240-4067-b898-421c26dd7eb0}}  summarizes some of the common approaches to improve the robustness of deep neural networks.


A Robust Deep Learning Approach for Position-Independent Smartphone-Based Human Activity Recognition

Abstract: Recently, modern smartphones equipped with a variety of embedded-sensors, such as accelerometers and gyroscopes, have been used as an alternative platform for human activity recognition (HAR), since they are cost-effective, unobtrusive and they facilitate real-time applications. However, the majority of the related works have proposed a position-dependent HAR, i.e., the target subject has to fix the smartphone in a pre-defined position. Few studies have tackled the problem of position-independent HAR. They have tackled the problem either using handcrafted features that are less influenced by the position of the smartphone or by building a position-aware HAR. The performance of these studies still needs more improvement to produce a reliable smartphone-based HAR. Thus, in this paper, we propose a deep convolution neural network model that provides a robust position-independent HAR system. We build and evaluate the performance of the proposed model using the RealWorld HAR public dataset. We find that our deep learning proposed model increases the overall performance compared to the state-of-the-art traditional machine learning method from 84% to 88% for position-independent HAR. In addition, the position detection performance of our model improves superiorly from 89% to 98%. Finally, the recognition time of the proposed model is evaluated in order to validate the applicability of the model for real-time applications.

Automatic Annotation for Human Activity Recognition in Free Living Using a Smartphone

Abstract: Data annotation is a time-consuming process posing major limitations to the development of Human Activity Recognition (HAR) systems. The availability of a large amount of labeled data is required for supervised Machine Learning (ML) approaches, especially in the case of online and personalized approaches requiring user specific datasets to be labeled. The availability of such datasets has the potential to help address common problems of smartphone-based HAR, such as inter-person variability. In this work, we present (i) an automatic labeling method facilitating the collection of labeled datasets in free-living conditions using the smartphone, and (ii) we investigate the robustness of common supervised classification approaches under instances of noisy data. We evaluated the results with a dataset consisting of 38 days of manually labeled data collected in free living. The comparison between the manually and the automatically labeled ground truth demonstrated that it was possible to obtain labels automatically with an 80–85% average precision rate. Results obtained also show how a supervised approach trained using automatically generated labels achieved an 84% f-score (using Neural Networks and Random Forests); however, results also demonstrated how the presence of label noise could lower the f-score up to 64–74% depending on the classification approach (Nearest Centroid and Multi-Class Support Vector Machine).

DeepSense: A Unified Deep Learning Framework for Time-Series Mobile Sensing Data Processing

Abstract: Mobile sensing applications usually require time-series inputs from sensors. Some applications, such as tracking, can use sensed acceleration and rate of rotation to calculate displacement based on physical system models. Other applications, such as activity recognition, extract manually designed features from sensor inputs for classification. Such applications face two challenges. On one hand, on-device sensor measurements are noisy. For many mobile applications, it is hard to find a distribution that exactly describes the noise in practice. Unfortunately, calculating target quantities based on physical system and noise models is only as accurate as the noise assumptions. Similarly, in classification applications, although manually designed features have proven to be effective, it is not always straightforward to find the most robust features to accommodate diverse sensor noise patterns and user behaviors. To this end, we propose DeepSense, a deep learning framework that directly addresses the aforementioned noise and feature customization challenges in a unified manner. DeepSense integrates convolutional and recurrent neural networks to exploit local interactions among similar mobile sensors, merge local interactions of different sensory modalities into global interactions, and extract temporal relationships to model signal dynamics. DeepSense thus provides a general signal estimation and classification framework that accommodates a wide range of applications. We demonstrate the effectiveness of DeepSense using three representative and challenging tasks: car tracking with motion sensors, heterogeneous human activity recognition, and user identification with biometric motion analysis. DeepSense significantly outperforms the state-of-the-art methods for all three tasks. In addition, DeepSense is feasible to implement on smartphones due to its moderate energy consumption and low latency

Comparison of different HAR applications.

Edge intelligence could be used to extract contextual information from sensor data and facilitate the research on Human Activity Recognition (HAR). HAR refers to the problem of recognising when, where, and what a person is doing {{cite:7180b3c}}, which could be potentially used in many applications, e.g., healthcare, fitness tracking, and activity monitoring {{cite:b6d7ca7}}, {{cite:9aba1fe}}.
Table {{table:be9d459d-d1a9-4201-a245-aeff1f2e64ae}}  compares existing HAR technologies, regarding to their frameworks, models, ML methods, and objects.
The challenges of HAR on edge platforms could summarised as follows.


distributed spectrum access for maximizing network utility without message exchanges

adaptive DQN aided multichannel access yielding a near-optimal performance

Deep Multi-User Reinforcement Learning for Distributed Dynamic Spectrum Access

Abstract: We consider the problem of dynamic spectrum access for network utility maximization in multichannel wireless networks. The shared bandwidth is divided into K orthogonal channels. In the beginning of each time slot, each user selects a channel and transmits a packet with a certain transmission probability. After each time slot, each user that has transmitted a packet receives a local observation indicating whether its packet was successfully delivered or not (i.e., ACK signal). The objective is a multi-user strategy for accessing the spectrum that maximizes a certain network utility in a distributed manner without online coordination or message exchanges between users. Obtaining an optimal solution for the spectrum access problem is computationally expensive in general due to the large state space and partial observability of the states. To tackle this problem, we develop a novel distributed dynamic spectrum access algorithm based on deep multi-user reinforcement leaning. Specifically, at each time slot, each user maps its current state to spectrum access actions based on a trained deep-Q network used to maximize the objective function. Game theoretic analysis of the system dynamics is developed for establishing design principles for the implementation of the algorithm. Experimental results demonstrate strong performance of the algorithm.

Deep Reinforcement Learning for Dynamic Multichannel Access in Wireless Networks

Abstract: We consider a dynamic multichannel access problem, where multiple correlated channels follow an unknown joint Markov model. A user at each time slot selects a channel to transmit data and receives a reward based on the success or failure of the transmission. The objective is to find a policy that maximizes the expected long-term reward. The problem is formulated as a partially observable Markov decision process (POMDP) with unknown system dynamics. To overcome the challenges of unknown system dynamics as well as prohibitive computation, we apply the concept of reinforcement learning and implement a Deep Q-Network (DQN) that can deal with large state space without any prior knowledge of the system dynamics. We provide an analytical study on the optimal policy for fixed-pattern channel switching with known system dynamics and show through simulations that DQN can achieve the same optimal performance without knowing the system statistics. We compare the performance of DQN with a Myopic policy and a Whittle Index-based heuristic through both simulations as well as real-data trace and show that DQN achieves near-optimal performance in more complex situations. Finally, we propose an adaptive DQN approach with the capability to adapt its learning in time-varying, dynamic scenarios.

Compelling Applications of Deep Reinforcement Learning in NGWN

Table {{table:a3e8a459-b188-492e-953f-91f64ff24ea1}}  lists some typical applications of deep reinforcement learning in NGWN.


Visual Relationship Detection with Language Priors

Abstract: Visual relationships capture a wide variety of interactions between pairs of objects in images (e.g. "man riding bicycle" and "man pushing bicycle"). Consequently, the set of possible relationships is extremely large and it is difficult to obtain sufficient training examples for all possible relationships. Because of this limitation, previous work on visual relationship detection has concentrated on predicting only a handful of relationships. Though most relationships are infrequent, their objects (e.g. "man" and "bicycle") and predicates (e.g. "riding" and "pushing") independently occur more frequently. We propose a model that uses this insight to train visual models for objects and predicates individually and later combines them together to predict multiple relationships per image. We improve on prior work by leveraging language priors from semantic word embeddings to finetune the likelihood of a predicted relationship. Our model can scale to predict thousands of types of relationships from a few examples. Additionally, we localize the objects in the predicted relationships as bounding boxes in the image. We further demonstrate that understanding relationships can improve content based image retrieval.

Scene Graph Generation by Iterative Message Passing

Abstract: Understanding a visual scene goes beyond recognizing individual objects in isolation. Relationships between objects also constitute rich semantic information about the scene. In this work, we explicitly model the objects and their relationships using scene graphs, a visually-grounded graphical structure of an image. We propose a novel end-to-end model that generates such structured scene representation from an input image. The model solves the scene graph inference problem using standard RNNs and learns to iteratively improves its predictions via message passing. Our joint inference model can take advantage of contextual cues to make better predictions on objects and their relationships. The experiments show that our model significantly outperforms previous methods for generating scene graphs using Visual Genome dataset and inferring support relations with NYU Depth v2 dataset.

Comparison between existing visual relationship datasets.

We list all relationship related datasets in Table {{table:389fe72d-7087-43ae-a252-f84a02fe8335}} .
Visual phrase dataset {{cite:b674fc0}} focused on relation phrase recognition and detection. In the visual phrase, a relation triplet is annotated as a whole target.
Visual phrase dataset {{cite:b674fc0}} contained 8 objects categories from Pascal VOC2008 {{cite:2471040}} and 17 relation phrases with 9 different relationships. Scene graph dataset was proposed in {{cite:8205589}} and mainly explored the ability of image retrieval by scene graph.
The VRD dataset {{cite:1545301}} intended to benchmark the relationship detection task, and contains 37,993 relation triplets with 6,672 unique triplets
Open image {{cite:7b1925b}} provided the largest amount of images. It was a brilliant resource for object detection and also presented a challenging task for relationship detection. PIC {{cite:0deacbf}} is the newest among all the datasets we listed. It proposed an interesting task for combination the instance segmentation and relationships. In stead of taking detection bounding boxes and classes as supervision, the segmentation masks of instances became the main goal in PIC.
Moreover, UnRel {{cite:00abce0}} dataset focus on some rare relationships and present a new task for relation presentation in weakly-supervision. While due to the main target of UnRel is representing uncommon relationships, the amount of annotations is limited. There are only 1071 images in UnRel dataset, which are hard to be adapted in a supervised task.

Visual Genome (VG) {{cite:da77153}} had the maximum amount of relation data with the most diverse objects and relation categories in all listed datasets. As shown in Table {{table:389fe72d-7087-43ae-a252-f84a02fe8335}} , millions of object labels, relation triplets were contained in VG. However, the relations in VG are extracted from image captions, which contains lots of noisy and duplicate relations. Thus VG150 is constructed by processing VG dataset and only high frequency relationships are kept. However, most of high frequency relationships are visual irrelevance as we mentioned before.


Continuous Integration in a SocialCoding World Empirical Evidence from GitHub

An Insight into the Pull Requests of GitHub

Continuous integration in a social-coding world: Empirical evidence from GitHub. **Updated version with corrections**

Abstract: Continuous integration is a software engineering practice of frequently merging all developer working copies with a shared main branch, e.g., several times a day. With the advent of GitHub, a platform well known for its "social coding" features that aid collaboration and sharing, and currently the largest code host in the open source world, collaborative software development has never been more prominent. In GitHub development one can distinguish between two types of developer contributions to a project: direct ones, coming from a typically small group of developers with write access to the main project repository, and indirect ones, coming from developers who fork the main repository, update their copies locally, and submit pull requests for review and merger. In this paper we explore how GitHub developers use continuous integration as well as whether the contribution type (direct versus indirect) and different project characteristics (e.g., main programming language, or project age) are associated with the success of the automatic builds.

Abstract: Given the increasing number of unsuccessful pull requests in GitHub projects, insights into the success and failure of these requests are essential for the developers. In this paper, we provide a comparative study between successful and unsuccessful pull requests made to 78 GitHub base projects by 20,142 developers from 103,192 forked projects. In the study, we analyze pull request discussion texts, project specific information (e.g., domain, maturity), and developer specific information (e.g., experience) in order to report useful insights, and use them to contrast between successful and unsuccessful pull requests. We believe our study will help developers overcome the issues with pull requests in GitHub, and project administrators with informed decision making.

A sample of related qualitative and quantitative work.Here, by “quantitative”, we mean using data mining with littleto no interaction with project personnel.

Step 1: Literature Overview and Data Extraction

For the literature exploration, we searched for keywords `pull', `request' and `GitHub' on Google Scholar from 2008 to 2016 and also obtained a dataset from 16 top software engineering conferences, 1992 to 2016 {{cite:36622f7}}, filtering out the work unrelated to GitHub pull requests.
Table {{table:5b530503-df14-4b33-8d73-103a8dd16a13}}  lists the remaining research papers that have studied pull requests in GitHub using either qualitative or quantitative methods.
Here, we distinguish qualitative and quantitative methods by whether or not there is human involvement during data collection process. Qualitative studies have human involvement and include interviews, controlled human experiments, and surveys.
We observe that all previous studies on pull request in GitHub, except for one, use either qualitative or quantitative methods.
The remaining study combines both with a very time consuming manual analysis for the qualitative part {{cite:bc5e224}}, which starts from the very beginning with no previous knowledge. This is quite different from ours; we leverage prior work and apply crowdsourcing directly on the results extracted from primary qualitative studies.

Table {{table:088abd7c-d913-42cf-afce-e8001c5f7f36}}  summarizes the features found to be relevant in determining pull request acceptance. This includes all quantitative papers from Table {{table:5b530503-df14-4b33-8d73-103a8dd16a13}}  that use features to predict the outcomes of pull requests, and the features explored in at least one of those papers.
In Table {{table:088abd7c-d913-42cf-afce-e8001c5f7f36}} :


Recent advances in features extraction and description algorithms: A comprehensive survey

Deep learning for sentiment analysis: A survey

Addresses the recent advancements for feature extraction and description algorithms in the area of computer vision.

Explores the existing deep learning approaches that can be fine-tuned to achieve the desired results.

Illustrates the recent research works for semantic analysis via deep learning approaches.

Abstract: Computer vision is one of the most active research fields in information technology today. Giving machines and robots the ability to see and comprehend the surrounding world at the speed of sight creates endless potential applications and opportunities. Feature detection and description algorithms can be indeed considered as the retina of the eyes of such machines and robots. However, these algorithms are typically computationally intensive, which prevents them from achieving the speed of sight real-time performance. In addition, they differ in their capabilities and some may favor and work better given a specific type of input compared to others. As such, it is essential to compactly report their pros and cons as well as their performances and recent advances. This paper is dedicated to provide a comprehensive overview on the state-of-the-art and recent advances in feature detection and description algorithms. Specifically, it starts by overviewing fundamental concepts. It then compares, reports and discusses their performance and capabilities. The Maximally Stable Extremal Regions algorithm and the Scale Invariant Feature Transform algorithms, being two of the best of their type, are selected to report their recent algorithmic derivatives.

A Survey on Deep Transfer Learning

Abstract: As a new classification platform, deep learning has recently received increasing attention from researchers and has been successfully applied to many domains. In some domains, like bioinformatics and robotics, it is very difficult to construct a large-scale well-annotated dataset due to the expense of data acquisition and costly annotation, which limits its development. Transfer learning relaxes the hypothesis that the training data must be independent and identically distributed (i.i.d.) with the test data, which motivates us to use transfer learning to solve the problem of insufficient training data. This survey focuses on reviewing the current researches of transfer learning by using deep neural network and its applications. We defined deep transfer learning, category and review the recent research works based on the techniques used in deep transfer learning.

A Survey on Deep Transfer Learning

Abstract: Deep learning has emerged as a powerful machine learning technique that learns multiple layers of representations or features of the data and produces state‐of‐the‐art prediction results. Along with the success of deep learning in many application domains, deep learning is also used in sentiment analysis in recent years. This paper gives an overview of deep learning and then provides a comprehensive survey of its current applications in sentiment analysis.

Recent surveys conducted in the field of computer vision.

Computer vision, emerged from artificial intelligence, is the study of analysis of images and videos to gain deep understanding to serve the purpose of a concerned task. Starting from the distinguishing handwritten text from typed text in the 1950s, it has grown exponentially. This field studies the image classification, object detection, object localicalization, object segmentation, image captioning, etc. This field is also treated as if it provides eyes to the computers {{cite:f3e4d75}}. Fig. {{figure:b0d3ac65-ebc6-488f-ab2a-add5a51dd6d4}}  describes the disruptive technologies developed in this field of study since its origin. It is composed of several image and video processing tasks with a wide area of applications in industrial inspection or quality control, surveillance and security, face recognition, gesture recognition, medical image analysis, virtual reality, and a lot more {{cite:e64366c}}. With such wide real-world applications of this field, most research communities are exploring its dimensions to understand its potential. Table {{table:f95314f1-30e0-479e-9e4f-2c927a457664}}  presents the recent survey articles that can be utilized to explore its application domain.


Blockchain and IoT integration: A systematic survey

A survey of blockchain applications in different domains

A review on the role of blockchain technology in the healthcare domain

Untangling blockchain: A data processing view of blockchain systems

Presents the research trends in blockchain for IoT enabled environments and challenges associated with the IoT and blockchain.

Presents the recent advancements in blockchain in the broad areas of healthcare, insurance, energy, copyright protection, and societal applications.

Highlights the application of blockchain in the healthcare sector along with the generalized implementation framework.

Proposed blockbench to analyse the performance of blockchain models for varying data processing workloads.

Blockchain and IoT Integration: A Systematic Survey

Abstract: The Internet of Things (IoT) refers to the interconnection of smart devices to collect data and make intelligent decisions. However, a lack of intrinsic security measures makes IoT vulnerable to privacy and security threats. With its “security by design,” Blockchain (BC) can help in addressing major security requirements in IoT. BC capabilities like immutability, transparency, auditability, data encryption and operational resilience can help solve most architectural shortcomings of IoT. This article presents a comprehensive survey on BC and IoT integration. The objective of this paper is to analyze the current research trends on the usage of BC-related approaches and technologies in an IoT context. This paper presents the following novelties, with respect to related work: (i) it covers different application domains, organizing the available literature according to this categorization, (ii) it introduces two usage patterns, i.e., device manipulation and data management (open marketplace solution), and (iii) it reports on the development level of some of the presented solutions. We also analyze the main challenges faced by the research community in the smooth integration of BC and IoT, and point out the main open issues and future research directions. Last but not least, we also present a survey about novel uses of BC in the machine economy.

A Survey of Blockchain Applications in Different Domains

Abstract: Blockchains have received much attention recently since they provide decentralized approaches to the creation and management of value. Many banks, Internet companies, car manufacturers, and even governments worldwide have incorporated or started considering blockchains to improve the security, scalability, and efficiency of their services. In this paper, we survey blockchain applications in different areas. These areas include cryptocurrency, healthcare, advertising, insurance, copyright protection, energy, and societal applications. Our work provides a timely summary for individuals and organizations interested in blockchains. We envision our study to motivate more blockchain applications.

A Review on the Role of Blockchain Technology in the Healthcare Domain

Abstract: Recently, there have been increasing calls for healthcare providers to provide controls for patients over their personal health records. Nevertheless, security issues concerning how different healthcare providers exchange healthcare information have caused a flop in the deployment of such systems. The ability to exchange data securely is important so that new borderless integrated healthcare services can be provided to patients. Due to its decentralized nature, blockchain technology is a suitable driver for the much-needed shift towards integrated healthcare, providing new insights and addressing some of the main challenges of many healthcare areas. Blockchain allows healthcare providers to record and manage peer-to-peer transactions through a network without central authority. In this paper, we discuss the concept of blockchain technology and hurdles in their adoption in the healthcare domain. Furthermore, a review is conducted on the latest implementations of blockchain technology in healthcare. Finally, a new case study of a blockchain-based healthcare platform is presented addressing the drawbacks of current designs, followed by recommendations for future blockchain researchers and developers.

Untangling Blockchain: A Data Processing View of Blockchain Systems

Abstract: Blockchain technologies are gaining massive momentum in the last few years. Blockchains are distributed ledgers that enable parties who do not fully trust each other to maintain a set of global states. The parties agree on the existence, values, and histories of the states. As the technology landscape is expanding rapidly, it is both important and challenging to have a firm grasp of what the core technologies have to offer, especially with respect to their data processing capabilities. In this paper, we first survey the state of the art, focusing on private blockchains (in which parties are authenticated). We analyze both in-production and research systems in four dimensions: distributed ledger, cryptography, consensus protocol, and smart contract. We then present BLOCKBENCH, a benchmarking framework for understanding performance of private blockchains against data processing workloads. We conduct a comprehensive evaluation of three major blockchain systems based on BLOCKBENCH, namely Ethereum, Parity, and Hyperledger Fabric. The results demonstrate several trade-offs in the design space, as well as big performance gaps between blockchain and database systems. Drawing from design principles of database systems, we discuss several research directions for bringing blockchain performance closer to the realm of databases.

Recent surveys conducted in blockchain technology.

Blockchain also known as the decentralized ledger, conceptualized by S. Nakomoto in 2008, is a cryptographic distributed database without any central controller {{cite:8f87dc1}} that features cryptography, immutability, provenance, decentralization, anonymity, and transparency. As illustrated in Fig. {{figure:71010912-3269-4de8-971d-6524288b9b04}}  the concept of blockchain initiated as the electronic transaction system while expanding its dimensions in broad application areas e.g. cryptocurrency, banking, supply chain, property records, healthcare, security, privacy, etc. {{cite:3d23762}}. The key components of blockchain network are node (user or computer within the blockchain), transaction (smallest building block of a blockchain system), block (a data structure used for keeping a set of transactions which is distributed to all nodes in the network), chain (a sequence of blocks in a specific order), miners (specific nodes which perform the block verification process), and consensus (a set of rules and arrangements to carry out blockchain operations). Table {{table:e720d2f8-92db-4640-b0a5-608538bcd1a4}}  describes the recent advancements in the blockchain technology with the help of survey papers.


DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images

Abstract: Understanding fashion images has been advanced by benchmarks with rich annotations such as DeepFashion, whose labels include clothing categories, landmarks, and consumer-commercial image pairs. However, DeepFashion has nonnegligible issues such as single clothing-item per image, sparse landmarks (4∼8 only), and no per-pixel masks, making it had significant gap from real-world scenarios. We fill in the gap by presenting DeepFashion2 to address these issues. It is a versatile benchmark of four tasks including clothes detection, pose estimation, segmentation, and retrieval. It has 801K clothing items where each item has rich annotations such as style, scale, view- point, occlusion, bounding box, dense landmarks (e.g. 39 for ‘long sleeve outwear’ and 15 for ‘vest’), and masks. There are also 873K Commercial-Consumer clothes pairs. The annotations of DeepFashion2 are much larger than its counterparts such as 8× of FashionAI Global Challenge. A strong baseline is proposed, called Match R- CNN, which builds upon Mask R-CNN to solve the above four tasks in an end-to-end manner. Extensive evaluations are conducted with different criterions in Deep- Fashion2. DeepFashion2 Dataset will be released at : https://github.com/switchablenorms/DeepFashion2

Cross-domain Image Retrieval with a Dual Attribute-aware Ranking Network

Abstract: We address the problem of cross-domain image retrieval, considering the following practical application: given a user photo depicting a clothing image, our goal is to retrieve the same or attribute-similar clothing items from online shopping stores. This is a challenging problem due to the large discrepancy between online shopping images, usually taken in ideal lighting/pose/background conditions, and user photos captured in uncontrolled conditions. To address this problem, we propose a Dual Attribute-aware Ranking Network (DARN) for retrieval feature learning. More specifically, DARN consists of two sub-networks, one for each domain, whose retrieval feature representations are driven by semantic attribute learning. We show that this attribute-guided learning is a key factor for retrieval accuracy improvement. In addition, to further align with the nature of the retrieval problem, we impose a triplet visual similarity constraint for learning to rank across the two sub-networks. Another contribution of our work is a large-scale dataset which makes the network learning feasible. We exploit customer review websites to crawl a large set of online shopping images and corresponding offline user photos with fine-grained clothing attributes, i.e., around 450,000 online shopping images and about 90,000 exact offline counterpart images of those online ones. All these images are collected from real-world consumer websites reflecting the diversity of the data modality, which makes this dataset unique and rare in the academic community. We extensively evaluate the retrieval performance of networks in different configurations. The top-20 retrieval accuracy is doubled when using the proposed DARN other than the current popular solution using pre-trained CNN features only (0.570 vs. 0.268).

VITON: An Image-Based Virtual Try-on Network

Abstract: We present an image-based VIirtual Try-On Network (VITON) without using 3D information in any form, which seamlessly transfers a desired clothing item onto the corresponding region of a person using a coarse-to-fine strategy. Conditioned upon a new clothing-agnostic yet descriptive person representation, our framework first generates a coarse synthesized image with the target clothing item overlaid on that same person in the same pose. We further enhance the initial blurry clothing area with a refinement network. The network is trained to learn how much detail to utilize from the target clothing item, and where to apply to the person in order to synthesize a photo-realistic image in which the target item deforms naturally with clear visual patterns. Experiments on our newly collected Zalando dataset demonstrate its promise in the image-based virtual try-on task over state-of-the-art generative models.1

Towards Multi-Pose Guided Virtual Try-On Network

Abstract: Virtual try-on systems under arbitrary human poses have significant application potential, yet also raise extensive challenges, such as self-occlusions, heavy misalignment among different poses, and complex clothes textures. Existing virtual try-on methods can only transfer clothes given a fixed human pose, and still show unsatisfactory performances, often failing to preserve person identity or texture details, and with limited pose diversity. This paper makes the first attempt towards a multi-pose guided virtual try-on system, which enables clothes to transfer onto a person with diverse poses. Given an input person image, a desired clothes image, and a desired pose, the proposed Multi-pose Guided Virtual Try-On Network (MG-VTON) generates a new person image after fitting the desired clothes into the person and manipulating the pose. MG-VTON is constructed with three stages: 1) a conditional human parsing network is proposed that matches both the desired pose and the desired clothes shape; 2) a deep Warping Generative Adversarial Network (Warp-GAN) that warps the desired clothes appearance into the synthesized human parsing map and alleviates the misalignment problem between the input human pose and the desired one; 3) a refinement render network recovers the texture details of clothes and removes artifacts, based on multi-pose composition masks. Extensive experiments on commonly-used datasets and our newly-collected largest virtual try-on benchmark demonstrate that our MG-VTON significantly outperforms all state-of-the-art methods both qualitatively and quantitatively, showing promising virtual try-on performances.

The utilized fashion-related datasets.

Table  {{table:92774256-8532-4d94-8e80-9db406897361}}  shows various fashion-related datasets that were used in detection and retrieval tasks. Specifically, original images were used to train detection models while cropped and deduplicated images were for the retrieval models.

Table  {{table:0357e329-5d68-4810-ba00-fe802dba0f61}}  shows the performances of retrieval models using ground-truth bounding boxes on the DeepFashion2 validation set. In addition to the DeepFashion2 train set, we accumulated publicly available datasets as listed in Table {{table:92774256-8532-4d94-8e80-9db406897361}}  to train our retrieval models. The default configurations used to train retrieval models is described in the Table {{table:988fa595-3757-4125-ab84-ed757b7c8fe5}} .


T-GCN: A Temporal Graph ConvolutionalNetwork for Traffic Prediction

Abstract: Accurate and real-time traffic forecasting plays an important role in the Intelligent Traffic System and is of great significance for urban traffic planning, traffic management, and traffic control. However, traffic forecasting has always been considered an open scientific issue, owing to the constraints of urban road network topological structure and the law of dynamic change with time, namely, spatial dependence and temporal dependence. To capture the spatial and temporal dependence simultaneously, we propose a novel neural network-based traffic forecasting method, the temporal graph convolutional network (T-GCN) model, which is in combination with the graph convolutional network (GCN) and gated recurrent unit (GRU). Specifically, the GCN is used to learn complex topological structures to capture spatial dependence and the gated recurrent unit is used to learn dynamic changes of traffic data to capture temporal dependence. Then, the T-GCN model is employed to traffic forecasting based on the urban road network. Experiments demonstrate that our T-GCN model can obtain the spatio-temporal correlation from traffic data and the predictions outperform state-of-art baselines on real-world traffic datasets. Our tensorflow implementation of the T-GCN is available at https://github.com/lehaifeng/T-GCN.

GMAN: A Graph Multi-Attention Network for Traffic Prediction

Abstract: Long-term traffic prediction is highly challenging due to the complexity of traffic systems and the constantly changing nature of many impacting factors. In this paper, we focus on the spatio-temporal factors, and propose a graph multi-attention network (GMAN) to predict traffic conditions for time steps ahead at different locations on a road network graph. GMAN adapts an encoder-decoder architecture, where both the encoder and the decoder consist of multiple spatio-temporal attention blocks to model the impact of the spatio-temporal factors on traffic conditions. The encoder encodes the input traffic features and the decoder predicts the output sequence. Between the encoder and the decoder, a transform attention layer is applied to convert the encoded traffic features to generate the sequence representations of future time steps as the input of the decoder. The transform attention mechanism models the direct relationships between historical and future time steps that helps to alleviate the error propagation problem among prediction time steps. Experimental results on two real-world traffic prediction tasks (i.e., traffic volume prediction and traffic speed prediction) demonstrate the superiority of GMAN. In particular, in the 1 hour ahead prediction, GMAN outperforms state-of-the-art methods by up to 4% improvement in MAE measure. The source code is available at https://github.com/zhengchuanpan/GMAN.

Graph WaveNet for Deep Spatial-Temporal Graph Modeling

Abstract: Spatial-temporal graph modeling is an important task to analyze the spatial relations and temporal trends of components in a system. Existing approaches mostly capture the spatial dependency on a fixed graph structure, assuming that the underlying relation between entities is pre-determined. However, the explicit graph structure (relation) does not necessarily reflect the true dependency and genuine relation may be missing due to the incomplete connections in the data. Furthermore, existing methods are ineffective to capture the temporal trends as the RNNs or CNNs employed in these methods cannot capture long-range temporal sequences. To overcome these limitations, we propose in this paper a novel graph neural network architecture, {Graph WaveNet}, for spatial-temporal graph modeling. By developing a novel adaptive dependency matrix and learn it through node embedding, our model can precisely capture the hidden spatial dependency in the data. With a stacked dilated 1D convolution component whose receptive field grows exponentially as the number of layers increases, Graph WaveNet is able to handle very long sequences. These two components are integrated seamlessly in a unified framework and the whole framework is learned in an end-to-end manner. Experimental results on two public traffic network datasets, METR-LA and PEMS-BAY, demonstrate the superior performance of our algorithm.

Some open source codes

Open-source implementations are helpful for researchers to compare their approaches. We provide the hyperlinks of public source codes of the literatures reviewed in this paper (as shown in Table {{table:447201d1-8ff1-4167-865e-1db0c10fc049}} ) to facilitate the baseline experiments in traffic domain.


Show and tell: A neural image caption generator

Abstract: Automatically describing the content of an image is a fundamental problem in artificial intelligence that connects computer vision and natural language processing. In this paper, we present a generative model based on a deep recurrent architecture that combines recent advances in computer vision and machine translation and that can be used to generate natural sentences describing an image. The model is trained to maximize the likelihood of the target description sentence given the training image. Experiments on several datasets show the accuracy of the model and the fluency of the language it learns solely from image descriptions. Our model is often quite accurate, which we verify both qualitatively and quantitatively. For instance, while the current state-of-the-art BLEU-1 score (the higher the better) on the Pascal dataset is 25, our approach yields 59, to be compared to human performance around 69. We also show BLEU-1 score improvements on Flickr30k, from 56 to 66, and on SBU, from 19 to 28. Lastly, on the newly released COCO dataset, we achieve a BLEU-4 of 27.7, which is the current state-of-the-art.

Generative Adversarial Text to Image Synthesis

Abstract: Automatic synthesis of realistic images from text would be interesting and useful, but current AI systems are still far from this goal. However, in recent years generic and powerful recurrent neural network architectures have been developed to learn discriminative text feature representations. Meanwhile, deep convolutional generative adversarial networks (GANs) have begun to generate highly compelling images of specific categories, such as faces, album covers, and room interiors. In this work, we develop a novel deep architecture and GAN formulation to effectively bridge these advances in text and image modeling, translating visual concepts from characters to pixels. We demonstrate the capability of our model to generate plausible images of birds and flowers from detailed text descriptions.

End-to-end GPU microservices in Camelot suite

According to the above concepts,
we choose user-facing applications that uses common deep learning techniques and implement them
in microservice architecture.
Table {{table:7b01d04c-5bfb-45b6-b9d6-247a0e75f1cf}}  lists the end-to-end user-facing applications that cover a wide spectrum of real applications based on GPU microservices.


Pre-trained embeddings + neural networks

Causal-driven rep. learning + Regression adjustment, TMLE

Learning Representations for Counterfactual Inference

Abstract: Observational studies are rising in importance due to the widespread accumulation of data in fields such as healthcare, education, employment and ecology. We consider the task of answering counterfactual questions such as, "Would this patient have lower blood sugar had she received a different medication?". We propose a new algorithmic framework for counterfactual inference which brings together ideas from domain adaptation and representation learning. In addition to a theoretical justification, we perform an empirical comparison with previous approaches to causal inference from observational data. Our deep learning algorithm significantly outperforms the previous state-of-the-art.

A Deep Causal Inference Approach to Measuring the Effects of Forming Group Loans in Online Non-profit Microfinance Platform

Abstract: Kiva is an online non-profit crowdsouring microfinance platform that raises funds for the poor in the third world. The borrowers on Kiva are small business owners and individuals in urgent need of money. To raise funds as fast as possible, they have the option to form groups and post loan requests in the name of their groups. While it is generally believed that group loans pose less risk for investors than individual loans do, we study whether this is the case in a philanthropic online marketplace. In particular, we measure the effect of group loans on funding time while controlling for the loan sizes and other factors. Because loan descriptions (in the form of texts) play an important role in lenders' decision process on Kiva, we make use of this information through deep learning in natural language processing. In this aspect, this is the first paper that uses one of the most advanced deep learning techniques to deal with unstructured data in a way that can take advantage of its superior prediction power to answer causal questions. We find that on average, forming group loans speeds up the funding time by about 3.3 days.

Estimating Causal Effects of Tone in Online Debates

Abstract: Statistical methods applied to social media posts shed light on the dynamics of online dialogue. For example, users' wording choices predict their persuasiveness and users adopt the language patterns of other dialogue participants. In this paper, we estimate the causal effect of reply tones in debates on linguistic and sentiment changes in subsequent responses. The challenge for this estimation is that a reply's tone and subsequent responses are confounded by the users' ideologies on the debate topic and their emotions. To overcome this challenge, we learn representations of ideology using generative models of text. We study debates from 4Forums.com and compare annotated tones of replying such as emotional versus factual, or reasonable versus attacking. We show that our latent confounder representation reduces bias in ATE estimation. Our results suggest that factual and asserting tones affect dialogue and provide a methodology for estimating causal effects from text.

Adapting Text Embeddings for Causal Inference

Abstract: Does adding a theorem to a paper affect its chance of acceptance? Does labeling a post with the author's gender affect the post popularity? This paper develops a method to estimate such causal effects from observational text data, adjusting for confounding features of the text such as the subject or writing quality. We assume that the text suffices for causal adjustment but that, in practice, it is prohibitively high-dimensional. To address this challenge, we develop causally sufficient embeddings, low-dimensional document representations that preserve sufficient information for causal identification and allow for efficient estimation of causal effects. Causally sufficient embeddings combine two ideas. The first is supervised dimensionality reduction: causal adjustment requires only the aspects of text that are predictive of both the treatment and outcome. The second is efficient language modeling: representations of text are designed to dispose of linguistically irrelevant information, and this information is also causally irrelevant. Our method adapts language models (specifically, word embeddings and topic models) to learn document embeddings that are able to predict both treatment and outcome. We study causally sufficient embeddings with semi-synthetic datasets and find that they improve causal estimation over related embedding methods. We illustrate the methods by answering the two motivating questions---the effect of a theorem on paper acceptance and the effect of a gender label on post popularity. Code and data available at https://github.com/vveitch/causal-text-embeddings-tf2}{github.com/vveitch/causal-text-embeddings-tf2

Example applications that infer the causal effects of treatment on outcome by measuring confounders (unobserved) from text data (observed). In doing so, these applications choose a representation of text (text rep.) and a method to adjust for confounding.

In Table {{table:1239afde-decf-44ac-8326-db44cb945d14}} , we gather and summarize applications that use text to adjust for potential confounding.
This encompasses both (a) text as a surrogate for confounders, or (b) the language itself as confounders.We acknowledge that Table {{table:1239afde-decf-44ac-8326-db44cb945d14}}  is by no means exhaustive. To construct Table {{table:1239afde-decf-44ac-8326-db44cb945d14}} , we started with three seed papers: {{cite:dd1c9ec}}, {{cite:abd367c}}, and {{cite:ba405be}}. We then examined papers cited by these papers, papers that cited these papers, and papers published by the papers' authors. We repeated this approach with the additional papers we found that adjusted for confounding with text. We also examined papers matching the query “causal" or “causality" in the ACL Anthology.


Webly Supervised Learning of Convolutional Networks

Abstract: We present an approach to utilize large amounts of web data for learning CNNs. Specifically inspired by curriculum learning, we present a two-step approach for CNN training. First, we use easy images to train an initial visual representation. We then use this initial CNN and adapt it to harder, more realistic images by leveraging the structure of data and categories. We demonstrate that our two-stage CNN outperforms a fine-tuned CNN trained on ImageNet on Pascal VOC 2012. We also demonstrate the strength of webly supervised learning by localizing objects in web images and training a R-CNN style detector. It achieves the best performance on VOC 2007 where no VOC training data is used. Finally, we show our approach is quite robust to noise and performs comparably even when we use image search results from March 2013 (pre-CNN image search era).

ConceptLearner: Discovering visual concepts from weakly labeled image collections

Abstract: Discovering visual knowledge from weakly labeled data is crucial to scale up computer vision recognition systems, since it is expensive to obtain fully labeled data for a large number of concept categories. In this paper, we propose ConceptLearner, which is a scalable approach to discover visual concepts from weakly labeled image collections. Thousands of visual concept detectors are learned automatically, without human in the loop for additional annotation. We show that these learned detectors could be applied to recognize concepts at image-level and to detect concepts at image region-level accurately. Under domain-specific supervision, we further evaluate the learned concepts for scene recognition on SUN database and for object detection on Pascal VOC 2007. ConceptLearner shows promising performance compared to fully supervised and weakly supervised methods.

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

Abstract: Object detection performance, as measured on the canonical PASCAL VOC dataset, has plateaued in the last few years. The best-performing methods are complex ensemble systems that typically combine multiple low-level image features with high-level context. In this paper, we propose a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%. Our approach combines two key insights: (1) one can apply high-capacity convolutional neural networks (CNNs) to bottom-up region proposals in order to localize and segment objects and (2) when labeled training data is scarce, supervised pre-training for an auxiliary task, followed by domain-specific fine-tuning, yields a significant performance boost. Since we combine region proposals with CNNs, we call our method R-CNN: Regions with CNN features. We also present experiments that provide insight into what the network learns, revealing a rich hierarchy of image features. Source code for the complete system is available at http://www.cs.berkeley.edu/~rbg/rcnn.

Comparison of supervision between our method and others. The supervision in our work only comes from labeling the CAD models and their texture when generating the synthetic images. The second column indicates whether in-domain data (VOC train+val set) is used in the methods. The amount of supervision used in each method increases from top to bottom.

In this paper, we address these shortcomings by proposing a two-stream DCNN architecture (See Figure {{figure:0bfcaa2f-5691-4f1e-bc3d-83e710ac0d05}} ) that decomposes the input into texture and shape feature streams. The texture stream learns realistic texture cues from images downloaded from the Internet. Web images (which contain noise from backgrounds and unrelated results) are collected by searching for the names of categories in image search engines, i.e. Google Image Search. We prune the noise data and use the cleaned data to train the texture-based stream for classification or detection. The shape stream is trained exclusively on shape information from 2D images rendered from CAD models annotated with category labels.
Note that images from web search also contain shape information, but due to the lack of bounding box annotations, it is not accurate enough for localization models. Synthetic images can also be generated from 3D CAD models with added texture mapping, but the result is non-photorealistic and lacks real-world variety. Therefore, synthetic images rendered by 3D CAD models can be viewed as primarily shape-oriented and the web-search images can be viewed as primarily texture-oriented. The outputs of the two streams are combined through averaging the two top layers' activations. Our method requires no tedious manual bounding box annotation of object instances and no per-image category labeling and can generate training data for almost any novel category. Table {{table:3ed855a4-7ae6-4b09-a77f-a41305ba7d31}}  shows a comparison of the amount of supervision with other methods. The only supervision in our work comes from labeling the CAD models as positive examples vs. “outliers” while downloading them, and choosing proper textures for each category while generating the synthetic images.


GRAF: Generative Radiance Fields for 3D-Aware Image Synthesis

Abstract: While 2D generative adversarial networks have enabled high-resolution image synthesis, they largely lack an understanding of the 3D world and the image formation process. Thus, they do not provide precise control over camera viewpoint or object pose. To address this problem, several recent approaches leverage intermediate voxel-based representations in combination with differentiable rendering. However, existing methods either produce low image resolution or fall short in disentangling camera and scene properties, e.g., the object identity may vary with the viewpoint. In this paper, we propose a generative model for radiance fields which have recently proven successful for novel view synthesis of a single scene. In contrast to voxel-based representations, radiance fields are not confined to a coarse discretization of the 3D space, yet allow for disentangling camera and scene properties while degrading gracefully in the presence of reconstruction ambiguity. By introducing a multi-scale patch-based discriminator, we demonstrate synthesis of high-resolution images while training our model from unposed 2D images alone. We systematically analyze our approach on several challenging synthetic and real-world datasets. Our experiments reveal that radiance fields are a powerful representation for generative image synthesis, leading to 3D consistent models that render with high fidelity.

BlockGAN: Learning 3D Object-aware Scene Representations from Unlabelled Images

Abstract: We present BlockGAN, an image generative model that learns object-aware 3D scene representations directly from unlabelled 2D images. Current work on scene representation learning either ignores scene background or treats the whole scene as one object. Meanwhile, work that considers scene compositionality treats scene objects only as image patches or 2D layers with alpha maps. Inspired by the computer graphics pipeline, we design BlockGAN to learn to first generate 3D features of background and foreground objects, then combine them into 3D features for the wholes cene, and finally render them into realistic images. This allows BlockGAN to reason over occlusion and interaction between objects' appearance, such as shadow and lighting, and provides control over each object's 3D pose and identity, while maintaining image realism. BlockGAN is trained end-to-end, using only unlabelled single images, without the need for 3D geometry, pose labels, object masks, or multiple views of the same scene. Our experiments show that using explicit 3D features to represent objects allows BlockGAN to learn disentangled representations both in terms of objects (foreground and background) and their properties (pose and identity).

Visual Object Networks: Image Generation with Disentangled 3D Representations

Abstract: Recent progress in deep generative models has led to tremendous breakthroughs in image generation. While being able to synthesize photorealistic images, existing models lack an understanding of our underlying 3D world. Different from previous works built on 2D datasets and models, we present a new generative model, Visual Object Networks (VONs), synthesizing natural images of objects with a disentangled 3D representation. Inspired by classic graphics rendering pipelines, we unravel the image formation process into three conditionally independent factors---shape, viewpoint, and texture---and present an end-to-end adversarial learning framework that jointly models 3D shape and 2D texture. Our model first learns to synthesize 3D shapes that are indistinguishable from real shapes. It then renders the object's 2.5D sketches (i.e., silhouette and depth map) from its shape under a sampled viewpoint. Finally, it learns to add realistic textures to these 2.5D sketches to generate realistic images. The VON not only generates images that are more realistic than the state-of-the-art 2D image synthesis methods but also enables many 3D operations such as changing the viewpoint of a generated image, shape and texture editing, linear interpolation in texture and shape space, and transferring appearance across different objects and viewpoints.

DeepVoxels: Learning Persistent 3D Feature Embeddings

Abstract: In this work, we address the lack of 3D understanding of generative neural networks by introducing a persistent 3D feature embedding for view synthesis. To this end, we propose DeepVoxels, a learned representation that encodes the view-dependent appearance of a 3D scene without having to explicitly model its geometry. At its core, our approach is based on a Cartesian 3D grid of persistent embedded features that learn to make use of the underlying 3D scene structure. Our approach combines insights from 3D geometric computer vision with recent advances in learning image-to-image mappings based on adversarial loss functions. DeepVoxels is supervised, without requiring a 3D reconstruction of the scene, using a 2D re-rendering loss and enforces perspective and multi-view geometry in a principled manner. We apply our persistent 3D scene representation to the problem of novel view synthesis demonstrating high-quality results for a variety of challenging scenes.

SynSin: End-to-End View Synthesis From a Single Image

Abstract: View synthesis allows for the generation of new views of a scene given one or more images. This is challenging; it requires comprehensively understanding the 3D scene from images. As a result, current methods typically use multiple images, train on ground-truth depth, or are limited to synthetic data. We propose a novel end-to-end model for this task using a single image at test time; it is trained on real images without any ground-truth 3D information. To this end, we introduce a novel differentiable point cloud renderer that is used to transform a latent 3D point cloud of features into the target view. The projected features are decoded by our refinement network to inpaint missing regions and generate a realistic output image. The 3D component inside of our generative model allows for interpretable manipulation of the latent feature space at test time, e.g. we can animate trajectories from a single image. Additionally, we can generate high resolution images and generalise to other input resolutions. We outperform baselines and prior work on the Matterport, Replica, and RealEstate10K datasets.

Key differences amongst 3D-aware methods. Adversarial losses are used by a range of methods that differ in the type of 3D feature representation and training supervision.

As summarized in Table {{table:9ad470ce-10bf-49cc-8090-d0e1aa4a8680}} , the works discussed in this section use a variety of 3D feature representations, and train their networks using paired input-output with known transformations or unlabeled and unpaired data. The use of a GAN loss is common to all these approaches. This is perhaps because traditional hand-designed losses such as the {{formula:7f0efce9-809d-4780-86ae-876cf0040f1d}}  loss or even perceptual loss are unable fully to capture what makes a synthesized image look unrealistic. Further, in the case where explicit task supervision is unavailable, BlockGAN {{cite:0f0c6ce}} shows that a GAN loss can help in learning disentangled features by ensuring that the outputs after projection and rendering look realistic. The learnability and flexibility of the GAN loss to the task at hand helps provide feedback, guiding how to change the generated image, and thus the upstream features, so that it looks as if it were sampled from the distribution of real images. This makes the GAN framework a powerful asset in the toolbox of any neural rendering practitioner.


Domain Adaptation in Multi-Channel Autoencoder based Features for Robust Face Anti-Spoofing

Abstract: While the performance of face recognition systems has improved significantly in the last decade, they are proved to be highly vulnerable to presentation attacks (spoofing). Most of the research in the field of face presentation attack detection (PAD), was focused on boosting the performance of the systems within a single database. Face PAD datasets are usually captured with RGB cameras, and have very limited number of both bona-fide samples and presentation attack instruments. Training face PAD systems on such data leads to poor performance, even in the closed-set scenario, especially when sophisticated attacks are involved. We explore two paths to boost the performance of the face PAD system against challenging attacks. First, by using multi-channel (RGB, Depth and NIR) data, which is still easily accessible in a number of mass production devices. Second, we develop a novel Autoencoders + MLP based face PAD algorithm. Moreover, instead of collecting more data for training of the proposed deep architecture, the domain adaptation technique is proposed, transferring the knowledge of facial appearance from RGB to multi-channel domain. We also demonstrate, that learning the features of individual facial regions, is more discriminative than the features learned from an entire face. The proposed system is tested on a very recent publicly available multi-channel PAD database with a wide variety of presentation attacks.

Generalizing Fingerprint Spoof Detector: Learning a One-Class Classifier

Abstract: Prevailing fingerprint recognition systems are vulnerable to spoof attacks. To mitigate these attacks, automated spoof detectors are trained to distinguish a set of live or bona fide fingerprints from a set of known spoof fingerprints. Despite their success, spoof detectors remain vulnerable when exposed to attacks from spoofs made with materials not seen during training of the detector. To alleviate this shortcoming, we approach spoof detection as a one-class classification problem. The goal is to train a spoof detector on only the live fingerprints such that once the concept of "live" has been learned, spoofs of any material can be rejected. We accomplish this through training multiple generative adversarial networks (GANS) on live fingerprint images acquired with the open source, dual-camera, 1900 ppi RaspiReader fingerprint reader. Our experimental results, conducted on 5.5K spoof images (from 12 materials) and 11.8K live images show that the proposed approach improves the cross-material spoof detection performance over state-of-the-art one-class and binary class spoof detectors on 11 of 12 testing materials and 7 of 12 testing materials, respectively.

One-class PAD methods across different modalities.

Unlike traditional classification problems, the motivation behind one-class classifiers is learning the structure of data samples belonging to a single class. Therefore, in case of PAD, one-class classifiers are trained only on bona fide samples. New and unseen samples are classified as PAs if their structure differs from those bona fide samples used in the training phase. In this context, the main challenge is to find an optimal threshold to ensure that sophisticated PAs can still be distinguished from those bona fides that deviate from normality. Due to the environmental conditions and interaction factors (data subject with respect to the capture device) a significant intra-class variation for the bona fide class must be expected. Since the majority of published PAD approaches are based on two-class classification, this section reviews one-class publications across modalities as summarised in Table {{table:306a97d4-bdee-49e3-ac2d-64df528c609b}} . Due to the different modalities and datasets used, a comparison of performance metrics is not included.


Very Deep Convolutional Networks for Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications

Abstract: We present a class of efficient models called MobileNets for mobile and embedded vision applications. MobileNets are based on a streamlined architecture that uses depth-wise separable convolutions to build light weight deep neural networks. We introduce two simple global hyper-parameters that efficiently trade off between latency and accuracy. These hyper-parameters allow the model builder to choose the right sized model for their application based on the constraints of the problem. We present extensive experiments on resource and accuracy tradeoffs and show strong performance compared to other popular models on ImageNet classification. We then demonstrate the effectiveness of MobileNets across a wide range of applications and use cases including object detection, finegrain classification, face attributes and large scale geo-localization.

Densely Connected Convolutional Networks

Abstract: Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections—one between each layer and its subsequent layer—our network has L(L+1)/2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less memory and computation to achieve high performance. Code and pre-trained models are available at https://github.com/liuzhuang13/DenseNet.

Pre-trained DNN models available from Keras that is used in this paper; Type: S - sequential, N - non-sequential

Eight DNNs are considered in this research and are shown in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}} . The size of a trained model and its corresponding weights, the number of layers in the DNN, the number of valid points for partitioning, and the type of the DNN is shown. These models are obtained from the Kerashttps://keras.io neural network library.

There are two types of DNNs - sequential and non-sequential, represented as S and N, respectively in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}} .
In a sequential DNN the input of one layer is connected to the next in a linear manner (Figure {{figure:27cb2e21-8b2d-49fe-80c0-3ea133051471}}  shows an example with six layers).
A non-sequential DNN on the other hand will have layers that may be connected to two or more layers (refer Figure {{figure:27cb2e21-8b2d-49fe-80c0-3ea133051471}}  for an example with 11 layers). Hence, there are multiple paths that connect the first and last layer of the DNN.

Distributing a sequential DNN across the cloud and the edge is straightforward. The DNN would need to be partitioned at a suitable layer that would yield maximum performance (for example, lowest end-to-end latency and/or least amount of data transferred from the edge to the cloud).
For example, consider VGG19 shown in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}} , which has 26 layers and has 25 suitable partitioning points.
Consider a DNN that has {{formula:f2accf71-272c-4be1-8b4e-992dcf079d81}}  layers. If the {{formula:c7d50b11-ddbc-4246-abaa-ab21445697f6}}  layer is identified as the partitioning point, then the partition running on the edge would consist of a sequence of the first {{formula:9cd11853-0bc1-459d-b358-5355ca975ac0}}  layers and the partition to be executed on the cloud would consist of {{formula:d6fe8907-3964-4f61-9fa5-fabad67da37d}}  layers. The output of the {{formula:f2e640c6-ce7d-4c35-89be-527ef67b603f}}  layer would need to be transferred from the edge to the cloud and provided as an input layer for the cloud partition.

Distributing a non-sequential DNN requires additional pre-processing. This is to ensure that a parallel path in a DNN is not partitioned as it may lead to synchronization issues that will incur communication overheads {{cite:27d67c6}}. Partitioning is avoided on the parallel paths by grouping parallel layers as a single entity, referred to as a block of layers (refer to Figure {{figure:27cb2e21-8b2d-49fe-80c0-3ea133051471}}  and Figure {{figure:27cb2e21-8b2d-49fe-80c0-3ea133051471}}  for an example - Layers 2-9 are treated as a single entity). Therefore, the number of partitioning points is reduced. For example, in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}}  ResNet50V2 has 192 layers, but with only 16 suitable partitioning points.

The methodology adopted for obtaining data from the DNNs shown in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}}  has the following four steps:

Step 1 - Pre-process the DNN and identify suitable partitioning points:
As discussed in the previous section, a DNN may be a sequential or non-sequential DNN. All layers of a sequential DNN can be partitioned. However, a non-sequential DNN may have parallel paths and therefore there will be fewer suitable partitioning points. In this step, the DNN is pre-processed to identify the valid partitioning points of the DNN. As shown in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}}  non-sequential DNNs have fewer partitioning points than the number of layers; a block of layers will need to be treated as a single entity to avoid partitioning any parallel paths in the DNN. This is to avoid synchronization issues {{cite:27d67c6}}.

The above benchmarking methodology is implemented in Python and requires Tensorflow 1.5+. Tensorflow {{cite:e1ac559}} is used to execute the pre-trained DNNs shown in Table {{table:62be1c0d-34b0-4723-a1e3-b798bd3dbaea}}  that is provided by the Kerashttps://keras.io, an open source neural network library. NumPy is used for processing multi-dimensional arrays that are produced as outputs of individual layers or a block of layers.