Bike Flow Prediction with Multi-Graph Convolutional Networks

Abstract: One fundamental issue in managing bike sharing systems is the bike flow prediction. Due to the hardness of predicting the flow for a single station, recent research works often predict the bike flow at cluster-level. While such studies gain satisfactory prediction accuracy, they cannot directly guide some fine-grained bike sharing system management issues at station-level. In this paper, we revisit the problem of the station-level bike flow prediction, aiming to boost the prediction accuracy leveraging the breakthroughs of deep learning techniques. We propose a new multi-graph convolutional neural network model to predict the bike flow at station-level, where the key novelty is viewing the bike sharing system from the graph perspective. More specifically, we construct multiple inter-station graphs for a bike sharing system. In each graph, nodes are stations, and edges are a certain type of relations between stations. Then, multiple graphs are constructed to reflect heterogeneous relationships (e.g., distance, ride record correlation). Afterward, we fuse the multiple graphs and then apply the convolutional layers on the fused graph to predict station-level future bike flow. In addition to the estimated bike flow value, our model also gives the prediction confidence interval so as to help the bike sharing system managers make decisions. Using New York City and Chicago bike sharing data for experiments, our model can outperform state-of-the-art station-level prediction models by reducing 25.1% and 17.0% of prediction error in New York City and Chicago, respectively.

Short-Term Forecasting of Passenger Demand under On-Demand Ride Services: A Spatio-Temporal Deep Learning Approach

Abstract: Short-term passenger demand forecasting is of great importance to the on-demand ride service platform, which can incentivize vacant cars moving from over-supply regions to over-demand regions. The spatial dependences, temporal dependences, and exogenous dependences need to be considered simultaneously, however, which makes short-term passenger demand forecasting challenging. We propose a novel deep learning (DL) approach, named the fusion convolutional long short-term memory network (FCL-Net), to address these three dependences within one end-to-end learning architecture. The model is stacked and fused by multiple convolutional long short-term memory (LSTM) layers, standard LSTM layers, and convolutional layers. The fusion of convolutional techniques and the LSTM network enables the proposed DL approach to better capture the spatio-temporal characteristics and correlations of explanatory variables. A tailored spatially aggregated random forest is employed to rank the importance of the explanatory variables. The ranking is then used for feature selection. The proposed DL approach is applied to the short-term forecasting of passenger demand under an on-demand ride service platform in Hangzhou, China. Experimental results, validated on real-world data provided by DiDi Chuxing, show that the FCL-Net achieves better predictive performance than traditional approaches including both classical time-series prediction models and neural network based algorithms (e.g., artificial neural network and LSTM). This paper is one of the first DL studies to forecast the short-term passenger demand of an on-demand ride service platform by examining the spatio-temporal correlations.

Deep and Confident Prediction for Time Series at Uber

Abstract: Reliable uncertainty estimation for time series prediction is critical in many fields, including physics, biology, and manufacturing. At Uber, probabilistic time series forecasting is used for robust prediction of number of trips during special events, driver incentive allocation, as well as real-time anomaly detection across millions of metrics. Classical time series models are often used in conjunction with a probabilistic formulation for uncertainty estimation. However, such models are hard to tune, scale, and add exogenous variables to. Motivated by the recent resurgence of Long Short Term Memory networks, we propose a novel end-to-end Bayesian deep model that provides time series prediction along with uncertainty estimation. We provide detailed experiments of the proposed solution on completed trips data, and successfully apply it to large-scale time series anomaly detection at Uber.

Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction

Abstract: Taxi demand prediction is an important building block to enabling intelligent transportation systems in a smart city. An accurate prediction model can help the city pre-allocate resources to meet travel demand and to reduce empty taxis on streets which waste energy and worsen the traffic congestion. With the increasing popularity of taxi requesting services such as Uber and Didi Chuxing (in China), we are able to collect large-scale taxi demand data continuously. How to utilize such big data to improve the demand prediction is an interesting and critical real-world problem. Traditional demand prediction methods mostly rely on time series forecasting techniques, which fail to model the complex non-linear spatial and temporal relations. Recent advances in deep learning have shown superior performance on traditionally challenging tasks such as image classification by learning the complex features and correlations from large-scale data. This breakthrough has inspired researchers to explore deep learning techniques on traffic prediction problems. However, existing methods on traffic prediction have only considered spatial relation (e.g., using CNN) or temporal relation (e.g., using LSTM) independently. We propose a Deep Multi-View Spatial-Temporal Network (DMVST-Net) framework to model both spatial and temporal relations. Specifically, our proposed model consists of three views: temporal view (modeling correlations between future demand values with near time points via LSTM), spatial view (modeling local spatial correlation via local CNN), and semantic view (modeling correlations among regions sharing similar temporal patterns). Experiments on large-scale real taxi demand data demonstrate effectiveness of our approach over state-of-the-art methods.

Deep Spatio-Temporal Residual Networks for Citywide Crowd Flows Prediction

Abstract: Forecasting the flow of crowds is of great importance to traffic management and public safety, yet a very challenging task affected by many complex factors, such as inter-region traffic, events and weather. In this paper, we propose a deep-learning-based approach, called ST-ResNet, to collectively forecast the in-flow and out-flow of crowds in each and every region through a city. We design an end-to-end structure of ST-ResNet based on unique properties of spatio-temporal data. More specifically, we employ the framework of the residual neural networks to model the temporal closeness, period, and trend properties of the crowd traffic, respectively. For each property, we design a branch of residual convolutional units, each of which models the spatial properties of the crowd traffic. ST-ResNet learns to dynamically aggregate the output of the three residual neural networks based on data, assigning different weights to different branches and regions. The aggregation is further combined with external factors, such as weather and day of the week, to predict the final traffic of crowds in each and every region. We evaluate ST-ResNet based on two types of crowd flows in Beijing and NYC, finding that its performance exceeds six well-know methods.

Context features and its modeling techniques in STCFP Studies. (T: Temperature; H: Humidity; V: Visibility; WS: Wind Speed; WD: Wind Degree; AQ: Air Quality; S: Weather State)

In general, weather context such as temperature, humidity, wind speed and weather conditions (and etc.) refers to short-term changes in the atmospherehttps://www.ncei.noaa.gov/news/weather-vs-climate and can indeed influence crowd flow {{cite:6b2cb95}}, {{cite:7eef09f}}, {{cite:038a1f6}}. For instance, rising temperatures will increase the use of bikesharing {{cite:6b2cb95}}. Heavy rains and strong winds will reduce the use of bikesharing and online ride-hailing {{cite:7eef09f}}. The crowd flow in different STCFP application scenarios may be affected by different kinds of weather context. To show the difference, we give weather context usage statistics in Table {{table:f2228b5c-3405-4c9d-85c2-e94781be3ba4}} . It is not surprising that temperature and weather state are the most important considerations because when a person is out, he will observe the weather condition and temperature to decide whether he goes out or not.


3D model generation based on 2D plan and section using Style Transfer

GAN allows for navigation in the latent space to create digital designs using vector arithmetic and interpolation techniques, then converting resulting images to 3D voxel structures

3D Model Generation on Architectural Plan and Section Training through Machine Learning

Abstract: Machine learning, especially the GAN (Generative Adversarial Network) model, has been developed tremendously in recent years. Since the NVIDIA Machine Learning group presented the StyleGAN in December 2018, it has become a new way for designers to make machines learn different or similar types of architectural photos, drawings, and renderings, then generate (a) similar fake images, (b) style-mixing images, and (c) truncation trick images. The author both collected and created input image data, and specially made architectural plan and section drawing inputs with a clear design purpose, then applied StyleGAN to train specific networks on these datasets. With the training process, we could look into the deep relationship between these input architectural plans or sections, then generate serialized transformation images (truncation trick images) to form the 3D (three-dimensional) model with a decent resolution (up to 1024 × 1024 × 1024 pixels). Though the results of the 3D model generation are difficult to use directly in 3D spatial modeling, these unexpected 3D forms still could inspire new design methods and greater possibilities of architectural plan and section design.

Machinic Interpolations: A GAN Pipeline for Integrating Lateral Thinking in Computational Tools of Architecture

Abstract: In this paper, we discuss a new tool pipeline that aims to re-integrate lateral thinking strategies in computational tools of architecture. We present a 4-step AI-driven pipeline, based on Generative Adversarial Networks (GANs), that draws from the ability to access the latent space of a machine and use this space as a digital design environment. We demonstrate examples of navigating in this space using vector arithmetic and interpolations as a method to generate a series of images that are then translated to 3D voxel structures. Through a gallery of forms, we show how this series of techniques could result in unexpected spaces and outputs beyond what could be produced by human capability alone.

The related works in the architecture fields. The applications in the architectural field are sorted into different categories in this table, including 2D to 3D transposition, 3D solid generation and NLP based 3D form generation. The category means the generation methodology: `2D to 3D' means the 3D form generation is based on 2D images; `3D Solid' means generating 3D form directly with DGMs including GAN, VAE, and diffusion model. `NLP' means the 3D form generation process includes text input to assist the output control. `3D representations' in architecture encompass explicit point cloud, voxel grid, and mesh. The Objective column explains the directions and objectives the research aims to address. The methodology column gives an overview to what kind of workflow the generation process proposed. This table compares the research in the architectural field.

Generated 3D Architecture: A PARADIGM SHIFT

Before the 3D algorithms that can automatically store and process the 3D data, the 3D generation methods were mostly developed based on 2D images.
The remarkable growth of the 3D generation in recent years has revealed the tremendous power of this field. Compared to 2D image generation, 3D generation is a daunting task regarding the aspects of the 3D dataset, computational consumption, feature learning, and probability distribution in 3D space.
We investigate the virtual architecture generation based on various DGMs both in methods of CV (Table  {{table:3c50de9e-8eda-429a-a7bb-1bac6e42f3c8}} , Table  {{table:629c4353-56a8-41c4-afaa-abfd7880db3c}} ) and architecture (Table  {{table:94d9bae0-27b3-4bd0-8bb8-9c304075242d}} ).
In the first part, we introduce some research on architectural designs concentrating on the 2D deep generative models aiming for 3D transposition, especially GANs.
Then, we divide deep learning generation approaches for 3D representations into four categories based on DGMs: (See Fig.  {{figure:df36acf2-d0cc-4883-a8b8-d8edddd93238}} ):


FaceNet: A unified embedding for face recognition and clustering

Abstract: Despite significant recent advances in the field of face recognition [10, 14, 15, 17], implementing face verification and recognition efficiently at scale presents serious challenges to current approaches. In this paper we present a system, called FaceNet, that directly learns a mapping from face images to a compact Euclidean space where distances directly correspond to a measure offace similarity. Once this space has been produced, tasks such as face recognition, verification and clustering can be easily implemented using standard techniques with FaceNet embeddings asfeature vectors. Our method uses a deep convolutional network trained to directly optimize the embedding itself, rather than an intermediate bottleneck layer as in previous deep learning approaches. To train, we use triplets of roughly aligned matching / non-matching face patches generated using a novel online triplet mining method. The benefit of our approach is much greater representational efficiency: we achieve state-of-the-artface recognition performance using only 128-bytes perface. On the widely used Labeled Faces in the Wild (LFW) dataset, our system achieves a new record accuracy of 99.63%. On YouTube Faces DB it achieves 95.12%. Our system cuts the error rate in comparison to the best published result [15] by 30% on both datasets.

Targeting Ultimate Accuracy: Face Recognition via Deep Embedding

Abstract: Face Recognition has been studied for many decades. As opposed to traditional hand-crafted features such as LBP and HOG, much more sophisticated features can be learned automatically by deep learning methods in a data-driven way. In this paper, we propose a two-stage approach that combines a multi-patch deep CNN and deep metric learning, which extracts low dimensional but very discriminative features for face verification and recognition. Experiments show that this method outperforms other state-of-the-art methods on LFW dataset, achieving 99.77% pair-wise verification accuracy and significantly better accuracy under other two more practical protocols. This paper also discusses the importance of data size and the number of patches, showing a clear path to practical high-performance face recognition systems in real world.

Robust Face Recognition via Multimodal Deep Face Representation

Abstract: Face images appearing in multimedia applications, e.g., social networks and digital entertainment, usually exhibit dramatic pose, illumination, and expression variations, resulting in considerable performance degradation for traditional face recognition algorithms. This paper proposes a comprehensive deep learning framework to jointly learn face representation using multimodal information. The proposed deep learning structure is composed of a set of elaborately designed convolutional neural networks (CNNs) and a three-layer stacked auto-encoder (SAE). The set of CNNs extracts complementary facial features from multimodal data. Then, the extracted features are concatenated to form a high-dimensional feature vector, whose dimension is compressed by SAE. All of the CNNs are trained using a subset of 9,000 subjects from the publicly available CASIA-WebFace database, which ensures the reproducibility of this work. Using the proposed single CNN architecture and limited training data, 98.43% verification rate is achieved on the LFW database. Benefitting from the complementary information contained in multimodal data, our small ensemble system achieves higher than 99.0% recognition rate on LFW using publicly available training set.

Large-Margin Softmax Loss for Convolutional Neural Networks

Abstract: Cross-entropy loss together with softmax is arguably one of the most common used supervision components in convolutional neural networks (CNNs). Despite its simplicity, popularity and excellent performance, the component does not explicitly encourage discriminative learning of features. In this paper, we propose a generalized large-margin softmax (L-Softmax) loss which explicitly encourages intra-class compactness and inter-class separability between learned features. Moreover, L-Softmax not only can adjust the desired margin but also can avoid overfitting. We also show that the L-Softmax loss can be optimized by typical stochastic gradient descent. Extensive experiments on four benchmark datasets demonstrate that the deeply-learned features with L-softmax loss become more discriminative, hence significantly boosting the performance on a variety of visual classification and verification tasks.

Neural Aggregation Network for Video Face Recognition

Abstract: This paper presents a Neural Aggregation Network (NAN) for video face recognition. The network takes a face video or face image set of a person with a variable number of face images as its input, and produces a compact, fixed-dimension feature representation for recognition. The whole network is composed of two modules. The feature embedding module is a deep Convolutional Neural Network (CNN) which maps each face image to a feature vector. The aggregation module consists of two attention blocks which adaptively aggregate the feature vectors to form a single feature inside the convex hull spanned by them. Due to the attention mechanism, the aggregation is invariant to the image order. Our NAN is trained with a standard classification or verification loss without any extra supervision signal, and we found that it automatically learns to advocate high-quality face images while repelling low-quality ones such as blurred, occluded and improperly exposed faces. The experiments on IJB-A, YouTube Face, Celebrity-1000 video face recognition benchmarks show that it consistently outperforms naive aggregation methods and achieves the state-of-the-art accuracy.

A summary of efficiency and accuracy comparisons between recent remarkable works on face recognition

Although different network structures are employed in different analysis tasks, we find that the features can be uniformly represented without significantly sacrificing the analysis accuracy.
Table {{table:c85df833-10c8-4364-bc80-e25762ee2d5a}}  lists the video retrieval performance of the deep learning features with off-the-shelf CNN model reported in CDVA benchmarks. From the perspective of the performance and feature compactness, the deep learning feature shows the competitive performance. With the advance of network structure and training scheme, the performance of deep features have also been dramatically improved, being state-of-the-art on several image retrieval benchmarks, such as Holiday, Oxford5K and Paris. The person/vehicle ReID tasks also benefit a lot from the success of deep networks, as shown in Tables {{table:2968f3c4-8b9b-4c24-bb0c-30cd1933eb47}} &{{table:195176c6-8c64-42eb-87eb-f75b2e1b6c91}} . In particular, we observe there is a trend that the recent methods employ features with much lower dimensions to produce the representation of an object, with the help of more powerful networks and well-defined loss functions. Similar trends can also be found in the recent efforts for face recognition, as shown in Table {{table:d7192be2-e7e7-4b08-aca0-53236f125cbe}} . In addition, the performance improvements originate from the introducing of object proposal in image retrieval has been witnessed in several benchmarks such as Oxford and Paris. The performance gains originate from the better localization ability provided by region proposal as shown in Fig. {{figure:0111dee7-d37a-4374-a21b-1e82171e8997}}  (c)&(d). However, the obvious demerit of region-based methods is the relatively high feature dimension due to the representation requirements of multiple targets in query or reference images. As such, the region proposal mechanism would be practically applicable with the representations of compact feature. In Fig. {{figure:0111dee7-d37a-4374-a21b-1e82171e8997}}  (a)&(b), the performance variations against the dimension are illustrated. All these experimental results demonstrate that these tasks can be successfully achieved with identical or similar feature dimension. For example, when the feature size reaches 512 dimensions, in most of the face recognition cases, competitive results can be obtained. Obviously, similar phenomenon can also be found when the dimension reaches 512 in CDVA, 512 in Person ReID, and 1024 in vehicle ReID. In a word, a converging point can be feasibly attained from the perspectives of feature dimensions, proposal numbers, network structure.
Another observation is that the performance variation along with the augment of proposals can also arrive at a saturation eventually as shown in Fig. {{figure:0111dee7-d37a-4374-a21b-1e82171e8997}}  (c)&(d). Such observations provide useful evidence for the further standardization of deep learning features, as discussed in Section 4.2.
In the future, it is also expected that more compact and discriminative feature representations will emerge due to the advance of the network architectures and optimization strategies.


UT Dallas / Chan Zuckerburg Initiative / UCLA

Scientific Claim Verification with VERT5ERINI

Abstract: This work describes the adaptation of a pretrained sequence-to-sequence model to the task of scientific claim verification in the biomedical domain. We propose VERT5ERINI that exploits T5 for abstract retrieval, sentence selection and label prediction, which are three critical sub-tasks of claim verification. We evaluate our pipeline on SCIFACT, a newly curated dataset that requires models to not just predict the veracity of claims but also provide relevant sentences from a corpus of scientific literature that support this decision. Empirically, our pipeline outperforms a strong baseline in each of the three steps. Finally, we show VERT5ERINI's ability to generalize to two new datasets of COVID-19 claims using evidence from the ever-expanding CORD-19 corpus.

A Paragraph-level Multi-task Learning Model for Scientific Fact-Verification

Abstract: Even for domain experts, it is a non-trivial task to verify a scientific claim by providing supporting or refuting evidence rationales. The situation worsens as misinformation is proliferated on social media or news websites, manually or programmatically, at every moment. As a result, an automatic fact-verification tool becomes crucial for combating the spread of misinformation. In this work, we propose a novel, paragraph-level, multi-task learning model for the SciFact task by directly computing a sequence of contextualized sentence embeddings from a BERT model and jointly training the model on rationale selection and stance prediction.

QMUL-SDS at SCIVER: Step-by-Step Binary Classification for Scientific Claim Verification

Abstract: Scientific claim verification is a unique challenge that is attracting increasing interest. The SCIVER shared task offers a benchmark scenario to test and compare claim verification approaches by participating teams and consists in three steps: relevant abstract selection, rationale selection and label prediction. In this paper, we present team QMUL-SDS's participation in the shared task. We propose an approach that performs scientific claim verification by doing binary classifications step-by-step. We trained a BioBERT-large classifier to select abstracts based on pairwise relevance assessments for each <claim, title of the abstract> and continued to train it to select rationales out of each retrieved abstract based on <claim, sentence>. We then propose a two-step setting for label prediction, i.e. first predicting "NOT_ENOUGH_INFO" or "ENOUGH_INFO", then label those marked as "ENOUGH_INFO" as either "SUPPORT" or "CONTRADICT". Compared to the baseline system, we achieve substantial improvements on the dev set. As a result, our team is the No. 4 team on the leaderboard.

Fact or Fiction: Verifying Scientific Claims

Abstract: We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that supports or refutes a given scientific claim, and to identify rationales justifying each decision. To study this task, we construct SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. We develop baseline models for SciFact, and demonstrate that these models benefit from combined training on a large dataset of claims about Wikipedia articles, together with the new SciFact data. We show that our claim verification system is able to identify plausible evidence for 23 / 36 claims relevant to COVID-19 on the CORD-19 corpus. Our results and experiments strongly suggest that our new task and data will support significant future research efforts.

CelebHair: A New Large-Scale Dataset for Hairstyle Recommendation based on CelebA

Abstract: In this paper, we present a new large-scale dataset for hairstyle recommendation, CelebHair, based on the celebrity facial attributes dataset, CelebA. Our dataset inherited the majority of facial images along with some beauty-related facial attributes from CelebA. Additionally, we employed facial landmark detection techniques to extract extra features such as nose length and pupillary distance, and deep convolutional neural networks for face shape and hairstyle classification. Empirical comparison has demonstrated the superiority of our dataset to other existing hairstyle-related datasets regarding variety, veracity, and volume. Analysis and experiments have been conducted on the dataset in order to evaluate its robustness and usability.

Systems that submitted to the SCIVER leaderboard. Alongside system names are affiliations of the associated team (if applicable) and any papers associated with the submission (if applicable).

Table {{table:0c08592c-2499-453e-b250-12ee469069a8}}  presents the submitted systems. As the online leaderboard is still accepting new submissions, we only include in this report the systems that were present by the shared task submission deadline.

A number of participating teams reported performance on individual pipeline components in publications associated with their systems (see Table {{table:0c08592c-2499-453e-b250-12ee469069a8}} ). Based on their reports and discussions with shared task participants, we highlight findings on several modeling decisions that have had significant impact on results in the shared task.Metrics reported in this section are self-reported by participants and have not been verified by the task organizers. Furthermore, metrics reported in this section are not directly comparable to those in Table {{table:6bc623d7-c068-4429-9cfb-3d066f5e0acc}}  as the main results evaluate end-to-end system performance while the metrics reported here are ablations with respect to a pipeline component.


Overcoming Exploration in Reinforcement Learning with Demonstrations

Abstract: Exploration in environments with sparse rewards has been a persistent problem in reinforcement learning (RL). Many tasks are natural to specify with a sparse reward, and manually shaping a reward function can result in suboptimal performance. However, finding a non-zero reward is exponentially more difficult with increasing task horizon or action dimensionality. This puts many real-world tasks out of practical reach of RL methods. In this work, we use demonstrations to overcome the exploration problem and successfully learn to perform long-horizon, multi-step robotics tasks with continuous control such as stacking blocks with a robot arm. Our method, which builds on top of Deep Deterministic Policy Gradients and Hindsight Experience Replay, provides an order of magnitude of speedup over RL on simulated robotics tasks. It is simple to implement and makes only the additional assumption that we can collect a small set of demonstrations. Furthermore, our method is able to solve tasks not solvable by either RL or behavior cloning alone, and often ends up outperforming the demonstrator policy.

Policy Distillation

Abstract: Abstract: Policies for complex visual tasks have been successfully learned with deep reinforcement learning, using an approach called deep Q-networks (DQN), but relatively large (task-specific) networks and extensive training are needed to achieve good performance. In this work, we present a novel method called policy distillation that can be used to extract the policy of a reinforcement learning agent and train a new network that performs at the expert level while being dramatically smaller and more efficient. Furthermore, the same method can be used to consolidate multiple task-specific policies into a single policy. We demonstrate these claims using the Atari domain and show that the multi-task distilled agent outperforms the single-task teachers as well as a jointly-trained DQN agent.

Actor-Mimic: Deep Multitask and Transfer Reinforcement Learning

Abstract: The ability to act in multiple environments and transfer previous knowledge to new situations can be considered a critical aspect of any intelligent agent. Towards this goal, we define a novel method of multitask and transfer learning that enables an autonomous agent to learn how to behave in multiple tasks simultaneously, and then generalize its knowledge to new domains. This method, termed "Actor-Mimic", exploits the use of deep reinforcement learning and model compression techniques to train a single policy network that learns how to act in a set of distinct tasks by using the guidance of several expert teachers. We then show that the representations learnt by the deep policy network are capable of generalizing to new tasks with no prior expert guidance, speeding up learning in novel environments. Although our method can in general be applied to a wide range of problems, we use Atari games as a testing environment to demonstrate these methods.

Distral: Robust Multitask Reinforcement Learning

Abstract: Most deep reinforcement learning algorithms are data inefficient in complex and rich environments, limiting their applicability to many scenarios. One direction for improving data efficiency is multitask learning with shared neural network parameters, where efficiency may be improved through transfer across related tasks. In practice, however, this is not usually observed, because gradients from different tasks can interfere negatively, making learning unstable and sometimes even less data efficient. Another issue is the different reward schemes between tasks, which can easily lead to one task dominating the learning of a shared model. We propose a new approach for joint training of multiple tasks, which we refer to as Distral (Distill & transfer learning). Instead of sharing parameters between the different workers, we propose to share a "distilled" policy that captures common behaviour across tasks. Each worker is trained to solve its own task while constrained to stay close to the shared policy, while the shared policy is trained by distillation to be the centroid of all task policies. Both aspects of the learning process are derived by optimizing a joint objective function. We show that our approach supports efficient transfer on complex 3D environments, outperforming several related methods. Moreover, the proposed learning process is more robust and more stable---attributes that are critical in deep reinforcement learning.

Applications of TL approaches in Game Playing.

Besides online chessboard games, TL approaches have also performed well in playing video games.
Current video game platforms are MineCraft, Atari, and Starcraft.
Especially, {{cite:15ae20c}} designed new RL tasks under the MineCraft platform for a better comparison of different RL algorithms.
A survey of AI for real-time strategy (RTS) games on the Starcraft platform can be found in {{cite:08095f7}}, with a dataset available from {{cite:360bdbf}}.
{{cite:ad9615d}} provides a comprehensive survey on DL applications in video game playing, which also covers TL and RL strategies from certain perspectives.
A large portion of TL approaches reviewed in this survey have been applied to the Atari {{cite:ec6c7b3}} and other game above-mentioned platforms. Especially, OpenAI trained an Dota2 agent which surpasses human experts {{cite:64fbad9}}.
Game applications of TL approaches mentioned in this survey are summarized in Table {{table:8896c22a-0560-4a7d-8e06-91c1c300d2e9}} .


A Graph Based Framework for Malicious Insider Threat Detection

Abstract: While most security projects have focused on fending off attacks coming from outside the organizational boundaries, a real threat has arisen from the people who are inside those perimeter protections. Insider threats have shown their power by hugely affecting national security, financial stability, and the privacy of many thousands of people. What is in the news is the tip of the iceberg, with much more going on under the radar, and some threats never being detected. We propose a hybrid framework based on graphical analysis and anomaly detection approaches, to combat this severe cybersecurity threat. Our framework analyzes heterogeneous data in isolating possible malicious users hiding behind others. Empirical results reveal this framework to be effective in distinguishing the majority of users who demonstrate typical behavior from the minority of users who show suspicious behavior.

Cyber Anomaly Detection Using Graph-node Role-dynamics

Abstract: Intrusion detection systems (IDSs) generate valuable knowledge about network security, but an abundance of false alarms and a lack of methods to capture the interdependence among alerts hampers their utility for network defense. Here, we explore a graph-based approach for fusing alerts generated by multiple IDSs (e.g., Snort, OSSEC, and Bro). Our approach generates a weighted graph of alert fields (not network topology) that makes explicit the connections between multiple alerts, IDS systems, and other cyber artifacts. We use this multi-modal graph to identify anomalous changes in the alert patterns of a network. To detect the anomalies, we apply the role-dynamics approach, which has successfully identified anomalies in social media, email, and IP communication graphs. In the cyber domain, each node (alert field) in the fused IDS alert graph is assigned a probability distribution across a small set of roles based on that node's features. A cyber attack should trigger IDS alerts and cause changes in the node features, but rather than track every feature for every alert-field node individually, roles provide a succinct, integrated summary of those feature changes. We measure changes in each node's probabilistic role assignment over time, and identify anomalies as deviations from expected roles. We test our approach using simulations including three weeks of normal background traffic, as well as cyber attacks that occur near the end of the simulations. This paper presents a novel approach to multi-modal data fusion and a novel application of role dynamics within the cyber-security domain. Our results show a drastic decrease in the false-positive rate when considering our anomaly indicator instead of the IDS alerts themselves, thereby reducing alarm fatigue and providing a promising avenue for threat intelligence in network defense.

Typical graph approach used in network security.

Network security has been widely addressed by network traffic analysis {{cite:a8f74dd}}, {{cite:e01f9fa}}. Generally, a traffic packet can be represented by a 5-tuple {source IP, target IP, source port, target port, protocol}, and a traffic flow consists of multiple packets with the same 5-tuple. The well-known Deep Packet Inspection (DPI) technology analyzes traffic by inspecting the payload of packets. However, payload-based methods can't be used in encrypted traffic. To overcome this limitation, header features (e.g., IP, packet length, protocol) and statistic features (e.g, average duration, maximum inter-arrival time) are utilized to extract malicious features. However, both of the above methods ignore the graph structural features of traffic. Naturally, the network traffic components (e.g., traffic flow, domain, IP, etc.) form an enterprise network, or the whole Internet can be abstracted as a Network Traffic Graph (NTG). Compared with traditional methods, graph mining methods based on NTG can avoid inspecting payloads and facilitate network security analysis {{cite:cf69865}}. In this section, we explicitly illustrate graph-based solutions for three main network security tasks, namely botnet detection, malicious domain detection, and intrusion detection. The characteristics of these solutions are summarized in Table {{table:5e8759f9-6a93-4a97-9f97-301efd0f69a1}} .


An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Abstract: While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.

Swin Transformer: Hierarchical Vision Transformer using Shifted Windows

Abstract: This paper presents a new vision Transformer, called Swin Transformer, that capably serves as a general-purpose backbone for computer vision. Challenges in adapting Transformer from language to vision arise from differences between the two domains, such as large variations in the scale of visual entities and the high resolution of pixels in images compared to words in text. To address these differences, we propose a hierarchical Transformer whose representation is computed with Shifted windows. The shifted windowing scheme brings greater efficiency by limiting self-attention computation to non-overlapping local windows while also allowing for cross-window connection. This hierarchical architecture has the flexibility to model at various scales and has linear computational complexity with respect to image size. These qualities of Swin Transformer make it compatible with a broad range of vision tasks, including image classification (87.3 top-1 accuracy on ImageNet-1K) and dense prediction tasks such as object detection (58.7 box AP and 51.1 mask AP on COCO test-dev) and semantic segmentation (53.5 mIoU on ADE20K val). Its performance surpasses the previous state-of-the-art by a large margin of +2.7 box AP and +2.6 mask AP on COCO, and +3.2 mIoU on ADE20K, demonstrating the potential of Transformer-based models as vision backbones. The hierarchical design and the shifted window approach also prove beneficial for all-MLP architectures. The code and models are publicly available at https://github.com/microsoft/Swin-Transformer.

Pretrained backbones.

See Table {{table:99bc302b-2aa0-4325-88d6-8009c0c62586}} .


Thinking Fast and Slow with Deep Learning and Tree Search

Abstract: Sequential decision making problems, such as structured prediction, robotic control, and game playing, require a combination of planning policies and generalisation of those plans. In this paper, we present Expert Iteration (ExIt), a novel reinforcement learning algorithm which decomposes the problem into separate planning and generalisation tasks. Planning new policies is performed by tree search, while a deep neural network generalises those plans. Subsequently, tree search is improved by using the neural network policy to guide search, increasing the strength of new plans. In contrast, standard deep Reinforcement Learning algorithms rely on a neural network not only to generalise plans, but to discover them too. We show that ExIt outperforms REINFORCE for training a neural network to play the board game Hex, and our final tree search agent, trained tabula rasa, defeats MoHex 1.0, the most recent Olympiad Champion player to be publicly released.

Solving Hard AI Planning Instances Using Curriculum-Driven Deep Reinforcement Learning

Abstract: Despite significant progress in general AI planning, certain domains remain out of reach of current AI planning systems. Sokoban is a PSPACE-complete planning task and represents one of the hardest domains for current AI planners. Even domain-specific specialized search methods fail quickly due to the exponential search complexity on hard instances. Our approach based on deep reinforcement learning augmented with a curriculum-driven method is the first one to solve hard instances within one day of training while other modern solvers cannot solve these instances within any reasonable time limit. In contrast to prior efforts, which use carefully handcrafted pruning techniques, our approach automatically uncovers domain structure. Our results reveal that deep RL provides a promising framework for solving previously unsolved AI planning problems, provided a proper training curriculum can be devised.

Overview of Explicit Planning/Given Transitionds Methods

The first approach in model-based learning is when the transition
and reward model is
provided clearly in the rules of the problem. This is the case, for example,
in games such as Go and chess. Table {{table:c0e818d8-0565-4970-808a-b91c97b29a4b}}  summarizes the
approaches of this subsection. Note the addition of the reinforcement
learning method in an extra column.


Training Verifiers to Solve Math Word Problems

Abstract: State-of-the-art language models can match human performance on many tasks, but they still struggle to robustly perform multi-step mathematical reasoning. To diagnose the failures of current models and support research, we introduce GSM8K, a dataset of 8.5K high quality linguistically diverse grade school math word problems. We find that even the largest transformer models fail to achieve high test performance, despite the conceptual simplicity of this problem distribution. To increase performance, we propose training verifiers to judge the correctness of model completions. At test time, we generate many candidate solutions and select the one ranked highest by the verifier. We demonstrate that verification significantly improves performance on GSM8K, and we provide strong empirical evidence that verification scales more effectively with increased data than a finetuning baseline.

Are NLP Models really able to Solve Simple Math Word Problems?

Abstract: The problem of designing NLP solvers for math word problems (MWP) has seen sustained research activity and steady gains in the test accuracy. Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered “solved” with the bulk of research attention moving to more complex MWPs. In this paper, we restrict our attention to English MWPs taught in grades four and lower. We provide strong evidence that the existing MWP solvers rely on shallow heuristics to achieve high performance on the benchmark datasets. To this end, we show that MWP solvers that do not have access to the question asked in the MWP can still solve a large fraction of MWPs. Similarly, models that treat MWPs as bag-of-words can also achieve surprisingly high accuracy. Further, we introduce a challenge dataset, SVAMP, created by applying carefully chosen variations over examples sampled from existing datasets. The best accuracy achieved by state-of-the-art models is substantially lower on SVAMP, thus showing that much remains to be done even for the simplest of the MWPs.

Program Induction by Rationale Generation: Learning to Solve and Explain Algebraic Word Problems

Abstract: Solving algebraic word problems requires executing a series of arithmetic operations—a program—to obtain a final answer. However, since programs can be arbitrarily complicated, inducing them directly from question-answer pairs is a formidable challenge. To make this task more feasible, we solve these problems by generating answer rationales, sequences of natural language and human-readable mathematical expressions that derive the final answer through a series of small steps. Although rationales do not explicitly specify programs, they provide a scaffolding for their structure via intermediate milestones. To evaluate our approach, we have created a new 100,000-sample dataset of questions, answers and rationales. Experimental results show that indirect supervision of program learning via answer rationales is a promising strategy for inducing arithmetic programs.

Solving General Arithmetic Word Problems

Abstract: This paper presents a novel approach to automatically solving arithmetic word problems. This is the first algorithmic approach that can handle arithmetic problems with multiple steps and operations, without depending on additional annotations or predefined templates. We develop a theory for expression trees that can be used to represent and evaluate the target arithmetic expressions; we use it to uniquely decompose the target arithmetic problem to multiple classification problems; we then compose an expression tree, combining these with world knowledge through a constrained inference framework. Our classifiers gain from the use of quantity schemas that supports better extraction of features. Experimental results show that our method outperforms existing systems, achieving state of the art performance on benchmark datasets of arithmetic word problems.

Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge

Abstract: We present a new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering. Together, these constitute the AI2 Reasoning Challenge (ARC), which requires far more powerful knowledge and reasoning than previous challenges such as SQuAD or SNLI. The ARC question set is partitioned into a Challenge Set and an Easy Set, where the Challenge Set contains only questions answered incorrectly by both a retrieval-based algorithm and a word co-occurence algorithm. The dataset contains only natural, grade-school science questions (authored for human tests), and is the largest public-domain set of this kind (7,787 questions). We test several baselines on the Challenge Set, including leading neural models from the SQuAD and SNLI tasks, and find that none are able to significantly outperform a random baseline, reflecting the difficult nature of this task. We are also releasing the ARC Corpus, a corpus of 14M science sentences relevant to the task, and implementations of the three neural baseline models tested. Can your model perform better? We pose ARC as a challenge to the community.

Did Aristotle Use a Laptop? A Question Answering Benchmark with Implicit Reasoning Strategies

Abstract: Abstract A key limitation in current datasets for multi-hop reasoning is that the required steps for answering the question are mentioned in it explicitly. In this work, we introduce StrategyQA, a question answering (QA) benchmark where the required reasoning steps are implicit in the question, and should be inferred using a strategy. A fundamental challenge in this setup is how to elicit such creative questions from crowdsourcing workers, while covering a broad range of potential strategies. We propose a data collection procedure that combines term-based priming to inspire annotators, careful control over the annotator population, and adversarial filtering for eliminating reasoning shortcuts. Moreover, we annotate each question with (1) a decomposition into reasoning steps for answering it, and (2) Wikipedia paragraphs that contain the answers to each step. Overall, StrategyQA includes 2,780 examples, each consisting of a strategy question, its decomposition, and evidence paragraphs. Analysis shows that questions in StrategyQA are short, topic-diverse, and cover a wide range of strategies. Empirically, we show that humans perform well (87%) on this task, while our best baseline reaches an accuracy of ∼ 66%.

Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them

Abstract: BIG-Bench (Srivastava et al., 2022) is a diverse evaluation suite that focuses on tasks believed to be beyond the capabilities of current language models. Language models have already made good progress on this benchmark, with the best model in the BIG-Bench paper outperforming average reported human-rater results on 65% of the BIG-Bench tasks via few-shot prompting. But on what tasks do language models fall short of average human-rater performance, and are those tasks actually unsolvable by current language models? In this work, we focus on a suite of 23 challenging BIG-Bench tasks which we call BIG-Bench Hard (BBH). These are the task for which prior language model evaluations did not outperform the average human-rater. We find that applying chain-of-thought (CoT) prompting to BBH tasks enables PaLM to surpass the average human-rater performance on 10 of the 23 tasks, and Codex (code-davinci-002) to surpass the average human-rater performance on 17 of the 23 tasks. Since many tasks in BBH require multi-step reasoning, few-shot prompting without CoT, as done in the BIG-Bench evaluations (Srivastava et al., 2022), substantially underestimates the best performance and capabilities of language models, which is better captured via CoT prompting. As further analysis, we explore the interaction between CoT and model scale on BBH, finding that CoT enables emergent task performance on several BBH tasks with otherwise flat scaling curves.

Details of ten evaluation datasets. Math: mathematical reasoning. CS: commonsense reasoning. Sym.: symbolic reasoning.

Our method is evaluated on ten benchmark datasets from three categories of reasoning problems:
mathematical reasoning, commonsense reasoning, and symbolic reasoning.
Table {{table:196388ef-7221-4b26-9445-9ff94fc6ed87}}  shows the statistics of evaluation datasets.


MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

Abstract: This paper presents our recent work on the design and development of a new, large scale dataset, which we name MS MARCO, for MAchine Reading COmprehension. This new dataset is aimed to overcome a number of well-known weaknesses of previous publicly available datasets for the same task of reading comprehension and question answering. In MS MARCO, all questions are sampled from real anonymized user queries. The context passages, from which answers in the dataset are derived, are extracted from real web documents using the most advanced version of the Bing search engine. The answers to the queries are human generated. Finally, a subset of these queries has multiple answers. We aim to release one million queries and the corresponding answers in the dataset, which, to the best of our knowledge, is the most comprehensive real-world dataset of its kind in both quantity and quality. We are currently releasing 100,000 queries with their corresponding answers to inspire work in reading comprehension and question answering along with gathering feedback from the research community.

ORCAS: 18 Million Clicked Query-Document Pairs for Analyzing Search

Abstract: Users of Web search engines reveal their information needs through queries and clicks, making click logs a useful asset for information retrieval. However, click logs have not been publicly released for academic use, because they can be too revealing of personally or commercially sensitive information. This paper describes a click data release related to the TREC Deep Learning Track document corpus. After aggregation and filtering, including a k -anonymity requirement, we find 1.4 million of the TREC DL URLs have 18 million connections to 10 million distinct queries. Our dataset of these queries and connections to TREC documents is of similar size to proprietary datasets used in previous papers on query mining and ranking. We perform some preliminary experiments using the click data to augment the TREC DL training data, offering by comparison: 28x more queries, with 49x more connections to 4.4x more URLs in the corpus. We present a description of the dataset's generation process, characteristics, use in ranking and other potential uses.

The five datasets used for model evaluation. “Avg. D/Q” denotes the average number of relevant docs per query.

As we are interested in exploring the performance of the deep retrieval model in a variety of out-of-domain settings, we choose to specifically focus on five datasets in our evaluation (summarized in Table {{table:382e646f-e647-40c2-9f68-fadfc871ab21}} ).

Complementarity of lexical and deep retrieval models

As with query/document expansion, deep retrieval model can narrow the vocabulary gap between queries and documents. One natural question is, are these models still complementary to each other? To answer this, we plot the unique relevant documents retrieved by BM25+Bo1, BM25+docT5query and NPR and their overlaps in Figure {{figure:923af0ef-991e-495e-bc03-84a10ce26c6f}}  for Robust04 and TREC-COVID (other three datasets only have around one relevance document per query, ref Table {{table:382e646f-e647-40c2-9f68-fadfc871ab21}} ). We see that each method is complementary to each other.
In general, NPR retrieves the largest number of unique relevant results, though it retrieves less relevant results than the other two methods.


A Streamlined Encoder/decoder Architecture for Melody Extraction

Abstract: Melody extraction in polyphonic musical audio is important for music signal processing. In this paper, we propose a novel streamlined encoder/decoder network that is designed for the task. We make two technical contributions. First, drawing inspiration from a state-of-the-art model for semantic pixel-wise segmentation, we pass through the pooling indices between pooling and un-pooling layers to localize the melody in frequency. We can achieve result close to the state-of-the-art with much fewer convolutional layers and simpler convolution modules. Second, we propose a way to use the bottleneck layer of the network to estimate the existence of a melody line for each time frame, and make it possible to use a simple argmax function instead of ad-hoc thresholding to get the final estimation of the melody line. Our experiments on both vocal melody extraction and general melody extraction validate the effectiveness of the proposed model.

Joint Detection and Classification of Singing Voice Melody Using Convolutional Recurrent Neural Networks

Abstract: Singing melody extraction essentially involves two tasks: one is detecting the activity of a singing voice in polyphonic music, and the other is estimating the pitch of a singing voice in the detected voiced segments. In this paper, we present a joint detection and classification (JDC) network that conducts the singing voice detection and the pitch estimation simultaneously. The JDC network is composed of the main network that predicts the pitch contours of the singing melody and an auxiliary network that facilitates the detection of the singing voice. The main network is built with a convolutional recurrent neural network with residual connections and predicts pitch labels that cover the vocal range with a high resolution, as well as non-voice status. The auxiliary network is trained to detect the singing voice using multi-level features shared from the main network. The two optimization processes are tied with a joint melody loss function. We evaluate the proposed model on multiple melody extraction and vocal detection datasets, including cross-dataset evaluation. The experiments demonstrate how the auxiliary network and the joint melody loss function improve the melody extraction performance. Furthermore, the results show that our method outperforms state-of-the-art algorithms on the datasets.

Classification-Based Singing Melody Extraction Using Deep Convolutional Neural Networks

Abstract: Singing melody extraction is the task that identifies the melody pitch contour of singing 1 voice from polyphonic music. Most of the traditional melody extraction algorithms are based on 2 calculating salient pitch candidates or separating the melody source from the mixture. Recently, 3 classification-based approach based on deep learning has drawn much attentions. In this paper, 4 we present a classification-based singing melody extraction model using deep convolutional neural 5 networks. The proposed model consists of a singing pitch extractor (SPE) and a singing voice activity 6 detector (SVAD). The SPE is trained to predict a high-resolution pitch label of singing voice from 7 a short segment of spectrogram. This allows the model to predict highly continuous curves. The 8 melody contour is smoothed further by post-processing the output of the melody extractor. The 9 SVAD is trained to determine if a long segment of mel-spectrogram contains a singing voice. This 10 often produces voice false alarm errors around the boundary of singing segments. We reduced them 11 by exploiting the output of the SPE. Finally, we evaluate the proposed melody extraction model on 12 several public datasets. The results show that the proposed model is comparable to state-of-the-art 13 algorithms. 14

HANME: Hierarchical Attention Network for Singing Melody Extraction

Abstract: Singing melody extraction in polyphonic musical audio is a very critical and challenging task in music information retrieval (MIR). Contextual frame-level information has proven its effectiveness in this task. However, existing works assign equal weight to each contextual frame, which may hinder the further improvement of the performance. To this end, we propose a hierarchical attention network for singing melody extraction (HANME) to extract the discriminative attention-aware features and alleviate the workload of the convolutional recurrent neural network (CRNN) for extracting local spatial and temporal features. Specifically, the first attention layer learns the context vector based on local spatial features extracted by residual convolutional neural network (CNN), and the second attention layer learns the temporal context vector based on long-term features extracted by Bidirectional Gated Recurrent Units (BiGRU). Due to the scarcity of labeled training data, we further propose a partial parameter adaptation approach to address the imbalance distribution of the labels for this task. We use the RWC dataset and part of vocal tracks of the MedleyDB dataset for training the model and evaluate the performance on the ADC2004, MIREX 05 and MedleyDB datasets. The experimental study demonstrates the superiority of our method compared with other state-of-the-art ones.

Multitask Learning for Fundamental Frequency Estimation in Music

Abstract: Fundamental frequency (f0) estimation from polyphonic music includes the tasks of multiple-f0, melody, vocal, and bass line estimation. Historically these problems have been approached separately, and only recently, using learning-based approaches. We present a multitask deep learning architecture that jointly estimates outputs for various tasks including multiple-f0, melody, vocal and bass line estimation, and is trained using a large, semi-automatically annotated dataset. We show that the multitask model outperforms its single-task counterparts, and explore the effect of various design decisions in our approach, and show that it performs better or at least competitively when compared against strong baseline methods.

Semi-supervised learning using teacher-student models for vocal melody extraction

Abstract: The lack of labeled data is a major obstacle in many music information retrieval tasks such as melody extraction, where labeling is extremely laborious or costly. Semi-supervised learning (SSL) provides a solution to alleviate the issue by leveraging a large amount of unlabeled data. In this paper, we propose an SSL method using teacher-student models for vocal melody extraction. The teacher model is pre-trained with labeled data and guides the student model to make identical predictions given unlabeled input in a self-training setting. We examine three setups of teacher-student models with different data augmentation schemes and loss functions. Also, considering the scarcity of labeled data in the test phase, we artificially generate large-scale testing data with pitch labels from unlabeled data using an analysis-synthesis method. The results show that the SSL method significantly increases the performance against supervised learning only and the improvement depends on the teacher-student models, the size of unlabeled data, the number of self-training iterations, and other training details. We also find that it is essential to ensure that the unlabeled audio has vocal parts. Finally, we show that the proposed SSL method enables a baseline convolutional recurrent neural network model to achieve performance comparable to state-of-the-arts.

Vocal melody extraction using patch-based CNN

Abstract: A patch-based convolutional neural network (CNN) model presented in this paper for vocal melody extraction in polyphonic music is inspired from object detection in image processing. The input of the model is a novel time-frequency representation which enhances the pitch contours and suppresses the harmonic components of a signal. This succinct data representation and the patch-based CNN model enable an efficient training process with limited labeled data. Experiments on various datasets show excellent speed and competitive accuracy comparing to other deep learning approaches.

The frequency range, resolution, number of classes, and voicing decision labels used by the classification based model. VM and GM stands for vocal melody and general melody extraction. NA stands for either not mentioned or not available. Voicing column indicates is the voicing label included with the pitch labels during training.

The number of pitch classes and the pitch resolution considered for target representation varies across the melody extraction models. Please note that increasing the number of pitch levels also increases pitch resolution and results in label imbalance problems if there are not adequate frames (examples) to represent the particular label. In contrast, considering fewer labels may solve the class imbalance problem but results in low pitch resolution contours. Also, we have to note that not all authors stack the voicing decision labels on top of the pitch labels. Some consider it a standalone task and train it as a separate binary classification task. Later, the voicing detection is combined with the melody extraction to get the overall clean melody output. The number of pitch classes, pitch resolution, and other parameters adopted for creating the target pitch classes for training various models is shown in Table {{table:6932f3b0-ae81-4a3c-badc-2a00e1f21228}} .


MTSamples Laparoscopic Gastric Bypass Consult (augmented)

MIMIC-CXR-JPG, a large publicly available database of labeled chest radiographs

Abstract: Chest radiography is an extremely powerful imaging modality, allowing for a detailed inspection of a patient's thorax, but requiring specialized training for proper interpretation. With the advent of high performance general purpose computer vision algorithms, the accurate automated analysis of chest radiographs is becoming increasingly of interest to researchers. However, a key challenge in the development of these techniques is the lack of sufficient data. Here we describe MIMIC-CXR-JPG v2.0.0, a large dataset of 377,110 chest x-rays associated with 227,827 imaging studies sourced from the Beth Israel Deaconess Medical Center between 2011 - 2016. Images are provided with 14 labels derived from two natural language processing tools applied to the corresponding free-text radiology reports. MIMIC-CXR-JPG is derived entirely from the MIMIC-CXR database, and aims to provide a convenient processed version of MIMIC-CXR, as well as to provide a standard reference for data splits and image labels. All images have been de-identified to protect patient privacy. The dataset is made freely available to facilitate and encourage a wide range of research in medical computer vision.

A new paradigm for accelerating clinical data science at Stanford Medicine

Abstract: Stanford Medicine is building a new data platform for our academic research community to do better clinical data science. Hospitals have a large amount of patient data and researchers have demonstrated the ability to reuse that data and AI approaches to derive novel insights, support patient care, and improve care quality. However, the traditional data warehouse and Honest Broker approaches that are in current use, are not scalable. We are establishing a new secure Big Data platform that aims to reduce time to access and analyze data. In this platform, data is anonymized to preserve patient data privacy and made available preparatory to Institutional Review Board (IRB) submission. Furthermore, the data is standardized such that analysis done at Stanford can be replicated elsewhere using the same analytical code and clinical concepts. Finally, the analytics data warehouse integrates with a secure data science computational facility to support large scale data analytics. The ecosystem is designed to bring the modern data science community to highly sensitive clinical data in a secure and collaborative big data analytics environment with a goal to enable bigger, better and faster science.

Information for each deidentification algorithm.

We implemented wrappers for five openly available deidentification algorithms: Philter-UCSF {{cite:92137eb}}, TiDE {{cite:b1bba94}}, PyDeID {{cite:8cba0f8}}, PhysioNet-deid {{cite:9d95db6}}, and Transformer-DeID {{cite:ecab937}}. These algorithms adopt various approaches to deidentification including the use of supervised machine learning (ML) with labeled training data, rule-based approaches, and dictionaries of known entities.
A comparison of these algorithms is provided in Table {{table:81e1f256-8818-4306-a9e1-23f8ca8af9d0}} . Similar to the datasets described above, each deidentification algorithm differs slightly in which entities are considered protected, including the annotation of professions, ages less than 90, non-patient names, large geographic locations such as country, lone years, and organizations. These differences are highlighted in Table {{table:4e930266-5bf9-490f-b6e5-5f5338cf84a9}} . We harmonize entity types output by each algorithm for consistency.


Select as many clients as possible within a specified deadline

Norm-based client selection to tackle the communication bottleneck

Local loss based client selection tradeoff between convergence speed and solution bias

Two-level importance sampling for clients and data

Bandwidth allocation under client energy constraints

Based on the correlation between local update and global update

Removing adverse local updates by comparing the gradients of the local and the global

Optimization of client sampling probabilities to address system and statistical heterogeneity

Client Selection for Federated Learning with Heterogeneous Resources in Mobile Edge

Abstract: We envision a mobile edge computing (MEC) framework for machine learning (ML) technologies, which leverages distributed client data and computation resources for training high-performance ML models while preserving client privacy. Toward this future goal, this work aims to extend Federated Learning (FL), a decentralized learning framework that enables privacy-preserving training of models, to work with heterogeneous clients in a practical cellular network. The FL protocol iteratively asks random clients to download a trainable model from a server, update it with own data, and upload the updated model to the server, while asking the server to aggregate multiple client updates to further improve the model. While clients in this protocol are free from disclosing own private data, the overall training process can become inefficient when some clients are with limited computational resources (i.e. requiring longer update time) or under poor wireless channel conditions (longer upload time). Our new FL protocol, which we refer to as FedCS, mitigates this problem and performs FL efficiently while actively managing clients based on their resource conditions. Specifically, FedCS solves a client selection problem with resource constraints, which allows the server to aggregate as many client updates as possible and to accelerate performance improvement in ML models. We conducted an experimental evaluation using publicly-available large-scale image datasets to train deep neural networks on MEC environment simulations. The experimental results show that FedCS is able to complete its training process in a significantly shorter time compared to the original FL protocol.

Optimal Client Sampling for Federated Learning

Abstract: It is well understood that client-master communication can be a primary bottleneck in Federated Learning. In this work, we address this issue with a novel client subsampling scheme, where we restrict the number of clients allowed to communicate their updates back to the master node. In each communication round, all participating clients compute their updates, but only the ones with "important" updates communicate back to the master. We show that importance can be measured using only the norm of the update and give a formula for optimal client participation. This formula minimizes the distance between the full update, where all clients participate, and our limited update, where the number of participating clients is restricted. In addition, we provide a simple algorithm that approximates the optimal formula for client participation, which only requires secure aggregation and thus does not compromise client privacy. We show both theoretically and empirically that for Distributed SGD (DSGD) and Federated Averaging (FedAvg), the performance of our approach can be close to full participation and superior to the baseline where participating clients are sampled uniformly. Moreover, our approach is orthogonal to and compatible with existing methods for reducing communication overhead, such as local methods and communication compression methods.

Client Selection in Federated Learning: Convergence Analysis and Power-of-Choice Selection Strategies

Abstract: Federated learning is a distributed optimization paradigm that enables a large number of resource-limited client nodes to cooperatively train a model without data sharing. Several works have analyzed the convergence of federated learning by accounting of data heterogeneity, communication and computation limitations, and partial client participation. However, they assume unbiased client participation, where clients are selected at random or in proportion of their data sizes. In this paper, we present the first convergence analysis of federated optimization for biased client selection strategies, and quantify how the selection bias affects convergence speed. We reveal that biasing client selection towards clients with higher local loss achieves faster error convergence. Using this insight, we propose Power-of-Choice, a communication- and computation-efficient client selection framework that can flexibly span the trade-off between convergence speed and solution bias. Our experiments demonstrate that Power-of-Choice strategies converge up to 3 $\times$ faster and give $10$% higher test accuracy than the baseline random selection.

Optimal Importance Sampling for Federated Learning

Abstract: Federated learning involves a mixture of centralized and decentralized processing tasks, where a server regularly selects a sample of the agents and these in turn sample their local data to compute stochastic gradients for their learning updates. The sampling of both agents and data is generally uniform; however, in this work we consider non-uniform sampling. We derive optimal importance sampling strategies for both agent and data selection and show that under convexity and Lipschitz assumptions, non-uniform sampling without replacement improves the performance of the original FedAvg algorithm. We run experiments on a regression and classification problem to illustrate the theoretical results.

Client Selection and Bandwidth Allocation in Wireless Federated Learning Networks: A Long-Term Perspective

Abstract: This paper studies federated learning (FL) in a classic wireless network, where learning clients share a common wireless link to a coordinating server to perform federated model training using their local data. In such wireless federated learning networks (WFLNs), optimizing the learning performance depends crucially on how clients are selected and how bandwidth is allocated among the selected clients in every learning round, as both radio and client energy resources are limited. While existing works have made some attempts to allocate the limited wireless resources to optimize FL, they focus on the problem in individual learning rounds, overlooking an inherent yet critical feature of federated learning. This paper brings a new long-term perspective to resource allocation in WFLNs, realizing that learning rounds are not only temporally interdependent but also have varying significance towards the final learning outcome. To this end, we first design data-driven experiments to show that different temporal client selection patterns lead to considerably different learning performance. With the obtained insights, we formulate a stochastic optimization problem for joint client selection and bandwidth allocation under long-term client energy constraints, and develop a new algorithm that utilizes only currently available wireless channel information but can achieve long-term performance guarantee. Further experiments show that our algorithm results in the desired temporal client selection pattern, is adaptive to changing network environments and far outperforms benchmarks that ignore the long-term effect of FL.

Fast-Convergent Federated Learning

Abstract: Federated learning has emerged recently as a promising solution for distributing machine learning tasks through modern networks of mobile devices. Recent studies have obtained lower bounds on the expected decrease in model loss that is achieved through each round of federated learning. However, convergence generally requires a large number of communication rounds, which induces delay in model training and is costly in terms of network resources. In this paper, we propose a fast-convergent federated learning algorithm, called FOLB, which performs intelligent sampling of devices in each round of model training to optimize the expected convergence speed. We first theoretically characterize a lower bound on improvement that can be obtained in each round if devices are selected according to the expected improvement their local models will provide to the current global model. Then, we show that FOLB obtains this bound through uniform sampling by weighting device updates according to their gradient information. FOLB is able to handle both communication and computation heterogeneity of devices by adapting the aggregations according to estimates of device's capabilities of contributing to the updates. We evaluate FOLB in comparison with existing federated learning algorithms and experimentally show its improvement in trained model accuracy, convergence speed, and/or model stability across various machine learning tasks and datasets.

Node Selection Toward Faster Convergence for Federated Learning on Non-IID Data

Abstract: Federated Learning (FL) is a distributed learning paradigm that enables a large number of resource-limited nodes to collaboratively train a model without data sharing. The non-independent-and-identically-distributed (non-i.i.d.) data samples invoke discrepancies between the global and local objectives, making the FL model slow to converge. In this paper, we proposed Optimal Aggregation algorithm for better aggregation, which finds out the optimal subset of local updates of participating nodes in each global round, by identifying and excluding the adverse local updates via checking the relationship between the local gradient and the global gradient. Then, we proposed a Probabilistic Node Selection framework (FedPNS) to dynamically change the probability for each node to be selected based on the output of Optimal Aggregation. FedPNS can preferentially select nodes that propel faster model convergence. The unbiasedness of the proposed FedPNS design is illustrated and the convergence rate improvement of FedPNS over the commonly adopted Federated Averaging (FedAvg) algorithm is analyzed theoretically. Experimental results demonstrate the effectiveness of FedPNS in accelerating the FL convergence rate, as compared to FedAvg with random node selection.

Tackling System and Statistical Heterogeneity for Federated Learning with Adaptive Client Sampling

Abstract: Federated learning (FL) algorithms usually sample a fraction of clients in each round (partial participation) when the number of participants is large and the server's communication bandwidth is limited. Recent works on the convergence analysis of FL have focused on unbiased client sampling, e.g., sampling uniformly at random, which suffers from slow wall-clock time for convergence due to high degrees of system heterogeneity and statistical heterogeneity. This paper aims to design an adaptive client sampling algorithm that tackles both system and statistical heterogeneity to minimize the wall-clock convergence time. We obtain a new tractable convergence bound for FL algorithms with arbitrary client sampling probabilities. Based on the bound, we analytically establish the relationship between the total learning time and sampling probabilities, which results in a non-convex optimization problem for training time minimization. We design an efficient algorithm for learning the unknown parameters in the convergence bound and develop a low-complexity algorithm to approximately solve the non-convex problem. Experimental results from both hardware prototype and simulation demonstrate that our proposed sampling scheme significantly reduces the convergence time compared to several baseline sampling schemes. Notably, our scheme in hardware prototype spends 73% less time than the uniform sampling baseline for reaching the same target loss.

Representative work of FL client selection algorithms, ordered by the publication year. We only list work with explicit utility measurement and scheduling decisions for clients. We tag whether the work designed for system heterogeneity and data heterogeneity.

In each training round, each client is measured for its utility/priority and the clients with the best utility measurements are selected for model training and aggregation. There are various methods for formulating clients' utility. Table {{table:75c961c9-0c71-4d9a-bfcd-0967578b16ab}}  summarizes the representative client selection algorithms that have explicit utility measurement and scheduling decisions. We tag the work designed specially for the system heterogeneity and the data heterogeneity. In this section, we elaborate on the utility functions. Table {{table:a0eddbf8-b1b2-43fc-b913-303ebd3970bd}}  tabulates the notations that are used in this paper for quick reference.


Deep Learning for Android Malware Defenses: A Systematic Literature Review

A Survey of Privacy Vulnerabilities of Mobile Device Sensors

A Systematic Survey on Android API Usage for Data-Driven Analytics with Smartphones

A systematic mapping study addressing the reliability of mobile applications: The need to move beyond testing reliability

Mobile app privacy in software engineering research: A systematic mapping study

Rebooting Research on Detecting Repackaged Android Apps: Literature Review and Benchmark

Abstract: Malicious applications (particularly those targeting the Android platform) pose a serious threat to developers and end-users. Numerous research efforts have been devoted to developing effective approaches to defend against Android malware. However, given the explosive growth of Android malware and the continuous advancement of malicious evasion technologies like obfuscation and reflection, Android malware defense approaches based on manual rules or traditional machine learning may not be effective. In recent years, a dominant research field called deep learning (DL), which provides a powerful feature abstraction ability, has demonstrated a compelling and promising performance in a variety of areas, like natural language processing and computer vision. To this end, employing DL techniques to thwart Android malware attacks has recently garnered considerable research attention. Yet, no systematic literature review focusing on DL approaches for Android malware defenses exists. In this article, we conducted a systematic literature review to search and analyze how DL approaches have been applied in the context of malware defenses in the Android environment. As a result, a total of 132 studies covering the period 2014–2021 were identified. Our investigation reveals that, while the majority of these sources mainly consider DL-based Android malware detection, 53 primary studies (40.1%) design defense approaches based on other scenarios. This review also discusses research trends, research focuses, challenges, and future research directions in DL-based Android malware defenses.

Abstract: The number of mobile devices, such as smartphones and smartwatches, is relentlessly increasing to almost 6.8 billion by 2022, and along with it, the amount of personal and sensitive data captured by them. This survey overviews the state of the art of what personal and sensitive user attributes can be extracted from mobile device sensors, emphasising critical aspects such as demographics, health and body features, activity and behaviour recognition, etc. In addition, we review popular metrics in the literature to quantify the degree of privacy, and discuss powerful privacy methods to protect the sensitive data while preserving data utility for analysis. Finally, open research questions a represented for further advancements in the field.

Abstract: Recent industrial and academic research has focused on data-driven analytics with smartphones by collecting user interaction, context, and device systems data through Application Programming interfaces (APIs) and sensors. The Android OS provides various APIs to collect such mobile usage and sensor data for third-party developers. Usage Statistics API (US API) and Accessibility Service API (AS API) are representative Android APIs for collecting app usage data and are used for various research purposes as they can collect fine-grained interaction data (e.g., app usage history, user interaction type). Furthermore, other sensor APIs help to collect a user's context and device state data, along with AS/US APIs. This review investigates mobile usage and sensor data-driven research using AS/US APIs, by categorizing the research purposes and the data types. In this paper, the surveyed studies are classified as follows: five themes and 21 subthemes, and a four-layer hierarchical data classification structure. This allows us to identify a data usage trend and derive insight into data collection according to research purposes. Several limitations and future research directions of mobile usage and sensor data-driven analytics research are discussed, including the impact of changes in the Android API versions on research, the privacy and data quality issues, and the mitigation of reproducibility risks with standardized data typology.

A Systematic Mapping Study Addressing the Reliability of Mobile Applications: The Need to Move Beyond Testing Reliability

Abstract: Intense competition in the mobile apps market means it is important to maintain high levels of app reliability to avoid losing users. Yet despite its importance, app reliability is underexplored in the research literature. To address this need, we identify, analyse, and classify the state-of-the-art in the field of mobile apps' reliability through a systematic mapping study. From the results of such a study, researchers in the field can identify pressing research gaps, and developers can gain knowledge about existing solutions, to potentially leverage them in practice. We found 87 relevant papers which were then analysed and classified based on their research focus, research type, contribution, research method, study settings, data, quality attributes and metrics used. Results indicate that there is a lack of research on understanding reliability with regard to context-awareness, self-healing, ageing and rejuvenation, and runtime event handling. These aspects have rarely been studied, or if studied, there is limited evaluation. We also identified several other research gaps including the need to conduct more research in real-world industrial projects. Furthermore, little attention has been paid towards quality standards while conducting research. Outcomes here show numerous opportunities for greater research depth and breadth on mobile app reliability.

Mobile App Privacy in Software Engineering Research: A Systematic Mapping Study

Abstract: Mobile applications (apps) have become deeply personal, constantly demanding access to privacy-sensitive information in exchange for more personalized user experiences. Such privacy-invading practices have generated major multidimensional and unconventional privacy concerns among app users. To address these concerns, the research on mobile app privacy has experienced rapid growth over the past decade. In general, this line of research is aimed at systematically exposing the privacy practices of apps and proposing solutions to protect the privacy of mobile app users. In this survey paper, we conduct a systematic mapping study of 54 Software Engineering (SE) primary studies on mobile app privacy. Our objectives are to a) explore trends in SE app privacy research, b) categorize existing evidence, and c) identify potential directions for future research. Our results show that existing literature can be divided into four main categories: privacy policy, requirements, user perspective, and leak detection. Furthermore, our survey reveals an imbalance between these categories; majority of existing research focuses on proposing tools for detecting privacy leaks, with less studies targeting privacy requirements and policy and even less on user perspective. Finally, our survey exposes several gaps in existing research and suggests areas for improvement.

Abstract: Repackaging is a serious threat to the Android ecosystem as it deprives app developers of their benefits, contributes to spreading malware on users’ devices, and increases the workload of market maintainers. In the space of six years, the research around this specific issue has produced 57 approaches which do not readily scale to millions of apps or are only evaluated on private datasets without, in general, tool support available to the community. Through a systematic literature review of the subject, we argue that the research is slowing down, where many state-of-the-art approaches have reported high-performance rates on closed datasets, which are unfortunately difficult to replicate and to compare against. In this work, we propose to reboot the research in repackaged app detection by providing a literature review that summarises the challenges and current solutions for detecting repackaged apps and by providing a large dataset that supports replications of existing solutions and implications of new research directions. We hope that these contributions will re-activate the direction of detecting repackaged apps and spark innovative approaches going beyond the current state-of-the-art.

The List of Selected Publications.

We were able to eventually collect 39 papers to answer our research question defined at the beginning of this study.
Table {{table:04f07bbb-0d49-43cf-bd66-974168293f31}}  enumerates the list of selected papers, including their publication year and venue.
Once the relevant papers are collected, we carefully read all of them and attempt to extract the relevant data (i.e., Step 7) from each paper to answer the research question.
Specifically, we aim to extract the following two types of information:
(1) Targeted Problems, which involve understanding the issues within the Android/iOS ecosystem that have been identified by our MSE researchers as problems needing resolution to create a more user-friendly mobile ecosystem, and (2) Fundamental Techniques,
aimed at discovering the techniques required to address the various challenges in the mobile community.
Considering that OpenHarmony may encounter similar issues to those faced by Android and iOS, we argue that insights gained from exploring these two aspects could prove valuable in shaping the roadmap for conducting software engineering research for OpenHarmony. Furthermore, similar to our approach in identifying relevant papers, we have conducted cross-checks of our observations, involving at least two authors, to ensure the reliability of these observations, thereby enhancing the trustworthiness of the research roadmap.