Arithmetic, Algebraic (linear, nonlinear)

Algebra, Calculus, Geometry, Statistics, Number Theory

Unit Dependency Graph and its Application to Arithmetic Word Problem Solving

Abstract: Math word problems provide a natural abstraction to a range of natural language understanding problems that involve reasoning about quantities, such as interpreting election results, news about casualties, and the financial section of a newspaper. Units associated with the quantities often provide information that is essential to support this reasoning. This paper proposes a principled way to capture and reason about units and shows how it can benefit an arithmetic word problem solver. This paper presents the concept of Unit Dependency Graphs (UDGs), which provides a compact representation of the dependencies between units of numbers mentioned in a given problem. Inducing the UDG alleviates the brittleness of the unit extraction system and allows for a natural way to leverage domain knowledge about unit compatibility, for word problem solving. We introduce a decomposed model for inducing UDGs with minimal additional annotations, and use it to augment the expressions used in the arithmetic word problem solver of (Roy and Roth 2015) via a constrained inference framework. We show that introduction of UDGs reduces the error of the solver by over 10 %, surpassing all existing systems for solving arithmetic word problems. In addition, it also makes the system more robust to adaptation to new vocabulary and equation forms .

Program Induction by Rationale Generation : Learning to Solve and Explain Algebraic Word Problems

Abstract: Solving algebraic word problems requires executing a series of arithmetic operations---a program---to obtain a final answer. However, since programs can be arbitrarily complicated, inducing them directly from question-answer pairs is a formidable challenge. To make this task more feasible, we solve these problems by generating answer rationales, sequences of natural language and human-readable mathematical expressions that derive the final answer through a series of small steps. Although rationales do not explicitly specify programs, they provide a scaffolding for their structure via intermediate milestones. To evaluate our approach, we have created a new 100,000-sample dataset of questions, answers and rationales. Experimental results show that indirect supervision of program learning via answer rationales is a promising strategy for inducing arithmetic programs.

MathQA: Towards Interpretable Math Word Problem Solving with Operation-Based Formalisms

Abstract: We introduce a large-scale dataset of math word problems and an interpretable neural math problem solver by learning to map problems to their operation programs. Due to annotation challenges, current datasets in this domain have been either relatively small in scale or did not offer precise operational annotations over diverse problem types. We introduce a new representation language to model operation programs corresponding to each math problem that aim to improve both the performance and the interpretability of the learned models. Using this representation language, we significantly enhance the AQUA-RAT dataset with fully-specified operational programs. We additionally introduce a neural sequence-to-program model with automatic problem categorization. Our experiments show improvements over competitive baselines in our dataset as well as the AQUA-RAT dataset. The results are still lower than human performance indicating that the dataset poses new challenges for future research. Our dataset is available at https://math-qa.github.io/math-QA/

Measuring Mathematical Problem Solving With the MATH Dataset

Abstract: Many intellectual endeavors require mathematical problem solving, but this skill remains beyond the capabilities of computers. To measure this ability in machine learning models, we introduce MATH, a new dataset of 12,500 challenging competition mathematics problems. Each problem in MATH has a full step-by-step solution which can be used to teach models to generate answer derivations and explanations. To facilitate future research and increase accuracy on MATH, we also contribute a large auxiliary pretraining dataset which helps teach models the fundamentals of mathematics. Even though we are able to increase accuracy on MATH, our results show that accuracy remains relatively low, even with enormous Transformer models. Moreover, we find that simply increasing budgets and model parameter counts will be impractical for achieving strong mathematical reasoning if scaling trends continue. While scaling Transformers is automatically solving most other text-based tasks, scaling is not currently solving MATH. To have more traction on mathematical problem solving we will likely need new algorithmic advancements from the broader research community.

A Diverse Corpus for Evaluating and Developing English Math Word Problem Solvers

Abstract: We present ASDiv (Academia Sinica Diverse MWP Dataset), a diverse (in terms of both language patterns and problem types) English math word problem (MWP) corpus for evaluating the capability of various MWP solvers. Existing MWP corpora for studying AI progress remain limited either in language usage patterns or in problem types. We thus present a new English MWP corpus with 2,305 MWPs that cover more text patterns and most problem types taught in elementary school. Each MWP is annotated with its problem type and grade level (for indicating the level of difficulty). Furthermore, we propose a metric to measure the lexicon usage diversity of a given MWP corpus, and demonstrate that ASDiv is more diverse than existing corpora. Experiments show that our proposed corpus reflects the true capability of MWP solvers more faithfully.

Are NLP Models really able to Solve Simple Math Word Problems?

Abstract: The problem of designing NLP solvers for math word problems (MWP) has seen sustained research activity and steady gains in the test accuracy. Since existing solvers achieve high performance on the benchmark datasets for elementary level MWPs containing one-unknown arithmetic word problems, such problems are often considered "solved" with the bulk of research attention moving to more complex MWPs. In this paper, we restrict our attention to English MWPs taught in grades four and lower. We provide strong evidence that the existing MWP solvers rely on shallow heuristics to achieve high performance on the benchmark datasets. To this end, we show that MWP solvers that do not have access to the question asked in the MWP can still solve a large fraction of MWPs. Similarly, models that treat MWPs as bag-of-words can also achieve surprisingly high accuracy. Further, we introduce a challenge dataset, SVAMP, created by applying carefully chosen variations over examples sampled from existing datasets. The best accuracy achieved by state-of-the-art models is substantially lower on SVAMP, thus showing that much remains to be done even for the simplest of the MWPs.

Semantically-Aligned Universal Tree-Structured Solver for Math Word Problems

Abstract: A practical automatic textual math word problems (MWPs) solver should be able to solve various textual MWPs while most existing works only focused on one-unknown linear MWPs. Herein, we propose a simple but efficient method called Universal Expression Tree (UET) to make the first attempt to represent the equations of various MWPs uniformly. Then a semantically-aligned universal tree-structured solver (SAU-Solver) based on an encoder-decoder framework is proposed to resolve multiple types of MWPs in a unified model, benefiting from our UET representation. Our SAU-Solver generates a universal expression tree explicitly by deciding which symbol to generate according to the generated symbols' semantic meanings like human solving MWPs. Besides, our SAU-Solver also includes a novel subtree-level semanticallyaligned regularization to further enforce the semantic constraints and rationality of the generated expression tree by aligning with the contextual information. Finally, to validate the universality of our solver and extend the research boundary of MWPs, we introduce a new challenging Hybrid Math Word Problems dataset (HMWP), consisting of three types of MWPs. Experimental results on several MWPs datasets show that our model can solve universal types of MWPs and outperforms several state-of-the-art models.

Ape210K: A Large-Scale and Template-Rich Dataset of Math Word Problems

Abstract: Automatic math word problem solving has attracted growing attention in recent years. The evaluation datasets used by previous works have serious limitations in terms of scale and diversity. In this paper, we release a new large-scale and template-rich math word problem dataset named Ape210K. It consists of 210K Chinese elementary school-level math problems, which is 9 times the size of the largest public dataset Math23K. Each problem contains both the gold answer and the equations needed to derive the answer. Ape210K is also of greater diversity with 56K templates, which is 25 times more than Math23K. Our analysis shows that solving Ape210K requires not only natural language understanding but also commonsense knowledge. We expect Ape210K to be a benchmark for math word problem solving systems. Experiments indicate that state-of-the-art models on the Math23K dataset perform poorly on Ape210K. We propose a copy-augmented and feature-enriched sequence to sequence (seq2seq) model, which outperforms existing models by 3.2% on the Math23K dataset and serves as a strong baseline of the Ape210K dataset. The gap is still significant between human and our baseline model, calling for further research efforts. We make Ape210K dataset publicly available at this https URL

Analysing Mathematical Reasoning Abilities of Neural Models

Abstract: Mathematical reasoning---a core ability within human intelligence---presents some unique challenges as a domain: we do not come to understand and solve mathematical problems primarily on the back of experience and evidence, but on the basis of inferring, learning, and exploiting laws, axioms, and symbol manipulation rules. In this paper, we present a new challenge for the evaluation (and eventually the design) of neural architectures and similar system, developing a task suite of mathematics problems involving sequential questions and answers in a free-form textual input/output format. The structured nature of the mathematics domain, covering arithmetic, algebra, probability and calculus, enables the construction of training and test splits designed to clearly illuminate the capabilities and failure-modes of different architectures, as well as evaluate their ability to compose and relate knowledge and learned processes. Having described the data generation process and its potential future expansions, we conduct a comprehensive analysis of models from two broad classes of the most powerful sequence-to-sequence architectures and find notable differences in their ability to resolve mathematical problems and generalize their knowledge.

Deep Learning for Symbolic Mathematics

Abstract: Neural networks have a reputation for being better at solving statistical or approximate problems than at performing calculations or working with symbolic data. In this paper, we show that they can be surprisingly good at more elaborated tasks in mathematics, such as symbolic integration and solving differential equations. We propose a syntax for representing mathematical problems, and methods for generating large datasets that can be used to train sequence-to-sequence models. We achieve results that outperform commercial Computer Algebra Systems such as Matlab or Mathematica.

Categorization of MWP Datasets available for training AI models based on (a) Mathematics Area, (b) Language, (c) Type, and (d) Annotation.

One of the current limitations of achieving performance similar to human-level mathematical intelligence is the availability of labeled datasets. Table {{table:59d7204f-7887-4df7-8b09-6bacc7ae2453}}  categorizes various MWPs datasets along with information like dataset size, language, type of dataset, mathematics area etc. Each dataset focus on specific areas of Mathematics, i.e., Arithmetic, Algebra, Calculus, etc. AI2, IL, ALGES, ALLArith, SVAMP are examples of datasets for arithmetic MWPs. Datasets such as Alg514, Dolphin1878, DRAW, Dolphin18K, Math23K, HMWP, and Ape210K focus on Algebraic problems. MAWPS, AQuA, MathQA, and ASDiv are examples of datasets covering MWPs from Arithmetic and Algebra. There have been efforts to create MWPs datasets for complex areas including Probability, Integration and Differentiation {{cite:61d89ec}} {{cite:fcbcb2a}}. {{table:59d7204f-7887-4df7-8b09-6bacc7ae2453}}  covers examples of MWP datasets in English and Chinese languages.

Mathematical Reasoning in Neural Approaches

Mathematical reasoning is core to human intelligence. As explained before, it is a complex phenomenon interfacing natural language understanding and visual understanding to invoke mathematical transformations. Over the past few decades, there has been a flurry of research to develop models with mathematical reasoning ability; however, the progress is limited. Larger datasets definitely help models learning to solve complex and niche problems {{cite:fcbcb2a}} {{cite:61d89ec}}, but a common purpose model to solve a diverse variety of math problems is still a distinct reality {{cite:272c679}} {{cite:94d3b2f}}. Smaller datasets overfit the models, and hence reasoning ability is questionable. Non-neural models could not improve performance with larger datasets {{cite:5bdd851}}.
On the other hand, neural models achieve much better performance on the smaller datasets and attain near-perfect performance over very large datasets; however, its BlackBox structure hinders interpreting its reasoning ability. There is a decent consensus among researchers to solve problems by first generating intermediate expression trees and evaluating them using solvers to predict the answer {{cite:8eb4eaa}} {{cite:105ec56}} {{cite:1bea915}}. As opposed to directly predicting the final answer, generating intermediate expression trees helps understand models' reasoning ability better. Currently, the efforts are mainly to solve MWPs of primary or secondary schools as shown in Table {{table:59d7204f-7887-4df7-8b09-6bacc7ae2453}} . The state of the current art is still very far from building mathematical intelligence that could also infer visual narratives and natural language and use domain-specific knowledge.


Applications of LDPC Codes to the Wiretap Channel

Abstract: With the advent of quantum key distribution (QKD) systems, perfect (i.e. information-theoretic) security can now be achieved for distribution of a cryptographic key. QKD systems and similar protocols use classical error-correcting codes for both error correction (for the honest parties to correct errors) and privacy amplification (to make an eavesdropper fully ignorant). From a coding perspective, a good model that corresponds to such a setting is the wire tap channel introduced by Wyner in 1975. In this paper, we study fundamental limits and coding methods for wire tap channels. We provide an alternative view of the proof for secrecy capacity of wire tap channels and show how capacity achieving codes can be used to achieve the secrecy capacity for any wiretap channel. We also consider binary erasure channel and binary symmetric channel special cases for the wiretap channel and propose specific practical codes. In some cases our designs achieve the secrecy capacity and in others the codes provide security at rates below secrecy capacity. For the special case of a noiseless main channel and binary erasure channel, we consider encoder and decoder design for codes achieving secrecy on the wiretap channel; we show that it is possible to construct linear-time decodable secrecy codes based on LDPC codes that achieve secrecy.

Performance Analysis and Design of Two Edge-Type LDPC Codes for the BEC Wiretap Channel

Abstract: We consider transmission over a wiretap channel where both the main channel and the wiretapper's channel are binary erasure channels (BEC). A code construction method is proposed using two edge-type low-density parity-check (LDPC) codes based on the coset encoding scheme. Using a single edge-type LDPC ensemble with a given threshold over the BEC, we give a construction for a two edge-type LDPC ensemble with the same threshold. If the given single edge-type LDPC ensemble has degree two variable nodes, our construction gives rise to degree one variable nodes in the code used over the main channel. This results in zero threshold over the main channel. In order to circumvent this problem, the degree distribution of the two edge-type LDPC ensemble is numerically optimized. We find that the resulting ensembles are able to perform close to the boundary of the rate-equivocation region of the wiretap channel. Further, a method to compute the ensemble average equivocation of two edge-type LDPC ensembles is provided by generalizing a recently published approach to measure the equivocation of single edge-type ensembles for transmission over the BEC in the point-to-point setting. From this analysis, we find that relatively simple constructions give very good secrecy performance.

Strong Secrecy on the Binary Erasure Wiretap Channel Using Large-Girth LDPC Codes

Abstract: For an arbitrary degree distribution pair (DDP), we construct a sequence of low-density parity-check (LDPC) code ensembles with girth growing logarithmically in block-length using Ramanujan graphs. When the DDP has minimum left degree at least three, we show using density evolution analysis that the expected bit-error probability of these ensembles, when passed through a binary erasure channel with erasure probability ϵ, decays as O(exp(-(c1)n(c2))) with the block-length n for positive constants c1 and c2, as long as ϵ is less than the erasure threshold ϵth of the DDP. This guarantees that the coset coding scheme using the dual sequence provides strong secrecy over the binary erasure wiretap channel for erasure probabilities greater than 1-ϵth.

Coding with Scrambling, Concatenation, and HARQ for the AWGN Wire-Tap Channel: A Security Gap Analysis

Abstract: This study examines the use of nonsystematic channel codes to obtain secure transmissions over the additive white Gaussian noise (AWGN) wire-tap channel. Unlike the previous approaches, we propose to implement nonsystematic coded transmission by scrambling the information bits, and characterize the bit error rate of scrambled transmissions through theoretical arguments and numerical simulations. We have focused on some examples of Bose-Chaudhuri-Hocquenghem (BCH) and low-density parity-check (LDPC) codes to estimate the security gap, which we have used as a measure of physical layer security, in addition to the bit error rate. Based on a number of numerical examples, we found that such a transmission technique can outperform alternative solutions. In fact, when an eavesdropper (Eve) has a worse channel than the authorized user (Bob), the security gap required to reach a given level of security is very small. The amount of degradation of Eve's channel with respect to Bob's that is needed to achieve sufficient security can be further reduced by implementing scrambling and descrambling operations on blocks of frames, rather than on single frames. While Eve's channel has a quality equal to or better than that of Bob's channel, we have shown that the use of a hybrid automatic repeat-request (HARQ) protocol with authentication still allows achieving a sufficient level of security. Finally, the secrecy performance of some practical schemes has also been measured in terms of the equivocation rate about the message at the eavesdropper and compared with that of ideal codes.

Performance assessment and design of finite length LDPC codes for the Gaussian wiretap channel

Abstract: In this work we study the reliability and secrecy performance achievable by practical LDPC codes over the Gaussian wiretap channel. While several works have already addressed this problem in asymptotic conditions, i.e., under the hypothesis of codewords of infinite length, only a few approaches exist for the finite length regime. We propose an approach to measure the performance of practical codes and compare it with that achievable in asymptotic conditions. Moreover, based on the secrecy metrics we adopt to achieve this target, we propose a code optimization algorithm which allows to design irregular LDPC codes able to approach the ultimate performance limits even at moderately small codeword lengths (in the order of 10000 bits).

LDPC Codes for Physical Layer Security

LDPC codes have been designed for the Gaussian wiretap channel. The physical-layer security
communication is realized in {{cite:1fae58f}} by punctured LDPC codes under
the criterion of bit-error rate (BER), where the secrecy
information bits are hidden in the punctured bits. Therefore, these
information bits are not transmitted through the channel but can be decoded
at the receiver side based on the non-punctured part of the codeword.
This coding scheme can yield a BER close to 0.5 at the eavesdropper's side
while significantly reduces the security gap defined in {{cite:1fae58f}} comparing to the non-punctured LDPC codes.
However, the punctured LDPC codes result in higher power transmission comparing to the
non-punctured LDPC codes.
To solve this problem, M. Baldi et al. propose a
nonsystematic coded transmission design
by scrambling the information bits {{cite:f7a38a5}}. It is shown
in {{cite:f7a38a5}} that this scrambling technique achieves
security gap comparable to that design based on puncturing but without
increasing the transmit power. This scrambling design has been extended
to parallel Rayleigh distributed channels {{cite:5468a8d}}.
By exploiting the equivocation rate of eavesdropper's channel as an optimization criterion,
M. Baldi et al. propose a code design algorithm in the finite codeword length
regime {{cite:c87180d}}. Based on this algorithm, irregular LDPC codes which approach the
ultimate performance limits with small codeword lengths are constructed.
A brief summary of above work is given in Table {{table:a979c12e-9f64-48b2-b393-8a5b452af64b}} .

As indicated in Table {{table:a979c12e-9f64-48b2-b393-8a5b452af64b}} , in terms of mutual information criterion,
current LDPC code designs can only achieve the weak secrecy for the special BEC model.
For strong secrecy, it further requires the main channel to be noiseless.
How to design LDPC codes which can achieve the weak/stong secrecy for a more general
channel such as the Gaussian wiretap channel is still a challenge problem.
In terms of BER criterion, the LDPC codes for MIMO and massive MIMO systems can
be investigated.


Symmetric
binary-input memoryless degraded wiretap channel

Achieving the Secrecy Capacity of Wiretap Channels Using Polar Codes

Abstract: Suppose that Alice wishes to send messages to Bob through a communication channel C1, but her transmissions also reach an eavesdropper Eve through another channel C2. This is the wiretap channel model introduced by Wyner in 1975. The goal is to design a coding scheme that makes it possible for Alice to communicate both reliably and securely. Reliability is measured in terms of Bob's probability of error in recovering the message, while security is measured in terms of the mutual information between the message and Eve's observations. Wyner showed that the situation is characterized by a single constant Cs, called the secrecy capacity, which has the following meaning: for all ε >; 0, there exist coding schemes of rate R ≥ Cs-ε that asymptotically achieve the reliability and security objectives. However, his proof of this result is based upon a random-coding argument. To date, despite consider able research effort, the only case where we know how to construct coding schemes that achieve secrecy capacity is when Eve's channel C2 is an erasure channel, or a combinatorial variation thereof. Polar codes were recently invented by Arikan; they approach the capacity of symmetric binary-input discrete memoryless channels with low encoding and decoding complexity. In this paper, we use polar codes to construct a coding scheme that achieves the secrecy capacity for a wide range of wiretap channels. Our construction works for any instantiation of the wiretap channel model, as long as both C1 and C2 are symmetric and binary-input, and C2 is degraded with respect to C1. Moreover, we show how to modify our construction in order to provide strong security, in the sense defined by Maurer, while still operating at a rate that approaches the secrecy capacity. In this case, we cannot guarantee that the reliability condition will also be satisfied unless the main channel C1 is noiseless, although we believe it can be always satisfied in practice.

Polar coding for secure transmission and key agreement

Abstract: Wyner's work on wiretap channels and the recent works on information theoretic security are based on random codes. Achieving information theoretical security with practical coding schemes is of definite interest. In this note, the attempt is to overcome this elusive task by employing the polar coding technique of Arikan. It is shown that polar codes achieve nontrivial perfect secrecy rates for binary-input degraded wiretap channels while enjoying their low encoding-decoding complexity. In the special case of symmetric main and eavesdropper channels, this coding technique achieves the secrecy capacity. Next, fading erasure wiretap channels are considered and a secret key agreement scheme is proposed, which requires only the statistical knowledge of the eavesdropper channel state information (CSI). The enabling factor is the creation of advantage over Eve, by blindly using the proposed scheme over each fading block, which is then exploited with privacy amplification techniques to generate secret keys.

Achieving secrecy capacity of the wiretap channel and broadcast channel with a confidential component

Abstract: We show that capacity of the general (not necessarily degraded or symmetric) wiretap channel under a “strong secrecy constraint” can be achieved using an explicit scheme based on polar codes. We also extend our construction to the case of broadcast channels with confidential messages defined by Csiszár and Körner, achieving the entire capacity region of this communication model. This submission is an extended abstract of the paper by the same authors (see arXiv:1410.3422).

Polar Codes for Physical Layer Security

For the strong security criterion, E. Sasoǧlu et al. design a multi-block polar coding
scheme {{cite:ec2d5ce}} which achieves both security and reliability for the same wiretap
channel model as in {{cite:1bccb8f}}. T. C. Gulcu et al. provide a simple coding scheme
based on polar codes to achieve the secrecy capacity of the general wiretap channel
(not necessarily degraded or symmetric) {{cite:d89fe56}}. This coding scheme is also extended to
achieve the capacity region of discrete memoryless BC-CM.
Independently, R. A. Chou et al. design a more general (holds for more general conditions as given in Fig. 1{{cite:c896a55}})
polar coding scheme for discrete memoryless BC-CM {{cite:c896a55}}. A brief summary of above work is given in Table {{table:7ed39cc9-356c-45b3-bf6e-c3d9868dcad7}} .

As indicated in Table {{table:7ed39cc9-356c-45b3-bf6e-c3d9868dcad7}} , most of current polar code designs require the
perfect channel knowledge of the eavesdropper at the transmitter to achieve the weak/strong secrecy.
How to extend the polar code designs to a more reasonable case where only channel
distribution knowledge of the eavesdropper is available at the transmitter is an
important research issue. This point also applies for lattice code designs.


Propose efficient schemes to combat the pilot contamination attack

Adjust the lengths
of the secrecy key based on the estimated information leakage

Secure Massive MIMO Transmission with an Active Eavesdropper

Abstract: In this paper, we investigate secure and reliable transmission strategies for multi-cell multi-user massive multiple-input multiple-output (MIMO) systems with a multi-antenna active eavesdropper. We consider a time-division duplex system where uplink training is required and an active eavesdropper can attack the training phase to cause pilot contamination at the transmitter. This forces the precoder used in the subsequent downlink transmission phase to implicitly beamform towards the eavesdropper, thus increasing its received signal power. Assuming matched filter precoding and artificial noise (AN) generation at the transmitter, we derive an asymptotic achievable secrecy rate when the number of transmit antennas approaches infinity. For the case of a single-antenna active eavesdropper, we obtain a closed-form expression for the optimal power allocation policy for the transmit signal and the AN, and find the minimum transmit power required to ensure reliable secure communication. Furthermore, we show that the transmit antenna correlation diversity of the intended users and the eavesdropper can be exploited in order to improve the secrecy rate. In fact, under certain orthogonality conditions of the channel covariance matrices, the secrecy rate loss introduced by the eavesdropper can be completely mitigated.

Secret Key Agreement with Large Antenna Arrays under the Pilot Contamination Attack

Abstract: We present a secret key agreement (SKA) protocol for a multi-user time-division duplex system where a base-station (BS) with a large antenna array (LAA) shares secret keys with users in the presence of non-colluding eavesdroppers. In the system, when the BS transmits random sequences to legitimate users for sharing common randomness, the eavesdroppers can attempt the pilot contamination attack (PCA) in which each of eavesdroppers transmits its target user's training sequence in hopes of acquiring possible information leak by steering beam towards the eavesdropper. We show that there exists a crucial complementary relation between the received signal strengths at the eavesdropper and its target user. This relation tells us that the eavesdropper inevitably leaves a trace that enables us to devise a way of measuring the amount of information leakage to the eavesdropper even if PCA parameters are unknown. To this end, we derive an estimator for the channel gain from the BS to the eavesdropper and propose a rate-adaptation scheme for adjusting the length of secret key under the PCA. Extensive analysis and evaluations are carried out under various setups, which show that the proposed scheme adequately takes advantage of the LAA to establish the secret keys under the PCA.

Secure Massive MIMO with Active Eavesdropper

This pilot contamination attack causes a serious secrecy threat to TDD-based massive MIMO systems.
On one hand, large antenna arrays beamforming leads to
the hardening of the channel, which prevents
the exploitation of channel fluctuations caused by
fading to improve the secrecy performance.
On the other hand, as illustrated in Figure {{figure:4fe4b004-a94d-4976-86a7-ac576385ecf4}} ,
the pilot contamination attack enables the transmitter to beamform
towards the the eavesdropper instead of the desired user.
If the eavesdropper's pilot power is sufficiently large, a positive secrecy rate
may not be achievable. This is significantly different from
the conventional idea that massive MIMO naturally facilitates secure communication
since the large antenna arrays can generate very narrow beams focusing on the desired users without
spilling over the signal power in other directions.
In the first time, Y. Wu et al. systematically analyze the secrecy threat caused by
the pilot contamination attack for multi-cell multi-user massive MIMO systems over correlated fading channels {{cite:52881db}}.
Then, a matched filter precoding and AN generation design
and a null space design are provided in {{cite:52881db}} to combat the pilot contamination attack for weakly correlated channels
and highly correlated channels, respectively.
A unified design which combines the matched filter precoding and AN generation design
and the null space design is also proposed.
Simulations indicate that these designs can guarantee
reliable secure communication under the pilot contamination attack, as shown in Fig. {{figure:3711f97a-7b70-4f4b-af26-38ca7f1bc3d9}} .
At the same time, Y. O. Basciftci et al. study
the pilot contamination attack problem for single-cell multi-user massive MIMO systems
over i.i.d. fading channels {{cite:d0c7122}}. It is proved in {{cite:d0c7122}} that if the pilot contamination attack
does not exist, the maximum secure degree of freedom (DoF) of massive MIMO systems
is the same as the maximum DoF of massive MIMO systems when the eavesdropper does not exist.
However, if the pilot contamination attack exists, the maximum secure DoF of massive MIMO systems
could be zero. To defense the pilot contamination attack,
Y. O. Basciftci et al. expand cardinality of the pilot signal set and hide the
pilot signal within the enlarged set. On the other hand,
S. Im et al. employ a secret key agreement
protocol for single-cell multi-user massive MIMO systems with
the pilot contamination attack {{cite:4f99ae9}}.
An estimator is designed at the BS side to evaluate the information leakage.
Then, the BS and the desired user perform the
reliable secure communication by adjusting the lengths
of the secrecy key based on the estimated information leakage.
A brief summary of above work is given in Table {{table:a787f104-c9a2-45ae-8130-6538e97b200b}} .


A transmitter, multiple users, and a
eavesdropper

Downlink multiple transmit antenna networks

Secrecy Sum Rate Maximization in Non-Orthogonal Multiple Access

Abstract: Non-orthogonal multiple access (NOMA) has been recognized as a promising technique for providing high data rates in 5G systems. This letter is to study physical layer security in a single-input single-output (SISO) NOMA system consisting of a transmitter, multiple legitimate users and an eavesdropper. The aim of this letter is to maximize the secrecy sum rate (SSR) of the NOMA system subject to the users' quality of service (QoS) requirements. We firstly identify the feasible region of the transmit power for satisfying all users' QoS requirements. Then we derive the closed-form expression of an optimal power allocation policy that maximizes the SSR. Numerical results are provided to show a significant SSR improvement by NOMA compared with conventional orthogonal multiple access (OMA).

Enhancing the Physical Layer Security of Non-Orthogonal Multiple Access in Large-Scale Networks

Abstract: This paper investigates the physical layer security of non-orthogonal multiple access (NOMA) in large-scale networks with invoking stochastic geometry. Both single-antenna and multiple-antenna aided transmission scenarios are considered, where the base station (BS) communicates with randomly distributed NOMA users. In the single-antenna scenario, we adopt a protected zone around the BS to establish an eavesdropper-exclusion area with the aid of careful channel ordering of the NOMA users. In the multiple-antenna scenario, artificial noise is generated at the BS for further improving the security of a beamforming-aided system. In order to characterize the secrecy performance, we derive new exact expressions of the security outage probability for both single-antenna and multiple-antenna aided scenarios. For the single-antenna scenario, we perform secrecy diversity order analysis of the selected user pair. The analytical results derived demonstrate that the secrecy diversity order is determined by the specific user having the worse channel condition among the selected user pair. For the multiple-antenna scenario, we derive the asymptotic secrecy outage probability, when the number of transmit antennas tends to infinity. Monte Carlo simulations are provided for verifying the analytical results derived and to show that: 1) the security performance of the NOMA networks can be improved by invoking the protected zone and by generating artificial noise at the BS and 2) the asymptotic secrecy outage probability is close to the exact secrecy outage probability.

On the Spectral Efficiency and Security Enhancements of NOMA Assisted Multicast-Unicast Streaming

Abstract: This paper considers the application of non-orthogonal multiple access (NOMA) to a multi-user network with mixed multicasting and unicasting traffic. The proposed design of beamforming and power allocation ensures that the unicasting performance is improved while maintaining the reception reliability of multicasting. Both analytical and simulation results are provided to demonstrate that the use of the NOMA assisted multicast-unicast scheme yields a significant improvement in spectral efficiency compared with orthogonal multiple access (OMA) schemes which realize multicasting and unicasting services separately. Since unicast messages are broadcast to all the users, how the use of NOMA can prevent those multicast receivers intercepting the unicasting messages is also investigated, where it is shown that the secrecy unicasting rate achieved by NOMA is always larger than or equal to that of OMA. Simulation results are provided to verify the developed analytical results and demonstrate the superior performance of the proposed NOMA scheme.

Physical Layer Security of NOMA

Y. Zhang et al. first study the secure NOMA transmission
for a single-antenna, one transmitter, multiple users, and one
eavesdropper system with perfect CSI of the users and no CSI of the eavesdropper {{cite:9892201}}.
Perfect successive interference cancellation (SIC) is performed at each user.
The closed-formed expression of the optimal power allocation policy which maximizes the secrecy sum rate under each user's QoS constraint
is derived. For the same CSI assumption, Y. Liu et al. further investigate
the secure transmission for NOMA networks {{cite:5123788}}, as shown in Fig. {{figure:acf9bd74-41e6-4b1a-981b-25925f2e07e3}} .
An eavesdropper-exclusion zone is established.
To reduce the SIC complexity at the receiver, a user paring scheme
is employed, where one user in the internal zone and one user in the external
zone are allocated the same resource slot. When the base station only has
single antenna, the secrecy outage probability is analyzed. Moreover,
the secrecy diversity order is obtained, which reveals that the user
with the weaker channel in the pair determines the secrecy diversity order.
When the base station is equipped with multiple antennas,
a matched filter precoding and AN generation design is employed to further increase
the secrecy performance. Based on this design, both the exact and asymptotic (in large system limit)
secrecy outage probability expressions are derived.
With perfect CSI and perfect SIC assumption, Z. Ding et al. consider a NOMA network with both multicasting and
unicasting transmissions {{cite:21fec99}}. For the unicasting transmission, it it proved that the secrecy rate for NOMA
is no less than that for orthogonal multiple access in the high SNR regime. Also, the
secrecy outage probability is studied. A brief summary of above work is given in Table {{table:4ac90879-c41c-4617-beee-5c4dcbdcc942}} .


A multiple-antenna eavesdropper, imperfect SIC

Improving Physical Layer Secrecy Using Full-Duplex Jamming Receivers

Abstract: This paper studies secrecy rate optimization in a wireless network with a single-antenna source, a multi-antenna destination and a multi-antenna eavesdropper. This is an unfavorable scenario for secrecy performance as the system is interference-limited. In the literature, assuming that the receiver operates in half duplex (HD) mode, the aforementioned problem has been addressed via use of cooperating nodes who act as jammers to confound the eavesdropper. This paper investigates an alternative solution, which assumes the availability of a full duplex (FD) receiver. In particular, while receiving data, the receiver transmits jamming noise to degrade the eavesdropper channel. The proposed self-protection scheme eliminates the need for external helpers and provides system robustness. For the case in which global channel state information is available, we aim to design the optimal jamming covariance matrix that maximizes the secrecy rate and mitigates loop interference associated with the FD operation. We consider both fixed and optimal linear receiver design at the destination, and show that the optimal jamming covariance matrix is rank-1, and can be found via an efficient 1-D search. For the case in which only statistical information on the eavesdropper channel is available, the optimal power allocation is studied in terms of ergodic and outage secrecy rates. Simulation results verify the analysis and demonstrate substantial performance gain over conventional HD operation at the destination.

Exploiting Full-Duplex Receivers for Achieving Secret Communications in Multiuser MISO Networks

Abstract: We consider a broadcast channel in which a multi-antenna transmitter (Alice) sends <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> confidential information signals to <inline-formula> <tex-math notation="LaTeX">$K$ </tex-math></inline-formula> legitimate users (Bobs) in the presence of <inline-formula> <tex-math notation="LaTeX">$L$ </tex-math></inline-formula> eavesdroppers (Eves). Alice uses multiple-input multiple-output (MIMO) precoding to generate the information signals along with her own (Tx-based) friendly jamming (FJ). Interference at each Bob is removed by MIMO zero-forcing. This, however, leaves a “vulnerability region” around each Bob, which can be exploited by a nearby Eve. We address this problem by augmenting Tx-based FJ (TxFJ) with Rx-based FJ (RxFJ), generated by each Bob. Specifically, each Bob uses self-interference suppression to transmit a friendly jamming signal, while simultaneously receiving an information signal over the same channel. We minimize the powers allocated to the information, TxFJ, and RxFJ signals under given guarantees on the individual secrecy rate for each Bob. The problem is solved for the cases when the eavesdropper’s channel state information is known/unknown. Simulations show the effectiveness of the proposed solution. Furthermore, we discuss how to schedule transmissions when the rate requirements need to be satisfied on average rather than instantaneously. Under special cases, a scheduling algorithm that serves only the strongest receivers is shown to outperform the one that schedules all receivers.

Physical Layer Security in Wireless Ad Hoc Networks Under A Hybrid Full-/Half-Duplex Receiver Deployment Strategy

Abstract: This paper studies physical layer security in a wireless ad hoc network with numerous legitimate transmitter-receiver pairs and eavesdroppers. A hybrid full-/half-duplex receiver deployment strategy is proposed to secure legitimate transmissions, by letting a fraction of legitimate receivers work in the full-duplex (FD) mode sending jamming signals to confuse eavesdroppers upon their information receptions, and letting the other receivers work in the half-duplex mode just receiving their desired signals. The objective of this paper is to choose properly the fraction of FD receivers for achieving the optimal network security performance. Both accurate expressions and tractable approximations for the connection outage probability and the secrecy outage probability of an arbitrary legitimate link are derived, based on which the area secure link number, network-wide secrecy throughput and network-wide secrecy energy efficiency are optimized respectively. Various insights into the optimal fraction are further developed and its closed-form expressions are also derived under perfect self-interference cancellation or in a dense network. It is concluded that the fraction of FD receivers triggers a non-trivial trade-off between reliability and secrecy, and the proposed strategy can significantly enhance the network security performance.

Safeguarding Decentralized Wireless Networks Using Full-Duplex Jamming Receivers

Abstract: In this paper, we study the benefits of full-duplex (FD) receiver jamming in enhancing the physical-layer security of a two-tier decentralized wireless network with each tier deployed with a large number of pairs of a single-antenna transmitter and a multi-antenna receiver. In the underlying tier, the transmitter sends unclassified information and the receiver works in the half-duplex (HD) mode receiving the desired signal. In the overlaid tier, the transmitter delivers confidential information in the presence of randomly located eavesdroppers, and the receiver works in the FD mode radiating jamming signals to confuse eavesdroppers and receiving the desired signal simultaneously. We provide a comprehensive performance analysis and network design under a stochastic geometry framework. Specifically, we consider the scenarios where each FD receiver uses single- and multi-antenna jamming, and analyze the connection probability and the secrecy outage probability of a typical FD receiver by deriving accurate expressions and more tractable approximations for the two probabilities. We also determine the optimal deployment density of the FD-mode tier to maximize the network-wide secrecy throughput subject to constraints including the given dual probabilities and the network-wide throughput of the HD-mode tier. Numerical results are demonstrated to verify our theoretical findings, and show that the network-wide secrecy throughput is significantly improved by properly deploying the FD-mode tier.

Secure Communications with Full Duplex Receivers

For a decentralized heterogeneous network which includes a half duplex receiver tier and a full duplex receiver tier,
T.-X. Zheng et al. derive the secrecy outage probability of a typical full duplex receiver
based on the stochastic geometry framework {{cite:ed992ff}}. In addition, the deployment of the full duplex receivers
is optimized for the network-wide secrecy throughput maximization.
T. Zhang et al. design the secrecy communication schemes for a cognitive wiretap channel
with a multiple antenna full duplex secondary receiver {{cite:e2eef5a}}.
A brief summary of above work is given in Table {{table:8486faab-0d9a-48fe-ba88-898f51c9adad}} .


Neural Machine Translation by Jointly Learning to Align and Translate

Abstract: Neural machine translation is a recently proposed approach to machine translation. Unlike the traditional statistical machine translation, the neural machine translation aims at building a single neural network that can be jointly tuned to maximize the translation performance. The models proposed recently for neural machine translation often belong to a family of encoder-decoders and consists of an encoder that encodes a source sentence into a fixed-length vector from which a decoder generates a translation. In this paper, we conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and propose to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. With this new approach, we achieve a translation performance comparable to the existing state-of-the-art phrase-based system on the task of English-to-French translation. Furthermore, qualitative analysis reveals that the (soft-)alignments found by the model agree well with our intuition.

DiSAN: Directional Self-Attention Network for RNN/CNN-Free Language Understanding

Abstract: Recurrent neural nets (RNN) and convolutional neural nets (CNN) are widely used on NLP tasks to capture the long-term and local dependencies, respectively. Attention mechanisms have recently attracted enormous interest due to their highly parallelizable computation, significantly less training time, and flexibility in modeling dependencies. We propose a novel attention mechanism in which the attention between elements from input sequence(s) is directional and multi-dimensional (i.e., feature-wise). A light-weight neural net, "Directional Self-Attention Network (DiSAN)", is then proposed to learn sentence embedding, based solely on the proposed attention without any RNN/CNN structure. DiSAN is only composed of a directional self-attention with temporal order encoded, followed by a multi-dimensional attention that compresses the sequence into a vector representation. Despite its simple form, DiSAN outperforms complicated RNN models on both prediction quality and time efficiency. It achieves the best test accuracy among all sentence encoding methods and improves the most recent best result by 1.02% on the Stanford Natural Language Inference (SNLI) dataset, and shows state-of-the-art test accuracy on the Stanford Sentiment Treebank (SST), Multi-Genre natural language inference (MultiNLI), Sentences Involving Compositional Knowledge (SICK), Customer Review, MPQA, TREC question-type classification and Subjectivity (SUBJ) datasets.

Reinforced Mnemonic Reader for Machine Reading Comprehension

Abstract: In this paper, we introduce the Reinforced Mnemonic Reader for machine reading comprehension tasks, which enhances previous attentive readers in two aspects. First, a reattention mechanism is proposed to refine current attentions by directly accessing to past attentions that are temporally memorized in a multi-round alignment architecture, so as to avoid the problems of attention redundancy and attention deficiency. Second, a new optimization approach, called dynamic-critical reinforcement learning, is introduced to extend the standard supervised method. It always encourages to predict a more acceptable answer so as to address the convergence suppression problem occurred in traditional reinforcement learning algorithms. Extensive experiments on the Stanford Question Answering Dataset (SQuAD) show that our model achieves state-of-the-art results. Meanwhile, our model outperforms previous systems by over 6% in terms of both Exact Match and F1 metrics on two adversarial SQuAD datasets.

Attention is All you Need

Abstract: The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.8 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention

Abstract: In this paper, we proposed a sentence encoding-based model for recognizing text entailment. In our approach, the encoding of sentence is a two-stage process. Firstly, average pooling was used over word-level bidirectional LSTM (biLSTM) to generate a first-stage sentence representation. Secondly, attention mechanism was employed to replace average pooling on the same sentence for better representations. Instead of using target sentence to attend words in source sentence, we utilized the sentence's first-stage representation to attend words appeared in itself, which is called "Inner-Attention" in our paper . Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus has proved the effectiveness of "Inner-Attention" mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin.

A summary of various soft attention mechanisms.

We summarize the widely-used attention mechanisms via categorizing them into different classes, which are demonstrated in Table {{table:942e99c2-bcd6-445d-ac61-b07fea3fb6d2}} .
By the purposes, the soft attention mechanisms could be coarsely categorized into two types: 1) context fusion and 2) sequence compression.
By the query sources, the mechanisms could also be categorized into two types: 1) exterior (query from arbitrary source) and 2) interior (query from sequence itself).
By the compatibility functions, the mechanisms could also be categorized into two types: 1) additive or multi-layer perceptron attention and 2) multiplicative or dot-product attention.


Learning Scheduling Algorithms for Data Processing Clusters

Abstract: Efficiently scheduling data processing jobs on distributed compute clusters requires complex algorithms. Current systems, however, use simple generalized heuristics and ignore workload characteristics, since developing and tuning a scheduling policy for each workload is infeasible. In this paper, we show that modern machine learning techniques can generate highly-efficient policies automatically. Decima uses reinforcement learning (RL) and neural networks to learn workload-specific scheduling algorithms without any human instruction beyond a high-level objective such as minimizing average job completion time. Off-the-shelf RL techniques, however, cannot handle the complexity and scale of the scheduling problem. To build Decima, we had to develop new representations for jobs' dependency graphs, design scalable RL models, and invent RL training methods for dealing with continuous stochastic job arrivals. Our prototype integration with Spark on a 25-node cluster shows that Decima improves the average job completion time over hand-tuned scheduling heuristics by at least 21%, achieving up to 2x improvement during periods of high cluster load.

Unveiling the potential of Graph Neural Networks for network modeling and optimization in SDN

Abstract: Network modeling is a critical component for building self-driving Software-Defined Networks, particularly to find optimal routing schemes that meet the goals set by administrators. However, existing modeling techniques do not meet the requirements to provide accurate estimations of relevant performance metrics such as delay and jitter. In this paper we propose a novel Graph Neural Network (GNN) model able to understand the complex relationship between topology, routing and input traffic to produce accurate estimates of the per-source/destination pair mean delay and jitter. GNN are tailored to learn and model information structured as graphs and as a result, our model is able to generalize over arbitrary topologies, routing schemes and variable traffic intensity. In the paper we show that our model provides accurate estimates of delay and jitter (worst case R2 = 0.86) when testing against topologies, routing and traffic not seen during training. In addition, we present the potential of the model for network operation by presenting several use-cases that show its effective use in per-source/destination pair delay/jitter routing optimization and its generalization capabilities by reasoning in topologies and routing schemes not seen during training.

Two categories of DL-based networked systems.

We first categorize current DL-based networked systems into local control systems (LCSes) and global control systems (GCSes) based on their observation horizon. We present several examples in both categories in Table {{table:a91c2634-4126-4fc3-a39f-47f0bee85f8c}} .


Learning Deterministic Policy with Target for Power Control in Wireless Networks

Abstract: Inter-Cell Interference Coordination (ICIC) is a promising way to improve energy efficiency in wireless networks, especially where small base stations are densely deployed. However, traditional optimization based ICIC schemes suffer from severe performance degradation with complex interference pattern. To address this issue, we propose a Deep Reinforcement Learning with Deterministic Policy and Target (DRL-DPT) framework for ICIC in wireless networks. DRL- DPT overcomes the main obstacles in applying reinforcement learning and deep learning in wireless networks, i.e. continuous state space, continuous action space and convergence. Firstly, a Deep Neural Network (DNN) is involved as the actor to obtain deterministic power control actions in continuous space. Then, to guarantee the convergence, an online training process is presented, which makes use of a dedicated reward function as the target rule and a policy gradient descent algorithm to adjust DNN weights. Experimental results show that the proposed DRL-DPT framework consistently outperforms existing schemes in terms of energy efficiency and throughput under different wireless interference scenarios. More specifically, it improves up to 15% of energy efficiency with faster convergence rate.

Handover Control in Wireless Systems via Asynchronous Multi-User Deep Reinforcement Learning

Abstract: In this paper, we propose a two-layer framework to learn the optimal handover (HO) controllers in possibly large-scale wireless systems supporting mobile Internet-of-Things (IoT) users or traditional cellular users, where the user mobility patterns could be heterogeneous. In particular, our proposed framework first partitions the user equipments (UEs) with different mobility patterns into clusters, where the mobility patterns are similar in the same cluster. Then, within each cluster, an asynchronous multi-user deep reinforcement learning scheme is developed to control the HO processes across the UEs in each cluster, in the goal of lowering the HO rate while ensuring certain system throughput. In this scheme, we use a deep neural network (DNN) as an HO controller learned by each UE via reinforcement learning in a collaborative fashion. Moreover, we use supervised learning in initializing the DNN controller before the execution of reinforcement learning to exploit what we already know with traditional HO schemes and to mitigate the negative effects of random exploration at the initial stage. Furthermore, we show that the adopted global-parameter-based asynchronous framework enables us to train faster with more UEs, which could nicely address the scalability issue to support large systems. Finally, simulation results demonstrate that the proposed framework can achieve better performance than the state-of-art on-line schemes, in terms of HO rates.

Deep Reinforcement Learning for Resource Management in Network Slicing

Abstract: Network slicing is born as an emerging business to operators by allowing them to sell the customized slices to various tenants at different prices. In order to provide better-performing and cost-efficient services, network slicing involves challenging technical issues and urgently looks forward to intelligent innovations to make the resource management consistent with users’ activities per slice. In that regard, deep reinforcement learning (DRL), which focuses on how to interact with the environment by trying alternative actions and reinforcing the tendency actions producing more rewarding consequences, is assumed to be a promising solution. In this paper, after briefly reviewing the fundamental concepts of DRL, we investigate the application of DRL in solving some typical resource management for network slicing scenarios, which include radio resource slicing and priority-based core network slicing, and demonstrate the advantage of DRL over several competing schemes through extensive simulations. Finally, we also discuss the possible challenges to apply DRL in network slicing from a general perspective.

Related work in using DRL for resource management.

The detailed mapping of DRL on RRM depends on specific problems and scenarios. A brief summary covering typical RRM issues is presented in Table {{table:1c42a641-9ab6-4ae2-bfef-fa5c7b0e5a1d}} . As we have mentioned in Section I, the intensive computation complexity and energy consumption could hinder the application of DRL in future wireless networks. In addition, the features of RRM and networks are not fully investigated.


RoBERTa: A Robustly Optimized BERT Pretraining Approach

Abstract: Language model pretraining has led to significant performance gains but careful comparison between different approaches is challenging. Training is computationally expensive, often done on private datasets of different sizes, and, as we will show, hyperparameter choices have significant impact on the final results. We present a replication study of BERT pretraining (Devlin et al., 2019) that carefully measures the impact of many key hyperparameters and training data size. We find that BERT was significantly undertrained, and can match or exceed the performance of every model published after it. Our best model achieves state-of-the-art results on GLUE, RACE and SQuAD. These results highlight the importance of previously overlooked design choices, and raise questions about the source of recently reported improvements. We release our models and code.

DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter

Abstract: As Transfer Learning from large-scale pre-trained models becomes more prevalent in Natural Language Processing (NLP), operating these large models in on-the-edge and/or under constrained computational training or inference budgets remains challenging. In this work, we propose a method to pre-train a smaller general-purpose language representation model, called DistilBERT, which can then be fine-tuned with good performances on a wide range of tasks like its larger counterparts. While most prior work investigated the use of distillation for building task-specific models, we leverage knowledge distillation during the pre-training phase and show that it is possible to reduce the size of a BERT model by 40%, while retaining 97% of its language understanding capabilities and being 60% faster. To leverage the inductive biases learned by larger models during pre-training, we introduce a triple loss combining language modeling, distillation and cosine-distance losses. Our smaller, faster and lighter model is cheaper to pre-train and we demonstrate its capabilities for on-device computations in a proof-of-concept experiment and a comparative on-device study.

Longformer: The Long-Document Transformer

Abstract: Transformer-based models are unable to process long sequences due to their self-attention operation, which scales quadratically with the sequence length. To address this limitation, we introduce the Longformer with an attention mechanism that scales linearly with sequence length, making it easy to process documents of thousands of tokens or longer. Longformer's attention mechanism is a drop-in replacement for the standard self-attention and combines a local windowed attention with a task motivated global attention. Following prior work on long-sequence transformers, we evaluate Longformer on character-level language modeling and achieve state-of-the-art results on text8 and enwik8. In contrast to most prior work, we also pretrain Longformer and finetune it on a variety of downstream tasks. Our pretrained Longformer consistently outperforms RoBERTa on long document tasks and sets new state-of-the-art results on WikiHop and TriviaQA. We finally introduce the Longformer-Encoder-Decoder (LED), a Longformer variant for supporting long document generative sequence-to-sequence tasks, and demonstrate its effectiveness on the arXiv summarization dataset.

ClimateBert: A Pretrained Language Model for Climate-Related Text

Abstract: Over the recent years, large pretrained language models (LM) have revolutionized the field of natural language processing (NLP). However, while pretraining on general language has been shown to work very well for common language, it has been observed that niche language poses problems. In particular, climate-related texts include specific language that common LMs can not represent accurately. We argue that this shortcoming of today's LMs limits the applicability of modern NLP to the broad field of text processing of climate-related texts. As a remedy, we propose CLIMATEBERT, a transformer-based language model that is further pretrained on over 2 million paragraphs of climate-related texts, crawled from various sources such as common news, research articles, and climate reporting of companies. We find that CLIMATEBERT leads to a 48% improvement on a masked language model objective which, in turn, leads to lowering error rates by 3.57% to 35.71% for various climate-related downstream tasks like text classification, sentiment analysis, and fact-checking.

Pretrained Transformer Language Models used for Classification tasks

Next, we are going to describe the various baselines and models that we use to conduct experiments using the datasets described above.
Most tasks are classification tasks that require in-domain finetuning.
For the text classification tasks in ClimaBench, we examine Transformer-based {{cite:f13a97f}} pre-trained language models like BERT {{cite:891b142}}, RoBERTa {{cite:c5c1c8f}}, distilled versions like DistilRoBERTa {{cite:736dcad}}, longer context models like Longformer {{cite:95840b3}}, and domain specific models like ClimateBERT {{cite:b8779a0}} and SciBERT {{cite:e50447a}}.
This helps us contrast the effects of model architecture, input length and in-domain pretraining on downstream tasks.
We provide more details about models in Appendix Section REF  and Table {{table:84558058-29db-4337-9b06-8c649790381b}} .
For a baseline, we consider a linear kernel Support Vector Machine (SVM) trained using TF-IDF transformed n-gram (1,2,3-gram) features.
We also include a simple Majority and Random class voting baselines.


Questions automatically generated from COCO captions

Synthetic question generation using relations

Counting objects on varying complexities

Questions demanding reasoning about text

A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Input

Abstract: We propose a method for automatically answering questions about images by bringing together recent advances from natural language processing and computer vision. We combine discrete reasoning with uncertain predictions by a multi-world approach that represents uncertainty about the perceived world in a bayesian framework. Our approach can handle human questions of high complexity about realistic scenes and replies with range of answer like counts, object classes, instances and lists of them. The system is directly trained from question-answer pairs. We establish a first benchmark for this task that can be seen as a modern attempt at a visual turing test.

Exploring Models and Data for Image Question Answering

Abstract: This work aims to address the problem of image-based question-answering (QA) with new models and datasets. In our work, we propose to use neural networks and visual semantic embeddings, without intermediate stages such as object detection and image segmentation, to predict answers to simple questions about images. Our model performs 1.8 times better than the only published results on an existing image QA dataset. We also present a question generation algorithm that converts image descriptions, which are widely available, into QA form. We used this algorithm to produce an order-of-magnitude larger dataset, with more evenly distributed answers. A suite of baseline results on this new dataset are also presented.

Visual Madlibs: Fill in the blank Image Generation and Question Answering

Abstract: In this paper, we introduce a new dataset consisting of 360,001 focused natural language descriptions for 10,738 images. This dataset, the Visual Madlibs dataset, is collected using automatically produced fill-in-the-blank templates designed to gather targeted descriptions about: people and objects, their appearances, activities, and interactions, as well as inferences about the general scene or its broader context. We provide several analyses of the Visual Madlibs dataset and demonstrate its applicability to two new description generation tasks: focused description generation, and multiple-choice question-answering for images. Experiments using joint-embedding and deep learning methods show promising results on these tasks.

Visual7W: Grounded Question Answering in Images

Abstract: We have seen great progress in basic perceptual tasks such as object recognition and detection. However, AI models still fail to match humans in high-level vision tasks due to the lack of capacities for deeper reasoning. Recently the new task of visual question answering (QA) has been proposed to evaluate a model's capacity for deep image understanding. Previous works have established a loose, global association between QA sentences and images. However, many questions and answers, in practice, relate to local regions in the images. We establish a semantic link between textual descriptions and image regions by object-level grounding. It enables a new type of QA with visual answers, in addition to textual answers used in previous work. We study the visual QA tasks in a grounded setting with a large collection of 7W multiple-choice QA pairs. Furthermore, we evaluate human performance and several baseline models on the QA tasks. Finally, we propose a novel LSTM model with spatial attention to tackle the 7W QA tasks.

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

Abstract: When building artificial intelligence systems that can reason and answer questions about visual data, we need diagnostic tests to analyze our progress and discover shortcomings. Existing benchmarks for visual question answering can help, but have strong biases that models can exploit to correctly answer questions without reasoning. They also conflate multiple sources of error, making it hard to pinpoint model weaknesses. We present a diagnostic dataset that tests a range of visual reasoning abilities. It contains minimal biases and has detailed annotations describing the kind of reasoning each question requires. We use this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.

TallyQA: Answering Complex Counting Questions

Abstract: Most counting questions in visual question answering (VQA) datasets are simple and require no more than object detection. Here, we study algorithms for complex counting questions that involve relationships between objects, attribute identification, reasoning, and more. To do this, we created TallyQA, the world's largest dataset for open-ended counting. We propose a new algorithm for counting that uses relation networks with region proposals. Our method lets relation networks be efficiently used with high-resolution imagery. It yields state-of-the-art results compared to baseline and recent systems on both TallyQA and the HowMany-QA benchmark.

Towards VQA Models That Can Read

Abstract: Studies have shown that a dominant class of questions asked by visually impaired users on images of their surroundings involves reading text in the image. But today’s VQA models can not read! Our paper takes a first step towards addressing this problem. First, we introduce a new “TextVQA” dataset to facilitate progress on this important problem. Existing datasets either have a small proportion of questions about text (e.g., the VQA dataset) or are too small (e.g., the VizWiz dataset). TextVQA contains 45,336 questions on 28,408 images that require reasoning about text to answer. Second, we introduce a novel model architecture that reads text in the image, reasons about it in the context of the image and the question, and predicts an answer which might be a deduction based on the text and the image or composed of the strings found in the image. Consequently, we call our approach Look, Read, Reason & Answer (LoRRA). We show that LoRRA outperforms existing state-of-the-art VQA models on our TextVQA dataset. We find that the gap between human performance and machine performance is significantly larger on TextVQA than on VQA 2.0, suggesting that TextVQA is well-suited to benchmark progress along directions complementary to VQA 2.0.

Overview of VQA datasets described in this paper.

Several VQA datasets have been introduced {{cite:471b98c}}, {{cite:8aa2b02}}, {{cite:0aabd63}}.
In these datasets, images are either taken from an existing vision dataset (e.g. MSCOCO; {{cite:8b3bef2}}) or are artificially created (e.g. Abstract Scenes; {{cite:8adffd4}}, computer graphics; {{cite:9fe8ebd}}, {{cite:f51cd5d}}). Further, questions are generated either automatically {{cite:9fe8ebd}}, {{cite:f51cd5d}}, {{cite:8aa2b02}}, {{cite:fb21f13}}, {{cite:15eb4b5}}, {{cite:2ae2a4b}}, from crowd workers {{cite:8adffd4}}, {{cite:a1f8fe2}}, {{cite:7541872}}, {{cite:8aa2b02}}, {{cite:9696125}}, {{cite:fbc0777}}, or from in-house participants {{cite:8aa2b02}}, {{cite:dfe524e}}. Unlike these datasets, questions in our dataset are carefully constructed by experts such that to answer them a detailed inspection of the image is necessary. Some prominent VQA datasets are listed in Table {{table:1974bffd-e480-4eba-8586-2ad4f0ad6c5a}} . Relevant ones to our work are described next.


Graph evolution: Densification and shrinking diameters

Abstract:
How do real graphs evolve over time? What are normal growth patterns in social, technological, and information networks? Many studies have discovered patterns in static graphs, identifying properties in a single snapshot of a large network or in a very small number of snapshots; these include heavy tails for in- and out-degree distributions, communities, small-world phenomena, and others. However, given the lack of information about network evolution over long periods, it has been hard to convert these findings into statements about trends over time.
 Here we study a wide range of real graphs, and we observe some surprising phenomena. First, most of these graphs densify over time with the number of edges growing superlinearly in the number of nodes. Second, the average distance between nodes often shrinks over time in contrast to the conventional wisdom that such distance parameters should increase slowly as a function of the number of nodes (like O(log n) or O(log(log n)).
 Existing graph generation models do not exhibit these types of behavior even at a qualitative level. We provide a new graph generator, based on a forest fire spreading process that has a simple, intuitive justification, requires very few parameters (like the flammability of nodes), and produces graphs exhibiting the full range of properties observed both in prior work and in the present study.
 We also notice that the forest fire model exhibits a sharp transition between sparse graphs and graphs that are densifying. Graphs with decreasing distance between the nodes are generated around this transition point.
 Last, we analyze the connection between the temporal evolution of the degree distribution and densification of a graph. We find that the two are fundamentally related. We also observe that real networks exhibit this type of relation between densification and the degree distribution.

Multi-scale Attributed Node Embedding

Abstract: We present network embedding algorithms that capture information about a node from the local distribution over node attributes around it, as observed over random walks following an approach similar to Skip-gram. Observations from neighbourhoods of different sizes are either pooled (AE) or encoded distinctly in a multi-scale approach (MUSAE). Capturing attribute-neighbourhood relationships over multiple scales is useful for a range of applications, including latent feature identification across disconnected networks with similar features. We prove theoretically that matrices of node-feature pointwise mutual information are implicitly factorized by the embeddings. Experiments show that our algorithms are computationally efficient and outperform comparable models on social networks and web graphs.

Defining and Evaluating Network Communities based on Ground-truth

Abstract: Nodes in real-world networks organize into densely linked communities where edges appear with high concentration among the members of the community. Identifying such communities of nodes has proven to be a challenging task mainly due to a plethora of definitions of a community, intractability of algorithms, issues with evaluation and the lack of a reliable gold-standard ground-truth. In this paper we study a set of 230 large real-world social, collaboration and information networks where nodes explicitly state their group memberships. For example, in social networks nodes explicitly join various interest based social groups. We use such groups to define a reliable and robust notion of ground-truth communities. We then propose a methodology which allows us to compare and quantitatively evaluate how different structural definitions of network communities correspond to ground-truth communities. We choose 13 commonly used structural definitions of network communities and examine their sensitivity, robustness and performance in identifying the ground-truth. We show that the 13 structural definitions are heavily correlated and naturally group into four classes. We find that two of these definitions, Conductance and Triad-participation-ratio, consistently give the best performance in identifying ground-truth communities. We also investigate a task of detecting communities given a single seed node. We extend the local spectral clustering algorithm into a heuristic parameter-free community detection method that easily scales to networks with more than hundred million nodes. The proposed method achieves 30% relative improvement over current local clustering methods.

Description of the datasets.

To evaluate the proposed method, we used 7 graphs which are formed by user-user interactions. These datasets are described in Table {{table:116fa5d3-23f1-41cd-99f9-9bffb91c3ca7}}  and are accessible through the SNAP {{cite:59db0d8}}. The smallest dataset consists of 22 470 nodes with 171 002 edges and the largest one consists of 317 080 nodes with 1 049 866 edges. In all these graphs, we consider the existing edges as positively signed and non-existing edges as negatively signed.


Material-specific high-resolution table-top extreme ultraviolet microscopy

Abstract: Microscopy with extreme ultraviolet (EUV) radiation holds promise for high-resolution imaging with excellent material contrast, due to the short wavelength and numerous element-specific absorption edges available in this spectral range. At the same time, EUV radiation has significantly larger penetration depths than electrons. It thus enables a nano-scale view into complex three-dimensional structures that are important for material science, semiconductor metrology, and next-generation nano-devices. Here, we present high-resolution and material-specific microscopy at 13.5 nm wavelength. We combine a highly stable, high photon-flux, table-top EUV source with an interferometrically stabilized ptychography setup. By utilizing structured EUV illumination, we overcome the limitations of conventional EUV focusing optics and demonstrate high-resolution microscopy at a half-pitch lateral resolution of 16 nm. Moreover, we propose mixed-state orthogonal probe relaxation ptychography, enabling robust phase-contrast imaging over wide fields of view and long acquisition times. In this way, the complex transmission of an integrated circuit is precisely reconstructed, allowing for the classification of the material composition of mesoscopic semiconductor systems.

Addressing phase-curvature in Fourier ptychography

Abstract: In Fourier ptychography, multiple low resolution images are captured and subsequently combined computationally into a high-resolution, large-field of view micrograph. A theoretical image-formation model based on the assumption of plane-wave illumination from various directions is commonly used, to stitch together the captured information into a high synthetic aperture. The underlying far-field (Fraunhofer) diffraction assumption connects the source, sample, and pupil planes by Fourier transforms. While computationally simple, this assumption neglects phase-curvature due to non-planar illumination from point sources as well as phase-curvature from finite-conjugate microscopes (e.g., using a single-lens for image-formation). We describe a simple, efficient, and accurate extension of Fourier ptychography by embedding the effect of phase-curvature into the underlying forward model. With the improved forward model proposed here, quantitative phase reconstruction is possible even for wide fields-of-views and without the need of image segmentation. Lastly, the proposed method is computationally efficient, requiring only two multiplications: prior and following the reconstruction.

High-speed multi-objective Fourier ptychographic microscopy.

Abstract: The ability of a microscope to rapidly acquire wide-field, high-resolution images is limited by both the optical performance of the microscope objective and the bandwidth of the detector. The use of multiple detectors can increase electronic-acquisition bandwidth, but the use of multiple parallel objectives is problematic since phase coherence is required across the multiple apertures. We report a new synthetic-aperture microscopy technique based on Fourier ptychography, where both the illumination and image-space numerical apertures are synthesized, using a spherical array of low-power microscope objectives that focus images onto mutually incoherent detectors. Phase coherence across apertures is achieved by capturing diffracted fields during angular illumination and using ptychographic reconstruction to synthesize wide-field, high-resolution, amplitude and phase images. Compared to conventional Fourier ptychography, the use of multiple objectives reduces image acquisition times by increasing the area for sampling the diffracted field. We demonstrate the proposed scaleable architecture with a nine-objective microscope that generates an 89-megapixel, 1.1 µm resolution image nine-times faster than can be achieved with a single-objective Fourier-ptychographic microscope. New calibration procedures and reconstruction algorithms enable the use of low-cost 3D-printed components for longitudinal biological sample imaging. Our technique offers a route to high-speed, gigapixel microscopy, for example, imaging the dynamics of large numbers of cells at scales ranging from sub-micron to centimetre, with an enhanced possibility to capture rare phenomena.

Overview of open data sets and PtyLab tutorials.

We publish a variety of CP and FP data sets and tutorials with the aim to introduce users to the functionality of PtyLab. Figure {{figure:b7265873-dba4-44e0-9eff-b9d6b6b7a9cd}}  depicts two such data sets. The top row shows a soft x-ray ({{formula:c50dd5b0-8911-4f6c-bdb1-60db17676bc8}} nm) data set collected at a synchrotron (experimental details in {{cite:36528ff}}). The bottom row depicts a visible light ({{formula:2f4d9952-b79d-4bee-9ff6-29419fbe0cb7}} nm) FP data set of lung carcinoma. For both data sets we show from left to right a single frame of the raw data, the recovered quantitative phase image (QPI) of the object (resolution test target and lung carcinoma histology slide), and reconstructed probe/pupil for the case of CP (top) and FP (bottom). Hue and brightness depict the phase and amplitude, respectively, of the complex-valued reconstructed quantities. A variety of additional data sets are published alongside PtyLab, which are summarized in table {{table:47b97eb1-4e40-4e38-86f6-32207472aae7}} . Each of these data sets comes with an online tutorial explaining suitable data analysis approaches, including self-calibration and regularization.


Image Classification, Text Prediction and Sentiment Analysis

Image Classification and Word Prediction

Image Classification and Breast Cancer Detection

Data Poisoning Attacks Against Federated Learning Systems

Abstract: Federated learning (FL) is an emerging paradigm for distributed training of large-scale deep neural networks in which participants' data remains on their own devices with only model updates being shared with a central server. However, the distributed nature of FL gives rise to new threats caused by potentially malicious participants. In this paper, we study targeted data poisoning attacks against FL systems in which a malicious subset of the participants aim to poison the global model by sending model updates derived from mislabeled data. We first demonstrate that such data poisoning attacks can cause substantial drops in classification accuracy and recall, even with a small percentage of malicious participants. We additionally show that the attacks can be targeted, i.e., they have a large negative impact only on classes that are under attack. We also study attack longevity in early/late round training, the impact of malicious participant availability, and the relationships between the two. Finally, we propose a defense strategy that can help identify malicious participants in FL to circumvent poisoning attacks, and demonstrate its effectiveness.

Attack of the Tails: Yes, You Really Can Backdoor Federated Learning

Abstract: Due to its decentralized nature, Federated Learning (FL) lends itself to adversarial attacks in the form of backdoors during training. The goal of a backdoor is to corrupt the performance of the trained model on specific sub-tasks (e.g., by classifying green cars as frogs). A range of FL backdoor attacks have been introduced in the literature, but also methods to defend against them, and it is currently an open question whether FL systems can be tailored to be robust against backdoors. In this work, we provide evidence to the contrary. We first establish that, in the general case, robustness to backdoors implies model robustness to adversarial examples, a major open problem in itself. Furthermore, detecting the presence of a backdoor in a FL model is unlikely assuming first order oracles or polynomial time. We couple our theoretical results with a new family of backdoor attacks, which we refer to as edge-case backdoors. An edge-case backdoor forces a model to misclassify on seemingly easy inputs that are however unlikely to be part of the training, or test data, i.e., they live on the tail of the input distribution. We explain how these edge-case backdoors can lead to unsavory failures and may have serious repercussions on fairness, and exhibit that with careful tuning at the side of the adversary, one can insert them across a range of machine learning tasks (e.g., image classification, OCR, text prediction, sentiment analysis).

How To Backdoor Federated Learning

Abstract: Federated learning enables thousands of participants to construct a deep learning model without sharing their private training data with each other. For example, multiple smartphones can jointly train a next-word predictor for keyboards without revealing what individual users type. We demonstrate that any participant in federated learning can introduce hidden backdoor functionality into the joint global model, e.g., to ensure that an image classifier assigns an attacker-chosen label to images with certain features, or that a word predictor completes certain sentences with an attacker-chosen word. We design and evaluate a new model-poisoning methodology based on model replacement. An attacker selected in a single round of federated learning can cause the global model to immediately reach 100% accuracy on the backdoor task. We evaluate the attack under different assumptions for the standard federated-learning tasks and show that it greatly outperforms data poisoning. Our generic constrain-and-scale technique also evades anomaly detection-based defenses by incorporating the evasion into the attacker's loss function during training.

Local Model Poisoning Attacks to Byzantine-Robust Federated Learning

Abstract: In federated learning, multiple client devices jointly learn a machine learning model: each client device maintains a local model for its local training dataset, while a master device maintains a global model via aggregating the local models from the client devices. The machine learning community recently proposed several federated learning methods that were claimed to be robust against Byzantine failures (e.g., system failures, adversarial manipulations) of certain client devices. In this work, we perform the first systematic study on local model poisoning attacks to federated learning. We assume an attacker has compromised some client devices, and the attacker manipulates the local model parameters on the compromised client devices during the learning process such that the global model has a large testing error rate. We formulate our attacks as optimization problems and apply our attacks to four recent Byzantine-robust federated learning methods. Our empirical results on four real-world datasets show that our attacks can substantially increase the error rates of the models learnt by the federated learning methods that were claimed to be robust against Byzantine failures of some client devices. We generalize two defenses for data poisoning attacks to defend against our local model poisoning attacks. Our evaluation results show that one defense can effectively defend against our attacks in some cases, but the defenses are not effective enough in other cases, highlighting the need for new defenses against our local model poisoning attacks to federated learning.

Comparison of Attacks Strategies

Comparison and Evaluation of Attack Strategies

Firstly, we compare attack strategies with the following fours characteristics (Table {{table:f2ca7ecd-8422-4431-9117-2d340e5c2a35}} )

As can be seen in Table {{table:f2ca7ecd-8422-4431-9117-2d340e5c2a35}} , the majority of the attack strategies utilize iid datasets for DPA (except {{cite:2fd741f}}) in order to distinguish between the benign and malicious updates {{cite:e70e7d0}}
{{cite:29b6853}}. In contrast, the majority of MPA use datasets that are non-iid, making them beneficial in real-life FL environments.


LoMar: A Local Defense Against Poisoning Attack on Federated Learning

Abstract: Federated learning (FL) provides a high efficient decentralized machine learning framework, where the training data remains distributed at remote clients in a network. Though FL enables a privacy-preserving mobile edge computing framework using IoT devices, recent studies have shown that this approach is susceptible to poisoning attacks from the side of remote clients. To address the poisoning attacks on FL, we provide a \textit{two-phase} defense algorithm called {Lo}cal {Ma}licious Facto{r} (LoMar). In phase I, LoMar scores model updates from each remote client by measuring the relative distribution over their neighbors using a kernel density estimation method. In phase II, an optimal threshold is approximated to distinguish malicious and clean updates from a statistical perspective. Comprehensive experiments on four real-world datasets have been conducted, and the experimental results show that our defense strategy can effectively protect the FL system. {Specifically, the defense performance on Amazon dataset under a label-flipping attack indicates that, compared with FG+Krum, LoMar increases the target label testing accuracy from $96.0\%$ to $98.8\%$, and the overall averaged testing accuracy from $90.1\%$ to $97.0\%$.

BaFFLe: Backdoor detection via Feedback-based Federated Learning

Abstract: Recent studies have shown that federated learning (FL) is vulnerable to poisoning attacks that inject a backdoor into the global model. These attacks are effective even when performed by a single client, and undetectable by most existing defensive techniques. In this paper, we propose Backdoor detection via Feedback-based Federated Learning (BAFFLE), a novel defense to secure FL against backdoor attacks. The core idea behind BAFFLE is to leverage data of multiple clients not only for training but also for uncovering model poisoning. We exploit the availability of diverse datasets at the various clients by incorporating a feedback loop into the FL process, to integrate the views of those clients when deciding whether a given model update is genuine or not. We show that this powerful construct can achieve very high detection rates against state-of-the-art backdoor attacks, even when relying on straightforward methods to validate the model. Through empirical evaluation using the CIFAR-10 and FEMNIST datasets, we show that by combining the feedback loop with a method that suspects poisoning attempts by assessing the per-class classification performance of the updated model, BAFFLE reliably detects state-of-the-art backdoor attacks with a detection accuracy of 100% and a false-positive rate below 5%. Moreover, we show that our solution can detect adaptive attacks aimed at bypassing the defense.

FLDetector: Defending Federated Learning Against Model Poisoning Attacks via Detecting Malicious Clients

Abstract: Federated learning (FL) is vulnerable to model poisoning attacks, in which malicious clients corrupt the global model via sending manipulated model updates to the server. Existing defenses mainly rely on Byzantine-robust FL methods, which aim to learn an accurate global model even if some clients are malicious. However, they can only resist a small number of malicious clients in practice. It is still an open challenge how to defend against model poisoning attacks with a large number of malicious clients. Our FLDetector addresses this challenge via detecting malicious clients. FLDetector aims to detect and remove the majority of the malicious clients such that a Byzantine-robust FL method can learn an accurate global model using the remaining clients. Our key observation is that, in model poisoning attacks, the model updates from a client in multiple iterations are inconsistent. Therefore, FLDetector detects malicious clients via checking their model-updates consistency. Roughly speaking, the server predicts a client's model update in each iteration based on its historical model updates using the Cauchy mean value theorem and L-BFGS, and flags a client as malicious if the received model update from the client and the predicted model update are inconsistent in multiple iterations. Our extensive experiments on three benchmark datasets show that FLDetector can accurately detect malicious clients in multiple state-of-the-art model poisoning attacks. After removing the detected malicious clients, existing Byzantine-robust FL methods can learn accurate global models.Our code is available at https://github.com/zaixizhang/FLDetector.

Can You Really Backdoor Federated Learning?

Abstract: The decentralized nature of federated learning makes detecting and defending against adversarial attacks a challenging task. This paper focuses on backdoor attacks in the federated learning setting, where the goal of the adversary is to reduce the performance of the model on targeted tasks while maintaining good performance on the main task. Unlike existing works, we allow non-malicious clients to have correctly labeled samples from the targeted tasks. We conduct a comprehensive study of backdoor attacks and defenses for the EMNIST dataset, a real-life, user-partitioned, and non-iid dataset. We observe that in the absence of defenses, the performance of the attack largely depends on the fraction of adversaries present and the "complexity'' of the targeted task. Moreover, we show that norm clipping and "weak'' differential privacy mitigate the attacks without hurting the overall performance. We have implemented the attacks and defenses in TensorFlow Federated (TFF), a TensorFlow framework for federated learning. In open-sourcing our code, our goal is to encourage researchers to contribute new attacks and defenses and evaluate them on standard federated datasets.

Comparison of Defense Strategies

Comparison and Evaluation of Defense Strategies

Table {{table:e89a702e-0c3e-4cb0-9044-d7748444e7e2}}  illustrates the comparison of defense strategies by employing a number of aspects discussed below.


Rainbow: Combining Improvements in Deep Reinforcement Learning

Abstract: The deep reinforcement learning community has made several independent improvements to the DQN algorithm. However, it is unclear which of these extensions are complementary and can be fruitfully combined. This paper examines six extensions to the DQN algorithm and empirically studies their combination. Our experiments show that the combination provides state-of-the-art performance on the Atari 2600 benchmark, both in terms of data efficiency and final performance. We also provide results from a detailed ablation study that shows the contribution of each component to overall performance.

Agent57: Outperforming the Atari Human Benchmark

Abstract: Atari games have been a long-standing benchmark in the reinforcement learning (RL) community for the past decade. This benchmark was proposed to test general competency of RL algorithms. Previous work has achieved good average performance by doing outstandingly well on many games of the set, but very poorly in several of the most challenging games. We propose Agent57, the first deep RL agent that outperforms the standard human benchmark on all 57 Atari games. To achieve this result, we train a neural network which parameterizes a family of policies ranging from very exploratory to purely exploitative. We propose an adaptive mechanism to choose which policy to prioritize throughout the training process. Additionally, we utilize a novel parameterization of the architecture that allows for more consistent and stable learning.

Estimated cost to reproduce a full evaluation on ALE. These are the estimated costs for a single run, some papers include multiple runs, or ablation studies.

Estimation of Cost to Reproduce for Selected Papers

For some older algorithms, it may be possible to reduce the costs given here by training on more modern hardware (for example, an A100 or TPU). However, if the algorithm is CPU constrained, this could increase, not decrease, the cost due to the underutilization of the device. Therefore we estimate reproduction costs with our best guess at the hardware that would have been used at the time of publication. A summary of the calculations and results is given in Table {{table:d27b5247-bc80-4eb8-8055-cedfc88bb463}} . In addition, we detail the assumptions made for each algorithm in the paragraphs below.