Ultrasensitive beam deflection measurement via interferometric weak value amplification.

Abstract: We report on the use of an interferometric weak value technique to amplify very small transverse deflections of an optical beam. By entangling the beam's transverse degrees of freedom with the which-path states of a Sagnac interferometer, it is possible to realize an optical amplifier for polarization independent deflections. The theory for the interferometric weak value amplification method is presented along with the experimental results, which are in good agreement. Of particular interest, we measured the angular deflection of a mirror down to 400+/-200 frad and the linear travel of a piezo actuator down to 14+/-7 fm.

Colloquium : Understanding quantum weak values: Basics and applications

Abstract: The Institute of Optics, University of Rochester, Rochester, New York 14627, USAand Department of Physics, University of Ottawa, Ottawa, Ontario, Canada(published 28 March 2014)Since its introduction 25 years ago, the quantum weak value has gradually transitioned from atheoretical curiosity to a practical laboratory tool. While its utility is apparent in the recent explosionof weak value experiments, its interpretation has historically been a subject of confusion. Here apragmaticintroductiontotheweakvalueintermsofmeasurablequantitiesispresented,alongwithanexplanation for how it can be determined in the laboratory. Further, its application to three distinctexperimental techniques is reviewed. First, as a large interaction parameter it can amplify smallsignals above technical background noise. Second, as a measurable complex value it enables noveltechniques for direct quantum state and geometric phase determination. Third, as a conditionedaverage of generalized observable eigenvalues it provides a measurable window into nonclassicalfeaturesofquantummechanics.Inthisselectivereview,asingleexperimentalconfigurationtodiscussand clarify each of these applications is used.

Continuous variable quantum cryptography: beating the 3 dB loss limit.

Abstract: We demonstrate that secure quantum key distribution systems based on continuous variable implementations can operate beyond the apparent 3 dB loss limit that is implied by the beam splitting attack. The loss limit was established for standard minimum uncertainty states such as coherent states. We show that, by an appropriate postselection mechanism, we can enter a region where Eve's knowledge on Alice's key falls behind the information shared between Alice and Bob, even in the presence of substantial losses.

Hybrid quantum repeater using bright coherent light.

Abstract: We describe a quantum repeater protocol for long-distance quantum communication. In this scheme, entanglement is created between qubits at intermediate stations of the channel by using a weak dispersive light-matter interaction and distributing the outgoing bright coherent-light pulses among the stations. Noisy entangled pairs of electronic spin are then prepared with high success probability via homodyne detection and postselection. The local gates for entanglement purification and swapping are deterministic and measurement-free, based upon the same coherent-light resources and weak interactions as for the initial entanglement distribution. Finally, the entanglement is stored in a nuclear-spin-based quantum memory. With our system, qubit-communication rates approaching 100 Hz over 1280 km with fidelities near 99% are possible for reasonable local gate errors.

One-step deterministic polarization-entanglement purification using spatial entanglement

Abstract: We present a one-step deterministic entanglement purification protocol with linear optics and postselection. Compared with the Simon-Pan protocol [C. Simon and J. W. Pan, Phys. Rev. Lett. 89, 257901 (2002)], this one-step protocol has some advantages. First, it can obtain a maximally entangled pair from each photon pair with only one step, instead of improving the fidelity of less-entangled photon pairs by performing the entanglement purification process repeatedly in other protocols. Second, it works in a deterministic way, not a probabilistic one, which greatly reduces the number of entanglement resources needed. Third, it does not require the polarization state be entangled; only spatial entanglement is needed. Moreover, it is feasible with current techniques [J. W. Pan, S. Gasparonl, R. Ursin, G. Weihs, and A. Zellinger, Nature (London) 423, 417 (2003)]. All these advantages make this one-step protocol more convenient than others in quantum-communication applications.

Measuring small longitudinal phase shifts: weak measurements or standard interferometry?

Abstract: Recently, weak measurements were used to measure small effects that are transverse to the propagation direction of a light beam. Here we address the question of whether weak measurements are also useful for measuring small longitudinal phase shifts. We show that standard interferometry greatly outperforms weak measurements in a scenario involving a purely real weak value. However, we also present an interferometric scheme based on a purely imaginary weak value, combined with a frequency-domain analysis, which may have the potential to outperform standard interferometry by several orders of magnitude.

Complex weak values in quantum measurement

Abstract: In the weak value formalism of Aharonov et al., the weak value A{sub w} of any observable A is generally a complex number. We derive a physical interpretation of its value in terms of the shift in the measurement pointer's mean position and mean momentum. In particular, we show that the mean position shift contains a term jointly proportional to the imaginary part of the weak value and the rate at which the pointer is spreading in space as it enters the measurement interaction.

Top ten citation analysis of documents on weak measurement research.

Table. {{table:0f0eb32d-47b8-4c0a-9450-8f0b6e73a958}}  shows the top ten citation analyses of documents on weak measurement research from 2000 to 2020. "Ultrasensitive Beam Deflection Measurement via Interferometric Weak Value Amplification{{cite:ebaf9e6}}" was the most popular article with the highest citations. In their work, they reported on the use of an interferometric weak value technique to amplify very small transverse deflections of an optical beam. This paper can be used as a representative work of quantum weak measurement to amplify a detector signal. In addition, the review "Colloquium: Understanding quantum weak values: Basics and applications{{cite:14702e4}}" had concluded its application to their distinct experimental techniques: weak-value amplification{{cite:de83f70}}, {{cite:743263d}}, {{cite:1ebaa88}}, direct quantum state and geometric determination{{cite:1eb9f9a}}, {{cite:ae20e71}}, {{cite:a3f2119}} and a measurable window into non-classical features of quantum mechanics{{cite:e7e4fd4}}, {{cite:ac3d31e}}, {{cite:6b46447}}.


DropEdge: Towards Deep Graph Convolutional Networks on Node Classification

Abstract: \emph{Over-fitting} and \emph{over-smoothing} are two main obstacles of developing deep Graph Convolutional Networks (GCNs) for node classification. In particular, over-fitting weakens the generalization ability on small dataset, while over-smoothing impedes model training by isolating output representations from the input features with the increase in network depth. This paper proposes DropEdge, a novel and flexible technique to alleviate both issues. At its core, DropEdge randomly removes a certain number of edges from the input graph at each training epoch, acting like a data augmenter and also a message passing reducer. Furthermore, we theoretically demonstrate that DropEdge either reduces the convergence speed of over-smoothing or relieves the information loss caused by it. More importantly, our DropEdge is a general skill that can be equipped with many other backbone models (e.g. GCN, ResGCN, GraphSAGE, and JKNet) for enhanced performance. Extensive experiments on several benchmarks verify that DropEdge consistently improves the performance on a variety of both shallow and deep GCNs. The effect of DropEdge on preventing over-smoothing is empirically visualized and validated as well. Codes are released on~\url{https://github.com/DropEdge/DropEdge}.

Inductive Representation Learning on Large Graphs

Abstract: Low-dimensional embeddings of nodes in large graphs have proved extremely useful in a variety of prediction tasks, from content recommendation to identifying protein functions. However, most existing approaches require that all nodes in the graph are present during training of the embeddings; these previous approaches are inherently transductive and do not naturally generalize to unseen nodes. Here we present GraphSAGE, a general, inductive framework that leverages node feature information (e.g., text attributes) to efficiently generate node embeddings for previously unseen data. Instead of training individual embeddings for each node, we learn a function that generates embeddings by sampling and aggregating features from a node's local neighborhood. Our algorithm outperforms strong baselines on three inductive node-classification benchmarks: we classify the category of unseen nodes in evolving information graphs based on citation and Reddit post data, and we show that our algorithm generalizes to completely unseen graphs using a multi-graph dataset of protein-protein interactions.

FastGCN: Fast Learning with Graph Convolutional Networks via Importance Sampling

Abstract: The graph convolutional networks (GCN) recently proposed by Kipf and Welling are an effective graph model for semi-supervised learning. This model, however, was originally designed to be learned with the presence of both training and test data. Moreover, the recursive neighborhood expansion across layers poses time and memory challenges for training with large, dense graphs. To relax the requirement of simultaneous availability of test data, we interpret graph convolutions as integral transforms of embedding functions under probability measures. Such an interpretation allows for the use of Monte Carlo approaches to consistently estimate the integrals, which in turn leads to a batched training scheme as we propose in this work---FastGCN. Enhanced with importance sampling, FastGCN not only is efficient for training but also generalizes well for inference. We show a comprehensive set of experiments to demonstrate its effectiveness compared with GCN and related models. In particular, training is orders of magnitude more efficient while predictions remain comparably accurate.

Graph-Revised Convolutional Network

Abstract: Graph Convolutional Networks (GCNs) have received increasing attention in the machine learning community for effectively leveraging both the content features of nodes and the linkage patterns across graphs in various applications. As real-world graphs are often incomplete and noisy, treating them as ground-truth information, which is a common practice in most GCNs, unavoidably leads to sub-optimal solutions. Existing efforts for addressing this problem either involve an over-parameterized model which is difficult to scale, or simply re-weight observed edges without dealing with the missing-edge issue. This paper proposes a novel framework called Graph-Revised Convolutional Network (GRCN), which avoids both extremes. Specifically, a GCN-based graph revision module is introduced for predicting missing edges and revising edge weights w.r.t. downstream tasks via joint optimization. A theoretical analysis reveals the connection between GRCN and previous work on multigraph belief propagation. Experiments on six benchmark datasets show that GRCN consistently outperforms strong baseline methods by a large margin, especially when the original graphs are severely incomplete or the labeled instances for model training are highly sparse.

Graph Structure Learning for Robust Graph Neural Networks

Abstract: Graph Neural Networks (GNNs) are powerful tools in representation learning for graphs. However, recent studies show that GNNs are vulnerable to carefully-crafted perturbations, called adversarial attacks. Adversarial attacks can easily fool GNNs in making predictions for downstream tasks. The vulnerability to adversarial attacks has raised increasing concerns for applying GNNs in safety-critical applications. Therefore, developing robust algorithms to defend adversarial attacks is of great significance. A natural idea to defend adversarial attacks is to clean the perturbed graph. It is evident that real-world graphs share some intrinsic properties. For example, many real-world graphs are low-rank and sparse, and the features of two adjacent nodes tend to be similar. In fact, we find that adversarial attacks are likely to violate these graph properties. Therefore, in this paper, we explore these properties to defend adversarial attacks on graphs. In particular, we propose a general framework Pro-GNN, which can jointly learn a structural graph and a robust graph neural network model from the perturbed graph guided by these properties. Extensive experiments on real-world graphs demonstrate that the proposed framework achieves significantly better performance compared with the state-of-the-art defense methods, even when the graph is heavily perturbed. We release the implementation of Pro-GNN to our DeepRobust repository for adversarial attacks and defenses (footnote: https://github.com/DSE-MSU/DeepRobust). The specific experimental settings to reproduce our results can be found in https://github.com/ChandlerBang/Pro-GNN.

Learning Discrete Structures for Graph Neural Networks

Abstract: Graph neural networks (GNNs) are a popular class of machine learning models whose major advantage is their ability to incorporate a sparse and discrete dependency structure between data points. Unfortunately, GNNs can only be used when such a graph-structure is available. In practice, however, real-world graphs are often noisy and incomplete or might not be available at all. With this work, we propose to jointly learn the graph structure and the parameters of graph convolutional networks (GCNs) by approximately solving a bilevel program that learns a discrete probability distribution on the edges of the graph. This allows one to apply GCNs not only in scenarios where the given graph is incomplete or corrupted but also in those where a graph is not available. We conduct a series of experiments that analyze the behavior of the proposed method and demonstrate that it outperforms related methods by a significant margin.

A taxonomy of robust Graph Neural Networks.

Nevertheless, traditional GNNs are usually problematic when deployed to sensitive areas such as finance and healthcare because of robustness reasons. Like most neural network-based learning approaches, the noise that appears in data will lead to a significant performance degeneration for GNNs, especially when encountering structural noise. Most GNN models utilize the message passing mechanism as their fundamental, as each node will iteratively update its embedding by aggregating with its neighbors' embeddings. However, this procedure can have cascading effects when the graph data is noised, since the error may be propagated through the entire graph. Several studies have been conducted to address the robustness issue of GNNs. There are three major types of approaches to tackle this problem, which can be categorized into sample-based, revision-based, and construction-based methods shown in Table {{table:dbd95134-28e8-4151-b461-3b9fe5b3c73f}} .

However, since GNNs learn representations based on message passing via edges and aggregating from neighbors, graph structures with noisy and incomplete information may degenerate the performance of GNNs in learning representations and downstream tasks. In practice, noisy structures are often inevitable when collecting graph data. For instance, network delay may result in missing edges and spammers may create edges that should not exist. To overcome this challenge, some robust GNNs have been proposed and we categorize these GNNs into three types: sample-based, revision-based, and construction-based, shown in Table {{table:dbd95134-28e8-4151-b461-3b9fe5b3c73f}} . The selected methods in our comparative study are highlighted in bold in this table.


Learning a Neural Semantic Parser from User Feedback

Abstract: We present an approach to rapidly and easily build natural language interfaces to databases for new domains, whose performance improves over time based on user feedback, and requires minimal intervention. To achieve this, we adapt neural sequence models to map utterances directly to SQL with its full expressivity, bypassing any intermediate meaning representations. These models are immediately deployed online to solicit feedback from real users to flag incorrect queries. Finally, the popularity of SQL facilitates gathering annotations for incorrect predictions using the crowd, which is directly used to improve our models. This complete feedback loop, without intermediate representations or database specific engineering, opens up new ways of building high quality semantic parsers. Experiments suggest that this approach can be deployed quickly for any new target domain, as we show by learning a semantic parser for an online academic database from scratch.

Improving Text-to-SQL Evaluation Methodology

Abstract: To be informative, an evaluation must measure how well systems generalize to realistic unseen data. We identify limitations of and propose improvements to current evaluations of text-to-SQL systems. First, we compare human-generated and automatically generated questions, characterizing properties of queries necessary for real-world applications. To facilitate evaluation on multiple datasets, we release standardized and improved versions of seven existing datasets and one new text-to-SQL dataset. Second, we show that the current division of data into training and test sets measures robustness to variations in the way questions are asked, but only partially tests how well systems generalize to new queries; therefore, we propose a complementary dataset split for evaluation of future work. Finally, we demonstrate how the common practice of anonymizing variables during evaluation removes an important challenge of the task. Our observations highlight key difficulties, and our methodology enables effective measurement of future development.

Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task

Abstract: We present Spider, a large-scale complex and cross-domain semantic parsing and text-to-SQL dataset annotated by 11 college students. It consists of 10,181 questions and 5,693 unique complex SQL queries on 200 databases with multiple tables covering 138 different domains. We define a new complex and cross-domain semantic parsing and text-to-SQL task so that different complicated SQL queries and databases appear in train and test sets. In this way, the task requires the model to generalize well to both new SQL queries and new database schemas. Therefore, Spider is distinct from most of the previous semantic parsing tasks because they all use a single database and have the exact same program in the train set and the test set. We experiment with various state-of-the-art models and the best model achieves only 9.7% exact matching accuracy on a database split setting. This shows that Spider presents a strong challenge for future research. Our dataset and task with the most recent updates are publicly available at https://yale-lily.github.io/seq2sql/spider.

Text-to-SQL Datasets

Nowadays, there are several semantic parsing datasets developed for SQL query mapping. All of the different datasets vary in several aspects. Table {{table:2b42a312-a23a-426a-8496-9e0ad51c4cdf}}  provides detailed statistics of the most used datasets among researchers. The early developed datasets concentrate on one domain and one database: ATIS {{cite:c324ba6}}, GeoQuery {{cite:4332de8}}, Restaurants {{cite:50260b8}}, {{cite:c324ba6}}, Academic {{cite:1bca168}}, Scholar {{cite:8051ac6}}, Yelp {{cite:adb1d3e}}, IMDB {{cite:adb1d3e}} and Advisinghttps://github.com/jkkummerfeld/text2sql-data, last visited: 05.05.2020 {{cite:0027a3a}}.


Simple Unsupervised Multi-Object Tracking

Abstract: Multi-object tracking has seen a lot of progress recently, albeit with substantial annotation costs for developing better and larger labeled datasets. In this work, we remove the need for annotated datasets by proposing an unsupervised re-identification network, thus sidestepping the labeling costs entirely, required for training. Given unlabeled videos, our proposed method (SimpleReID) first generates tracking labels using SORT and trains a ReID network to predict the generated labels using crossentropy loss. We demonstrate that SimpleReID performs substantially better than simpler alternatives, and we recover the full performance of its supervised counterpart consistently across diverse tracking frameworks. The observations are unusual because unsupervised ReID is not expected to excel in crowded scenarios with occlusions, and drastic viewpoint changes. By incorporating our unsupervised SimpleReID with CenterTrack trained on augmented still images, we establish a new state-of-the-art performance on popular datasets like MOT16/17 without using tracking supervision, beating current best (CenterTrack) by 0.2-0.3 MOTA and 4.4-4.8 IDF1 scores. We further provide evidence for limited scope for improvement in IDF1 scores beyond our unsupervised ReID in the studied settings. Our investigation suggests reconsideration towards more sophisticated, supervised, end-to-end trackers by showing promise in simpler unsupervised alternatives.

Semi-TCL: Semi-Supervised Track Contrastive Representation Learning

Abstract: Online tracking of multiple objects in videos requires strong capacity of modeling and matching object appearances. Previous methods for learning appearance embedding mostly rely on instance-level matching without considering the temporal continuity provided by videos. We design a new instance-to-track matching objective to learn appearance embedding that compares a candidate detection to the embedding of the tracks persisted in the tracker. It enables us to learn not only from videos labeled with complete tracks, but also unlabeled or partially labeled videos. We implement this learning objective in a unified form following the spirit of constrastive loss. Experiments on multiple object tracking datasets demonstrate that our method can effectively learning discriminative appearance embeddings in a semi-supervised fashion and outperform state of the art methods on representative benchmarks.

Tracking Without Bells and Whistles

Abstract: The problem of tracking multiple objects in a video sequence poses several challenging tasks. For tracking-by-detection, these include object re-identification, motion prediction and dealing with occlusions. We present a tracker (without bells and whistles) that accomplishes tracking without specifically targeting any of these tasks, in particular, we perform no training or optimization on tracking data. To this end, we exploit the bounding box regression of an object detector to predict the position of an object in the next frame, thereby converting a detector into a Tracktor. We demonstrate the potential of Tracktor and provide a new state-of-the-art on three multi-object tracking benchmarks by extending it with a straightforward re-identification and camera motion compensation. We then perform an analysis on the performance and failure cases of several state-of-the-art tracking methods in comparison to our Tracktor. Surprisingly, none of the dedicated tracking methods are considerably better in dealing with complex tracking scenarios, namely, small and occluded objects or missing detections. However, our approach tackles most of the easy tracking scenarios. Therefore, we motivate our approach as a new tracking paradigm and point out promising future research directions. Overall, Tracktor yields superior tracking performance than any current tracking method and our analysis exposes remaining and unsolved tracking challenges to inspire future research directions.

RelationTrack: Relation-Aware Multiple Object Tracking With Decoupled Representation

Abstract: Existing online multiple object tracking (MOT) algorithms often consist of two subtasks, detection and re-identification (ReID). In order to enhance the inference speed and reduce the complexity, current methods commonly integrate these double subtasks into a unified framework. Nevertheless, detection and ReID demand diverse features. This issue results in an optimization contradiction during the training procedure. With the target of alleviating this contradiction, we devise a module named Global Context Disentangling (GCD) that decouples the learned representation into detection-specific and ReID-specific embeddings. As such, this module provides an implicit manner to balance the different requirements of these two subtasks. Moreover, we observe that preceding MOT methods typically leverage local information to associate the detected targets and neglect to consider the global semantic relation. To resolve this limitation, we develop a module, referred to as Guided Transformer Encoder (GTE), by combining the powerful reasoning ability of Transformer encoder and deformable attention. Unlike previous works, GTE avoids analyzing all the pixels and only attends to capture the relation between query nodes and a few self-adaptively selected key samples. Therefore, it is computationally efficient. Extensive experiments have been conducted on the MOT16, MOT17 and MOT20 benchmarks to demonstrate the superiority of the proposed MOT framework, namely RelationTrack. The experimental results indicate that RelationTrack has surpassed preceding methods significantly and established a new state-of-the-art performance, e.g., IDF1 of 70.5% and MOTA of 67.2% on MOT20.

Learning a Proposal Classifier for Multiple Object Tracking

Abstract: The recent trend in multiple object tracking (MOT) is heading towards leveraging deep learning to boost the tracking performance. However, it is not trivial to solve the data-association problem in an end-to-end fashion. In this paper, we propose a novel proposal-based learnable framework, which models MOT as a proposal generation, proposal scoring and trajectory inference paradigm on an affinity graph. This framework is similar to the two-stage object detector Faster RCNN, and can solve the MOT problem in a data-driven way. For proposal generation, we propose an iterative graph clustering method to reduce the computational cost while maintaining the quality of the generated proposals. For proposal scoring, we deploy a trainable graph-convolutional-network (GCN) to learn the structural patterns of the generated proposals and rank them according to the estimated quality scores. For trajectory inference, a simple deoverlapping strategy is adopted to generate tracking output while complying with the constraints that no detection can be assigned to more than one track. We experimentally demonstrate that the proposed method achieves a clear performance improvement in both MOTA and IDF1 with respect to previous state-of-the-art on two public benchmarks. Our code is available at https://github.com/daip13/LPC_MOT.git.

Split and Connect: A Universal Tracklet Booster for Multi-Object Tracking

Abstract: Multi-object tracking (MOT) is an essential task in the computer vision field. With the fast development of deep learning technology in recent years, MOT has achieved great improvement. However, some challenges still remain, such as sensitiveness to occlusion, instability under different lighting conditions, and non-robustness to deformable objects, causing incorrect temporal associations. To address such common challenges in most of the existing trackers, in this paper, a tracklet booster (TBooster) algorithm is proposed to correct the association errors resulting from existing trackers. The correction of the association error from TBooster has two folds: split tracklets on potential ID-change positions and then connect multiple tracklets into one if they are from the same object. To achieve this goal, the TBooster consists of two components, i.e., Splitter and Connector. In Splitter, an architecture with stacked temporal dilated convolution blocks is employed for the splitting position prediction via label smoothing strategy with adaptive Gaussian kernels. In Connector, a multi-head self-attention-based encoder is exploited for the tracklet embedding, which is further used to connect tracklets into full tracks. We conduct sufficient experiments on MOT17 and MOT20 benchmark datasets and achieve promising results. Combined with the proposed tracklet booster, existing trackers can achieve large improvements on the IDF1 score, which shows the effectiveness of the proposed TBooster.

Learning a Neural Solver for Multiple Object Tracking

Abstract: Graphs offer a natural way to formulate Multiple Object Tracking (MOT) within the tracking-by-detection paradigm. However, they also introduce a major challenge for learning methods, as defining a model that can operate on such structured domain is not trivial. As a consequence, most learning-based work has been devoted to learning better features for MOT and then using these with well-established optimization frameworks. In this work, we exploit the classical network flow formulation of MOT to define a fully differentiable framework based on Message Passing Networks (MPNs). By operating directly on the graph domain, our method can reason globally over an entire set of detections and predict final solutions. Hence, we show that learning in MOT does not need to be restricted to feature extraction, but it can also be applied to the data association step. We show a significant improvement in both MOTA and IDF1 on three publicly available benchmarks. Our code is available at https://bit.ly/motsolv.

GCNNMatch: Graph Convolutional Neural Networks for Multi-Object Tracking via Sinkhorn Normalization

Abstract: This paper proposes a novel method for online Multi-Object Tracking (MOT) using Graph Convolutional Neural Network (GCNN) based feature extraction and end-to-end feature matching for object association. The Graph based approach incorporates both appearance and geometry of objects at past frames as well as the current frame into the task of feature learning. This new paradigm enables the network to leverage the "context" information of the geometry of objects and allows us to model the interactions among the features of multiple objects. Another central innovation of our proposed framework is the use of the Sinkhorn algorithm for end-to-end learning of the associations among objects during model training. The network is trained to predict object associations by taking into account constraints specific to the MOT task. Experimental results demonstrate the efficacy of the proposed approach in achieving top performance on the MOT16 & 17 Challenge problems among state-of-the-art online and supervised approaches. The code is available at this https URL.

Summary of embedding methods on MOT20 benchmark.

In-Depth Analysis on State-of-the-Art Methods

We review the state-of-the-art embedding methods of MOT on widely used datasets, including MOT17, MOT20, and KITTI. We include methods published in top CV venues over the past three years. We mainly focus on three evaluation metrics: MOTA, IDF1, and HOTA. The performances on three benchmarks are reported in Table {{table:2d30c75c-66ef-4928-93cf-65b93068ee93}} , Table {{table:55f807fd-8599-4d60-a7a7-ba453b314aa4}} , and Table {{table:faacd719-1c55-44fe-8816-1736bc6f201d}} . The embedding methods in the table include “Patch-Level Box Image Embedding" (Patch), “Single-Frame Detection Embedding" (S-Fr), “Single-Frame Detection Embedding" (X-Fr), “Correlation-Based Embedding" (Corre), “Sequential Embedding" (Seq), “Sequential Embedding" (Tracklet) and “Sequential Embedding" (X-Track). The used detection can be Public (Pub.) or Private (Private). Modality (Mod.) in the KITTI benchmark includes Vision (V) and LiDAR (L). The Object (Obj.) category includes Car (C) and Person (P). We analyze the top performance for each category of embedding methods as follows.


Molecular De Novo Design through Deep Reinforcement Learning

Abstract: This work introduces a method to tune a sequence-based generative model for molecular de novo design that through augmented episodic likelihood can learn to generate structures with certain specified desirable properties. We demonstrate how this model can execute a range of tasks such as generating analogues to a query structure and generating compounds predicted to be active against a biological target. As a proof of principle, the model is first trained to generate molecules that do not contain sulphur. As a second example, the model is trained to generate analogues to the drug Celecoxib, a technique that could be used for scaffold hopping or library expansion starting from a single molecule. Finally, when tuning the model towards generating compounds predicted to be active against the dopamine receptor type 2, the model generates structures of which more than 95% are predicted to be active, including experimentally confirmed actives that have not been included in either the generative model nor the activity prediction model.

Controlled Molecule Generator for Optimizing Multiple Chemical Properties

Abstract: Generating a novel and optimized molecule with desired chemical properties is an essential part of the drug discovery process. Failure to meet one of the required properties can frequently lead to failure in a clinical test which is costly. In addition, optimizing these multiple properties is a challenging task because the optimization of one property is prone to changing other properties. In this paper, we pose this multi-property optimization problem as a sequence translation process and propose a new optimized molecule generator model based on the Transformer with two constraint networks: property prediction and similarity prediction. We further improve the model by incorporating score predictions from these constraint networks in a modified beam search algorithm. The experiments demonstrate that our proposed model outperforms state-of-the-art models by a significant margin for optimizing multiple properties simultaneously.

Retrieval-based Controllable Molecule Generation

Abstract: Generating new molecules with specified chemical and biological properties via generative models has emerged as a promising direction for drug discovery. However, existing methods require extensive training/fine-tuning with a large dataset, often unavailable in real-world generation tasks. In this work, we propose a new retrieval-based framework for controllable molecule generation. We use a small set of exemplar molecules, i.e., those that (partially) satisfy the design criteria, to steer the pre-trained generative model towards synthesizing molecules that satisfy the given design criteria. We design a retrieval mechanism that retrieves and fuses the exemplar molecules with the input molecule, which is trained by a new self-supervised objective that predicts the nearest neighbor of the input molecule. We also propose an iterative refinement process to dynamically update the generated molecules and retrieval database for better generalization. Our approach is agnostic to the choice of generative models and requires no task-specific fine-tuning. On various tasks ranging from simple design criteria to a challenging real-world scenario for designing lead compounds that bind to the SARS-CoV-2 main protease, we demonstrate our approach extrapolates well beyond the retrieval database, and achieves better performance and wider applicability than previous methods. Code is available at https://github.com/NVlabs/RetMol.

Controllable molecule generation models.

Controllable Molecule Generation.
Although compounds are selected based on their projected bioactivities, their absorption, distribution, metabolism, excretion, and toxicity (ADMET) properties are frequently difficult to predict and adjust, causing bottlenecks in downstream investigations and applications. It would be more productive if candidate molecules with adequate chemical properties were developed at the outset of the molecule design process. Most recent
research {{cite:f4c8b0c}}, {{cite:c7c08e8}}, {{cite:d1a2f43}}, {{cite:f827363}}, {{cite:d2ea6ac}}, {{cite:6049b80}}, {{cite:de28891}}, {{cite:5f1a3b8}} proposes to synthesize molecules in a controlled manner using generative models, which we summarize four different generation approaches shown in Table {{table:3d8f560b-b75d-4506-b1f4-10694527542a}}  (Appendix). On the one hand, reinforcement-learning (RL) and supervised-learning (SL) approaches necessitate extensive task-specific fine-tuning. Optimization-based techniques, on the other hand, train latent-space property predictors to uncover latent information relating to the target molecules. However, in real-world circumstances, we only have a limited amount of active molecules accessible for training. To address these problems, RetMol {{cite:5f1a3b8}} proposes a retrieval-based framework for controllable molecule generation. {{cite:6049b80}} develops CMG extending the self-attention technique Transformer to a molecular sequence by incorporating molecule properties and additional regularization network.


Leveraging Shape Completion for 3D Siamese Tracking

Abstract: Point clouds are challenging to process due to their sparsity, therefore autonomous vehicles rely more on appearance attributes than pure geometric features. However, 3D LIDAR perception can provide crucial information for urban navigation in challenging light or weather conditions. In this paper, we investigate the versatility of Shape Completion for 3D Object Tracking in LIDAR point clouds. We design a Siamese tracker that encodes model and candidate shapes into a compact latent representation. We regularize the encoding by enforcing the latent representation to decode into an object model shape. We observe that 3D object tracking and 3D shape completion complement each other. Learning a more meaningful latent representation shows better discriminatory capabilities, leading to improved tracking performance. We test our method on the KITTI Tracking set using car 3D bounding boxes. Our model reaches a 76.94% Success rate and 81.38% Precision for 3D Object Tracking, with the shape completion regularization leading to an improvement of 3% in both metrics.

P2B: Point-to-Box Network for 3D Object Tracking in Point Clouds

Abstract: Towards 3D object tracking in point clouds, a novel point-to-box network termed P2B is proposed in an end-to-end learning manner. Our main idea is to first localize potential target centers in 3D search area embedded with target information. Then point-driven 3D target proposal and verification are executed jointly. In this way, the time-consuming 3D exhaustive search can be avoided. Specifically, we first sample seeds from the point clouds in template and search area respectively. Then, we execute permutation-invariant feature augmentation to embed target clues from template into search area seeds and represent them with target-specific features. Consequently, the augmented search area seeds regress the potential target centers via Hough voting. The centers are further strengthened with seed-wise targetness scores. Finally, each center clusters its neighbors to leverage the ensemble power for joint 3D target proposal and verification. We apply PointNet++ as our backbone and experiments on KITTI tracking dataset demonstrate P2B’s superiority (~10%’s improvement over state-of-the-art). Note that P2B can run with 40FPS on a single NVIDIA 1080Ti GPU. Our code and model are available at https://github.com/HaozheQi/P2B.

3D Object Tracking with Transformer

Abstract: Feature fusion and similarity computation are two core problems in 3D object tracking, especially for object tracking using sparse and disordered point clouds. Feature fusion could make similarity computing more efficient by including target object information. However, most existing LiDAR-based approaches directly use the extracted point cloud feature to compute similarity while ignoring the attention changes of object regions during tracking. In this paper, we propose a feature fusion network based on transformer architecture. Benefiting from the self-attention mechanism, the transformer encoder captures the inter- and intra- relations among different regions of the point cloud. By using cross-attention, the transformer decoder fuses features and includes more target cues into the current point cloud feature to compute the region attentions, which makes the similarity computing more efficient. Based on this feature fusion network, we propose an end-to-end point cloud object tracking framework, a simple yet effective method for 3D object tracking using point clouds. Comprehensive experimental results on the KITTI dataset show that our method achieves new state-of-the-art performance. Code is available at: https://github.com/3bobo/lttr.

Box-Aware Feature Enhancement for Single Object Tracking on Point Clouds

Abstract: Current 3D single object tracking approaches track the target based on a feature comparison between the target template and the search area. However, due to the common occlusion in LiDAR scans, it is non-trivial to conduct accurate feature comparisons on severe sparse and incomplete shapes. In this work, we exploit the ground truth bounding box given in the first frame as a strong cue to enhance the feature description of the target object, enabling a more accurate feature comparison in a simple yet effective way. In particular, we first propose the BoxCloud, an informative and robust representation, to depict an object using the point-to-box relation. We further design an efficient box-aware feature fusion module, which leverages the aforementioned BoxCloud for reliable feature matching and embedding. Integrating the proposed general components into an existing model P2B [27], we construct a superior box-aware tracker (BAT)1. Experiments confirm that our proposed BAT outperforms the previous state-of-the-art by a large margin on both KITTI and NuScenes benchmarks, achieving a 12.8% improvement in terms of precision while running ∼20% faster.

Speed comparison on all test frames in the Car category from KITTI. Bold denotes the best result.

Inference Speed. In addition to tracking accuracy comparisons, we also compare the inference speed of our OSP2B with SOTA methods. For a fair comparison, the average running time of each tracker is calculated on all test frames in the Car category from KITTI. OSP2B runs at 34 Fps on a single NVIDIA 1080Ti GPU, including 7.6 ms for processing point cloud, 21.1 ms for network forward propagation, and 0.8 ms for post-processing. The running speeds of other methods under the same workstation are reported in Table {{table:f1921c61-97f4-4fbf-af3f-68bcb86faca7}} . Thanks to the efficient one-stage point-to-box prediction head, our OSP2B achieves the fastest inference speed.


HoloClean: Holistic Data Repairs with Probabilistic Inference

Abstract: We introduce HoloClean, a framework for holistic data repairing driven by probabilistic inference. HoloClean unifies existing qualitative data repairing approaches, which rely on integrity constraints or external data sources, with quantitative data repairing methods, which leverage statistical properties of the input data. Given an inconsistent dataset as input, HoloClean automatically generates a probabilistic program that performs data repairing. Inspired by recent theoretical advances in probabilistic inference, we introduce a series of optimizations which ensure that inference over HoloClean's probabilistic model scales to instances with millions of tuples. We show that HoloClean scales to instances with millions of tuples and find data repairs with an average precision of ~90% and an average recall of above ~76% across a diverse array of datasets exhibiting different types of errors. This yields an average F1 improvement of more than 2x against state-of-the-art methods.

Advancements of Outlier Detection: A Survey

Abstract: Outlier detection is an important research problem in data mining that aims to discover useful abnormal and irregular patterns hidden in large datasets. In this paper, we present a survey of outlier detection techniques to reflect the recent advancements in this field. The survey will not only cover the traditional outlier detection methods for static and low dimensional datasets but also review the more recent developments that deal with more complex outlier detection problems for dynamic/streaming and high-dimensional datasets.

CleanML: A Study for Evaluating the Impact of Data Cleaning on ML Classification Tasks

Abstract: Data quality affects machine learning (ML) model performances, and data scientists spend considerable amount of time on data cleaning before model training. However, to date, there does not exist a rigorous study on how exactly cleaning affects ML -- ML community usually focuses on developing ML algorithms that are robust to some particular noise types of certain distributions, while database (DB) community has been mostly studying the problem of data cleaning alone without considering how data is consumed by downstream ML analytics. We propose a CleanML study that systematically investigates the impact of data cleaning on ML classification tasks. The open-source and extensible CleanML study currently includes 14 real-world datasets with real errors, five common error types, seven different ML models, and multiple cleaning algorithms for each error type (including both commonly used algorithms in practice as well as state-of-the-art solutions in academic literature). We control the randomness in ML experiments using statistical hypothesis testing, and we also control false discovery rate in our experiments using the Benjamini-Yekutieli (BY) procedure. We analyze the results in a systematic way to derive many interesting and nontrivial observations. We also put forward multiple research directions for researchers.

ZeroER: Entity Resolution using Zero Labeled Examples

Abstract: Entity resolution (ER) refers to the problem of matching records in one or more relations that refer to the same real-world entity. While supervised machine learning (ML) approaches achieve the state-of-the-art results, they require a large amount of labeled examples that are expensive to obtain and often times infeasible. We investigate an important problem that vexes practitioners: is it possible to design an effective algorithm for ER that requires Zero labeled examples, yet can achieve performance comparable to supervised approaches? In this paper, we answer in the affirmative through our proposed approach dubbed ZeroER. Our approach is based on a simple observation -- the similarity vectors for matches should look different from that of unmatches. Operationalizing this insight requires a number of technical innovations. First, we propose a simple yet powerful generative model based on Gaussian Mixture Models for learning the match and unmatch distributions. Second, we propose an adaptive regularization technique customized for ER that ameliorates the issue of feature overfitting. Finally, we incorporate the transitivity property into the generative model in a novel way resulting in improved accuracy. On five benchmark ER datasets, we show that ZeroER greatly outperforms existing unsupervised approaches and achieves comparable performance to supervised approaches.

Confident Learning: Estimating Uncertainty in Dataset Labels

Abstract: Learning exists in the context of data, yet notions of confidence typically focus on model predictions, not label quality. Confident learning (CL) is an alternative approach which focuses instead on label quality by characterizing and identifying label errors in datasets, based on the principles of pruning noisy data, counting with probabilistic thresholds to estimate noise, and ranking examples to train with confidence. Whereas numerous studies have developed these principles independently, here, we combine them, building on the assumption of a class-conditional noise process to directly estimate the joint distribution between noisy (given) labels and uncorrupted (unknown) labels. This results in a generalized CL which is provably consistent and experimentally performant. We present sufficient conditions where CL exactly finds label errors, and show CL performance exceeding seven recent competitive approaches for learning with noisy labels on the CIFAR dataset. Uniquely, the CL framework is not coupled to a specific data modality or model (e.g., we use CL to find several label errors in the presumed error-free MNIST dataset and improve sentiment classification on text data in Amazon Reviews). We also employ CL on ImageNet to quantify ontological class overlap (e.g., estimating 645 "missile" images are mislabeled as their parent class "projectile"), and moderately increase model accuracy (e.g., for ResNet) by cleaning data prior to training. These results are replicable using the open-source cleanlab release.

Picket: Guarding Against Corrupted Data in Tabular Data during Learning and Inference

Abstract: Data corruption is an impediment to modern machine learning deployments. Corrupted data can severely bias the learned model and can also lead to invalid inferences. We present, Picket, a simple framework to safeguard against data corruptions during both training and deployment of machine learning models over tabular data. For the training stage, Picket identifies and removes corrupted data points from the training data to avoid obtaining a biased model. For the deployment stage, Picket flags, in an online manner, corrupted query points to a trained machine learning model that due to noise will result in incorrect predictions. To detect corrupted data, Picket uses a self-supervised deep learning model for mixed-type tabular data, which we call PicketNet. To minimize the burden of deployment, learning a PicketNet model does not require any human-labeled data. Picket is designed as a plugin that can increase the robustness of any machine learning pipeline. We evaluate Picket on a diverse array of real-world data considering different corruption models that include systematic and adversarial noise during both training and testing. We show that Picket consistently safeguards against corrupted data during both training and deployment of various models ranging from SVMs to neural networks, beating a diverse array of competing methods that span from data quality validation models to robust outlier-detection models.

Examined error detection and repair methods (The index (Idx) and abbreviation (Abbr) are used to refer to the detection and repair methods in the figures of Section )

In REIN, we selected 19 publicly-available error detection methods, which deal with the most common attribute and class errors in tabular dataAttribute errors occur in the training features, while class errors occur in the labels. Table {{table:04763b82-5d58-4f51-9b9e-b7fc5dbfc97c}}  lists the error detection methods and their targeted error types. Moreover, the table comprises the configurations and/or signals, i.e., patterns, constraints, helping functions, and knowledge bases, necessary for running each detection method. In REIN, we classify the error detection methods according to their methodology into two main categories, including (I) Non-learning methods and (II) ML-supported methods. As its name implies, the former category includes the methods and tools which detect errors using either a set of user-provided knowledge base, business rules, integrity constraints, or using a set of statistical measures. Each of these methods and tools typically tackle specific error types, e.g., duplicates, outliers, or missing values. The second category comprises the methods, e.g., Picket, ED2, and RAHA, which formulate the error detection task as a classification problem. These methods initially extract a set of features for each attribute. Such auto-generated features enable a classifier to differentiate between clean and dirty data samples. To train such a classifier, some training samples are selected to be labeled by an oracle. Below, we introduce the error detectors in each category.

The first method in Table {{table:04763b82-5d58-4f51-9b9e-b7fc5dbfc97c}}  is KATARA {{cite:c08ff0c}} which aligns the input dirty dataset with crowdsourced knowledge bases to identify and correct data samples that violate semantic patterns. To detect rule and pattern violations, NADEEF {{cite:5f09d8d}} treats data quality rules holistically via providing an interface for implementing denial constraints and other user-defined functions. Another relevant work is HoloClean {{cite:91771d7}} which combines qualitative and quantitative signals, e.g., denial constraints and correlations, in a statistical model that enables detecting and repairing missing values and rule/constraint violations. To identify inconsistencies and pattern violations, the OpenRefine tool {{cite:93013c4}} enables users to visually explore the dirty datasets through faceting and filtering operations. FAHES {{cite:42a11d5}} is another tool which detects disguised missing values, e.g., "999999" for a phone number. To this end, FAHES employs a syntactic pattern detection module for categorical data and a density-based outlier detection module for numerical data. To detect explicit missing values, REIN implements a method to find empty or NAN entries.

In this section, we present a representative set of common ML models utilized for assessing the performance of error detection and repair methods. Table {{table:9bb9b220-148c-4e77-99c8-fe768132024a}}  summarizes the algorithms and whether they are used for classification (C), regression (R), or unsupervised clustering (UC) tasks. As listed in the table, REIN examines 12 classifiers, 11 regression models, six clustering algorithms, and two AutoML algorithms. Such vital algorithms are broadly applicable in various real-world application domains, e.g., cybersecurity systems, smart cities, healthcare, e-commerce, agriculture, and many more {{cite:c39f5ed}}. The rationale behind involving two AutoML algorithms is to evaluate the performance of fully automated ML pipelines, consisting of data cleaning and model building. We are interested in checking whether such algorithms are able to find the best possible combination of model architectures and hyperparameters based on dirty or automatically-repaired datasets. For most of these models, REIN exploits the Python implementation of Scikit-learn {{cite:748a7d5}} library for training and testing. For hyperparameter tuning, REIN leverages a Bayesian-based informed search method, referred to as Optuna {{cite:0b9dea7}}. However, we did not use Optuna with the AutoML algorithms, since they can automatically select the best hyperparameters. Moreover, we did not use the internal processing pipelines of these algorithms, since we mainly focus on the examined cleaners (listed in Table {{table:04763b82-5d58-4f51-9b9e-b7fc5dbfc97c}} ).


Database resources of the National Center for Biotechnology Information

Abstract: In addition to maintaining the GenBank® nucleic acid sequence database, the National Center for Biotechnology Information (NCBI, http://www.ncbi.nlm.nih.gov) provides analysis and retrieval resources for the data in GenBank and other biological data made available through the NCBI Web site. NCBI resources include Entrez, the Entrez Programming Utilities, MyNCBI, PubMed, PubMed Central, PubReader, Gene, the NCBI Taxonomy Browser, BLAST, BLAST Link, Primer-BLAST, COBALT, RefSeq, UniGene, HomoloGene, ProtEST, dbMHC, dbSNP, dbVar, Epigenomics, the Genetic Testing Registry, Genome and related tools, the Map Viewer, Trace Archive, Sequence Read Archive, BioProject, BioSample, ClinVar, MedGen, HIV-1/Human Protein Interaction Database, Gene Expression Omnibus, Probe, Online Mendelian Inheritance in Animals, the Molecular Modeling Database, the Conserved Domain Database, the Conserved Domain Architecture Retrieval Tool, Biosystems, Protein Clusters and the PubChem suite of small molecule databases. Augmenting many of the Web applications are custom implementations of the BLAST program optimized to search specialized data sets. All these resources can be accessed through the NCBI home page.

UniProt: the universal protein knowledgebase in 2021

Abstract: Abstract The aim of the UniProt Knowledgebase is to provide users with a comprehensive, high-quality and freely accessible set of protein sequences annotated with functional information. In this article, we describe significant updates that we have made over the last two years to the resource. The number of sequences in UniProtKB has risen to approximately 190 million, despite continued work to reduce sequence redundancy at the proteome level. We have adopted new methods of assessing proteome completeness and quality. We continue to extract detailed annotations from the literature to add to reviewed entries and supplement these in unreviewed entries with annotations provided by automated systems such as the newly implemented Association-Rule-Based Annotator (ARBA). We have developed a credit-based publication submission interface to allow the community to contribute publications and annotations to UniProt entries. We describe how UniProtKB responded to the COVID-19 pandemic through expert curation of relevant entries that were rapidly made available to the research community through a dedicated portal. UniProt resources are available under a CC-BY (4.0) license via the web at https://www.uniprot.org/.

The 26 probe sets ranked by their feature importance regarding the predictability of CatBoost. Information such as, probe set's ID, corresponding gene symbols or NCBI accession numbers, blood malignancies and/or other types of cancer they are associated with, as well as, general annotations about the probe sets and the role of the gene products are presented here.

From these 34 probe sets we keep only those for which we cannot find any bibliographic reference regarding their correlation to AML, Table {{table:5be43652-e166-4efe-8b06-13871cd0c8fb}} . The only correlated to AML feature we include in our final machine learning model is the age of each individual.

From the 34 probe sets we exclude all which are correlated from bibliographic references to AML so we keep only the 26 probe sets of Table {{table:5be43652-e166-4efe-8b06-13871cd0c8fb}} . The tuned CatBoost model which we use for the diagnosis of AML (CatBoost26) has 100 iterators and depth 11 with learning rate 0.1.


Equality of Opportunity in Supervised Learning

Abstract: We propose a criterion for discrimination against a specified sensitive attribute in supervised learning, where the goal is to predict some target based on available features. Assuming data about the predictor, target, and membership in the protected group are available, we show how to optimally adjust any learned predictor so as to remove discrimination according to our definition. Our framework also improves incentives by shifting the cost of poor classification from disadvantaged groups to the decision maker, who can respond by improving the classification accuracy. In line with other studies, our notion is oblivious: it depends only on the joint statistics of the predictor, the target and the protected attribute, but not on interpretation of individualfeatures. We study the inherent limits of defining and identifying biases based on such oblivious measures, outlining what can and cannot be inferred from different oblivious tests. We illustrate our notion using a case study of FICO credit scores.

On Fairness and Calibration

Abstract: The machine learning community has become increasingly concerned with the potential for bias and discrimination in predictive models. This has motivated a growing line of work on what it means for a classification procedure to be "fair." In this paper, we investigate the tension between minimizing error disparity across different population groups while maintaining calibrated probability estimates. We show that calibration is compatible only with a single error constraint (i.e. equal false-negatives rates across groups), and show that any algorithm that satisfies this relaxation is no better than randomizing a percentage of predictions for an existing classifier. These unsettling findings, which extend and generalize existing results, are empirically confirmed on several datasets.

Proxy Fairness

Abstract: We consider the problem of improving fairness when one lacks access to a dataset labeled with protected groups, making it difficult to take advantage of strategies that can improve fairness but require protected group labels, either at training or runtime. To address this, we investigate improving fairness metrics for proxy groups, and test whether doing so results in improved fairness for the true sensitive groups. Results on benchmark and real-world datasets demonstrate that such a proxy fairness strategy can work well in practice. However, we caution that the effectiveness likely depends on the choice of fairness metric, as well as how aligned the proxy groups are with the true protected groups in terms of the constrained model parameters.

Active Fairness in Algorithmic Decision Making

Abstract: Society increasingly relies on machine learning models for automated decision making. Yet, efficiency gains from automation have come paired with concern for algorithmic discrimination that can systematize inequality. Recent work has proposed optimal post-processing methods that randomize classification decisions for a fraction of individuals, in order to achieve fairness measures related to parity in errors and calibration. These methods, however, have raised concern due to the information inefficiency, intra-group unfairness, and Pareto sub-optimality they entail. The present work proposes an alternative active framework for fair classification, where, in deployment, a decision-maker adaptively acquires information according to the needs of different groups or individuals, towards balancing disparities in classification performance. We propose two such methods, where information collection is adapted to group- and individual-level needs respectively. We show on real-world datasets that these can achieve: 1) calibration and single error parity (e.g., equal opportunity); and 2) parity in both false positive and false negative rates (i.e., equal odds). Moreover, we show that by leveraging their additional degree of freedom, active approaches can substantially outperform randomization-based classifiers previously considered optimal, while avoiding limitations such as intra-group unfairness.

Auditing and Achieving Intersectional Fairness in Classification Problems

Abstract: Machine learning algorithms are extensively used to make increasingly more consequential decisions about people, so achieving optimal predictive performance can no longer be the only focus. A particularly important consideration is fairness with respect to race, gender, or any other sensitive attribute. This paper studies intersectional fairness, where intersections of multiple sensitive attributes are considered. Prior research has mainly focused on fairness with respect to a single sensitive attribute, with intersectional fairness being comparatively less studied despite its critical importance for the safety of modern machine learning systems. We present a comprehensive framework for auditing and achieving intersectional fairness in classification problems: we define a suite of metrics to assess intersectional fairness in the data or model outputs by extending known single-attribute fairness metrics, and propose methods for robustly estimating them even when some intersectional subgroups are underrepresented. Furthermore, we develop post-processing techniques to mitigate any detected intersectional bias in a classification model. Our techniques do not rely on any assumptions regarding the underlying model and preserve predictive performance at a guaranteed level of fairness. Finally, we give guidance on a practical implementation, showing how the proposed methods perform on a real-world dataset.

Wasserstein Fair Classification

Abstract: We propose an approach to fair classification that enforces independence between the classifier outputs and sensitive information by minimizing Wasserstein-1 distances. The approach has desirable theoretical properties and is robust to specific choices of the threshold used to obtain class predictions from model outputs. We introduce different methods that enable hiding sensitive information at test time or have a simple and fast implementation. We show empirical performance against different fairness baselines on several benchmark fairness datasets.

Intra-Processing Methods for Debiasing Neural Networks

Abstract: As deep learning models become tasked with more and more decisions that impact human lives, such as criminal recidivism, loan repayment, and face recognition for law enforcement, bias is becoming a growing concern. Debiasing algorithms are typically split into three paradigms: pre-processing, in-processing, and post-processing. However, in computer vision or natural language applications, it is common to start with a large generic model and then fine-tune to a specific use-case. Pre- or in-processing methods would require retraining the entire model from scratch, while post-processing methods only have black-box access to the model, so they do not leverage the weights of the trained model. Creating debiasing algorithms specifically for this fine-tuning use-case has largely been neglected. In this work, we initiate the study of a new paradigm in debiasing research, intra-processing, which sits between in-processing and post-processing methods. Intra-processing methods are designed specifically to debias large models which have been trained on a generic dataset and fine-tuned on a more specific task. We show how to repurpose existing in-processing methods for this use-case, and we also propose three baseline algorithms: random perturbation, layerwise optimization, and adversarial fine-tuning. All of our techniques can be used for all popular group fairness measures such as equalized odds or statistical parity difference. We evaluate these methods across three popular datasets from the AIF360 toolkit, as well as on the CelebA faces dataset. Our code is available at https://github.com/abacusai/intraprocessing_debiasing.

Fair Regression with Wasserstein Barycenters

Abstract: We study the problem of learning a real-valued function that satisfies the Demographic Parity constraint. It demands the distribution of the predicted output to be independent of the sensitive attribute. We consider the case that the sensitive attribute is available for prediction. We establish a connection between fair regression and optimal transport theory, based on which we derive a close form expression for the optimal fair predictor. Specifically, we show that the distribution of this optimum is the Wasserstein barycenter of the distributions induced by the standard regression function on the sensitive groups. This result offers an intuitive interpretation of the optimal fair prediction and suggests a simple post-processing algorithm to achieve fairness. We establish risk and distribution-free fairness guarantees for this procedure. Numerical experiments indicate that our method is very effective in learning fair models, with a relative increase in error rate that is inferior to the relative gain in fairness.

Fairness via Representation Neutralization

Abstract: Existing bias mitigation methods for DNN models primarily work on learning debiased encoders. This process not only requires a lot of instance-level annotations for sensitive attributes, it also does not guarantee that all fairness sensitive information has been removed from the encoder. To address these limitations, we explore the following research question: Can we reduce the discrimination of DNN models by only debiasing the classification head, even with biased representations as inputs? To this end, we propose a new mitigation technique, namely, Representation Neutralization for Fairness (RNF) that achieves fairness by debiasing only the task-specific classification head of DNN models. To this end, we leverage samples with the same ground-truth label but different sensitive attributes, and use their neutralized representations to train the classification head of the DNN model. The key idea of RNF is to discourage the classification head from capturing spurious correlation between fairness sensitive information in encoder representations with specific class labels. To address low-resource settings with no access to sensitive attribute annotations, we leverage a bias-amplified model to generate proxy annotations for sensitive attributes. Experimental results over several benchmark datasets demonstrate our RNF framework to effectively reduce discrimination of DNN models with minimal degradation in task-specific performance.

Attributing Fair Decisions with Attention Interventions

Abstract: The widespread use of Artificial Intelligence (AI) in consequential domains, such as health-care and parole decision-making systems, has drawn intense scrutiny on the fairness of these methods. However, ensuring fairness is often insufficient as the rationale for a contentious decision needs to be audited, understood, and defended. We propose that the attention mechanism can be used to ensure fair outcomes while simultaneously providing feature attributions to account for how a decision was made. Toward this goal, we design an attention-based model that can be leveraged as an attribution framework. It can identify features responsible for both performance and fairness of the model through attention interventions and attention weight manipulation. Using this attribution framework, we then design a post-processing bias mitigation strategy and compare it with a suite of baselines. We demonstrate the versatility of our approach by conducting experiments on two distinct data types, tabular and textual.

A Confidence-Based Approach for Balancing Fairness and Accuracy

Abstract:
We study three classical machine learning algorithms in the context of algorithmic fairness: adaptive boosting, support vector machines, and logistic regression. Our goal is to maintain the high accuracy of these learning algorithms while reducing the degree to which they discriminate against individuals because of their membership in a protected group.
Our first contribution is a method for achieving fairness by shifting the decision boundary for the protected group. The method is based on the theory of margins for boosting. Our method performs comparably to or outperforms previous algorithms in the fairness literature in terms of accuracy and low discrimination, while simultaneously allowing for a fast and transparent quantification of the trade-off between bias and error.
Our second contribution addresses the shortcomings of the bias-error trade-off studied in most of the algorithmic fairness literature. We demonstrate that even hopelessly naive modifications of a biased algorithm, which cannot be reasonably said to be fair, can still achieve low bias and high accuracy. To help to distinguish between these naive algorithms and more sensible algorithms we propose a new measure of fairness, called resilience to random bias (RRB). We demonstrate that RRB distinguishes well between our naive and sensible fairness algorithms. RRB together with bias and accuracy provides a more complete picture of the fairness of an algorithm.

Achieving non-discrimination in prediction

Abstract: Discrimination-aware classification is receiving an increasing attention in data science fields. The pre-process methods for constructing a discrimination-free classifier first remove discrimination from the training data, and then learn the classifier from the cleaned data. However, they lack a theoretical guarantee for the potential discrimination when the classifier is deployed for prediction. In this paper, we fill this gap by mathematically bounding the probability of the discrimination in prediction being within a given interval in terms of the training data and classifier. We adopt the causal model for modeling the data generation mechanism, and formally defining discrimination in population, in a dataset, and in prediction. We obtain two important theoretical results: (1) the discrimination in prediction can still exist even if the discrimination in the training data is completely removed; and (2) not all pre-process methods can ensure non-discrimination in prediction even though they can achieve non-discrimination in the modified training data. Based on the results, we develop a two-phase framework for constructing a discrimination-free classifier with a theoretical guarantee. The experiments demonstrate the theoretical results and show the effectiveness of our two-phase framework.

Fairness Through Computationally-Bounded Awareness

Abstract: We study the problem of fair classification within the versatile framework of Dwork et al. [ITCS '12], which assumes the existence of a metric that measures similarity between pairs of individuals. Unlike earlier work, we do not assume that the entire metric is known to the learning algorithm; instead, the learner can query this arbitrary metric a bounded number of times. We propose a new notion of fairness called metric multifairness and show how to achieve this notion in our setting. Metric multifairness is parameterized by a similarity metric $d$ on pairs of individuals to classify and a rich collection ${\cal C}$ of (possibly overlapping) "comparison sets" over pairs of individuals. At a high level, metric multifairness guarantees that similar subpopulations are treated similarly, as long as these subpopulations are identified within the class ${\cal C}$.

FairMod - Making Predictive Models Discrimination Aware

Abstract: Predictive models such as decision trees and neural networks may produce discrimination in their predictions. This paper proposes a method to post-process the predictions of a predictive model to make the processed predictions non-discriminatory. The method considers multiple protected variables together. Multiple protected variables make the problem more challenging than a simple protected variable. The method uses a well-cited discrimination metric and adapts it to allow the specification of explanatory variables, such as position, profession, education, that describe the contexts of the applications. It models the post-processing of predictions problem as a nonlinear optimization problem to find best adjustments to the predictions so that the discrimination constraints of all protected variables are all met at the same time. The proposed method is independent of classification methods. It can handle the cases that existing methods cannot handle: satisfying multiple protected attributes at the same time, allowing multiple explanatory attributes, and being independent of classification model types. An evaluation using four real world data sets shows that the proposed method is as effectively as existing methods, in addition to its extra power.

Leveraging Labeled and Unlabeled Data for Consistent Fair Binary Classification

Abstract: We study the problem of fair binary classification using the notion of Equal Opportunity. It requires the true positive rate to distribute equally across the sensitive groups. Within this setting we show that the fair optimal classifier is obtained by recalibrating the Bayes classifier by a group-dependent threshold. We provide a constructive expression for the threshold. This result motivates us to devise a plug-in classification procedure based on both unlabeled and labeled datasets. While the latter is used to learn the output conditional probability, the former is used for calibration. The overall procedure can be computed in polynomial time and it is shown to be statistically consistent both in terms of the classification error and fairness measure. Finally, we present numerical experiments which indicate that our method is often superior or competitive with the state-of-the-art methods on benchmark datasets.

Path-Specific Counterfactual Fairness

Abstract: We consider the problem of learning fair decision systems in complex scenarios in which a sensitive attribute might affect the decision along both fair and unfair pathways. We introduce a causal approach to disregard effects along unfair pathways that simplifies and generalizes previous literature. Our method corrects observations adversely affected by the sensitive attribute, and uses these to form a decision. This avoids disregarding fair information, and does not require an often intractable computation of the path-specific effect. We leverage recent developments in deep learning and approximate inference to achieve a solution that is widely applicable to complex, non-linear scenarios.

FAE: A Fairness-Aware Ensemble Framework

Abstract: Automated decision making based on big data and machine learning (ML) algorithms can result in discriminatory decisions against certain protected groups defined upon personal data like gender, race, sexual orientation etc. Such algorithms designed to discover patterns in big data might not only pick up any encoded societal biases in the training data, but even worse, they might reinforce such biases resulting in more severe discrimination. The majority of thus far proposed fairness-aware machine learning approaches focus solely on the pre-, in- or post-processing steps of the machine learning process, that is, input data, learning algorithms or derived models, respectively. However, the fairness problem cannot be isolated to a single step of the ML process. Rather, discrimination is often a result of complex interactions between big data and algorithms, and therefore, a more holistic approach is required. The proposed FAE (Fairness-Aware Ensemble) framework combines fairness-related interventions at both pre- and postprocessing steps of the data analysis process. In the preprocessing step, we tackle the problems of under-representation of the protected group (group imbalance) and of class-imbalance by generating balanced training samples. In the post-processing step, we tackle the problem of class overlapping by shifting the decision boundary in the direction of fairness.

Bias Mitigation Post-processing for Individual and Group Fairness

Abstract: Whereas previous post-processing approaches for increasing the fairness of predictions of biased classifiers address only group fairness, we propose a method for increasing both individual and group fairness. Our novel framework includes an individual bias detector used to prioritize data samples in a bias mitigation algorithm aiming to improve the group fairness measure of disparate impact. We show superior performance to previous work in the combination of classification accuracy, individual fairness and group fairness on several real-world datasets in applications such as credit, employment, and criminal justice.

One-vs.-One Mitigation of Intersectional Bias: A General Method to Extend Fairness-Aware Binary Classification

Abstract: With the widespread adoption of machine learning in the real world, the impact of the discriminatory bias has attracted attention. In recent years, various methods to mitigate the bias have been proposed. However, most of them have not considered intersectional bias, which brings unfair situations where people belonging to specific subgroups of a protected group are treated worse when multiple sensitive attributes are taken into consideration. To mitigate this bias, in this paper, we propose a method called One-vs.-One Mitigation by applying a process of comparison between each pair of subgroups related to sensitive attributes to the fairness-aware machine learning for binary classification. We compare our method and the conventional fairness-aware binary classification methods in comprehensive settings using three approaches (pre-processing, in-processing, and post-processing), six metrics (the ratio and difference of demographic parity, equalized odds, and equal opportunity), and two real-world datasets (Adult and COMPAS). As a result, our method mitigates the intersectional bias much better than conventional methods in all the settings. With the result, we open up the potential of fairness-aware binary classification for solving more realistic problems occurring when there are multiple sensitive attributes.

Priority-based Post-Processing Bias Mitigation for Individual and Group Fairness

Abstract: Previous post-processing bias mitigation algorithms on both group and individual fairness don't work on regression models and datasets with multi-class numerical labels. We propose a priority-based post-processing bias mitigation on both group and individual fairness with the notion that similar individuals should get similar outcomes irrespective of socio-economic factors and more the unfairness, more the injustice. We establish this proposition by a case study on tariff allotment in a smart grid. Our novel framework establishes it by using a user segmentation algorithm to capture the consumption strategy better. This process ensures priority-based fair pricing for group and individual facing the maximum injustice. It upholds the notion of fair tariff allotment to the entire population taken into consideration without modifying the in-built process for tariff calculation. We also validate our method and show superior performance to previous work on a real-world dataset in criminal sentencing.

Publications on Post-processing bias mitigation methods.

Post-processing bias mitigation methods are applied once a classification model has been successfully trained.
With 35 publications that apply post-processing methods (Table {{table:acb0d274-e00f-485a-b4b1-0a92a84aaab1}} ), post-processing methods are the least frequently applied of those covered in this survey.


Knowledge Distillation of Russian Language Models with Reduction of Vocabulary

Abstract: Today, transformer language models serve as a core component for majority of natural language processing tasks. Industrial application of such models requires minimization of computation time and memory footprint. Knowledge distillation is one of approaches to address this goal. Existing methods in this field are mainly focused on reducing the number of layers or dimension of embeddings/hidden representations. Alternative option is to reduce the number of tokens in vocabulary and therefore the embeddings matrix of the student model. The main problem with vocabulary minimization is mismatch between input sequences and output class distributions of a teacher and a student models. As a result, it is impossible to directly apply KL-based knowledge distillation. We propose two simple yet effective alignment techniques to make knowledge distillation to the students with reduced vocabulary. Evaluation of distilled models on a number of common benchmarks for Russian such as Russian SuperGLUE, SberQuAD, RuSentiment, ParaPhaser, Collection-3 demonstrated that our techniques allow to achieve compression from 17× to 49×, while maintaining quality of 1.7× compressed student with the full-sized vocabulary, but reduced number of Transformer layers only. We make our code and distilled models available.

Adaptation of Deep Bidirectional Multilingual Transformers for Russian Language

Abstract: The paper introduces methods of adaptation of multilingual masked language models for a specific language. Pre-trained bidirectional language models show state-of-the-art performance on a wide range of tasks including reading comprehension, natural language inference, and sentiment analysis. At the moment there are two alternative approaches to train such models: monolingual and multilingual. While language specific models show superior performance, multilingual models allow to perform a transfer from one language to another and solve tasks for different languages simultaneously. This work shows that transfer learning from a multilingual model to monolingual model results in significant growth of performance on such tasks as reading comprehension, paraphrase detection, and sentiment analysis. Furthermore, multilingual initialization of monolingual model substantially reduces training time. Pre-trained models for the Russian language are open sourced.

Parameters of different backbone models considered in this article.

We performed the experiments on multiple backbones from HuggingFace Transformers library {{cite:2bb473e}}, which all have similar BERT-like architecture: bert-base-multilingual-cased {{cite:ab3bbec}}, DeepPavlov/distilrubert-tiny-cased-conversational {{cite:db625ff}}, ai-forever/ruBert-base {{cite:5b2bdaf}} and DeepPavlov/rubert-base-conversational-cased {{cite:ac83d06}}. The models ai-forever/ruBert-base and DeepPavlov/rubert-base-conversational-cased are similar, but they have a slightly different number of parameters because of different tokenization. We describe the difference between these backbones in Table {{table:e2871635-52e6-42a7-bf6a-2ca945d38900}} .


A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages

Abstract: We use the multilingual OSCAR corpus, extracted from Common Crawl via language classification, filtering and cleaning, to train monolingual contextualized word embeddings (ELMo) for five mid-resource languages. We then compare the performance of OSCAR-based and Wikipedia-based ELMo embeddings for these languages on the part-of-speech tagging and parsing tasks. We show that, despite the noise in the Common-Crawl-based OSCAR data, embeddings trained on OSCAR perform much better than monolingual embeddings trained on Wikipedia. They actually equal or improve the current state of the art in tagging and parsing for all five languages. In particular, they also improve over multilingual Wikipedia-based contextual embeddings (multilingual BERT), which almost always constitutes the previous state of the art, thereby showing that the benefit of a larger, more diverse corpus surpasses the cross-lingual benefit of multilingual embedding architectures.

1.5 billion words Arabic Corpus

Abstract: This study is an attempt to build a contemporary linguistic corpus for Arabic language. The corpus produced, is a text corpus includes more than five million newspaper articles. It contains over a billion and a half words in total, out of which, there is about three million unique words. The data were collected from newspaper articles in ten major news sources from eight Arabic countries, over a period of fourteen years. The corpus was encoded with two types of encoding, namely: UTF-8, and Windows CP-1256. Also it was marked with two mark-up languages, namely: SGML, and XML.

CCNet: Extracting High Quality Monolingual Datasets from Web Crawl Data

Abstract: Pre-training text representations have led to significant improvements in many areas of natural language processing. The quality of these models benefits greatly from the size of the pretraining corpora as long as its quality is preserved. In this paper, we describe an automatic pipeline to extract massive high-quality monolingual datasets from Common Crawl for a variety of languages. Our pipeline follows the data processing introduced in fastText (Mikolov et al., 2017; Grave et al., 2018), that deduplicates documents and identifies their language. We augment this pipeline with a filtering step to select documents that are close to high quality corpora like Wikipedia.

Shamela: A Large-Scale Historical Arabic Corpus

Abstract: Arabic is a widely-spoken language with a rich and long history spanning more than fourteen centuries. Yet existing Arabic corpora largely focus on the modern period or lack sufficient diachronic information. We develop a large-scale, historical corpus of Arabic of about 1 billion words from diverse periods of time. We clean this corpus, process it with a morphological analyzer, and enhance it by detecting parallel passages and automatically dating undated texts. We demonstrate its utility with selected case-studies in which we show its application to the digital humanities.

Overview of the data-sources used to develop the proposed corpus

The proposed Arabic corpus consists of 20 constituent data-sources, which are listed in Table {{table:cf453d8e-20e9-4929-af0e-505bb3a5ceb1}} , and their statistics are reported in Table {{table:486e633a-c3ab-436b-a91d-ac53a0661b77}} . The sources belong to news, academic, social, religious, cultural, and other related domains. These sources represent a sizeable proportion of most widely spoken Arabic dialects, such as MEGLN along with CA.