security and privacy, energy consumption

Resource Allocation for Edge Computing in IoT Networks via Reinforcement Learning

Abstract: In this paper, we consider resource allocation for edge computing in internet of things (IoT) networks. Specifically, each end device is considered as an agent, which makes its decisions on whether offloading the computation tasks to the edge devices or not. To minimize the long-term weighted sum cost which includes the power consumption and the task execution latency, we consider the channel conditions between the end devices and the gateway, the computation task queue as well as the remaining computation resource of the end devices as the network states. The problem of making a series of decisions at the end devices is modelled as a Markov decision process and solved by the reinforcement learning approach. Therefore, we propose a near optimal task offloading algorithm based on ∊-greedy Q-learning. Simulations validate the feasibility of our proposed algorithm, which achieves a better trade-off between the power consumption and the task execution latency compared to these of edge computing and local computing modes.

Optimal Auction for Edge Computing Resource Management in Mobile Blockchain Networks: A Deep Learning Approach

Abstract: Blockchain has recently been applied in many applications such as bitcoin, smart grid, and Internet of Things (IoT) as a public ledger of transactions. However, the use of blockchain in mobile environments is still limited because the mining process consumes too much computing and energy resources on mobile devices. Edge computing offered by the Edge Computing Service Provider (ECSP) can be adopted as a viable solution for offloading the mining tasks from the mobile devices, i.e., miners, in the mobile blockchain environment. However, a mechanism for edge resource allocation to maximize the revenue for the ECSP and to ensure incentive compatibility and individual rationality is still open. In this paper, we develop an optimal auction based on deep learning for the edge resource allocation. Specifically, we construct a multi-layer neural network architecture based on an analytical solution of the optimal auction. The neural networks first perform monotone transformations of the miners' bids. Then, they calculate allocation and conditional payment rules for the miners. We use valuations of the miners as the training data to adjust parameters of the neural networks so as to optimize the loss function which is the expected, negated revenue of the ECSP.We show the experimental results to confirm the benefits of using the deep learning for deriving the optimal auction for mobile blockchain with high revenue.

A Novel Two-Layered Reinforcement Learning for Task Offloading with Tradeoff between Physical Machine Utilization Rate and Delay

Abstract: Mobile devices could augment their ability via cloud resources in mobile cloud computing environments. This paper developed a novel two-layered reinforcement learning (TLRL) algorithm to consider task offloading for resource-constrained mobile devices. As opposed to existing literature, the utilization rate of the physical machine and the delay for offloaded tasks are taken into account simultaneously by introducing a weighted reward. The high dimensionality of the state space and action space might affect the speed of convergence. Therefore, a novel reinforcement learning algorithm with a two-layered structure is presented to address this problem. First, k clusters of the physical machines are generated based on the k-nearest neighbors algorithm (k-NN). The first layer of TLRL is implemented by a deep reinforcement learning to determine the cluster to be assigned for the offloaded tasks. On this basis, the second layer intends to further specify a physical machine for task execution. Finally, simulation examples are carried out to verify that the proposed TLRL algorithm is able to speed up the optimal policy learning and can deal with the tradeoff between physical machine utilization rate and delay.

Deep Learning for Secure Mobile Edge Computing

Abstract: Mobile edge computing (MEC) is a promising approach for enabling cloud-computing capabilities at the edge of cellular networks. Nonetheless, security is becoming an increasingly important issue in MEC-based applications. In this paper, we propose a deep-learning-based model to detect security threats. The model uses unsupervised learning to automate the detection process, and uses location information as an important feature to improve the performance of detection. Our proposed model can be used to detect malicious applications at the edge of a cellular network, which is a serious security threat. Extensive experiments are carried out with 10 different datasets, the results of which illustrate that our deep-learning-based model achieves an average gain of 6% accuracy compared with state-of-the-art machine learning algorithms.

Optimized Computation Offloading Performance in Virtual Edge Computing Systems Via Deep Reinforcement Learning

Abstract: To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is a promising paradigm by providing computing capabilities in close proximity within a sliced radio access network (RAN), which supports both traditional communication and MEC services. Nevertheless, the design of computation offloading policies for a virtual MEC system remains challenging. Specifically, whether to execute a computation task at the mobile device or to offload it for MEC server execution should adapt to the time-varying network dynamics. This paper considers MEC for a representative mobile user in an ultradense sliced RAN, where multiple base stations (BSs) are available to be selected for computation offloading. The problem of solving an optimal computation offloading policy is modeled as a Markov decision process, where our objective is to maximize the long-term utility performance whereby an offloading decision is made based on the task queue state, the energy queue state as well as the channel qualities between mobile user and BSs. To break the curse of high dimensionality in state space, we first propose a double deep ${Q}$ -network (DQN)-based strategic computation offloading algorithm to learn the optimal policy without knowing a priori knowledge of network dynamics. Then motivated by the additive structure of the utility function, a ${Q}$ -function decomposition technique is combined with the double DQN, which leads to a novel learning algorithm for the solving of stochastic computation offloading. Numerical experiments show that our proposed learning algorithms achieve a significant improvement in computation offloading performance compared with the baseline policies.

Mobile Edge Computation Offloading Using Game Theory and Reinforcement Learning

Abstract: Due to the ever-increasing popularity of resource-hungry and delay-constrained mobile applications, the computation and storage capabilities of remote cloud has partially migrated towards the mobile edge, giving rise to the concept known as Mobile Edge Computing (MEC). While MEC servers enjoy the close proximity to the end-users to provide services at reduced latency and lower energy costs, they suffer from limitations in computational and radio resources, which calls for fair efficient resource management in the MEC servers. The problem is however challenging due to the ultra-high density, distributed nature, and intrinsic randomness of next generation wireless networks. In this article, we focus on the application of game theory and reinforcement learning for efficient distributed resource management in MEC, in particular, for computation offloading. We briefly review the cutting-edge research and discuss future challenges. Furthermore, we develop a game-theoretical model for energy-efficient distributed edge server activation and study several learning techniques. Numerical results are provided to illustrate the performance of these distributed learning techniques. Also, open research issues in the context of resource management in MEC servers are discussed.

Performance Optimization in Mobile-Edge Computing via Deep Reinforcement Learning

Abstract: To improve the quality of computation experience for mobile devices, mobile-edge computing (MEC) is emerging as a promising paradigm by providing computing capabilities within radio access networks in close proximity. Nevertheless, the design of computation offloading policies for a MEC system remains challenging. Specifically, whether to execute an arriving computation task at local mobile device or to offload a task for cloud execution should adapt to the environmental dynamics in a smarter manner. In this paper, we consider MEC for a representative mobile user in an ultra dense network, where one of multiple base stations (BSs) can be selected for computation offloading. The problem of solving an optimal computation offloading policy is modelled as a Markov decision process, where our objective is to minimize the long-term cost and an offloading decision is made based on the channel qualities between the mobile user and the BSs, the energy queue state as well as the task queue state. To break the curse of high dimensionality in state space, we propose a deep $Q$-network-based strategic computation offloading algorithm to learn the optimal policy without having a priori knowledge of the dynamic statistics. Numerical experiments provided in this paper show that our proposed algorithm achieves a significant improvement in average cost compared with baseline policies.

Security in Mobile Edge Caching with Reinforcement Learning

Abstract: Mobile edge computing usually uses cache to support multimedia contents in 5G mobile Internet to reduce the computing overhead and latency. Mobile edge caching (MEC) systems are vulnerable to various attacks such as denial of service attacks and rogue edge attacks. This article investigates the attack models in MEC systems, focusing on both the mobile offloading and the caching procedures. In this paper, we propose security solutions that apply reinforcement learning (RL) techniques to provide secure offloading to the edge nodes against jamming attacks. We also present light-weight authentication and secure collaborative caching schemes to protect data privacy. We evaluate the performance of the RL-based security solution for mobile edge caching and discuss the challenges that need to be addressed in the future.

LBS for Addressing Issues in CC and EC.

In this subsection, we review the works on the use of LBS in CC and EC as summarized in Table {{table:1182427e-73f3-450e-9939-7341b670e5a8}} .


Blockchain-Assisted Crowdsourced Energy Systems

Abstract: Crowdsourcing relies on people’s contributions to meet product- or system-level objectives. Crowdsourcing-based methods have been implemented in various cyber-physical systems and realtime markets. This paper explores a framework for Crowdsourced Energy Systems (CES), where small-scale energy generation or energy trading is crowdsourced from distributed energy resources, electric vehicles, and shapable loads. The merits/pillars of energy crowdsourcing are discussed. Then, an operational model for CESs in distribution networks with different types of crowdsourcees is proposed. The model yields a market equilibrium depicting traditional and distributed generator and load setpoints. Given these setpoints, crowdsourcing incentives are designed to steer crowdsourcees to the equilibrium. As the number of crowdsourcees and energy trading transactions scales up, a secure energy trading platform is required. To that end, the presented framework is integrated with a lightweight Blockchain implementation and smart contracts. Numerical tests are provided to showcase the overall implementation.

Blockchain Based Decentralized Management of Demand Response Programs in Smart Energy Grids

Abstract: In this paper, we investigate the use of decentralized blockchain mechanisms for delivering transparent, secure, reliable, and timely energy flexibility, under the form of adaptation of energy demand profiles of Distributed Energy Prosumers, to all the stakeholders involved in the flexibility markets (Distribution System Operators primarily, retailers, aggregators, etc.). In our approach, a blockchain based distributed ledger stores in a tamper proof manner the energy prosumption information collected from Internet of Things smart metering devices, while self-enforcing smart contracts programmatically define the expected energy flexibility at the level of each prosumer, the associated rewards or penalties, and the rules for balancing the energy demand with the energy production at grid level. Consensus based validation will be used for demand response programs validation and to activate the appropriate financial settlement for the flexibility providers. The approach was validated using a prototype implemented in an Ethereum platform using energy consumption and production traces of several buildings from literature data sets. The results show that our blockchain based distributed demand side management can be used for matching energy demand and production at smart grid level, the demand response signal being followed with high accuracy, while the amount of energy flexibility needed for convergence is reduced.

Blockchain Based Sustainable Local Energy Trading Considering Home Energy Management and Demurrage Mechanism

Abstract: With the increase in local energy generation from Renewable Energy Sources (RESs), the concept of decentralized peer-to-peer Local Energy Market (LEM) is becoming popular. In this paper, a blockchain-based LEM is investigated, where consumers and prosumers in a small community trade energy without the need for a third party. In the proposed model, a Home Energy Management (HEM) system and demurrage mechanism are introduced, which allow both the prosumers and consumers to optimize their energy consumption and to minimize electricity costs. This method also allows end-users to shift their load to off-peak hours and to use cheap energy from the LEM. The proposed solution shows how energy consumption and electricity cost are optimized using HEM and demurrage mechanism. It also provides economic benefits at both the community and end-user levels and provides sufficient energy to the LEM. The simulation results show that electricity cost is reduced up to 44.73% and 28.55% when the scheduling algorithm is applied using the Critical Peak Price (CPP) and Real-Time Price (RTP) schemes, respectively. Similarly, 65.15% and 35.09% of costs are reduced when CPP and RTP are applied with demurrage mechanism. Moreover, 51.80% and 44.37% electricity costs reduction is observed when CPP and RTP are used with both demurrage and scheduling algorithm. We also carried out security vulnerability analysis to ensure that our energy trading smart contract is secure and bug-free against the common vulnerabilities and attacks.

Blockchain For Intelligent Energy Management

We summarize the design goals, targets, and other blockchain-related attributes of existing works discussed in this section, and report the results in Table {{table:196688cf-4942-4060-b12b-358ef0d42e32}} .


Trajectory prediction & plan recognition

Real-Time Locating System in Production Management

Abstract: Real-time monitoring and optimization of production and logistics processes significantly improve the efficiency of production systems. Advanced production management solutions require real-time information about the status of products, production, and resources. As real-time locating systems (also referred to as indoor positioning systems) can enrich the available information, these systems started to gain attention in industrial environments in recent years. This paper provides a review of the possible technologies and applications related to production control and logistics, quality management, safety, and efficiency monitoring. This work also provides a workflow to clarify the steps of a typical real-time locating system project, including the cleaning, pre-processing, and analysis of the data to provide a guideline and reference for research and development of indoor positioning-based manufacturing solutions.

Towards Efficient Human-Robot Collaboration With Robust Plan Recognition and Trajectory Prediction

Abstract: Human-robot collaboration (HRC) is becoming increasingly important as the paradigm of manufacturing is shifting from mass production to mass customization. The introduction of HRC can significantly improve the flexibility and intelligence of automation. To efficiently finish tasks in HRC systems, the robots need to not only predict the future movements of human, but also more high-level plans, i.e., the sequence of actions to finish the tasks. However, due to the stochastic and time-varying nature of human collaborators, it is quite challenging for the robot to efficiently and accurately identify such task plans and respond in a safe manner. To address this challenge, we propose an integrated human-robot collaboration framework. Both plan recognition and trajectory prediction modules are included for the generation of safe and efficient robotic motions. Such a framework enables the robots to perceive, predict and adapt their actions to the human's work plan and intelligently avoid collisions with the human. Moreover, by explicitly leveraging the hierarchical relationship between plans and trajectories, more robust plan recognition performance can be achieved. Physical experiments were conducted on an industrial robot to verify the proposed framework. The results show that the proposed framework could accurately recognize the human workers’ plans and thus significantly improve the time efficiency of the HRC team even in the presence of motion classification noises.

Unsupervised Movement Detection in Indoor Positioning Systems

Abstract: Consider indoor positioning systems (IPS) in production halls where objects equipped with sensors send their current position. Beside its large volume, the analyzation of the resulting raw data is challenging due to the susceptibility towards noise. Reasons are accuracy issues and undesired awakenings of sensors that occur due to the dynamics of logistic processes (e.g.~vibrations of passing forklifts). We propose a tailor-made statistical procedure for these challenges and combine visual analytics with movement detection. Contrary to common stay-point algorithms, we do not only distinguish between stops and moves, but also consider undesired awakenings. This leads to a more detailed interpretation scheme offering usages for online (e.g.~monitoring of orders) and offline applications (e.g.~detection of problematic areas). The approach does not require other information than the raw IPS output and enables an ad-hoc analysis. We underline our findings in an extensive case study with real IPS data of our industry partner.

Performance Analysis of IoT-Based Sensor, Big Data Processing, and Machine Learning Model for Real-Time Monitoring System in Automotive Manufacturing

Abstract: With the increase in the amount of data captured during the manufacturing process, monitoring systems are becoming important factors in decision making for management. Current technologies such as Internet of Things (IoT)-based sensors can be considered a solution to provide efficient monitoring of the manufacturing process. In this study, a real-time monitoring system that utilizes IoT-based sensors, big data processing, and a hybrid prediction model is proposed. Firstly, an IoT-based sensor that collects temperature, humidity, accelerometer, and gyroscope data was developed. The characteristics of IoT-generated sensor data from the manufacturing process are: real-time, large amounts, and unstructured type. The proposed big data processing platform utilizes Apache Kafka as a message queue, Apache Storm as a real-time processing engine and MongoDB to store the sensor data from the manufacturing process. Secondly, for the proposed hybrid prediction model, Density-Based Spatial Clustering of Applications with Noise (DBSCAN)-based outlier detection and Random Forest classification were used to remove outlier sensor data and provide fault detection during the manufacturing process, respectively. The proposed model was evaluated and tested at an automotive manufacturing assembly line in Korea. The results showed that IoT-based sensors and the proposed big data processing system are sufficiently efficient to monitor the manufacturing process. Furthermore, the proposed hybrid prediction model has better fault prediction accuracy than other models given the sensor data as input. The proposed system is expected to support management by improving decision-making and will help prevent unexpected losses caused by faults during the manufacturing process.

A list of prominent studies that consider different manufacturing applications

Table {{table:0549eaf9-50ec-49c3-9881-2c636a21171e}}  presents a list of prominent research papers in the literature, categorized according to their applications.


Attention-based network and CXR synthesis process for data augmentation

Conditional GANs for multi-class segmentation of heart,clavicles and lungs

GANs with U-Net autoencoder and CNN discriminator and encoder for one-class learning

Generates high resolution CXRs using multi-scale, patch based GANs

Proposes an iterative gradient based input preprocessing for improved performance

U-Net for bone generation with CT projection images, used for CXR enhancement

Novel CycleGAN model to decompose CXR images incorporating CT projection images

Novel reconstruction algorithm for CXR enhancement

2D-to-3D encoder-decoder network for generating 3D spine models from CXR studies

XLSor: A Robust and Accurate Lung Segmentor on Chest X-Rays Using Criss-Cross Attention and Customized Radiorealistic Abnormalities Generation

Abstract: This paper proposes a novel framework for lung segmentation in chest X-rays. It consists of two key contributions, a criss-cross attention based segmentation network and radiorealistic chest X-ray image synthesis (i.e. a synthesized radiograph that appears anatomically realistic) for data augmentation. The criss-cross attention modules capture rich global contextual information in both horizontal and vertical directions for all the pixels thus facilitating accurate lung segmentation. To reduce the manual annotation burden and to train a robust lung segmentor that can be adapted to pathological lungs with hazy lung boundaries, an image-to-image translation module is employed to synthesize radiorealistic abnormal CXRs from the source of normal ones for data augmentation. The lung masks of synthetic abnormal CXRs are propagated from the segmentation results of their normal counterparts, and then serve as pseudo masks for robust segmentor training. In addition, we annotate 100 CXRs with lung masks on a more challenging NIH Chest X-ray dataset containing both posterioranterior and anteroposterior views for evaluation. Extensive experiments validate the robustness and effectiveness of the proposed framework. The code and data can be found from https://github.com/rsummers11/CADLab/tree/master/Lung_Segmentation_XLSor .

Image-to-Images Translation for Multi-Task Organ Segmentation and Bone Suppression in Chest X-Ray Radiography

Abstract:
Chest X-ray radiography is one of the earliest medical imaging technologies and remains one of the most widely-used for diagnosis, screening, and treatment follow up of diseases related to lungs and heart. The literature in this field of research reports many interesting studies dealing with the challenging tasks of bone suppression and organ segmentation but performed separately, limiting any learning that comes with the consolidation of parameters that could optimize both processes. This study, and for the first time, introduces a multitask deep learning model that generates simultaneously the bone-suppressed image and the organ-segmented image, enhancing the accuracy of tasks, minimizing the number of parameters needed by the model and optimizing the processing time, all by exploiting the interplay between the network parameters to benefit the performance of both tasks. The architectural design of this model, which relies on a conditional generative adversarial network, reveals the process on how the well-established <italic>pix2pix</italic> network (<italic>image-to-image</italic> network) is modified to fit the need for multitasking and extending it to the new <italic>image-to-imag<underline>es</underline></italic> architecture. The developed source code of this multitask model is shared publicly on Github as the first attempt for providing the two-task <italic>pix2pix</italic> extension, a supervised/paired/aligned/registered <italic>image-to-imag<underline>es</underline></italic> translation which would be useful in many multitask applications. Dilated convolutions are also used to improve the results through a more effective receptive field assessment. The comparison with state-of-the-art algorithms along with ablation study and a demonstration video<xref ref-type="fn" rid="fn1">1</xref> are provided to evaluate the efficacy and gauge the merits of the proposed approach.<fn id="fn1">1
<uri>https://youtu.be/J8Uth26_7rQhttps://youtu.be/J8Uth26_7rQ</uri>
</fn>

Abnormal Chest X-ray Identification With Generative Adversarial One-Class Classifier

Abstract: Being one of the most common diagnostic imaging tests, chest radiography requires timely reporting of potential findings in the images. In this paper, we propose an end-to-end architecture for abnormal chest X-ray identification using generative adversarial one-class learning. Unlike previous approaches, our method takes only normal chest X-ray images as input. The architecture is composed of three deep neural networks, each of which learned by competing while collaborating among them to model the underlying content structure of the normal chest X-rays. Given a chest X-ray image in the testing phase, if it is normal, the learned architecture can well model and reconstruct the content; if it is abnormal, since the content is unseen in the training phase, the model would perform poorly in its reconstruction. It thus enables distinguishing abnormal chest X-rays from normal ones. Quantitative and qualitative experiments demonstrate the effectiveness and efficiency of our approach, where an AUC of 0.841 is achieved on the challenging NIH Chest X-ray dataset in a one-class learning setting, with the potential in reducing the workload for radiologists.

Multi-scale GANs for Memory-efficient Generation of High Resolution Medical Images

Abstract: Currently generative adversarial networks (GANs) are rarely applied to medical images of large sizes, especially 3D volumes, due to their large computational demand. We propose a novel multi-scale patch-based GAN approach to generate large high resolution 2D and 3D images. Our key idea is to first learn a low-resolution version of the image and then generate patches of successively growing resolutions conditioned on previous scales. In a domain translation use-case scenario, 3D thorax CTs of size 512x512x512 and thorax X-rays of size 2048x2048 are generated and we show that, due to the constant GPU memory demand of our method, arbitrarily large images of high resolution can be generated. Moreover, compared to common patch-based approaches, our multi-resolution scheme enables better image quality and prevents patch artifacts.

Improved Inference via Deep Input Transfer

Abstract: Although numerous improvements have been made in the field of image segmentation using convolutional neural networks, the majority of these improvements rely on training with larger datasets, model architecture modifications, novel loss functions, and better optimizers. In this paper, we propose a new segmentation performance boosting paradigm that relies on optimally modifying the network's input instead of the network itself. In particular, we leverage the gradients of a trained segmentation network with respect to the input to transfer it to a space where the segmentation accuracy improves. We test the proposed method on three publicly available medical image segmentation datasets: the ISIC 2017 Skin Lesion Segmentation dataset, the Shenzhen Chest X-Ray dataset, and the CVC-ColonDB dataset, for which our method achieves improvements of 5.8%, 0.5%, and 4.8% in the average Dice scores, respectively.

Bone Structures Extraction and Enhancement in Chest Radiographs via CNN Trained on Synthetic Data

Abstract: In this paper, we present a deep learning-based image processing technique for extraction of bone structures in chest radiographs using a U-Net FCNN. The U-Net was trained to accomplish the task in a fully supervised setting. To create the training image pairs, we employed simulated X-Ray or Digitally Reconstructed Radiographs (DRR), derived from 664 CT scans belonging to the LIDC-IDRI dataset. Using HU based segmentation of bone structures in the CT domain, a synthetic 2D "Bone x-ray" DRR is produced and used for training the network. For the reconstruction loss, we utilize two loss functions- L1 Loss and perceptual loss. Once the bone structures are extracted, the original image can be enhanced by fusing the original input x-ray and the synthesized "Bone X-ray". We show that our enhancement technique is applicable to real x-ray data, and display our results on the NIH Chest X-Ray-14 dataset.

Encoding CT Anatomy Knowledge for Unpaired Chest X-ray Image Decomposition

Abstract: Although chest X-ray (CXR) offers a 2D projection with overlapped anatomies, it is widely used for clinical diagnosis. There is clinical evidence supporting that decomposing an X-ray image into different components (e.g., bone, lung and soft tissue) improves diagnostic value. We hereby propose a decomposition generative adversarial network (DecGAN) to anatomically decompose a CXR image but with unpaired data. We leverage the anatomy knowledge embedded in CT, which features a 3D volume with clearly visible anatomies. Our key idea is to embed CT priori decomposition knowledge into the latent space of unpaired CXR autoencoder. Specifically, we train DecGAN with a decomposition loss, adversarial losses, cycle-consistency losses and a mask loss to guarantee that the decomposed results of the latent space preserve realistic body structures. Extensive experiments demonstrate that DecGAN provides superior unsupervised CXR bone suppression results and the feasibility of modulating CXR components by latent space disentanglement. Furthermore, we illustrate the diagnostic value of DecGAN and demonstrate that it outperforms the state-of-the-art approaches in terms of predicting 11 out of 14 common lung diseases.

Improved digital chest tomosynthesis image quality by use of a projection-based dual-energy virtual monochromatic convolutional neural network with super resolution

Abstract: We developed a novel dual-energy (DE) virtual monochromatic (VM) very-deep super-resolution (VDSR) method with an unsharp masking reconstruction algorithm (DE–VM–VDSR) that uses projection data to improve the nodule contrast and reduce ripple artifacts during chest digital tomosynthesis (DT). For estimating the residual errors from high-resolution and multiscale VM images from the projection space, the DE–VM–VDSR algorithm employs a training network (mini-batch stochastic gradient-descent algorithm with momentum) and a hybrid super-resolution (SR) image [simultaneous algebraic reconstruction technique (SART) total-variation (TV) first-iterative shrinkage–thresholding algorithm (FISTA); SART–TV–FISTA] that involves subjective reconstruction with bilateral filtering (BF) [DE–VM–VDSR with BF]. DE-DT imaging was accomplished by pulsed X-ray exposures rapidly switched between low (60 kV, 37 projection) and high (120 kV, 37 projection) tube-potential kVp by employing a 40° swing angle. This was followed by comparison of images obtained employing the conventional polychromatic filtered backprojection (FBP), SART, SART–TV–FISTA, and DE–VM–SART–TV–FISTA algorithms. The improvements in contrast, ripple artifacts, and resolution were compared using the signal-difference-to-noise ratio (SDNR), Gumbel distribution of the largest variations, radial modulation transfer function (radial MTF) for a chest phantom with simulated ground-glass opacity (GGO) nodules, and noise power spectrum (NPS) for uniform water phantom. The novel DE–VM–VDSR with BF improved the overall performance in terms of SDNR (DE–VM–VDSR with BF: 0.1603, without BF: 0.1517; FBP: 0.0521; SART: 0.0645; SART–TV–FISTA: 0.0984; and DE–VM–SART–TV–FISTA: 0.1004), obtained a Gumbel distribution that yielded good images showing the type of simulated GGO nodules used in the chest phantom, and reduced the ripple artifacts. The NPS of DE–VM–VDSR with BF showed the lowest noise characteristics in the high-frequency region (~0.8 cycles/mm). The DE–VM–VDSR without BF yielded an improved resolution relative to that of the conventional reconstruction algorithms for radial MTF analysis (0.2–0.3 cycles/mm). Finally, based on the overall image quality, DE–VM–VDSR with BF improved the contrast and reduced the high-frequency ripple artifacts and noise.

Inferring the 3D Standing Spine Posture from 2D Radiographs

Abstract: The treatment of degenerative spinal disorders requires an understanding of the individual spinal anatomy and curvature in 3D. An upright spinal pose (i.e. standing) under natural weight bearing is crucial for such bio-mechanical analysis. 3D volumetric imaging modalities (e.g. CT and MRI) are performed in patients lying down. On the other hand, radiographs are captured in an upright pose, but result in 2D projections. This work aims to integrate the two realms, i.e. it combines the upright spinal curvature from radiographs with the 3D vertebral shape from CT imaging for synthesizing an upright 3D model of spine, loaded naturally. Specifically, we propose a novel neural network architecture working vertebra-wise, termed \emph{TransVert}, which takes orthogonal 2D radiographs and infers the spine's 3D posture. We validate our architecture on digitally reconstructed radiographs, achieving a 3D reconstruction Dice of $95.52\%$, indicating an almost perfect 2D-to-3D domain translation. Deploying our model on clinical radiographs, we successfully synthesise full-3D, upright, patient-specific spine models for the first time.

Image Generation Studies (Section ).Tasks: DA=Domain Adaptation, IC=Interval Change, IG=Image Generation, IL=Image-level Predictions, LC=Localization, PR=Preprocessing, RE=Registration, SE=Segmentation, SR=Super Resolution. Bold font in tasks implies that this additional task is central to the work and the study also appears in another table in this paper.Labels: BS=Bone Suppression, C=ChestX-Ray14, CL=Clavicle, CM=Cardiomegaly, CV=COVID, E=Edema, H=Heart, L=Lung, LO=Lesion or Opacity, PE=Effusion, PT=Pneumothorax, T=Triage/Abnormal, TB=Tuberculosis, Z=Other.Datasets: C=ChestX-ray14, CC=COVID-CXR, J=JSRT+SCR, MO=Montgomery, O=Open-i, PL=PLCO, PP=Ped-pneumonia, PR=Private, RP=RSNA-Pneumonia, S=Shenzen, SM=Simulated CXR from CT, X=CheXpert.

There are 35 studies identified in this work whose main focus is Image Generation, as detailed in Table {{table:7994e188-5c73-49d3-9524-b1449cb89824}} . Image generation techniques have been harnessed for a wide variety of purposes including data augmentation {{cite:571b0b0}}, visualization {{cite:7a168b5}}, {{cite:ef20edc}}, abnormality detection through reconstruction {{cite:3ffed8a}}, {{cite:0e2974c}}, domain adaptation {{cite:4071bf0}} or image enhancement techniques {{cite:b2e2c86}}.


Local Gradients Smoothing: Defense Against Localized Adversarial Attacks

Abstract: Deep neural networks (DNNs) have shown vulnerability to adversarial attacks, i.e., carefully perturbed inputs designed to mislead the network at inference time. Recently introduced localized attacks, Localized and Visible Adversarial Noise (LaVAN) and Adversarial patch, pose a new challenge to deep learning security by adding adversarial noise only within a specific region without affecting the salient objects in an image. Driven by the observation that such attacks introduce concentrated high-frequency changes at a particular image location, we have developed an effective method to estimate noise location in gradient domain and transform those high activation regions caused by adversarial noise in image domain while having minimal effect on the salient object that is important for correct classification. Our proposed Local Gradients Smoothing (LGS) scheme achieves this by regularizing gradients in the estimated noisy region before feeding the image to DNN for inference. We have shown the effectiveness of our method in comparison to other defense methods including Digital Watermarking, JPEG compression, Total Variance Minimization (TVM) and Feature squeezing on ImageNet dataset. In addition, we systematically study the robustness of the proposed defense mechanism against Back Pass Differentiable Approximation (BPDA), a state of the art attack recently developed to break defenses that transform an input sample to minimize the adversarial effect. Compared to other defense mechanisms, LGS is by far the most resistant to BPDA in localized adversarial attack setting.

BlurNet: Defense by Filtering the Feature Maps

Abstract: Recently, the field of adversarial machine learning has been garnering attention by showing that state-of-the-art deep neural networks are vulnerable to adversarial examples, stemming from small perturbations being added to the input image. Adversarial examples are generated by a malicious adversary by obtaining access to the model parameters, such as gradient information, to alter the input or by attacking a substitute model and transferring those malicious examples over to attack the victim model. Specifically, one of these attack algorithms, Robust Physical Perturbations $(RP_{2})$, generates adversarial images of stop signs with black and white stickers to achieve high targeted misclassification rates against standard-architecture traffic sign classifiers. In this paper, we propose BlurNet, a defense against the RP2 attack. First, we motivate the defense with a frequency analysis of the first layer feature maps of the network on the LISA dataset, which shows that high frequency noise is introduced into the input image by the RP2 algorithm. To remove the high frequency noise, we introduce a depthwise convolution layer of standard blur kernels after the first layer. We perform a blackbox transfer attack to show that low-pass filtering the feature maps is more beneficial than filtering the input. We then present various regularization schemes to incorporate this low-pass filtering behavior into the training regime of the network and perform white-box attacks. We conclude with an adaptive attack evaluation to show that the success rate of the attack drops from 90% to 20% with total variation regularization, one of the proposed defenses.

Detecting Localized Adversarial Examples: A Generic Approach using Critical Region Analysis

Abstract: Deep neural networks (DNNs) have been applied in a wide range of applications, e.g., face recognition and image classification; however, they are vulnerable to adversarial examples. By adding a small amount of imperceptible perturbations, an attacker can easily manipulate the outputs of a DNN. Particularly, the localized adversarial examples only perturb a small and contiguous region of the target object, so that they are robust and effective in both digital and physical worlds. Although the localized adversarial examples have more severe real-world impacts than traditional pixel attacks, they have not been well addressed in the literature. In this paper, we propose a generic defense system called TaintRadar to accurately detect localized adversarial examples via analyzing critical regions that have been manipulated by attackers. The main idea is that when removing critical regions from input images, the ranking changes of adversarial labels will be larger than those of benign labels. Compared with existing defense solutions, TaintRadar can effectively capture sophisticated localized partial attacks, e.g., the eye-glasses attack, while not requiring additional training or fine-tuning of the original model’s structure. Comprehensive experiments have been conducted in both digital and physical worlds to verify the effectiveness and robustness of our defense.

Segment and Complete: Defending Object Detectors against Adversarial Patch Attacks with Robust Patch Detection

Abstract: Object detection plays a key role in many security-critical systems. Adversarial patch attacks, which are easy to implement in the physical world, pose a serious threat to state-of-the-art object detectors. Developing reliable defenses for object detectors against patch attacks is critical but severely understudied. In this paper, we propose Segment and Complete defense (SAC), a general framework for defending object detectors against patch attacks through detection and removal of adversarial patches. We first train a patch segmenter that outputs patch masks which provide pixel-level localization of adversarial patches. We then propose a self adversarial training algorithm to robustify the patch segmenter. In addition, we design a robust shape completion algorithm, which is guaranteed to remove the entire patch from the images if the outputs of the patch segmenter are within a certain Hamming distance of the ground-truth patch masks. Our experiments on COCO and xView datasets demonstrate that SAC achieves superior robustness even under strong adaptive attacks with no reduction in performance on clean images, and generalizes well to unseen patch shapes, attack budgets, and unseen attack methods. Furthermore, we present the APRICOT-Mask dataset, which augments the APRICOT dataset with pixel-level annotations of adversarial patches. We show SAC can significantly reduce the targeted attack success rate of physical patch attacks. Our code is available at https://github.com/joellliu/SegmentAndComplete.

DIFFender: Diffusion-Based Adversarial Defense against Patch Attacks in the Physical World

Abstract: Adversarial attacks, particularly patch attacks, pose significant threats to the robustness and reliability of deep learning models. Developing reliable defenses against patch attacks is crucial for real-world applications, yet current research in this area is unsatisfactory. In this paper, we propose DIFFender, a novel defense method that leverages a text-guided diffusion model to defend against adversarial patches. DIFFender includes two main stages: patch localization and patch restoration. In the localization stage, we find and exploit an intriguing property of the diffusion model to precisely identify the locations of adversarial patches. In the restoration stage, we employ the diffusion model to reconstruct the adversarial regions in the images while preserving the integrity of the visual content. Thanks to the former finding, these two stages can be simultaneously guided by a unified diffusion model. Thus, we can utilize the close interaction between them to improve the whole defense performance. Moreover, we propose a few-shot prompt-tuning algorithm to fine-tune the diffusion model, enabling the pre-trained diffusion model to adapt to the defense task easily. We conduct extensive experiments on image classification, face recognition, and further in the physical world, demonstrating that our proposed method exhibits superior robustness under strong adaptive attacks and generalizes well across various scenarios, diverse classifiers, and multiple patch attack methods.

Defending Against Physically Realizable Attacks on Image Classification

Abstract: We study the problem of defending deep neural network approaches for image classification from physically realizable attacks. First, we demonstrate that the two most scalable and effective methods for learning robust models, adversarial training with PGD attacks and randomized smoothing, exhibit very limited effectiveness against three of the highest profile physical attacks. Next, we propose a new abstract adversarial model, rectangular occlusion attacks, in which an adversary places a small adversarially crafted rectangle in an image, and develop two approaches for efficiently computing the resulting adversarial examples. Finally, we demonstrate that adversarial training using our new attack yields image classification models that exhibit high robustness against the physically realizable attacks we study, offering the first effective generic defense against such attacks.

Certified Defenses for Adversarial Patches

Abstract: Adversarial patch attacks were recently recognized as the most practical threat model against real-world computer vision systems. Most published defenses against patch attacks are based on preprocessing input images to mitigate adversarial noise. The first contribution of this paper is a set of experiments demonstrating that such defense strategies can easily be broken by white-box adversaries. Motivated by this finding, we present an extension of certified defense algorithms and propose significantly faster variants for robust training against patch attacks. Finally, we experiment with different patch shapes for testing, and observe that robustness to such attacks transfers surprisingly well.

(De)Randomized Smoothing for Certifiable Defense against Patch Attacks

Abstract: Patch adversarial attacks on images, in which the attacker can distort pixels within a region of bounded size, are an important threat model since they provide a quantitative model for physical adversarial attacks. In this paper, we introduce a certifiable defense against patch attacks that guarantees for a given image and patch attack size, no patch adversarial examples exist. Our method is related to the broad class of randomized smoothing robustness schemes which provide high-confidence probabilistic robustness certificates. By exploiting the fact that patch attacks are more constrained than general sparse attacks, we derive meaningfully large robustness certificates. Additionally, the algorithm we propose is de-randomized, providing deterministic certificates. To the best of our knowledge, there exists only one prior method for certifiable defense against patch attacks, which relies on interval bound propagation. While this sole existing method performs well on MNIST, it has several limitations: it requires computationally expensive training, does not scale to ImageNet, and performs poorly on CIFAR-10. In contrast, our proposed method effectively addresses all of these issues: our classifier can be trained quickly, achieves high clean and certified robust accuracy on CIFAR-10, and provides certificates at the ImageNet scale. For example, for a 5*5 patch attack on CIFAR-10, our method achieves up to around 57.8% certified accuracy (with a classifier around 83.9% clean accuracy), compared to at most 30.3% certified accuracy for the existing method (with a classifier with around 47.8% clean accuracy), effectively establishing a new state-of-the-art. Code is available at this https URL.

Adversarial YOLO: Defense Human Detection Patch Attacks via Detecting Adversarial Patches

Abstract: The security of object detection systems has attracted increasing attention, especially when facing adversarial patch attacks. Since patch attacks change the pixels in a restricted area on objects, they are easy to implement in the physical world, especially for attacking human detection systems. The existing defenses against patch attacks are mostly applied for image classification problems and have difficulty resisting human detection attacks. Towards this critical issue, we propose an efficient and effective plug-in defense component on the YOLO detection system, which we name Ad-YOLO. The main idea is to add a patch class on the YOLO architecture, which has a negligible inference increment. Thus, Ad-YOLO is expected to directly detect both the objects of interest and adversarial patches. To the best of our knowledge, our approach is the first defense strategy against human detection attacks. We investigate Ad-YOLO's performance on the YOLOv2 baseline. To improve the ability of Ad-YOLO to detect variety patches, we first use an adversarial training process to develop a patch dataset based on the Inria dataset, which we name Inria-Patch. Then, we train Ad-YOLO by a combination of Pascal VOC, Inria, and Inria-Patch datasets. With a slight drop of $0.70\%$ mAP on VOC 2007 test set, Ad-YOLO achieves $80.31\%$ AP of persons, which highly outperforms $33.93\%$ AP for YOLOv2 when facing white-box patch attacks. Furthermore, compared with YOLOv2, the results facing a physical-world attack are also included to demonstrate Ad-YOLO's excellent generalization ability.

Meta Adversarial Training against Universal Patches

Abstract: Recently demonstrated physical-world adversarial attacks have exposed vulnerabilities in perception systems that pose severe risks for safety-critical applications such as autonomous driving. These attacks place adversarial artifacts in the physical world that indirectly cause the addition of a universal patch to inputs of a model that can fool it in a variety of contexts. Adversarial training is the most effective defense against image-dependent adversarial attacks. However, tailoring adversarial training to universal patches is computationally expensive since the optimal universal patch depends on the model weights which change during training. We propose meta adversarial training (MAT), a novel combination of adversarial training with meta-learning, which overcomes this challenge by meta-learning universal patches along with model training. MAT requires little extra computation while continuously adapting a large set of patches to the current model. MAT considerably increases robustness against universal patch attacks on image classification and traffic-light detection.

Efficient Certified Defenses Against Patch Attacks on Image Classifiers

Abstract: Adversarial patches pose a realistic threat model for physical world attacks on autonomous systems via their perception component. Autonomous systems in safety-critical domains such as automated driving should thus contain a fail-safe fallback component that combines certifiable robustness against patches with efficient inference while maintaining high performance on clean inputs. We propose BagCert, a novel combination of model architecture and certification procedure that allows efficient certification. We derive a loss that enables end-to-end optimization of certified robustness against patches of different sizes and locations. On CIFAR10, BagCert certifies 10.000 examples in 43 seconds on a single GPU and obtains 86% clean and 60% certified accuracy against 5x5 patches.

ScaleCert: Scalable Certified Defense against Adversarial Patches with Sparse Superficial Layers

Abstract: Adversarial patch attacks that craft the pixels in a confined region of the input images show their powerful attack effectiveness in physical environments even with noises or deformations. Existing certified defenses towards adversarial patch attacks work well on small images like MNIST and CIFAR-10 datasets, but achieve very poor certified accuracy on higher-resolution images like ImageNet. It is urgent to design both robust and effective defenses against such a practical and harmful attack in industry-level larger images. In this work, we propose the certified defense methodology that achieves high provable robustness for high-resolution images and largely improves the practicality for real adoption of the certified defense. The basic insight of our work is that the adversarial patch intends to leverage localized superficial important neurons (SIN) to manipulate the prediction results. Hence, we leverage the SIN-based DNN compression techniques to significantly improve the certified accuracy, by reducing the adversarial region searching overhead and filtering the prediction noises. Our experimental results show that the certified accuracy is increased from 36.3% (the state-of-the-art certified detection) to 60.4% on the ImageNet dataset, largely pushing the certified defenses for practical use.

Defending Against Multiple and Unforeseen Adversarial Videos

Abstract: Adversarial robustness of deep neural networks has been actively investigated. However, most existing defense approaches are limited to a specific type of adversarial perturbations. Specifically, they often fail to offer resistance to multiple attack types simultaneously, i.e., they lack multi-perturbation robustness. Furthermore, compared to image recognition problems, the adversarial robustness of video recognition models is relatively unexplored. While several studies have proposed how to generate adversarial videos, only a handful of approaches about defense strategies have been published in the literature. In this paper, we propose one of the first defense strategies against multiple types of adversarial videos for video recognition. The proposed method, referred to as MultiBN, performs adversarial training on multiple adversarial video types using multiple independent batch normalization (BN) layers with a learning-based BN selection module. With a multiple BN structure, each BN brach is responsible for learning the distribution of a single perturbation type and thus provides more precise distribution estimations. This mechanism benefits dealing with multiple perturbation types. The BN selection module detects the attack type of an input video and sends it to the corresponding BN branch, making MultiBN fully automatic and allowing end-to-end training. Compared to present adversarial training approaches, the proposed MultiBN exhibits stronger multi-perturbation robustness against different and even unforeseen adversarial video types, ranging from Lp-bounded attacks and physically realizable attacks. This holds true on different datasets and target models. Moreover, we conduct an extensive analysis to study the properties of the multiple BN structure.

Towards Practical Certifiable Patch Defense with Vision Transformer

Abstract: Patch attacks, one of the most threatening forms of physical attack in adversarial examples, can lead networks to induce misclassification by modifying pixels arbitrarily in a continuous region. Certifiable patch defense can guarantee robustness that the classifier is not affected by patch attacks. Existing certifiable patch defenses sacrifice the clean accuracy of classifiers and only obtain a low certified accuracy on toy datasets. Furthermore, the clean and certified accuracy of these methods is still significantly lower than the accuracy of normal classification networks, which limits their application in practice. To move towards a practical certifiable patch defense, we introduce Vision Transformer (ViT) into the framework of Derandomized Smoothing (DS). Specifically, we propose a progressive smoothed image modeling task to train Vision Transformer, which can capture the more discriminable local context of an image while preserving the global semantic information. For efficient inference and deployment in the real world, we innovatively reconstruct the global self-attention structure of the original ViT into isolated band unit self-attention. On ImageNet, under 2% area patch attacks our method achieves 41.70% certified accuracy, a nearly 1-fold increase over the previous best method (26.00%). Simultaneously, our method achieves 78.58% clean accuracy, which is quite close to the normal ResNet-101 accuracy. Extensive experiments show that our method obtains state-of-the-art clean and certified accuracy with inferring efficiently on CIFAR-10 and ImageNet.

Certified Patch Robustness via Smoothed Vision Transformers

Abstract: Certified patch defenses can guarantee robustness of an image classifier to arbitrary changes within a bounded contiguous region. But, currently, this robustness comes at a cost of degraded standard accuracies and slower inference times. We demonstrate how using vision transformers enables significantly better certified patch robustness that is also more computationally efficient and does not incur a substantial drop in standard accuracy. These improvements stem from the inherent ability of the vision transformer to gracefully handle largely masked images.11Our code is available at https://github.com/MadryLab/smoothed-vit..

Certified Defences Against Adversarial Patch Attacks on Semantic Segmentation

Abstract: Adversarial patch attacks are an emerging security threat for real world deep learning applications. We present Demasked Smoothing, the first approach (up to our knowledge) to certify the robustness of semantic segmentation models against this threat model. Previous work on certifiably defending against patch attacks has mostly focused on image classification task and often required changes in the model architecture and additional training which is undesirable and computationally expensive. In Demasked Smoothing, any segmentation model can be applied without particular training, fine-tuning, or restriction of the architecture. Using different masking strategies, Demasked Smoothing can be applied both for certified detection and certified recovery. In extensive experiments we show that Demasked Smoothing can on average certify 64% of the pixel predictions for a 1% patch in the detection task and 48% against a 0.5% patch for the recovery task on the ADE20K dataset.

Knowledge Enhanced Machine Learning Pipeline against Diverse Adversarial Attacks

Abstract: Despite the great successes achieved by deep neural networks (DNNs), recent studies show that they are vulnerable against adversarial examples, which aim to mislead DNNs by adding small adversarial perturbations. Several defenses have been proposed against such attacks, while many of them have been adaptively attacked. In this work, we aim to enhance the ML robustness from a different perspective by leveraging domain knowledge: We propose a Knowledge Enhanced Machine Learning Pipeline (KEMLP) to integrate domain knowledge (i.e., logic relationships among different predictions) into a probabilistic graphical model via first-order logic rules. In particular, we develop KEMLP by integrating a diverse set of weak auxiliary models based on their logical relationships to the main DNN model that performs the target task. Theoretically, we provide convergence results and prove that, under mild conditions, the prediction of KEMLP is more robust than that of the main DNN model. Empirically, we take road sign recognition as an example and leverage the relationships between road signs and their shapes and contents as domain knowledge. We show that compared with adversarial training and other baselines, KEMLP achieves higher robustness against physical attacks, $\mathcal{L}_p$ bounded attacks, unforeseen attacks, and natural corruptions under both whitebox and blackbox settings, while still maintaining high clean accuracy.

DDDM: a Brain-Inspired Framework for Robust Classification

Abstract: Despite their outstanding performance in a broad spectrum of real-world tasks, deep artificial neural networks are sensitive to input noises, particularly adversarial perturbations. On the contrary, human and animal brains are much less vulnerable. In contrast to the one-shot inference performed by most deep neural networks, the brain often solves decision-making with an evidence accumulation mechanism that may trade time for accuracy when facing noisy inputs. The mechanism is well described by the Drift-Diffusion Model (DDM). In the DDM, decision-making is modeled as a process in which noisy evidence is accumulated toward a threshold. Drawing inspiration from the DDM, we propose the Dropout-based Drift-Diffusion Model (DDDM) that combines test-phase dropout and the DDM for improving the robustness for arbitrary neural networks. The dropouts create temporally uncorrelated noises in the network that counter perturbations, while the evidence accumulation mechanism guarantees a reasonable decision accuracy. Neural networks enhanced with the DDDM tested in image, speech, and text classification tasks all significantly outperform their native counterparts, demonstrating the DDDM as a task-agnostic defense against adversarial attacks.

Representative physical defense methods mentioned in this paper

In this section, we review the literature in computer vision for physically adversarial defenses. To help readers achieve a comprehensive understanding of the physically adversarial defenses, we organize the methods from three stages for the DNN models, i.e., pre-processing, in-processing, and post-processing. The taxonomy is given in Fig.{{figure:50008442-3923-4140-9d12-40439fde3c29}} . Specifically, in section 3.1, we discuss the mechanism of pre-processing defenses. In Section 3.2, we divide the in-processing defense methods into adversarial training, architecture modification, and certified defenses, respectively. In section 3.3, the post-processing methods are discussed. In addition, Table {{table:1b8c8373-d3bf-4ad5-be35-2dbb0852643d}}  lists a summary for the referred physically adversarial defenses. From the table, we can see that the existing physically adversarial defenses are mainly proposed against adversarial patches. This is reasonable because adversarial patches are currently the main form of physically adversarial attacks.


Efficient and Effective Text Encoding for Chinese LLaMA and Alpaca

Abstract: Large Language Models (LLMs), such as ChatGPT and GPT-4, have dramatically transformed natural language processing research and shown promising strides towards Artificial General Intelligence (AGI). Nonetheless, the high costs associated with training and deploying LLMs present substantial obstacles to transparent, accessible academic research. While several large language models, such as LLaMA, have been open-sourced by the community, these predominantly focus on English corpora, limiting their usefulness for other languages. In this paper, we propose a method to augment LLaMA with capabilities for understanding and generating Chinese text and its ability to follow instructions. We achieve this by extending LLaMA's existing vocabulary with an additional 20,000 Chinese tokens, thereby improving its encoding efficiency and semantic understanding of Chinese. We further incorporate secondary pre-training using Chinese data and fine-tune the model with Chinese instruction datasets, significantly enhancing the model's ability to comprehend and execute instructions. Our experimental results indicate that the newly proposed model markedly enhances the original LLaMA's proficiency in understanding and generating Chinese content. Additionally, the results on the C-Eval dataset yield competitive performance among the models with several times the size of ours. We have made our pre-trained models, training scripts, and other resources available through GitHub, fostering open research for our community. Chinese LLaMA series: \url{https://github.com/ymcui/Chinese-LLaMA-Alpaca} and Chinese Llama-2 series: \url{https://github.com/ymcui/Chinese-LLaMA-Alpaca-2}

Exploring the Impact of Instruction Data Scaling on Large Language Models: An Empirical Study on Real-World Use Cases

Abstract: The success of ChatGPT has recently attracted numerous efforts to replicate it, with instruction-tuning strategies being a key factor in achieving remarkable results. Instruction-tuning not only significantly enhances the model's performance and generalization but also makes the model's generated results more consistent with human speech patterns. However current research rarely studies the impact of different amounts of instruction data on model performance, especially in the real-world use cases. In this paper we explore the performance of large language models based on instruction tuning across different scales of instruction data. An evaluation dataset consisting of 12 major online use cases is constructed in the experiment. With Bloomz-7B1-mt as the base model, the results show that 1) merely increasing the amount of instruction data leads to continuous improvement in tasks such as open-ended generation, 2) in tasks such as math and code, the model performance curve remains quite flat while increasing data size. We further analyze the possible causes of these phenomena and propose potential future research directions such as effectively selecting high-quality training data, scaling base models and training methods specialized for hard tasks. We will release our training and evaluation datasets, as well as model checkpoints.

EduChat: A Large-Scale Language Model-based Chatbot System for Intelligent Education

Abstract: EduChat (https://www.educhat.top/) is a large-scale language model (LLM)-based chatbot system in the education domain. Its goal is to support personalized, fair, and compassionate intelligent education, serving teachers, students, and parents. Guided by theories from psychology and education, it further strengthens educational functions such as open question answering, essay assessment, Socratic teaching, and emotional support based on the existing basic LLMs. Particularly, we learn domain-specific knowledge by pre-training on the educational corpus and stimulate various skills with tool use by fine-tuning on designed system prompts and instructions. Currently, EduChat is available online as an open-source project, with its code, data, and model parameters available on platforms (e.g., GitHub https://github.com/icalk-nlp/EduChat, Hugging Face https://huggingface.co/ecnu-icalk ). We also prepare a demonstration of its capabilities online (https://vimeo.com/851004454). This initiative aims to promote research and applications of LLMs for intelligent education.

CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation

Abstract: Pre-trained models for Natural Languages (NL) like BERT and GPT have been recently shown to transfer well to Programming Languages (PL) and largely benefit a broad set of code-related tasks. Despite their success, most current methods either rely on an encoder-only (or decoder-only) pre-training that is suboptimal for generation (resp. understanding) tasks or process the code snippet in the same way as NL, neglecting the special characteristics of PL such as token types. We present CodeT5, a unified pre-trained encoder-decoder Transformer model that better leverages the code semantics conveyed from the developer-assigned identifiers. Our model employs a unified framework to seamlessly support both code understanding and generation tasks and allows for multi-task learning. Besides, we propose a novel identifier-aware pre-training task that enables the model to distinguish which code tokens are identifiers and to recover them when they are masked. Furthermore, we propose to exploit the user-written code comments with a bimodal dual generation task for better NL-PL alignment. Comprehensive experiments show that CodeT5 significantly outperforms prior methods on understanding tasks such as code defect detection and clone detection, and generation tasks across various directions including PL-NL, NL-PL, and PL-PL. Further analysis reveals that our model can better capture semantic information from code. Our code and pre-trained models are released at https://github.com/salesforce/CodeT5.

WizardCoder: Empowering Code Large Language Models with Evol-Instruct

Abstract: Code Large Language Models (Code LLMs), such as StarCoder, have demonstrated exceptional performance in code-related tasks. However, most existing models are solely pre-trained on extensive raw code data without instruction fine-tuning. In this paper, we introduce WizardCoder, which empowers Code LLMs with complex instruction fine-tuning, by adapting the Evol-Instruct method to the domain of code. Through comprehensive experiments on four prominent code generation benchmarks, namely HumanEval, HumanEval+, MBPP, and DS-1000, we unveil the exceptional capabilities of our model. It surpasses all other open-source Code LLMs by a substantial margin. Moreover, our model even outperforms the largest closed LLMs, Anthropic's Claude and Google's Bard, on HumanEval and HumanEval+. Our code, model weights, and data are public at https://github.com/nlpxucan/WizardLM

Judging LLM-as-a-judge with MT-Bench and Chatbot Arena

Abstract: Evaluating large language model (LLM) based chat assistants is challenging due to their broad capabilities and the inadequacy of existing benchmarks in measuring human preferences. To address this, we explore using strong LLMs as judges to evaluate these models on more open-ended questions. We examine the usage and limitations of LLM-as-a-judge, including position, verbosity, and self-enhancement biases, as well as limited reasoning ability, and propose solutions to mitigate some of them. We then verify the agreement between LLM judges and human preferences by introducing two benchmarks: MT-bench, a multi-turn question set; and Chatbot Arena, a crowdsourced battle platform. Our results reveal that strong LLM judges like GPT-4 can match both controlled and crowdsourced human preferences well, achieving over 80% agreement, the same level of agreement between humans. Hence, LLM-as-a-judge is a scalable and explainable way to approximate human preferences, which are otherwise very expensive to obtain. Additionally, we show our benchmark and traditional benchmarks complement each other by evaluating several variants of LLaMA and Vicuna. The MT-bench questions, 3K expert votes, and 30K conversations with human preferences are publicly available at https://github.com/lm-sys/FastChat/tree/main/fastchat/llm_judge.

DIALOGPT : Large-Scale Generative Pre-training for Conversational Response Generation

Abstract: We present a large, tunable neural conversational response generation model, DIALOGPT (dialogue generative pre-trained transformer). Trained on 147M conversation-like exchanges extracted from Reddit comment chains over a period spanning from 2005 through 2017, DialoGPT extends the Hugging Face PyTorch transformer to attain a performance close to human both in terms of automatic and human evaluation in single-turn dialogue settings. We show that conversational systems that leverage DialoGPT generate more relevant, contentful and context-consistent responses than strong baseline systems. The pre-trained model and training pipeline are publicly released to facilitate research into neural response generation and the development of more intelligent open-domain dialogue systems.

CodeApex evaluates 14 LLMs along with their variants.

CodeApex totally evaluates 14 LLMs, comprising a total of 37 variants.
The tested LLMs and their variants are listed in Table {{table:dbd98647-3ec8-4b89-a6f6-3f401d4bca7e}} .
The variants of the models encompass the chatbot mode and the completion mode, as well as different parameter sizes and versions within the same base model.
In the programming comprehension task, we conduct experiments on 11 LLMs, including both open-source models and API-based models.
In code generation task, we evaluate 12 accessible state-of-art LLMs, including both general-purpose and code-specialized models.
The largest open-source models have 13 billion parameters, while the largest parameterized model based on API is GPT3.5-turbo.


Nonlocality of Majorana Modes in Hybrid Nanowires

Abstract: Spatial separation of Majorana zero modes distinguishes trivial from topological midgap states and is key to topological protection in quantum computing applications. Although signatures of Majorana zero modes in tunneling spectroscopy have been reported in numerous studies, a quantitative measure of the degree of separation, or nonlocality, of the emergent zero modes has not been reported. Here, we present results of an experimental study of nonlocality of emergent zero modes in superconductor-semiconductor hybrid nanowire devices. The approach takes advantage of recent theory showing that nonlocality can be measured from splitting due to hybridization of the zero mode in resonance with a quantum dot state at one end of the nanowire. From these splittings as well as anticrossing of the dot states, measured for even and odd occupied quantum dot states, we extract both the degree of nonlocality of the emergent zero mode, as well as the spin canting angles of the nonlocal zero mode. Depending on the device measured, we obtain either a moderate degree of nonlocality, suggesting a partially separated Andreev subgap state, or a highly nonlocal state consistent with a well-developed Majorana mode.

Scaling of Majorana Zero-Bias Conductance Peaks

Abstract: We report an experimental study of the scaling of zero-bias conductance peaks compatible with Majorana zero modes as a function of magnetic field, tunnel coupling, and temperature in one-dimensional structures fabricated from an epitaxial semiconductor-superconductor heterostructure. Results are consistent with theory, including a peak conductance that is proportional to tunnel coupling, saturates at $2e^2/h$, decreases as expected with field-dependent gap, and collapses onto a simple scaling function in the dimensionless ratio of temperature and tunnel coupling.

Parity independence of the zero-bias conductance peak in a nanowire based topological superconductor-quantum dot hybrid device

Abstract: We explore the signatures of Majorana fermions in a nanowire based topological superconductor-quantum dot-topological superconductor hybrid device by charge transport measurements. The device is made from an epitaxially grown InSb nanowire with two superconductor Nb contacts on a Si/SiO$_2$ substrate. At low temperatures, a quantum dot is formed in the segment of the InSb nanowire between the two Nb contacts and the two Nb contacted segments of the InSb nanowire show superconductivity due to the proximity effect. At zero magnetic field, well defined Coulomb diamonds and the Kondo effect are observed in the charge stability diagram measurements in the Coulomb blockade regime of the quantum dot. Under the application of a finite, sufficiently strong magnetic field, a zero-bias conductance peak structure is observed in the same Coulomb blockade regime. It is found that the zero-bias conductance peak is present in many consecutive Coulomb diamonds, irrespective of the even-odd parity of the quasi-particle occupation number in the quantum dot. In addition, we find that the zero-bias conductance peak is in most cases accompanied by two differential conductance peaks, forming a triple-peak structure, and the separation between the two side peaks in bias voltage shows oscillations closely correlated to the background Coulomb conductance oscillations of the device. The observed zero-bias conductance peak and the associated triple-peak structure are in line with the signatures of Majorana fermion physics in a nanowire based topological superconductor-quantum dot-topological superconductor system, in which the two Majorana bound states adjacent to the quantum dot are hybridized into a pair of quasi-particle states with finite energies and the other two Majorana bound states remain as the zero-energy modes located at the two ends of the entire InSb nanowire.

Ubiquitous non-Majorana Zero-Bias Conductance Peaks in Nanowire Devices

Abstract: We perform tunneling measurements on indium antimonide nanowire/superconductor hybrid devices fabricated for the studies of Majorana bound states. At finite magnetic field, resonances that strongly resemble Majorana bound states, including zero-bias pinning, become common to the point of ubiquity. Since Majorana bound states are predicted in only a limited parameter range in nanowire devices, we seek an alternative explanation for the observed zero-bias peaks. With the help of a self-consistent Poission-Schr\"odinger multiband model developed in parallel, we identify several families of trivial subgap states which overlap and interact, giving rise to a crowded spectrum near zero energy and zero-bias conductance peaks in experiments. These findings advance the search for Majorana bound states through improved understanding of broader phenomena found in superconductor-semiconductor systems.

Signatures of Majorana fermions in hybrid superconductor-semiconductor nanowire devices

Abstract: Majorana fermions are particles identical to their own antiparticles. They have been theoretically predicted to exist in topological superconductors. We report electrical measurements on InSb nanowires contacted with one normal (Au) and one superconducting electrode (NbTiN). Gate voltages vary electron density and define a tunnel barrier between normal and superconducting contacts. In the presence of magnetic fields of order 100 mT we observe bound, mid-gap states at zero bias voltage. These bound states remain fixed to zero bias even when magnetic fields and gate voltages are changed over considerable ranges. Our observations support the hypothesis of Majorana fermions in nanowires coupled to superconductors.

Majorana bound states in a coupled quantum-dot hybrid-nanowire system

Abstract: Hybrid nanowires combining semiconductor and superconductor materials appear well suited for the creation, detection, and control of Majorana bound states (MBSs). We demonstrate the emergence of MBSs from coalescing Andreev bound states (ABSs) in a hybrid InAs nanowire with epitaxial Al, using a quantum dot at the end of the nanowire as a spectrometer. Electrostatic gating tuned the nanowire density to a regime of one or a few ABSs. In an applied axial magnetic field, a topological phase emerges in which ABSs move to zero energy and remain there, forming MBSs. We observed hybridization of the MBS with the end-dot bound state, which is in agreement with a numerical model. The ABS/MBS spectra provide parameters that are useful for understanding topological superconductivity in this system.

Bookmark of the works we considered in experimental testings.

In table {{table:1cbbd90a-caaa-4756-aa69-31664aab00e2}}  below we show the bookmark for the works we considered in experimental testings. 16 figures (sample data) out of 10 works are tested in total.


PoS Tagging, Entity Extraction, Relation Extraction, Natural Language Inference, Question Answering, Text Summarization, Common Sense Reasoning

Question Answering, Natural Language Generation, Text Summarization

Natural Language Inference, Paraphrase Identification, Commonsense Reasoning, Question Answering, Parts of Speech Tagging, Sentiment Analysis, Text Summarization

German, Spanish, Russian, Turkish, Catalan

Eleven languages including Hindi, Arabic and Italian

Ten languages including Chinese and Japanese.

ChatGPT Beyond English: Towards a Comprehensive Evaluation of Large Language Models in Multilingual Learning

Abstract: Over the last few years, large language models (LLMs) have emerged as the most important breakthroughs in natural language processing (NLP) that fundamentally transform research and developments in the field. ChatGPT represents one of the most exciting LLM systems developed recently to showcase impressive skills for language generation and highly attract public attention. Among various exciting applications discovered for ChatGPT in English, the model can process and generate texts for multiple languages due to its multilingual training data. Given the broad adoption of ChatGPT for English in different problems and areas, a natural question is whether ChatGPT can also be applied effectively for other languages or it is necessary to develop more language-specific technologies. The answer to this question requires a thorough evaluation of ChatGPT over multiple tasks with diverse languages and large datasets (i.e., beyond reported anecdotes), which is still missing or limited in current research. Our work aims to fill this gap for the evaluation of ChatGPT and similar LLMs to provide more comprehensive information for multilingual NLP applications. While this work will be an ongoing effort to include additional experiments in the future, our current paper evaluates ChatGPT on 7 different tasks, covering 37 diverse languages with high, medium, low, and extremely low resources. We also focus on the zero-shot learning setting for ChatGPT to improve reproducibility and better simulate the interactions of general users. Compared to the performance of previous models, our extensive experimental results demonstrate a worse performance of ChatGPT for different NLP tasks and languages, calling for further research to develop better models and understanding for multilingual learning.

Is ChatGPT a Highly Fluent Grammatical Error Correction System? A Comprehensive Evaluation

Abstract: ChatGPT, a large-scale language model based on the advanced GPT-3.5 architecture, has shown remarkable potential in various Natural Language Processing (NLP) tasks. However, there is currently a dearth of comprehensive study exploring its potential in the area of Grammatical Error Correction (GEC). To showcase its capabilities in GEC, we design zero-shot chain-of-thought (CoT) and few-shot CoT settings using in-context learning for ChatGPT. Our evaluation involves assessing ChatGPT's performance on five official test sets in three different languages, along with three document-level GEC test sets in English. Our experimental results and human evaluations demonstrate that ChatGPT has excellent error detection capabilities and can freely correct errors to make the corrected sentences very fluent, possibly due to its over-correction tendencies and not adhering to the principle of minimal edits. Additionally, its performance in non-English and low-resource settings highlights its potential in multilingual GEC tasks. However, further analysis of various types of errors at the document-level has shown that ChatGPT cannot effectively correct agreement, coreference, tense errors across sentences, and cross-sentence boundary errors.

On the Multilingual Capabilities of Very Large-Scale English Language Models

Abstract: Generative Pre-trained Transformers (GPTs) have recently been scaled to unprecedented sizes in the history of machine learning. These models, solely trained on the language modeling objective, have been shown to exhibit outstanding zero, one, and few-shot learning capabilities in a number of different tasks. Nevertheless, aside from anecdotal experiences, little is known regarding their multilingual capabilities, given the fact that the pre-training corpus is almost entirely composed of English text. In this work, we investigate its potential and limits in three tasks: extractive question-answering, text summarization and natural language generation for five different languages, as well as the effect of scale in terms of model size. Our results show that GPT-3 can be almost as useful for many languages as it is for English, with room for improvement if optimization of the tokenization is addressed.

MEGA: Multilingual Evaluation of Generative AI

Abstract: Generative AI models have shown impressive performance on many Natural Language Processing tasks such as language understanding, reasoning, and language generation. An important question being asked by the AI community today is about the capabilities and limits of these models, and it is clear that evaluating generative AI is very challenging. Most studies on generative LLMs have been restricted to English and it is unclear how capable these models are at understanding and generating text in other languages. We present the first comprehensive benchmarking of generative LLMs - MEGA, which evaluates models on standard NLP benchmarks, covering 16 NLP datasets across 70 typologically diverse languages. We compare the performance of generative LLMs including Chat-GPT and GPT-4 to State of the Art (SOTA) non-autoregressive models on these tasks to determine how well generative models perform compared to the previous generation of LLMs. We present a thorough analysis of the performance of models across languages and tasks and discuss challenges in improving the performance of generative LLMs on low-resource languages. We create a framework for evaluating generative LLMs in the multilingual setting and provide directions for future progress in the field.

Evaluating ChatGPT's Performance for Multilingual and Emoji-based Hate Speech Detection

Abstract: Hate speech is a severe issue that affects many online platforms. So far, several studies have been performed to develop robust hate speech detection systems. Large language models like ChatGPT have recently shown a great promise in performing several tasks, including hate speech detection. However, it is crucial to comprehend the limitations of these models to build robust hate speech detection systems. To bridge this gap, our study aims to evaluate the strengths and weaknesses of the ChatGPT model in detecting hate speech at a granular level across 11 languages. Our evaluation employs a series of functionality tests that reveals various intricate failures of the model which the aggregate metrics like macro F1 or accuracy are not able to unfold. In addition, we investigate the influence of complex emotions, such as the use of emojis in hate speech, on the performance of the ChatGPT model. Our analysis highlights the shortcomings of the generative models in detecting certain types of hate speech and highlighting the need for further research and improvements in the workings of these models.

Are Large Language Model-based Evaluators the Solution to Scaling Up Multilingual Evaluation?

Abstract: Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks, yet their evaluation, particularly in languages beyond the top 20, remains inadequate due to existing benchmarks and metrics limitations. Employing LLMs as evaluators to rank or score other models’ outputs emerges as a viable solution, addressing the constraints tied to human annotators and established benchmarks. In this study, we explore the potential of LLM-based evaluators in enhancing multilingual evaluation by calibrating them against 20K human judgments across three text-generation tasks, five metrics, and eight languages. Our analysis reveals a bias in LLM-based evaluators towards higher scores, underscoring the necessity of calibration with native speaker judgments, especially in low-resource and non-Latin script languages, to ensure accurate evaluation of LLM performance across diverse languages.

Summary of research works exploring GLLMs for NLP tasks in multilingual settings. Here, ZS represents zero-shot, and FS represents few-shot.

Research works exploring GLLMs in multilingual settings. Recently, some of the research works focused on evaluating GLLMs across various non-English languages. The evaluation is done on various tasks like parts of speech tagging {{cite:8dbe182}}, {{cite:2063b21}}, named entity recognition {{cite:8dbe182}}, {{cite:2542748}}, relation extraction {{cite:8dbe182}}, natural language inference {{cite:8dbe182}}, {{cite:2063b21}}, {{cite:2542748}}, question answering {{cite:8dbe182}}, {{cite:02c324e}}, {{cite:2063b21}}, {{cite:401b5ff}}, {{cite:2542748}}, text summarization {{cite:8dbe182}}, {{cite:02c324e}}, {{cite:2063b21}}, {{cite:2542748}}, commonsense reasoning {{cite:8dbe182}}, {{cite:2063b21}}, grammar error correction {{cite:bff5edb}}, text generation {{cite:02c324e}}, {{cite:d5c17a2}}, paraphrase identification {{cite:2063b21}}, sentiment analysis {{cite:2063b21}}, {{cite:f43ce11}}, {{cite:2542748}}, language identification {{cite:f43ce11}}, machine translation {{cite:f43ce11}}, {{cite:2542748}}, genre identification {{cite:60cb905}}, hate speech detection {{cite:df88fab}} and toxicity detection {{cite:2542748}}. Most of the research focused on general domain datasets, except a few focused on other domains like social media {{cite:df88fab}}, {{cite:2542748}} and news {{cite:2542748}}. Table {{table:7ea67cfa-25a5-471e-8968-bffdff6f83ca}}  presents a summary of research works exploring GLLMs for NLP tasks in multilingual settings.


unrelated (75%), disagree (10%), agree (9%), discuss (6%)

comment (41%), unrelated (38%), support (13%), refute (8%)

argument against (56%), argument for (44%)

The Argument Reasoning Comprehension Task: Identification and Reconstruction of Implicit Warrants

Abstract: Reasoning is a crucial part of natural language argumentation. To comprehend an argument, one must analyze its warrant, which explains why its claim follows from its premises. As arguments are highly contextualized, warrants are usually presupposed and left implicit. Thus, the comprehension does not only require language understanding and logic skills, but also depends on common sense. In this paper we develop a methodology for reconstructing warrants systematically. We operationalize it in a scalable crowdsourcing process, resulting in a freely licensed dataset with warrants for 2k authentic arguments from news comments. On this basis, we present a new challenging task, the argument reasoning comprehension task. Given an argument with a claim and a premise, the goal is to choose the correct implicit warrant from two options. Both warrants are plausible and lexically close, but lead to contradicting claims. A solution to this task will define a substantial step towards automatic warrant reconstruction. However, experiments with several neural attention and language models reveal that current approaches do not suffice.

Seeing Things from a Different Angle:Discovering Diverse Perspectives about Claims

Abstract: One key consequence of the information revolution is a significant increase and a contamination of our information supply. The practice of fact checking won’t suffice to eliminate the biases in text data we observe, as the degree of factuality alone does not determine whether biases exist in the spectrum of opinions visible to us. To better understand controversial issues, one needs to view them from a diverse yet comprehensive set of perspectives. For example, there are many ways to respond to a claim such as “animals should have lawful rights”, and these responses form a spectrum of perspectives, each with a stance relative to this claim and, ideally, with evidence supporting it. Inherently, this is a natural language understanding task, and we propose to address it as such. Specifically, we propose the task of substantiated perspective discovery where, given a claim, a system is expected to discover a diverse set of well-corroborated perspectives that take a stance with respect to the claim. Each perspective should be substantiated by evidence paragraphs which summarize pertinent results and facts. We construct PERSPECTRUM, a dataset of claims, perspectives and evidence, making use of online debate websites to create the initial data collection, and augmenting it using search engines in order to expand and diversify our dataset. We use crowd-sourcing to filter out noise and ensure high-quality data. Our dataset contains 1k claims, accompanied with pools of 10k and 8k perspective sentences and evidence paragraphs, respectively. We provide a thorough analysis of the dataset to highlight key underlying language understanding challenges, and show that human baselines across multiple subtasks far outperform ma-chine baselines built upon state-of-the-art NLP techniques. This poses a challenge and opportunity for the NLP community to address.

A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking

Abstract: Automated fact-checking based on machine learning is a promising approach to identify false information distributed on the web. In order to achieve satisfactory performance, machine learning methods require a large corpus with reliable annotations for the different tasks in the fact-checking process. Having analyzed existing fact-checking corpora, we found that none of them meets these criteria in full. They are either too small in size, do not provide detailed annotations, or are limited to a single domain. Motivated by this gap, we present a new substantially sized mixed-domain corpus with annotations of good quality for the core fact-checking tasks: document retrieval, evidence extraction, stance detection, and claim validation. To aid future corpus construction, we describe our methodology for corpus creation and annotation, and demonstrate that it results in substantial inter-annotator agreement. As baselines for future research, we perform experiments on our corpus with a number of model architectures that reach high performance in similar problem settings. Finally, to support the development of future models, we provide a detailed error analysis for each of the tasks. Our results show that the realistic, multi-domain setting defined by our data poses new challenges for the existing models, providing opportunities for considerable improvement by future systems.

Will-They-Won't-They: A Very Large Dataset for Stance Detection on Twitter

Abstract: We present a new challenging stance detection dataset, called Will-They-Won't-They (WT-WT), which contains 51,284 tweets in English, making it by far the largest available dataset of the type. All the annotations are carried out by experts; therefore, the dataset constitutes a high-quality and reliable benchmark for future research in stance detection. Our experiments with a wide range of recent state-of-the-art stance detection systems show that the dataset poses a strong challenge to existing models in this domain.

Cross-topic Argument Mining from Heterogeneous Sources Using Attention-based Neural Networks

Abstract: Argument mining is a core technology for automating argument search in large document collections. Despite its usefulness for this task, most current approaches to argument mining are designed for use only with specific text types and fall short when applied to heterogeneous texts. In this paper, we propose a new sentential annotation scheme that is reliably applicable by crowd workers to arbitrary Web texts. We source annotations for over 25,000 instances covering eight controversial topics. The results of cross-topic experiments show that our attention-based neural network generalizes best to unseen topics and outperforms vanilla BiLSTM models by 6% in accuracy and 11% in F-score.

Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations

Abstract: Stance detection is an important component of understanding hidden influences in everyday life. Since there are thousands of potential topics to take a stance on, most with little to no training data, we focus on zero-shot stance detection: classifying stance from no training examples. In this paper, we present a new dataset for zero-shot stance detection that captures a wider range of topics and lexical variation than in previous datasets. Additionally, we propose a new model for stance detection that implicitly captures relationships between topics using generalized topic representations and show that this model improves performance on a number of challenging linguistic phenomena.

Stance Detection Benchmark datasets and their characteristics (sorted by source, then alphabetically). This table is based on {{cite:25fe434}}.

We use a SD benchmark {{cite:6ae40fe}}, {{cite:25fe434}} which covers 16 datasets altogether in English for research on (cross-domain) stance detection.
We use this benchmark (Table {{table:0f3990bc-ee28-4643-8b13-51e10d7fa436}} ) because it shows a large diversity regarding text sources, the number of targets, the number of annotated instances, and label imbalance.
Thus, it provides a suitable testbed to evaluate the effectiveness of our context injection approach.
More information about the details of each dataset can be found in the Appendix REF .