Embeds the multi-bit WM in the weights of the selected layer(s) of the target DNN by adding an additive binary cross-entropy loss as the WM regularization term.

Leverages backdoor images (misclassified) and random labels as the WM key set to embed the zero-bit WM in the pertinent DNN.

Retains the accuracy of the model; Robust against model compression and fine-tuning attacks.

Preserve the accuracy of the model; Yields a high detection rate.

Vulnerable to WM overwriting attacks. WM is not data-aware.

Susceptible to WM removal attacks; High overhead to embed the WM.

Embedding Watermarks into Deep Neural Networks

Abstract: Deep neural networks have recently achieved significant progress. Sharing trained models of these deep neural networks is very important in the rapid progress of researching or developing deep neural network systems. At the same time, it is necessary to protect the rights of shared trained models. To this end, we propose to use a digital watermarking technology to protect intellectual property or detect intellectual property infringement of trained models. Firstly, we formulate a new problem: embedding watermarks into deep neural networks. We also define requirements, embedding situations, and attack types for watermarking to deep neural networks. Secondly, we propose a general framework to embed a watermark into model parameters using a parameter regularizer. Our approach does not hurt the performance of networks into which a watermark is embedded. Finally, we perform comprehensive experiments to reveal the potential of watermarking to deep neural networks as a basis of this new problem. We show that our framework can embed a watermark in the situations of training a network from scratch, fine-tuning, and distilling without hurting the performance of a deep neural network. The embedded watermark does not disappear even after fine-tuning or parameter pruning; the watermark completely remains even after removing 65% of parameters were pruned. The implementation of this research is: https://github.com/yu4u/dnn-watermark

Turning Your Weakness Into a Strength: Watermarking Deep Neural Networks by Backdooring

Abstract: Deep Neural Networks have recently gained lots of success after enabling several breakthroughs in notoriously challenging problems. Training these networks is computationally expensive and requires vast amounts of training data. Selling such pre-trained models can, therefore, be a lucrative business model. Unfortunately, once the models are sold they can be easily copied and redistributed. To avoid this, a tracking mechanism to identify models as the intellectual property of a particular vendor is necessary. In this work, we present an approach for watermarking Deep Neural Networks in a black-box way. Our scheme works for general classification tasks and can easily be combined with current learning algorithms. We show experimentally that such a watermark has no noticeable impact on the primary task that the model is designed for and evaluate the robustness of our proposal against a multitude of practical attacks. Moreover, we provide a theoretical analysis, relating our approach to previous work on backdooring.

Summary of contemporary DNN watermarking methods.

In this section, we survey the present white-box and black-box DNN watermarking papers. To provide a fair comparison, we deploy the requirements for an effective DNN watermarking methodology as shown in Table {{table:68fdc24a-0d7c-4e9a-beb9-29e11964776f}} . For details about the definition of each criterion, we refer the readers to the paper {{cite:4aef5b5}}. We summarize the workflow, advantages, and disadvantages of each watermarking method in Table {{table:9814082a-8a27-42e5-a4a3-c3e3378277b0}} . The quantitative performance comparison of these techniques is given in Section .
Potential watermark (WM) removal attacks are discussed in Section REF .

In this section, we provide a quantitative comparison of the papers summarized in Table {{table:9814082a-8a27-42e5-a4a3-c3e3378277b0}} . We implement the white-box weights watermarking {{cite:b5d6c52}} using their open-sourced code in {{cite:325cfe4}}. All the other watermarking techniques are implemented based on the work flow and experimental setup described in the original papers. It is worth noting that {{cite:47f4cb3}} does not provide the decision threshold for WM detection. We use as {{formula:d07a3c73-8a66-4ee2-b62c-905c9f9388e9}}  accuracy on the WM key set as the WM detection threshold in our experiments to provide a fair comparison with other watermarking methods. We explicitly compare the watermarking performance regarding each metric discussed in Table {{table:68fdc24a-0d7c-4e9a-beb9-29e11964776f}}  as follows.


Towards Causal VQA: Revealing and Reducing Spurious Correlations by Invariant and Covariant Semantic Editing

Abstract: Despite significant success in Visual Question Answering (VQA), VQA models have been shown to be notoriously brittle to linguistic variations in the questions. Due to deficiencies in models and datasets, today’s models often rely on correlations rather than predictions that are causal w.r.t. data. In this paper, we propose a novel way to analyze and measure the robustness of the state of the art models w.r.t semantic visual variations as well as propose ways to make models more robust against spurious correlations. Our method performs automated semantic image manipulations and tests for consistency in model predictions to quantify the model robustness as well as generate synthetic data to counter these problems. We perform our analysis on three diverse, state of the art VQA models and diverse question types with a particular focus on challenging counting questions. In addition, we show that models can be made significantly more robust against inconsistent predictions using our edited data. Finally, we show that results also translate to real-world error cases of state of the art models, which results in improved overall performance

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

Abstract: Benefiting from large-scale pre-training, we have witnessed significant performance boost on the popular Visual Question Answering (VQA) task. Despite rapid progress, it remains unclear whether these state-of-the-art (SOTA) models are robust when encountering examples in the wild. To study this, we introduce Adversarial VQA, a new large-scale VQA benchmark, collected iteratively via an adversarial human-and-model-in-the-loop procedure. Through this new benchmark, we discover several interesting findings. (i) Surprisingly, we find that during dataset collection, non-expert annotators can easily attack SOTA VQA models successfully. (ii) Both large-scale pre-trained models and adversarial training methods achieve far worse performance on the new benchmark than over standard VQA v2 dataset, revealing the fragility of these models while demonstrating the effectiveness of our adversarial dataset. (iii) When used for data augmentation, our dataset can effectively boost model performance on other robust VQA benchmarks. We hope our Adversarial VQA dataset can shed new light on robustness study in the community and serve as a valuable benchmark for future work.

Current causal image question answering datasets.

Image question answering benchmarks evaluate models' capability of answering natural language questions based on a corresponded image. Recent image question answering benchmarks try to collect or generate balanced QA pairs to make the dataset distribution more balanced in question distribution. VQA v2.0 {{cite:4410ae9}} collects complementary QA pairs by replacing the image and the answer in a QA pairs. VQA-CP {{cite:1636945}} resplited the VQA v1 dataset and VQA v2 dataset to construct two new dataset VQA-CP v1 and VQA-CP v2. As Fig. {{figure:efea3a79-4bb1-4103-a391-ed52dd2c9799}}  shows, {{cite:c679a1f}} constructed IV-VQA and CV-VQA datasets using semantic editing to generate image and reexamine the image by human. {{cite:879de9d}} proposed a human-machine adversarial to collect robust QA pairs. Fig. {{figure:631d90f9-2d5b-4287-80e5-eaa92aa770ab}}  illustrates the adversarial data collecting procedure. In Table {{table:bc78acc8-c7bf-454b-8206-c5570bfe4a2f}} , we summarize these datasets in terms of image source, split numbers, collected or not, and rebalanced or not.


Image processing + GIS functionalities, RF, SVMR, RP, k-NN.

WorldView-3 satellite, LiDAR airborne imagery.

Counting and age estimation of the oil palm tree.

Identification and quantification of palm trees

Image Segmentation Parameter Selection and Ant Colony Optimization for Date Palm Tree Detection and Mapping from Very-High-Spatial-Resolution Aerial Imagery

Abstract: Accurate mapping of date palm trees is essential for their sustainable management, yield estimation, and environmental studies. In this study, we integrated geographic object-based image analysis, class-specific accuracy measures, fractional factorial design, metaheuristic feature-selection technique, and rule-based classification to detect and map date palm trees from very-high-spatial-resolution (VHSR) aerial images of two study areas. First, multiresolution segmentation was optimized through the synergy of the F1-score accuracy measure and the robust Taguchi design. Second, ant colony optimization (ACO) was adopted to select the most significant features. Out of 31 features, only 12 significant color invariants and textural features were selected. Third, based on the selected features, the rule-based classification with the aid of a decision tree algorithm was applied to extract date palm trees. The proposed methodology was developed on a subset of the first study area, and ultimately applied to the second study area to investigate its efficiency and transferability. To evaluate the proposed classification scheme, various supervised object-based algorithms, namely random forest (RF), support vector machine (SVM), and k-nearest neighbor (k-NN), were applied to the first study area. The result of image segmentation optimization demonstrated that segmentation optimization based on an integrated F1-score class-specific accuracy measure and Taguchi statistical design showed improvement compared with objective function, along with the Taguchi design. Moreover, the result of the feature selection by ACO outperformed, with almost 88% overall accuracy, several feature-selection techniques, such as chi-square, correlation-based feature selection, gain ratio, information gain, support vector machine, and principal component analysis. The integrated framework for palm tree detection outperformed RF, SVM, and k-NN classification algorithms with an overall accuracy of 91.88% and 87.03%, date palm class-specific accuracies of 0.91 and 0.89, and kappa coefficients of 0.90 and 0.85 for the first and second study areas, respectively. The proposed integrated methodology demonstrated a highly efficient and promising tool to detect and map date palm trees from VHSR aerial images.

Oil Palm Counting and Age Estimation from WorldView-3 Imagery and LiDAR Data Using an Integrated OBIA Height Model and Regression Analysis

Abstract: The current study proposes a new method for oil palm age estimation and counting from Worldview-3 satellite image and light detection and range (LiDAR) airborne imagery. A support vector machine algorithm (SVM) of object-based image analysis (OBIA) was implemented for oil palm counting. The sensitivity analysis was conducted on four SVM kernel types with associated segmentation parameters to obtain the optimal crown coverage delineation. Extracting tree’s crown was integrated with height model and multiregression methods to accurately estimate the age of trees. The multiregression model with multikernel sizes was examined to achieve the most optimized model for age estimation. Applied models were trained and examined over five different oil palm plantations. The results of oil palm counting had an overall accuracy of 98.80%, while the overall accuracy of age estimation showed 84.91%, over all blocks. The relationship between tree’s height and age was significant which supports the polynomial regression function (PRF) model with a kernel size for under 10–12-year-old oil palm trees, while exponential regression function (ERF) is more fitted for older trees (i.e., 22 years old). Overall, recent remote sensing dataset and machine learning techniques are useful in monitoring and detecting oil palm plantation to maximize productivity.

Identifying and Quantifying the Abundance of Economically Important Palms in Tropical Moist Forest Using UAV Imagery

Abstract: Sustainable management of non-timber forest products such as palm fruits is crucial for the long-term conservation of intact forest. A major limitation to expanding sustainable management of palms has been the need for precise information about the resources at scales of tens to hundreds of hectares, while typical ground-based surveys only sample small areas. In recent years, small unmanned aerial vehicles (UAVs) have become an important tool for mapping forest areas as they are cheap and easy to transport, and they provide high spatial resolution imagery of remote areas. We developed an object-based classification workflow for RGB UAV imagery which aims to identify and delineate palm tree crowns in the tropical rainforest by combining image processing and GIS functionalities using color and textural information in an integrative way to show one of the potential uses of UAVs in tropical forests. Ten permanent forest plots with 1170 reference palm trees were assessed from October to December 2017. The results indicate that palm tree crowns could be clearly identified and, in some cases, quantified following the workflow. The best results were obtained using the random forest classifier with an 85% overall accuracy and 0.82 kappa index.

Overview of the main results in considered studies based on ML algorithms for palm tree detection

The 22 selected studies guided the review by reading full text focusing on the following attributes; the applications studied, the data sets used, the CNN architectures and DL environments, and the results obtained. Thus, the CNN architecture reported for each article is the most effective architecture studied, particularly in ablation studies that compare several architectures. Results about the traditional machine learning-based methods and the deep learning-based methods are presented respectively in table {{table:35b78c49-a743-4829-aed9-a7ef04bf23c6}} ,  and  (table  continue).

From the previous studies in (Table {{table:35b78c49-a743-4829-aed9-a7ef04bf23c6}} ), the disadvantage is the detection's accuracy. Recently, numerous researchers have started extensively using deep learning methods because it shows excellent achievement in different applications. The aim of CNN-based methods for object detection techniques is to increase the efficiency of palm tree detection, as described in (Table ). The main interest concerning CNN for object detection is that features can be learned automatically from images, which means that we do not need to have prior knowledge about the feature representation of the object or go through a complex handcrafted feature extraction process. It is kind of a fully data-driven scheme. The DL-based method was found to have outperformed other traditional tree detection methods. However, DL requires a massive volume of the sampling data set and complex data models to train it, which can be very costly. Furthermore, GPUs and hundreds of expensive machines would make costs go up. Additionally, with the introduction of transfer learning into DL models {{cite:fbb1a27}}, users can use a small dataset to refine an already formed network with countless images. This is quicker than training a CNN from scratch. Despite the fact that good tree detection results are obtained in small study areas (e.g., less than 5 km2), the actual DL-based tree detection methods are not appropriate within the large-scale study area because palm trees are very similar to other types of vegetation in different areas.


nuScenes: A Multimodal Dataset for Autonomous Driving

Abstract: Robust detection and tracking of objects is crucial for the deployment of autonomous vehicle technology. Image based benchmark datasets have driven development in computer vision tasks such as object detection, tracking and segmentation of agents in the environment. Most autonomous vehicles, however, carry a combination of cameras and range sensors such as lidar and radar. As machine learning based methods for detection and tracking become more prevalent, there is a need to train and evaluate such methods on datasets containing range sensor data along with images. In this work we present nuTonomy scenes (nuScenes), the first dataset to carry the full autonomous vehicle sensor suite: 6 cameras, 5 radars and 1 lidar, all with full 360 degree field of view. nuScenes comprises 1000 scenes, each 20s long and fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It has 7x as many annotations and 100x as many images as the pioneering KITTI dataset. We define novel 3D detection and tracking metrics. We also provide careful dataset analysis as well as baselines for lidar and image based detection and tracking. Data, development kit and more information are available online.

Panoptic nuScenes: A Large-Scale Benchmark for LiDAR Panoptic Segmentation and Tracking

Abstract: Panoptic scene understanding and tracking of dynamic agents are essential for robots and automated vehicles to navigate in urban environments. As LiDARs provide accurate illumination-independent geometric depictions of the scene, performing these tasks using LiDAR point clouds provides reliable predictions. However, existing datasets lack diversity in the type of urban scenes and have a limited number of dynamic object instances which hinders both learning of these tasks as well as credible benchmarking of the developed methods. In this paper, we introduce the large-scale Panoptic nuScenes benchmark dataset that extends our popular nuScenes dataset with point-wise groundtruth annotations for semantic segmentation, panoptic segmentation, and panoptic tracking tasks. To facilitate comparison, we provide several strong baselines for each of these tasks on our proposed dataset. Moreover, we analyze the drawbacks of the existing metrics for panoptic tracking and propose the novel instance-centric PAT metric that addresses the concerns. We present exhaustive experiments that demonstrate the utility of Panoptic nuScenes compared to existing datasets and make the online evaluation server available at \url{nuScenes.org}. We believe that this extension will accelerate the research of novel methods for scene understanding of dynamic urban environments.

Scalability in Perception for Autonomous Driving: Waymo Open Dataset

Abstract: The research community has increasing interest in autonomous driving research, despite the resource intensity of obtaining representative real world data. Existing self-driving datasets are limited in the scale and variation of the environments they capture, even though generalization within and between operating regions is crucial to the over-all viability of the technology. In an effort to help align the research community’s contributions with real-world self-driving problems, we introduce a new large scale, high quality, diverse dataset. Our new dataset consists of 1150 scenes that each span 20 seconds, consisting of well synchronized and calibrated high quality LiDAR and camera data captured across a range of urban and suburban geographies. It is 15x more diverse than the largest camera+LiDAR dataset available based on our proposed diversity metric. We exhaustively annotated this data with 2D (camera image) and 3D (LiDAR) bounding boxes, with consistent identifiers across frames. Finally, we provide strong baselines for 2D as well as 3D detection and tracking tasks. We further study the effects of dataset size and generalization across geographies on 3D detection methods. Find data, code and more up-to-date information at http://www.waymo.com/open.

KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D

Abstract: For the last few decades, several major subfields of artificial intelligence including computer vision, graphics, and robotics have progressed largely independently from each other. Recently, however, the community has realized that progress towards robust intelligent systems such as self-driving cars requires a concerted effort across the different fields. This motivated us to develop KITTI-360, successor of the popular KITTI dataset. KITTI-360 is a suburban driving dataset which comprises richer input modalities, comprehensive semantic instance annotations and accurate localization to facilitate research at the intersection of vision, graphics and robotics. For efficient annotation, we created a tool to label 3D scenes with bounding primitives and developed a model that transfers this information into the 2D image domain, resulting in over 150k images and 1B 3D points with coherent semantic instance annotations across 2D and 3D. Moreover, we established benchmarks and baselines for several tasks relevant to mobile perception, encompassing problems from computer vision, graphics, and robotics on the same dataset, e.g., semantic scene understanding, novel view synthesis and semantic SLAM. KITTI-360 will enable progress at the intersection of these research areas and thus contribute towards solving one of today's grand challenges: the development of fully autonomous self-driving systems.

The ApolloScape Open Dataset for Autonomous Driving and Its Application

Abstract: Autonomous driving has attracted tremendous attention especially in the past few years. The key techniques for a self-driving car include solving tasks like 3D map construction, self-localization, parsing the driving road and understanding objects, which enable vehicles to reason and act. However, large scale data set for training and system evaluation is still a bottleneck for developing robust perception models. In this paper, we present the ApolloScape dataset [1] and its applications for autonomous driving. Compared with existing public datasets from real scenes, e.g., KITTI [2] or Cityscapes [3] , ApolloScape contains much large and richer labelling including holistic semantic dense point cloud for each site, stereo, per-pixel semantic labelling, lanemark labelling, instance segmentation, 3D car instance, high accurate location for every frame in various driving videos from multiple sites, cities and daytimes. For each task, it contains at lease 15x larger amount of images than SOTA datasets. To label such a complete dataset, we develop various tools and algorithms specified for each task to accelerate the labelling process, such as joint 3D-2D segment labeling, active labelling in videos etc. Depend on ApolloScape, we are able to develop algorithms jointly consider the learning and inference of multiple tasks. In this paper, we provide a sensor fusion scheme integrating camera videos, consumer-grade motion sensors (GPS/IMU), and a 3D semantic map in order to achieve robust self-localization and semantic segmentation for autonomous driving. We show that practically, sensor fusion and joint learning of multiple tasks are beneficial to achieve a more robust and accurate system. We expect our dataset and proposed relevant algorithms can support and motivate researchers for further development of multi-sensor fusion and multi-task learning in the field of computer vision.

Argoverse: 3D Tracking and Forecasting With Rich Maps

Abstract: We present Argoverse, a dataset designed to support autonomous vehicle perception tasks including 3D tracking and motion forecasting. Argoverse includes sensor data collected by a fleet of autonomous vehicles in Pittsburgh and Miami as well as 3D tracking annotations, 300k extracted interesting vehicle trajectories, and rich semantic maps. The sensor data consists of 360 degree images from 7 cameras with overlapping fields of view, forward-facing stereo imagery, 3D point clouds from long range LiDAR, and 6-DOF pose. Our 290km of mapped lanes contain rich geometric and semantic metadata which are not currently available in any public dataset. All data is released under a Creative Commons license at Argoverse.org. In baseline experiments, we use map information such as lane direction, driveable area, and ground height to improve the accuracy of 3D object tracking. We use 3D object tracking to mine for more than 300k interesting vehicle trajectories to create a trajectory forecasting benchmark. Motion forecasting experiments ranging in complexity from classical methods (k-NN) to LSTMs demonstrate that using detailed vector maps with lane-level information substantially reduces prediction error. Our tracking and forecasting experiments represent only a superficial exploration of the potential of rich maps in robotic perception. We hope that Argoverse will enable the research community to explore these problems in greater depth.

One Million Scenes for Autonomous Driving: ONCE Dataset

Abstract: Current perception models in autonomous driving have become notorious for greatly relying on a mass of annotated data to cover unseen cases and address the long-tail problem. On the other hand, learning from unlabeled large-scale collected data and incrementally self-training powerful recognition models have received increasing attention and may become the solutions of next-generation industry-level powerful and robust perception models in autonomous driving. However, the research community generally suffered from data inadequacy of those essential real-world scene data, which hampers the future exploration of fully/semi/self-supervised methods for 3D perception. In this paper, we introduce the ONCE (One millioN sCenEs) dataset for 3D object detection in the autonomous driving scenario. The ONCE dataset consists of 1 million LiDAR scenes and 7 million corresponding camera images. The data is selected from 144 driving hours, which is 20x longer than the largest 3D autonomous driving dataset available (e.g. nuScenes and Waymo), and it is collected across a range of different areas, periods and weather conditions. To facilitate future research on exploiting unlabeled data for 3D detection, we additionally provide a benchmark in which we reproduce and evaluate a variety of self-supervised and semi-supervised methods on the ONCE dataset. We conduct extensive analyses on those methods and provide valuable observations on their performance related to the scale of used data. Data, code, and more information are available at https://once-for-auto-driving.github.io/index.html.

A*3D Dataset: Towards Autonomous Driving in Challenging Environments

Abstract: With the increasing global popularity of self-driving cars, there is an immediate need for challenging real-world datasets for benchmarking and training various computer vision tasks such as 3D object detection. Existing datasets either represent simple scenarios or provide only day-time data. In this paper, we introduce a new challenging A*3D dataset which consists of RGB images and LiDAR data with a significant diversity of scene, time, and weather. The dataset consists of high-density images (≈ 10 times more than the pioneering KITTI dataset), heavy occlusions, a large number of nighttime frames (≈ 3 times the nuScenes dataset), addressing the gaps in the existing datasets to push the boundaries of tasks in autonomous driving research to more challenging highly diverse environments. The dataset contains 39K frames, 7 classes, and 230K 3D object annotations. An extensive 3D object detection benchmark evaluation on the A*3D dataset for various attributes such as high density, day-time/night-time, gives interesting insights into the advantages and limitations of training and testing 3D object detection in real-world setting.

A2D2: Audi Autonomous Driving Dataset

Abstract: Research in machine learning, mobile robotics, and autonomous driving is accelerated by the availability of high quality annotated data. To this end, we release the Audi Autonomous Driving Dataset (A2D2). Our dataset consists of simultaneously recorded images and 3D point clouds, together with 3D bounding boxes, semantic segmentation, instance segmentation, and data extracted from the automotive bus. Our sensor suite consists of six cameras and five LiDAR units, providing full 360 degree coverage. The recorded data is time synchronized and mutually registered. Annotations are for non-sequential frames: 41,277 frames with semantic segmentation image and point cloud labels, of which 12,497 frames also have 3D bounding box annotations for objects within the field of view of the front camera. In addition, we provide 392,556 sequential frames of unannotated sensor data for recordings in three cities in the south of Germany. These sequences contain several loops. Faces and vehicle number plates are blurred due to GDPR legislation and to preserve anonymity. A2D2 is made available under the CC BY-ND 4.0 license, permitting commercial use subject to the terms of the license. Data and further information are available at http://www.a2d2.audi.

Overview of widely-used autonomous driving datasets with multimodal sensors. C denotes camera and L denotes LiDAR. Most datasets provide bounding annotations for 3D detection, yet only a few of them provide semantic labels for the LiDAR point cloud due to the high cost. Note that ApolloScape {{cite:d89ba2c}} only provides 3D semantic labels for the static environments.

Autonomous Driving Dataset and Benchmark.
Autonomous driving research thrives on high-quality datasets, which serve as the lifeblood for training and evaluating perception {{cite:7a70c03}}, prediction {{cite:b35a4f1}}, and planning algorithms {{cite:0adf609}}. In 2012, the pioneering KITTI dataset sparked a revolution in autonomous driving research, unlocking a multitude of tasks including object detection, tracking, mapping, and optical/depth estimation {{cite:932c65d}}, {{cite:dfdf0be}}, {{cite:5983812}}, {{cite:3812d39}}, {{cite:c4b5741}}. Since then, the research community has embraced the challenge, giving rise to a wealth of datasets. These datasets push the boundaries of autonomous driving research by addressing challenges posed by multimodal fusion {{cite:7a70c03}}, multi-tasking learning {{cite:1ca4d88}}, {{cite:13a8e18}}, adverse weather {{cite:8d33a58}}, collaborative driving {{cite:3dd3f15}}, {{cite:391f31a}}, {{cite:9cc3f0b}}, repeated driving {{cite:2ca6ee4}}, and dense traffic scenarios {{cite:a461b60}}, {{cite:d46dda3}}, etc.
There are several impactful and widely-used driving datasets such as KITTI-360 {{cite:13a8e18}}, nuScenes {{cite:7a70c03}}, and Waymo {{cite:f957fa1}}. They provide LiDAR and camera recordings as well as point cloud semantics and bounding annotations, as summarized in Tab. {{table:6f5276ea-209f-45ae-92cd-19bb2e58aa3e}} . Therefore, we can create accurate ground truth labels for SSC by aggregating multiple semantic point clouds and leveraging the 3D boxes to handle dynamic objects.


Very Deep Convolutional Networks for Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

Densely Connected Convolutional Networks

Abstract: Recent work has shown that convolutional networks can be substantially deeper, more accurate, and efficient to train if they contain shorter connections between layers close to the input and those close to the output. In this paper, we embrace this observation and introduce the Dense Convolutional Network (DenseNet), which connects each layer to every other layer in a feed-forward fashion. Whereas traditional convolutional networks with L layers have L connections—one between each layer and its subsequent layer—our network has L(L+1)/2 direct connections. For each layer, the feature-maps of all preceding layers are used as inputs, and its own feature-maps are used as inputs into all subsequent layers. DenseNets have several compelling advantages: they alleviate the vanishing-gradient problem, strengthen feature propagation, encourage feature reuse, and substantially reduce the number of parameters. We evaluate our proposed architecture on four highly competitive object recognition benchmark tasks (CIFAR-10, CIFAR-100, SVHN, and ImageNet). DenseNets obtain significant improvements over the state-of-the-art on most of them, whilst requiring less memory and computation to achieve high performance. Code and pre-trained models are available at https://github.com/liuzhuang13/DenseNet.

Neural network models' properties

For evaluation purposes, we implemented Algorithm REF  as
a proof-of-concept tool. The tool is written in Python 3.7.10, and
uses the TensorFlow 2.5 and Keras 2.4 frameworks. For our models, we
used Resnet-10, Resnet-100, VGG16-10, VGG16-200 and Densenet, as
described in Table {{table:9131ceb5-3819-4f56-84a9-7bab51fa69f5}} , all trained using the
CIFAR10 data set. All experiments mentioned in this section were run
using the Google Colab Pro environment, with an
NVIDIA-SMI 470.74 GPU and a single-core Intel(R) Xeon(R) CPU @
2.20GHz. The


Surgical Treatment of Atrial Fibrillation: A Review

Abstract: Atrial fibrillation is the most commonly sustained arrhythmia in man. While it affects millions of patients worldwide, its incidence will markedly increase with an aging population. Primary goals of AF therapy are to (1) reduce embolic complications, particularly stroke, (2) alleviate symptoms, and (3) prevent long-term heart remodelling. These have been proven to be a challenge as there are major limitations in our knowledge of the pathological and electrophysiological mechanisms underlying AF. Although advances continue to be made in the medical management of this condition, pharmacotherapy is often unsuccessful. Because of the high recurrence rate of AF despite antiarrhythmic drug therapy for maintenance of sinus rhythm and the adverse effects of these drugs, there has been growing interest in nonpharmacological strategies. Surgery for treatment of AF has been around for some time. The Cox-Maze procedure is the gold standard for the surgical treatment of atrial fibrillation and has more than 90% success in eliminating atrial fibrillation. Although the cut and sew maze is very effective, it has been superseded by newer operations that rely on alternate energy sources to create lines of conduction block. In addition, the evolution of improved ablation technology and instrumentation has facilitated the development of minimally invasive approaches. In this paper, the rationale for surgical ablation for atrial fibrillation and the different surgical techniques that were developed will be explored. In addition, it will detail the new approaches to surgical ablation of atrial fibrillation that employ alternate energy sources.

Advancements in Imaging for Atrial Fibrillation Ablation: Is There a Potential to Improve Procedural Outcomes?

Abstract: Since the introduction of atrial fibrillation (AF) ablation in the 1990s, the procedure has continuously evolved, with gradual improvements in outcomes and safety. Recent technological advancements include the introduction of contact force catheters and high-resolution electroanatomical mapping systems, while imaging modalities including transesophageal echocardiography and fluoroscopy have become integral parts of AF ablation procedures. Further, intraprocedural intracardiac echocardiography and the integration of cardiac magnetic resonance and computed tomography images with electroanatomical mapping have shown promise to improve procedural outcomes by reducing radiation exposure and procedural times. However, available data on procedural utility and the reduction in AF recurrence rates associated with these modalities are mixed. This review therefore aims to discuss the current common imaging modalities used in AF ablation and their potential impact on outcomes. In particular, imaging is discussed with respect to the important information it offers before, during, and after the procedure. Perspectives on the future of imaging in AF ablation are also shared.

Summary of the review papers related to AF.EAM: electroanatomical mapping;JAF: Journal of Atrial Fibrillation;JACC: Journal of the American College of Cardiology;RMPBM: Magnetic Resonance Materials in Physics, Biology and Medicine;JICRM: The Journal of Innovations in Cardiac Rhythm Management;FCM: Frontiers in Cardiovascular Medicine;CET: Cardiovascular Engineering and Technology.

Table {{table:07a0da3c-7b3a-4dfa-969f-bb83723450ab}}  lists existing review papers related to AF.
One can see that most current AF related review papers focused on a clinical survey instead of the methodology of image computing, such as segmentation or quantification algorithms.
Only two reviews, {{cite:98c2c07}} and {{cite:b79eeb9}}, are similar to ours in terms of topic (LGE MRI) and style (technical).
However, only conventional thresholding methods or only DL-based methods were reviewed in each work.
Fig. {{figure:296a9f66-8d31-4ce9-8610-5f27c014c3c2}}  visualizes the scopes of current reviews as well as this review, and one can see that the scopes are different though with partial overlaps.
Besides, our review organizes the related works according to the clinical pipeline (see Fig. {{figure:eebbc57b-0885-4b75-b27b-b7de6f785943}} ), resulting in an intuitive structure of the paper.


Appropriate TrustCalibrated Trust, Trust, Reliance

Explanation Satisfaction,Satisfaction Scale, Comprehensibility,Explanation Goodness,In {{cite:578cd95}} there is two different scales to measure the Explanation Satisfaction Scale and the Explanation Goodness Checklist, however they are in this paper seen as the same criterion Causability,System Causability Scale

How much a user can distinguish between correct and erroneous predictions and act based on it.

How much small changes in the input causes changes in the prediction.

The degree of how much the users feel they understand the system, the explanations, and the user interface.

Machine Learning Interpretability: A Survey on Methods and Metrics

Abstract: Machine learning systems are becoming increasingly ubiquitous. These systems’s adoption has been expanding, accelerating the shift towards a more algorithmic society, meaning that algorithmically informed decisions have greater potential for significant social impact. However, most of these accurate decision support systems remain complex black boxes, meaning their internal logic and inner workings are hidden to the user and even experts cannot fully understand the rationale behind their predictions. Moreover, new regulations and highly regulated domains have made the audit and verifiability of decisions mandatory, increasing the demand for the ability to question, understand, and trust machine learning systems, for which interpretability is indispensable. The research community has recognized this interpretability problem and focused on developing both interpretable models and explanation methods over the past few years. However, the emergence of these methods shows there is no consensus on how to assess the explanation quality. Which are the most suitable metrics to assess the quality of an explanation? The aim of this article is to provide a review of the current state of the research field on machine learning interpretability while focusing on the societal impact and on the developed methods and metrics. Furthermore, a complete literature review is presented in order to identify future directions of work on this field.

Towards A Rigorous Science of Interpretable Machine Learning

Abstract: As machine learning systems become ubiquitous, there has been a surge of interest in interpretable machine learning: systems that provide explanation for their outputs. These explanations are often used to qualitatively assess other criteria such as safety or non-discrimination. However, despite the interest in interpretability, there is very little consensus on what interpretable machine learning is and how it should be measured. In this position paper, we first define interpretability and describe when interpretability is needed (and when it is not). Next, we suggest a taxonomy for rigorous evaluation and expose open questions towards a more rigorous science of interpretable machine learning.

Metrics for Explainable AI: Challenges and Prospects

Abstract: The question addressed in this paper is: If we present to a user an AI system that explains how it works, how do we know whether the explanation works and the user has achieved a pragmatic understanding of the AI? In other words, how do we know that an explanainable AI system (XAI) is any good? Our focus is on the key concepts of measurement. We discuss specific methods for evaluating: (1) the goodness of explanations, (2) whether users are satisfied by explanations, (3) how well users understand the AI systems, (4) how curiosity motivates the search for explanations, (5) whether the user's trust and reliance on the AI are appropriate, and finally, (6) how the human-XAI work system performs. The recommendations we present derive from our integration of extensive research literatures and our own psychometric evaluations.

The identified explanation criteria.

When the final sample was selected, the surveys were analysed to find evaluation criteria (definitions, usage, and quality threshold value). Each evaluation criterion was initially documented separately and then grouped based on definition (see table {{table:0c06ea0f-4d67-4dae-bcd0-38971133e581}} ).
The analysis also included several additional steps:

we grouped the criteria on how they were defined: how the criteria were collected, if the criteria were objective or subjective, and what the criteria were supposed to measure. The most common or well-established names in the surveys were used as the name of the criteria (groups). Some of the criteria were vaguely described, lacking in usage description. Although several other criteria could be used in an evaluation and were connected to the resulting criteria, they were excluded due to the lack of (method) description of how to use or quantify them in evaluations. After the analysis, we identified eleven criteria (see table {{table:0c06ea0f-4d67-4dae-bcd0-38971133e581}} ) possible to use with some form of measured value.
No threshold values to reach an acceptable quality level could be found, which cause another challenge for comparative evaluations. Without a threshold value for an acceptable level of quality, it is possible to compare if an explanation method is, e.g., creating a higher level of appropriate trust than another method. However, the comparison does not indicate if any of the two methods have reached an acceptable level of appropriate trust.


This Email Could Save Your Life: Introducing the Task of Email Subject Line Generation

Abstract: Given the overwhelming number of emails, an effective subject line becomes essential to better inform the recipient of the email’s content. In this paper, we propose and study the task of email subject line generation: automatically generating an email subject line from the email body. We create the first dataset for this task and find that email subject line generation favor extremely abstractive summary which differentiates it from news headline generation or news single document summarization. We then develop a novel deep learning method and compare it to several baselines as well as recent state-of-the-art text summarization systems. We also investigate the efficacy of several automatic metrics based on correlations with human judgments and propose a new automatic evaluation metric. Our system outperforms competitive baselines given both automatic and human evaluations. To our knowledge, this is the first work to tackle the problem of effective email subject line generation.

On the Use of ArXiv as a Dataset

Abstract: The arXiv has collected 1.5 million pre-print articles over 28 years, hosting literature from scientific fields including Physics, Mathematics, and Computer Science. Each pre-print features text, figures, authors, citations, categories, and other metadata. These rich, multi-modal features, combined with the natural graph structure---created by citation, affiliation, and co-authorship---makes the arXiv an exciting candidate for benchmarking next-generation models. Here we take the first necessary steps toward this goal, by providing a pipeline which standardizes and simplifies access to the arXiv's publicly available data. We use this pipeline to extract and analyze a 6.7 million edge citation graph, with an 11 billion word corpus of full-text research articles. We present some baseline classification results, and motivate application of more exciting generative graph models.

Don’t Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization

Abstract: We introduce “extreme summarization”, a new single-document summarization task which does not favor extractive strategies and calls for an abstractive modeling approach. The idea is to create a short, one-sentence news summary answering the question “What is the article about?”. We collect a real-world, large-scale dataset for this task by harvesting online articles from the British Broadcasting Corporation (BBC). We propose a novel abstractive model which is conditioned on the article’s topics and based entirely on convolutional neural networks. We demonstrate experimentally that this architecture captures long-range dependencies in a document and recognizes pertinent content, outperforming an oracle extractive system and state-of-the-art abstractive approaches when evaluated automatically and by humans.

Teaching Machines to Read and Comprehend

Abstract: Teaching machines to read natural language documents remains an elusive challenge. Machine reading systems can be tested on their ability to answer questions posed on the contents of documents that they have seen, but until now large scale training and test datasets have been missing for this type of evaluation. In this work we define a new methodology that resolves this bottleneck and provides large scale supervised reading comprehension data. This allows us to develop a class of attention based deep neural networks that learn to read real documents and answer complex questions with minimal prior knowledge of language structure.

SAMSum Corpus: A Human-annotated Dialogue Dataset for Abstractive Summarization

Abstract: This paper introduces the SAMSum Corpus, a new dataset with abstractive dialogue summaries. We investigate the challenges it poses for automated summarization by testing several models and comparing their results with those obtained on a corpus of news articles. We show that model-generated summaries of dialogues achieve higher ROUGE scores than the model-generated summaries of news – in contrast with human evaluators’ judgement. This suggests that a challenging task of abstractive dialogue summarization requires dedicated models and non-standard quality measures. To our knowledge, our study is the first attempt to introduce a high-quality chat-dialogues corpus, manually annotated with abstractive summarizations, which can be used by the research community for further studies.

SGD-X: A Benchmark for Robust Generalization in Schema-Guided Dialogue Systems

Abstract: Zero/few-shot transfer to unseen services is a critical challenge in task-oriented dialogue research. The Schema-Guided Dialogue (SGD) dataset introduced a paradigm for enabling models to support any service in zero-shot through schemas, which describe service APIs to models in natural language. We explore the robustness of dialogue systems to linguistic variations in schemas by designing SGD-X - a benchmark extending SGD with semantically similar yet stylistically diverse variants for every schema. We observe that two top state tracking models fail to generalize well across schema variants, measured by joint goal accuracy and a novel metric for measuring schema sensitivity. Additionally, we present a simple model-agnostic data augmentation method to improve schema robustness.

Character-level Convolutional Networks for Text Classification

Abstract: This article offers an empirical exploration on the use of character-level convolutional networks (ConvNets) for text classification. We constructed several large-scale datasets to show that character-level convolutional networks could achieve state-of-the-art or competitive results. Comparisons are offered against traditional models such as bag of words, n-grams and their TFIDF variants, and deep learning models such as word-based ConvNets and recurrent neural networks.

Data sources from which riSum is sampled and the minimum (Min), median (Med), and maximum (Max) sampled document length (in words). 10 documents were sampled without replacement from each of the 10 data sources.

To create riSum, a total of 100 documents are chosen from 10 existing datasets of different domains to ensure the data is as diverse as possible.
The documents are uniformly sampled from each dataset, restricting to documents with a word count between 100 and 500 words (Table {{table:58279ef1-11ae-44fc-9e2f-994e2eb7c320}} ).


Novel Modulation Techniques using Isomers as Messenger Molecules for Nano Communication Networks via Diffusion

Abstract: In this paper, we propose three novel modulation techniques, i.e., concentration-based, molecular-type-based, and molecular-ratio-based, using isomers as messenger molecules for nano communication networks via diffusion. To evaluate achievable rate performance, we compare the proposed techniques with conventional insulin-based concepts under practical scenarios. Analytical and numerical results confirm that the proposed modulation techniques using isomers achieve higher data transmission rate performance than the insulin based concepts. We also investigate the tradeoff between messenger sizes and modulation orders and provide guidelines for selecting from among several possible candidates.

Nanoscale Communication with Brownian Motion

Abstract: In this paper, the problem of communicating using chemical messages propagating using Brownian motion, rather than electromagnetic messages propagating as waves in free space or along a wire, is considered. This problem is motivated by nanotechnological and biotechnological applications, where the energy cost of electromagnetic communication might be prohibitive. Models are given for communication using particles that propagate with Brownian motion, and achievable capacity results are given. Under conservative assumptions, it is shown that rates exceeding one bit per particle are achievable.

Diffusion-Based Nanonetworking: A New Modulation Technique and Performance Analysis

Abstract: In this letter, a new molecular modulation scheme for nanonetworks is proposed. To evaluate the scheme, a system model based on the Poisson distribution is introduced. The error probability of the proposed scheme as well as that of two previously known schemes, the concentration and molecular shift keying modulations, are derived for the Poisson model by taking into account the error propagation effect of previously decoded symbols. The proposed scheme is shown to outperform the previously introduced schemes. This is due to the fact that the decoding of the current symbol in the proposed scheme does not encounter propagation of error, as the decoding of the current symbol does not depend on the previously transmitted and decoded symbols. Finally, fundamental limits on the probability of error of a practical set of encoders and decoders are derived using information theoretical tools.

Comparison matrix of modulation schemes.

In the rest of this section we survey some of the recent work on modulation techniques for MC. Generally, these works can be broken down into three main categories: modulation schemes, inter-symbol-interference (ISI) mitigation schemes, and optimal detection at the receiver. Table {{table:ec4fccd5-0039-4b5f-b999-4aa1a2fcdd40}}  summarizes the different modulation schemes proposed in the literature.


How Robust is GPT-3.5 to Predecessors? A Comprehensive Study on Language Understanding Tasks

Abstract: The GPT-3.5 models have demonstrated impressive performance in various Natural Language Processing (NLP) tasks, showcasing their strong understanding and reasoning capabilities. However, their robustness and abilities to handle various complexities of the open world have yet to be explored, which is especially crucial in assessing the stability of models and is a key aspect of trustworthy AI. In this study, we perform a comprehensive experimental analysis of GPT-3.5, exploring its robustness using 21 datasets (about 116K test samples) with 66 text transformations from TextFlint that cover 9 popular Natural Language Understanding (NLU) tasks. Our findings indicate that while GPT-3.5 outperforms existing fine-tuned models on some tasks, it still encounters significant robustness degradation, such as its average performance dropping by up to 35.74\% and 43.59\% in natural language inference and sentiment analysis tasks, respectively. We also show that GPT-3.5 faces some specific robustness challenges, including robustness instability, prompt sensitivity, and number sensitivity. These insights are valuable for understanding its limitations and guiding future research in addressing these challenges to enhance GPT-3.5's overall performance and generalization abilities.

On the Robustness of ChatGPT: An Adversarial and Out-of-distribution Perspective

Abstract: ChatGPT is a recent chatbot service released by OpenAI and is receiving increasing attention over the past few months. While evaluations of various aspects of ChatGPT have been done, its robustness, i.e., the performance to unexpected inputs, is still unclear to the public. Robustness is of particular concern in responsible AI, especially for safety-critical applications. In this paper, we conduct a thorough evaluation of the robustness of ChatGPT from the adversarial and out-of-distribution (OOD) perspective. To do so, we employ the AdvGLUE and ANLI benchmarks to assess adversarial robustness and the Flipkart review and DDXPlus medical diagnosis datasets for OOD evaluation. We select several popular foundation models as baselines. Results show that ChatGPT shows consistent advantages on most adversarial and OOD classification and translation tasks. However, the absolute performance is far from perfection, which suggests that adversarial and OOD robustness remains a significant threat to foundation models. Moreover, ChatGPT shows astounding performance in understanding dialogue-related texts and we find that it tends to provide informal suggestions for medical tasks instead of definitive answers. Finally, we present in-depth discussions of possible research directions.

On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex

Abstract: Semantic parsing is a technique aimed at constructing a structured representation of the meaning of a natural-language question. Recent advances in language models trained on code have shown superior performance in generating these representations compared to language models trained solely on natural language text. The existing fine-tuned neural semantic parsers are vulnerable to adversarial attacks on natural-language inputs. While it has been established that the robustness of smaller semantic parsers can be enhanced through adversarial training, this approach is not feasible for large language models in real-world scenarios, as it requires both substantial computational resources and expensive human annotation on in-domain semantic parsing data. This paper presents the first empirical study on the adversarial robustness of a prompt-based semantic parser based on CODEX, a stateof-the-art (SOTA) language model trained on code. Our results demonstrate that the large language model of code is vulnerable to carefully crafted adversarial examples. To overcome this challenge, we propose methods for enhancing robustness without requiring substantial amounts of labelled data or intensive computational resources.

PromptBench: Towards Evaluating the Robustness of Large Language Models on Adversarial Prompts

Abstract: The increasing reliance on Large Language Models (LLMs) across academia and industry necessitates a comprehensive understanding of their robustness to prompts. In response to this vital need, we introduce PromptBench, a robustness benchmark designed to measure LLMs' resilience to adversarial prompts. This study uses a plethora of adversarial textual attacks targeting prompts across multiple levels: character, word, sentence, and semantic. The adversarial prompts, crafted to mimic plausible user errors like typos or synonyms, aim to evaluate how slight deviations can affect LLM outcomes while maintaining semantic integrity. These prompts are then employed in diverse tasks, such as sentiment analysis, natural language inference, reading comprehension, machine translation, and math problem-solving. Our study generates 4788 adversarial prompts, meticulously evaluated over 8 tasks and 13 datasets. Our findings demonstrate that contemporary LLMs are not robust to adversarial prompts. Furthermore, we present comprehensive analysis to understand the mystery behind prompt robustness and its transferability. We then offer insightful robustness analysis and pragmatic recommendations for prompt composition, beneficial to both researchers and everyday users. Code is available at: https://github.com/microsoft/promptbench.

Exploring the Robustness of Large Language Models for Solving Programming Problems

Abstract: Using large language models (LLMs) for source code has recently gained attention. LLMs, such as Transformer-based models like Codex and ChatGPT, have been shown to be highly capable of solving a wide range of programming problems. However, the extent to which LLMs understand problem descriptions and generate programs accordingly or just retrieve source code from the most relevant problem in training data based on superficial cues has not been discovered yet. To explore this research question, we conduct experiments to understand the robustness of several popular LLMs, CodeGen and GPT-3.5 series models, capable of tackling code generation tasks in introductory programming problems. Our experimental results show that CodeGen and Codex are sensitive to the superficial modifications of problem descriptions and significantly impact code generation performance. Furthermore, we observe that Codex relies on variable names, as randomized variables decrease the solved rate significantly. However, the state-of-the-art (SOTA) models, such as InstructGPT and ChatGPT, show higher robustness to superficial modifications and have an outstanding capability for solving programming problems. This highlights the fact that slight modifications to the prompts given to the LLMs can greatly affect code generation performance, and careful formatting of prompts is essential for high-quality code generation, while the SOTA models are becoming more robust to perturbations.

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations

Abstract: Despite significant progress having been made in question answering on tabular data (Table QA), it’s unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e.g., replacing key question entities or shuffling table columns. To systematically study the robustness of Table QA models, we propose a benchmark called RobuT, which builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and includes human-annotated adversarial perturbations in terms of table header, table content, and question. Our results indicate that both state-of-the-art Table QA models and large language models (e.g., GPT-3) with few-shot learning falter in these adversarial sets. We propose to address this problem by using large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models.

Is Information Extraction Solved by ChatGPT? An Analysis of Performance, Evaluation Criteria, Robustness and Errors

Abstract: ChatGPT has stimulated the research boom in the field of large language models. In this paper, we assess the capabilities of ChatGPT from four perspectives including Performance, Evaluation Criteria, Robustness and Error Types. Specifically, we first evaluate ChatGPT's performance on 17 datasets with 14 IE sub-tasks under the zero-shot, few-shot and chain-of-thought scenarios, and find a huge performance gap between ChatGPT and SOTA results. Next, we rethink this gap and propose a soft-matching strategy for evaluation to more accurately reflect ChatGPT's performance. Then, we analyze the robustness of ChatGPT on 14 IE sub-tasks, and find that: 1) ChatGPT rarely outputs invalid responses; 2) Irrelevant context and long-tail target types greatly affect ChatGPT's performance; 3) ChatGPT cannot understand well the subject-object relationships in RE task. Finally, we analyze the errors of ChatGPT, and find that"unannotated spans"is the most dominant error type. This raises concerns about the quality of annotated data, and indicates the possibility of annotating data with ChatGPT. The data and code are released at Github site.

Evaluating the Logical Reasoning Ability of ChatGPT and GPT-4

Abstract: Harnessing logical reasoning ability is a comprehensive natural language understanding endeavor. With the release of Generative Pretrained Transformer 4 (GPT-4), highlighted as"advanced"at reasoning tasks, we are eager to learn the GPT-4 performance on various logical reasoning tasks. This report analyses multiple logical reasoning datasets, with popular benchmarks like LogiQA and ReClor, and newly-released datasets like AR-LSAT. We test the multi-choice reading comprehension and natural language inference tasks with benchmarks requiring logical reasoning. We further construct a logical reasoning out-of-distribution dataset to investigate the robustness of ChatGPT and GPT-4. We also make a performance comparison between ChatGPT and GPT-4. Experiment results show that ChatGPT performs significantly better than the RoBERTa fine-tuning method on most logical reasoning benchmarks. With early access to the GPT-4 API we are able to conduct intense experiments on the GPT-4 model. The results show GPT-4 yields even higher performance on most logical reasoning datasets. Among benchmarks, ChatGPT and GPT-4 do relatively well on well-known datasets like LogiQA and ReClor. However, the performance drops significantly when handling newly released and out-of-distribution datasets. Logical reasoning remains challenging for ChatGPT and GPT-4, especially on out-of-distribution and natural language inference datasets. We release the prompt-style logical reasoning datasets as a benchmark suite and name it LogiEval.

A comprehensive evaluation of ChatGPT's zero-shot Text-to-SQL capability

Abstract: This paper presents the first comprehensive analysis of ChatGPT's Text-to-SQL ability. Given the recent emergence of large-scale conversational language model ChatGPT and its impressive capabilities in both conversational abilities and code generation, we sought to evaluate its Text-to-SQL performance. We conducted experiments on 12 benchmark datasets with different languages, settings, or scenarios, and the results demonstrate that ChatGPT has strong text-to-SQL abilities. Although there is still a gap from the current state-of-the-art (SOTA) model performance, considering that the experiment was conducted in a zero-shot scenario, ChatGPT's performance is still impressive. Notably, in the ADVETA (RPL) scenario, the zero-shot ChatGPT even outperforms the SOTA model that requires fine-tuning on the Spider dataset by 4.1\%, demonstrating its potential for use in practical applications. To support further research in related fields, we have made the data generated by ChatGPT publicly available at https://github.com/THU-BPM/chatgpt-sql.

Summary of research works exploring GLLMs robustness to out-of-distribution instances, adversarial prompts and adversarial inputs. Here ZS represents zero-shot, and FS represents few-shot.

Overview. GPT-3 family large language models achieve impressive performances in zero and few-shot settings in many NLP tasks. In some tasks like text classification {{cite:cfeb568}}, relation extraction {{cite:e3d3a16}}, etc. GLLMs without any explicit fine-tuning outperform state-of-the-art fine-tuned models. For example, Sun et al. {{cite:cfeb568}} demonstrated that InstructGPT, with the advanced prompting strategy, achieves SOTA results using just 16 examples on four text classification datasets. Similarly, Wan et al. {{cite:e3d3a16}} achieved SOTA results in relation extraction with the GPT-RE framework. However, to increase the reliability of these models in real-world applications, especially in critical domains like medicine, it is essential to systematically study the robustness of these models in various scenarios. Adversarial robustness refers to the model’s ability to maintain good performance even in the case of deliberately crafted instances {{cite:566121a}}, {{cite:529fb33}}. These instances are called adversarial instances and are carefully designed by making subtle changes in the original inputs to deceive the model. Out-of-distribution (OOD) instances refer to examples that differ significantly from the data distribution used to train the model {{cite:bd2fae0}}. These instances fall outside the range of the model's training data and present challenges to the model's performance and generalization ability. Some of the recent research works focused on evaluating the robustness of GLLMs to out-of-distribution instances {{cite:f3c3cd3}}, {{cite:349b667}}, adversarial prompts {{cite:82caf44}}, {{cite:f1ae20d}}, {{cite:37425e4}} and adversarial inputs {{cite:b658ec6}}, {{cite:3ed9671}}, {{cite:f6f9f24}}, {{cite:79fe067}} in one or more natural language processing tasks. Table {{table:8c4ead6d-a01c-4e18-bb28-429d85f89cc7}}  presents a summary of research works assessing GLLMs robustness to out-of-distribution instances, adversarial prompts and adversarial inputs.


Energy-Efficient Control with Harvesting Predictions for Solar-Powered Wireless Sensor Networks

Abstract: Wireless sensor networks equipped with rechargeable batteries are useful for outdoor environmental monitoring. However, the severe energy constraints of the sensor nodes present major challenges for long-term applications. To achieve sustainability, solar cells can be used to acquire energy from the environment. Unfortunately, the energy supplied by the harvesting system is generally intermittent and considerably influenced by the weather. To improve the energy efficiency and extend the lifetime of the networks, we propose algorithms for harvested energy prediction using environmental shadow detection. Thus, the sensor nodes can adjust their scheduling plans accordingly to best suit their energy production and residual battery levels. Furthermore, we introduce clustering and routing selection methods to optimize the data transmission, and a Bayesian network is used for warning notifications of bottlenecks along the path. The entire system is implemented on a real-time Texas Instruments CC2530 embedded platform, and the experimental results indicate that these mechanisms sustain the networks’ activities in an uninterrupted and efficient manner.

Task Scheduling for Simultaneous IoT Sensing and Energy Harvesting: A Survey and Critical Analysis

Abstract: The Internet of Things (IoT) has important applications in our daily lives including health and fitness tracking, environmental monitoring and transportation. However, sensor nodes in IoT suffer from the limited lifetime of batteries resulting from their finite energy availability. A promising solution is to harvest energy from environmental sources, such as solar, kinetic, thermal and radio frequency, for perpetual and continuous operation of IoT sensor nodes. In addition to energy generation, recently energy harvesters have been used for context detection, eliminating the need for additional activity sensors (e.g. accelerometers), saving space, cost, and energy consumption. Using energy harvesters for simultaneous sensing and energy harvesting enables energy positive sensing -- an important and emerging class of sensors, which harvest higher energy than required for signal acquisition and the additional energy can be used to power other components of the system. Although simultaneous sensing and energy harvesting is an important step forward towards autonomous self-powered sensor nodes, the energy and information availability can be still intermittent, unpredictable and temporally misaligned with various computational tasks on the sensor node. This paper provides a comprehensive survey on task scheduling algorithms for the emerging class of energy harvesting-based sensors (i.e., energy positive sensors) to achieve the sustainable operation of IoT. We discuss inherent differences between conventional sensing and energy positive sensing and provide an extensive critical analysis for devising new task scheduling algorithms incorporating this new class of sensors. Finally, we outline future research directions towards the implementation of autonomous and self-powered IoT.

Previous algorithms in the literature for predicting the harvested energy in solar powered iot sensor node

Algorithms for energy prediction in ehiot

In order to ensure eno, the sensor nodes need to consume the harvested energy in such a way that the current node operation is not affected, and the future tasks do not run out of energy. Therefore, information about the future harvested energy is important to schedule the energy consumption proactively for sustainable operation of the system. In the literature, there are various harvested energy prediction models that utilise previous energy samples, weather conditions and seasonal trends to compute the future harvested energy in energy harvesting based iot. Table {{table:39a3fc9b-f069-49bb-8824-13ab49d22063}}  comprehensively presents some of the harvested energy prediction algorithms that employ statistical, probabilistic and machine learning models to predict the future harvested energy.
On a normal sunny day, the harvested energy from a solar powered node is highest at noon, and decreases at dawn and dusk, finally reaching zero at night {{cite:16d61d5}}, due to the non-availability of sunlight. Knowing this overall pattern of seh, the future solar harvested energy can be predicted using the previous energy pattern. Kansal et al. {{cite:42eb52c}} present a harvested energy prediction model based on ewma for solar powered iot. Their model relies on the intuition that the harvested energy in a particular day at a given time slot is similar to that of the energy harvested in the previous days at the corresponding time slots. Therefore, the harvested energy in a particular time slot is calculated by accumulating the weighted average of harvested energy in the previous days in the same time.
ewma algorithm awards higher weight to the recent energy values and exponentially decreases weight for the previous energy samples to calculate the future harvested energy in solar powered iot. The weight is calculated dynamically using the real previous energy traces, which provide the lowest value of error. However, {{cite:42eb52c}} gives significant prediction error when there is a sudden change in the weather. It is due to the reason that the ewma scheme does not take the seasonal weather trends and diurnal cycles into account. Hassan et al. {{cite:644e6c5}} propose an energy prediction model for solar powered iot, which takes into account the sudden changes in the environment. It also takes into account the seasonal and diurnal cycles of the solar energy.
However, this scheme is more computationally complex as it takes multiple parameters into account and costs more energy as well as processing time, incurring delay in the system. Another technique which considers weather conditions for predicting the harvested energy in solar powered iot is presented in {{cite:8f0b8de}}. It presents a model to predict solar as well as wind harvested energy. The method takes the data from the weather forecast stations to predict the energy to be harvested in future time slots. However, this scheme is also computationally complex and depends on another source, which increases the cost of the system. Additionally, receiving weather data and processing it on an energy-constrained miniaturized sensor node hinders the execution of other time-critical tasks.
Piorno et al. {{cite:16d61d5}} present a prediction algorithm for seh which depends on ewma and takes into account the sudden and abrupt changes in weather conditions. They propose to use the weighting factor depending on the solar conditions of the current day relative to the previous days.
Cui {{cite:fcdf38e}} proposes a seh prediction algorithm for sensor nodes in ehiot. It employs a recurrent long short term memory neural network to forecast the harvested energy. However, this is a complex method that has higher cost in terms of energy, time, memory requirement and computational resources.

In summary, there are various energy prediction algorithms for solar powered iot, which take into account the previous harvested energy values, weather forecasts and previous energy profiles, to correctly predict the future harvested energy, as shown in Table {{table:39a3fc9b-f069-49bb-8824-13ab49d22063}} . However, these algorithms, when implemented on the node, consume a significant amount of harvested energy during their execution. Therefore, in addition to prediction accuracy, the cost in terms of energy and computational complexity must also be explored. Such an analysis will provide the real picture about the models and will identify the algorithm which provides best energy prediction results, while executing within the limited energy and computational resources.


Analyses the security threats, challenges, and mechanisms in all edge paradigms.

Explores dependability and deployment challenges in edge computing. Considers dependability in a wider meaning that includes security.

Discusses MEC from the perspectives of security, dependability, and performance but keeps the discussion decoupled from the architecture.

Presents new challenges in physical security and scalable authentication, considering both centralized and decentralized security mechanisms.

Mobile Edge Computing, Fog et al.: A Survey and Analysis of Security Threats and Challenges

Abstract: For various reasons, the cloud computing paradigm is unable to meet certain requirements (e.g. low latency and jitter, context awareness, mobility support) that are crucial for several applications (e.g. vehicular networks, augmented reality). To fulfil these requirements, various paradigms, such as fog computing, mobile edge computing, and mobile cloud computing, have emerged in recent years. While these edge paradigms share several features, most of the existing research is compartmentalised; no synergies have been explored. This is especially true in the field of security, where most analyses focus only on one edge paradigm, while ignoring the others. The main goal of this study is to holistically analyse the security threats, challenges, and mechanisms inherent in all edge paradigms, while highlighting potential synergies and venues of collaboration. In our results, we will show that all edge paradigms should consider the advances in other paradigms.

Dependability in Edge Computing

Abstract: Edge computing is the practice of placing computing resources at the edges of the Internet in close proximity to devices and information sources. This, much like a cache on a CPU, increases bandwidth and reduces latency for applications but at a potential cost of dependability and capacity. This is because these edge devices are often not as well maintained, dependable, powerful, or robust as centralized server-class cloud resources. This article explores dependability and deployment challenges in the field of edge computing, what aspects are solvable with today's technology, and what aspects call for new solutions. The first issue addressed is failures, both hard (crash, hang, etc.) and soft (performance-related), and real-time constraint violation. In this domain, edge computing bolsters real-time system capacity through reduced end-to-end latency. However, much like cache misses, overloaded or malfunctioning edge computers can drive latency beyond tolerable limits. Second, decentralized management and device tampering can lead to chain of trust and security or privacy violations. Authentication, access control, and distributed intrusion detection techniques have to be extended from current cloud deployments and need to be customized for the edge ecosystem. The third issue deals with handling multi-tenancy in the typically resource-constrained edge devices and the need for standardization to allow for interoperability across vendor products. We explore the key challenges in each of these three broad issues as they relate to dependability of edge computing and then hypothesize about promising avenues of work in this area.

Academic Surveys Related to MEC Security

Table {{table:720e2c97-c95c-446b-9aaa-8ed8d87e59a2}}  lists recent surveys on security and privacy for MEC. Papers that refer to general MEC security aspects are also considered. Not many papers are fully dedicated to MEC but consider it together with other technologies such as cloud or fog (sometimes only marginally such as in e.g., {{cite:5454e73}}). As mentioned in some of these (e.g., {{cite:4b82dbd}}, {{cite:9966be3}}), some specialized work on the security for MEC has been performed. However, the number of papers referring to particular aspects regarding general edge technologies (and, to some extent, applicable to MEC too) or general security issues that are not MEC specific is large and thus out of the goal of this paper. For example, a large number of papers are dedicated to defences against (D)DoS in MEC (e.g., {{cite:2b03208}}, {{cite:68ace28}}, {{cite:104cce9}}, {{cite:2b59eb1}}) or usage of ML for MEC (e.g., {{cite:43f7cc9}}, {{cite:7267ad3}}).
Security in 5G network slicing has been analyzed in {{cite:d217349}}, and some aspects are relevant for isolation in MEC too.


Interpreted Execution of Business Process Models on Blockchain

Abstract: Blockchain technology provides a tamper-proof mechanism to execute inter-organizational business processes involving mutually untrusted parties. Existing approaches to blockchain-based process execution are based on code generation. In these approaches, a process model is compiled into one or more smart contracts, which are then deployed on a blockchain platform. Given the immutability of the deployed smart contracts, these compiled approaches ensure that all process instances conform to the process model. However, this advantage comes at the price of inflexibility. Any changes to the process model require the redeployment of the smart contracts (a costly operation). In addition, changes cannot be applied to running process instances. To address this lack of flexibility, this paper presents an interpreter of BPMN process models based on dynamic data structures. The proposed interpreter is embedded in a business process execution system with a modular multi-layered architecture, supporting the creation, execution, monitoring and dynamic update of process instances. For efficiency purposes, the interpreter relies on compact bitmap-based encodings of process models. An experimental evaluation shows that the proposed interpreted approach achieves comparable or lower costs relative to existing compiled approaches.

iContractBot: A Chatbot for Smart Contracts’ Specification and Code Generation

Abstract: Recently, Blockchain technology adoption has expanded to many application areas due to the evolution of smart contracts. However, developing smart contracts is non-trivial and challenging due to the lack of tools and expertise in this field. A promising solution to overcome this issue is to use Model-Driven Engineering (MDE), however, using models still involves a learning curve and might not be suitable for non-technical users. To tackle this challenge, chatbot or conversational interfaces can be used to assess the non-technical users to specify a smart contract in gradual and interactive manner. In this paper, we propose iContractBot, a chatbot for modeling and developing smart contracts. Moreover, we investigate how to integrate iContractBot with iContractML, a domainspecific modeling language for developing smart contracts, and instantiate intention models from the chatbot. The iContractBot framework provides a domain-specific language (DSL) based on the user intention and performs model-to-text transformation to generate the smart contract code. A smart contract use case is presented to demonstrate how iContractBot can be utilized for creating models and generating the deployment artifacts for smart contracts based on a simple conversation.

Selected academic, model-driven approaches for blockchain application development that apply code generation.

In Table {{table:df69865e-08c3-4c5a-93bd-598d90ec5bf1}} , the final selection of academic approaches from (S-11) is shown. We further evaluated the approaches regarding the required user expertise - see column Expertise. Approaches where a user must not write any code and only basic understanding of blockchain concepts is required, we consider suitable for non-technical users. In contrast, approaches that require understanding of advanced concepts or chain-specific features, e.g. gas costs in Ethereum, we consider suitable for non-programmers. Finally, if the user has to write any code, the approach is only suitable for programmers.


Spatial As Deep: Spatial CNN for Traffic Scene Understanding

Abstract: Convolutional neural networks (CNNs) are usually built by stacking convolutional operations layer-by-layer. Although CNN has shown strong capability to extract semantics from raw pixels, its capacity to capture spatial relationships of pixels across rows and columns of an image is not fully explored. These relationships are important to learn semantic objects with strong shape priors but weak appearance coherences, such as traffic lanes, which are often occluded or not even painted on the road surface as shown in Fig. 1 (a). In this paper, we propose Spatial CNN (SCNN), which generalizes traditional deep layer-by-layer convolutions to slice-byslice convolutions within feature maps, thus enabling message passings between pixels across rows and columns in a layer. Such SCNN is particular suitable for long continuous shape structure or large objects, with strong spatial relationship but less appearance clues, such as traffic lanes, poles, and wall. We apply SCNN on a newly released very challenging traffic lane detection dataset and Cityscapse dataset. The results show that SCNN could learn the spatial relationship for structure output and significantly improves the performance. We show that SCNN outperforms the recurrent neural network (RNN) based ReNet and MRF+CNN (MRFNet) in the lane detection dataset by 8.7% and 4.6% respectively. Moreover, our SCNN won the 1st place on the TuSimple Benchmark Lane Detection Challenge, with an accuracy of 96.53%.

Ultra Fast Structure-aware Deep Lane Detection

Abstract: Modern methods mainly regard lane detection as a problem of pixel-wise segmentation, which is struggling to address the problem of challenging scenarios and speed. Inspired by human perception, the recognition of lanes under severe occlusion and extreme lighting conditions is mainly based on contextual and global information. Motivated by this observation, we propose a novel, simple, yet effective formulation aiming at extremely fast speed and challenging scenarios. Specifically, we treat the process of lane detection as a row-based selecting problem using global features. With the help of row-based selecting, our formulation could significantly reduce the computational cost. Using a large receptive field on global features, we could also handle the challenging scenarios. Moreover, based on the formulation, we also propose a structural loss to explicitly model the structure of lanes. Extensive experiments on two lane detection benchmark datasets show that our method could achieve the state-of-the-art performance in terms of both speed and accuracy. A light-weight version could even achieve 300+ frames per second with the same resolution, which is at least 4x faster than previous state-of-the-art methods. Our code will be made publicly available.

PolyLaneNet: Lane Estimation via Deep Polynomial Regression

Abstract: One of the main factors that contributed to the large advances in autonomous driving is the advent of deep learning. For safer self-driving vehicles, one of the problems that has yet to be solved completely is lane detection. Since methods for this task have to work in real-time (+30 FPS), they not only have to be effective (i.e., have high accuracy) but they also have to be efficient (i.e., fast). In this work, we present a novel method for lane detection that uses as input an image from a forward-looking camera mounted in the vehicle and outputs polynomials representing each lane marking in the image, via deep polynomial regression. The proposed method is shown to be competitive with existing state-of-the-art methods in the TuSimple dataset while maintaining its efficiency (115 FPS). Additionally, extensive qualitative results on two additional public datasets are presented, alongside with limitations in the evaluation metrics used by recent works for lane detection. Finally, we provide source code and trained models that allow others to replicate all the results shown in this paper, which is surprisingly rare in state-of-the-art lane detection methods. The full source code and pretrained models are available at https://github.com/lucastabelini/PolyLaneNet.

Keep your Eyes on the Lane: Real-time Attention-guided Lane Detection

Abstract: Modern lane detection methods have achieved remarkable performances in complex real-world scenarios, but many have issues maintaining real-time efficiency, which is important for autonomous vehicles. In this work, we pro-pose LaneATT: an anchor-based deep lane detection model, which, akin to other generic deep object detectors, uses the anchors for the feature pooling step. Since lanes follow a regular pattern and are highly correlated, we hypothesize that in some cases global information may be crucial to infer their positions, especially in conditions such as occlusion, missing lane markers, and others. Thus, this work proposes a novel anchor-based attention mechanism that aggregates global information. The model was evaluated extensively on three of the most widely used datasets in the literature. The results show that our method outperforms the current state-of-the-art methods showing both higher efficacy and efficiency. Moreover, an ablation study is performed along with a discussion on efficiency trade-off options that are useful in practice. Code and models are available at https://github.com/lucastabelini/LaneATT.

Target lane detection methods and its selection reason. Acc. is the accuracy of the TuSimple Challenge dataset {{cite:a92ffd7}} in the reference papers.

White-Box DRP Attack We implement the DRP attack {{cite:967ba21}}. While the original DRP attack uses the lane bending objective function, we apply a newly-designed attack objective introduced in §REF  to conduct a fair comparison with other attacks and to deal with the output space different from the original DRP attack, which outputs detected lane lines in the bird's-eye view. All target lane detection methods in Table {{table:5632f6fa-ea41-4ea7-bf93-7ecdd7cbaa8c}}  output detected lane lines in the driver's view.

We evaluate the robustness of 4 major types of lane detection approaches against 3 adversarial attacks: white-box DRP, black-box DRP, black-box drawing-lane-line attacks.
For each approach, we select a representative model for each approach as shown in Table {{table:5632f6fa-ea41-4ea7-bf93-7ecdd7cbaa8c}}  with the selection reasons. The pretrained weights of all models are obtained from the authors' or publicly available websitesWe obtained the pretrained models from:
LaneATT
https://github.com/lucastabelini/LaneATT
SCNN
https://github.com/harryhan618/SCNN_Pytorch
UltraFast
https://github.com/cfzd/Ultra-Fast-Lane-Detection
PolyLaneNet
https://github.com/lucastabelini/PolyLaneNet

.
All pretrained weights are training with the TuSimple Challenge training dataset {{cite:a92ffd7}}. In all our experiments, we use a machine with the AMD Ryzen 9 3950X processor, 128GB memory, and NVIDIA RTX 3090 GPU.

Conventional Evaluation Based on Accuracy and F1 Score

Results.
Table {{table:247b4260-408a-4963-8efe-4dfd90d95c60}}  shows the accuracy and F1 score metrics under the 3 types of adversarial attacks: white-box DRP, black-box DRP, and black-box drawing-lane-line attacks. In the benign scenarios, the accuracy is dropped from the reported number listed in Table {{table:5632f6fa-ea41-4ea7-bf93-7ecdd7cbaa8c}} . This indicates that the ego lane's lines are more difficult to detect correctly than other lane lines. Nevertheless, the LaneATT has only a slight decrease from 95.63% to 94%.
LaneATT also achieves the highest accuracy and F1 score in both the benign scenarios and all attack scenarios except for the white-box DRP attack. Contrarily, UltraFast and PolyLaneNet are the least robust models under the conventional metrics in this evaluation as they have the lowest accuracy and F1 score not only in benign scenarios but also in attack scenarios.


perception and acceptability of text messaging intervention for diabetes care

eHealth systems for supporting diabetes self-management

Perceptions and Acceptability of Text Messaging for Diabetes Care in Primary Care in Argentina: Exploratory Study

Abstract: Background Engagement in self-care behaviors that are essential to optimize diabetes care is challenging for many patients with diabetes. mHealth interventions have been shown to be effective in improving health care outcomes in diabetes. However, more research is needed on patient perceptions to support these interventions, especially in resource settings in low- and middle-income countries. Objective The goal of the research was to explore perceptions and acceptability of a short message service (SMS) text messaging intervention for diabetes care in underserved people with diabetes in Argentina. Methods A qualitative exploratory methodology was adopted as part of the evaluation of a program to strengthen diabetes services in primary care clinics located in low-resource settings. The diabetes program included a text messaging intervention for people with diabetes. A total of 24 semistructured telephone interviews were conducted with people with diabetes. Results Twenty-four middle-aged persons with diabetes were interviewed. Acceptability was considered adequate in terms of its actual use, frequency, and the role of texts as a reminder. We found that text messages could be a mediating device in the patient’s learning processes. Also, being exposed to the texts seemed to help bring about changes in risk perception and care practices and to function as psychosocial support. Another relevant finding was the role of text messaging as a potential facilitator in diabetes care. In this sense, we observed a strong association between receiving text messages and having a better patient-physician relationship. Additionally, social barriers that affect diabetes care such as socioeconomic and psychosocial vulnerability were identified. Conclusions Our findings show positive contributions of a text messaging intervention for the care of people with diabetes. We consider that an SMS strategy has potential to be replicated in other contexts. However, further studies are needed to explore its sustainability and long-term impact from the perspective of patients.

eHealth technologies to support nutrition and physical activity behaviors in diabetes self-management

Abstract: Diabetes is a chronic, complex condition requiring sound knowledge and self-management skills to optimize glycemic control and health outcomes. Dietary intake and physical activity are key diabetes self-management (DSM) behaviors that require tailored education and support. Electronic health (eHealth) technologies have a demonstrated potential for assisting individuals with DSM behaviors. This review provides examples of technologies used to support nutrition and physical activity behaviors in the context of DSM. Technologies covered include those widely used for DSM, such as web-based programs and mobile phone and smartphone applications. In addition, examples of novel tools such as virtual and augmented reality, video games, computer vision for dietary carbohydrate monitoring, and wearable devices are provided. The challenges to, and facilitators for, the use of eHealth technologies in DSM are discussed. Strategies to support the implementation of eHealth technologies within practice and suggestions for future research to enhance nutrition and physical activity behaviors as a part of broader DSM are provided.

Details of reviewed literature and identified themes.

Common themes related to the study objective were identified and mapped. The most common theme was related to social pitfalls, followed by motivational and legal challenges. Figure {{figure:39721eeb-2462-4653-8343-7102d6152e57}}  visualizes the distribution of the themes. Table {{table:b268ebb5-f0f0-4a2c-8632-6a808436e3c8}}  describes the included papers, their subject of study and the themes identified.