Fast and Simultaneous Determination of Soil Properties Using Laser-Induced Breakdown Spectroscopy (LIBS): A Case Study of Typical Farmland Soils in China

Abstract: Accurate management of soil nutrients and fast and simultaneous acquisition of soil properties are crucial in the development of sustainable agriculture. However, the conventional methods of soil analysis are generally labor-intensive, environmentally unfriendly, as well as time- and cost-consuming. Laser-induced breakdown spectroscopy (LIBS) is a “superstar” technique that has yielded outstanding results in the elemental analysis of a wide range of materials. However, its application for analysis of farmland soil faces the challenges of matrix effects, lack of large-scale soil samples with distinct origin and nature, and problems with simultaneous determination of multiple soil properties. Therefore, LIBS technique, in combination with partial least squares regression (PLSR), was applied to simultaneously determinate soil pH, cation exchange capacity (CEC), soil organic matter (SOM), total nitrogen (TN), total phosphorus (TP), total potassium (TK), available phosphorus (AP), and available potassium (AK) in 200 soils from different farmlands in China. The prediction performances of full spectra and characteristic lines were evaluated and compared. Based on full spectra, the estimates of pH, CEC, SOM, TN, and TK achieved excellent prediction abilities with the residual prediction deviation (RPDV) values > 2.0 and the estimate of TP featured good performance with RPDV value of 1.993. However, using characteristic lines only improved the predicted accuracy of SOM, but reduced the prediction accuracies of TN, TP, and TK. In addition, soil AP and AK were predicted poorly with RPDV values of < 1.4 based on both full spectra and characteristic lines. The weak correlations between conventionally analyzed soil AP and AK and soil LIBS spectra are responsible for the poor prediction abilities of AP and AK contents. Findings from this study demonstrated that the LIBS technique combined with multivariate methods is a promising alternative for fast and simultaneous detection of some properties (i.e., pH and CEC) and nutrient contents (i.e., SOM, TN, TP, and TK) in farmland soils because of the extraordinary prediction performances achieved for these attributes.

Deep Spectral CNN for Laser Induced Breakdown Spectroscopy

Abstract: This work proposes a spectral convolutional neural network (CNN) operating on laser induced breakdown spectroscopy (LIBS) signals to learn to (1) disentangle spectral signals from the sources of sensor uncertainty (i.e., pre-process) and (2) get qualitative and quantitative measures of chemical content of a sample given a spectral signal (i.e., calibrate). Once the spectral CNN is trained, it can accomplish either task through a single feed-forward pass, with real-time benefits and without any additional side information requirements including dark current, system response, temperature and detector-to-target range. Our experiments demonstrate that the proposed method outperforms the existing approaches used by the Mars Science Lab for pre-processing and calibration for remote sensing observations from the Mars rover, 'Curiosity'.

Machine Learning Allows Calibration Models to Predict Trace Element Concentration in Soil with Generalized LIBS Spectra

Abstract: Calibration models have been developed for determination of trace elements, silver for instance, in soil using laser-induced breakdown spectroscopy (LIBS). The major concern is the matrix effect. Although it affects the accuracy of LIBS measurements in a general way, the effect appears accentuated for soil because of large variation of chemical and physical properties among different soils. The purpose is to reduce its influence in such way an accurate and soil-independent calibration model can be constructed. At the same time, the developed model should efficiently reduce experimental fluctuations affecting measurement precision. A univariate model first reveals obvious influence of matrix effect and important experimental fluctuation. A multivariate model has been then developed. A key point is the introduction of generalized spectrum where variables representing the soil type are explicitly included. Machine learning has been used to develop the model. After a necessary pretreatment where a feature selection process reduces the dimension of raw spectrum accordingly to the number of available spectra, the data have been fed in to a back-propagation neuronal networks (BPNN) to train and validate the model. The resulted soilindependent calibration model allows average relative error of calibration (REC) and average relative error of prediction (REP) within the range of 5-6%.

Feature extraction methods used for LIBS spectral analysis

There are findings that matrix effects and interference factors could be mitigated by a small fraction of these emission lines using feature selection methods {{cite:839e5d4}}. It is also showed that the feature selection is effective in avoiding the influence of spectral interference and reducing the influence of self-absorption {{cite:339697c}}. Therefore, feature engineering is useful to alleviate matrix effects. Table {{table:3ff67fd9-5093-4c2a-8da7-64d1dae2b8b6}}  summarizes the feature selection methods used for LIBS spectral analysis.


Random Forest, J48, Logistic Regression, Naive Bayes

Evaluation of Machine Learning-based Anomaly Detection Algorithms on an Industrial Modbus/TCP Data Set

Abstract: In the context of the Industrial Internet of Things, communication technology, originally used in home and office environments, is introduced into industrial applications. Commercial off-the-shelf products, as well as unified and well-established communication protocols make this technology easy to integrate and use. Furthermore, productivity is increased in comparison to classic industrial control by making systems easier to manage, set up and configure. Unfortunately, most attack surfaces of home and office environments are introduced into industrial applications as well, which usually have very few security mechanisms in place. Over the last years, several technologies tackling that issue have been researched. In this work, machine learning-based anomaly detection algorithms are employed to find malicious traffic in a synthetically generated data set of Modbus/TCP communication of a fictitious industrial scenario. The applied algorithms are Support Vector Machine (SVM), Random Forest, k-nearest neighbour and k-means clustering. Due to the synthetic data set, supervised learning is possible. Support Vector Machine and k-nearest neighbour perform well with different data sets, while k-nearest neighbour and k-means clustering do not perform satisfactorily.

SCADA System Testbed for Cybersecurity Research Using Machine Learning Approach

Abstract: This paper presents the development of a Supervisory Control and Data Acquisition (SCADA) system testbed used for cybersecurity research. The testbed consists of a water storage tank’s control system, which is a stage in the process of water treatment and distribution. Sophisticated cyber-attacks were conducted against the testbed. During the attacks, the network traffic was captured, and features were extracted from the traffic to build a dataset for training and testing different machine learning algorithms. Five traditional machine learning algorithms were trained to detect the attacks: Random Forest, Decision Tree, Logistic Regression, Naïve Bayes and KNN. Then, the trained machine learning models were built and deployed in the network, where new tests were made using online network traffic. The performance obtained during the training and testing of the machine learning models was compared to the performance obtained during the online deployment of these models in the network. The results show the efficiency of the machine learning models in detecting the attacks in real time. The testbed provides a good understanding of the effects and consequences of attacks on real SCADA environments.

Detecting Cyberattacks in Industrial Control Systems Using Convolutional Neural Networks

Abstract: This paper presents a study on detecting cyberattacks on industrial control systems (ICS) using unsupervised deep neural networks, specifically, convolutional neural networks. The study was performed on a SecureWater Treatment testbed (SWaT) dataset, which represents a scaled-down version of a real-world industrial water treatment plant. e suggest a method for anomaly detection based on measuring the statistical deviation of the predicted value from the observed value.We applied the proposed method by using a variety of deep neural networks architectures including different variants of convolutional and recurrent networks. The test dataset from SWaT included 36 different cyberattacks. The proposed method successfully detects the vast majority of the attacks with a low false positive rate thus improving on previous works based on this data set. The results of the study show that 1D convolutional networks can be successfully applied to anomaly detection in industrial control systems and outperform more complex recurrent networks while being much smaller and faster to train.

Summary of current work on Intrusion Detection Systems in Industrial Control Systems

There has been a substantial increase in machine learning based IDSs for a range of ICS systems. Table {{table:1820e2a6-eff0-4238-9881-c0a1e1cf94d1}}  presents a summary of the existing ICS systems and associated supervised learning approaches to attack detection and classification in these contexts. To date, there has been less focus on AML in this context. Such research has mainly focused on email spam classifiers, malware detection, and very recently there has been interest in AML against network IDSs for traditional networks (e.g. {{cite:2447f4c}}, {{cite:1b2b662}}, {{cite:31cafb6}}).

In the context of ICSs, there exist only a handful of investigations into AML attacks. Specifically, Zizzo et al. {{cite:8113cf3}} showcased a simple AML attack against an Long Short-Term Memory (LSTM) classifier which was applied on an ICS dataset. However, this work is at a preliminary stage as the adversarial samples were generated by manually selecting the feature/acutator values to be perturbed. Yaghoubi and Fainekos {{cite:f633ba4}} proposed a gradient based search approach which was evaluated on a Simulink model of a steam condenser. However, this approach is efficient only against a handful of systems that may specifically employ Recurrent Neural Networks (RNN) with smooth activation functions. Finally, Erba et al. {{cite:2e4f9a4}} demonstrated two types of real-time evasion attacks, again using Recurrent Neural Network models, and used an autoencoder to generate adversarial samples, which is computationally complex. Neither of these aforementioned works investigate defence methods against AML. Conclusively, it is evident that there is room to investigate AML and the defence against such attacks for current IDSs in ICS systems that are supported by supervised learning. Moreover, as Table {{table:1820e2a6-eff0-4238-9881-c0a1e1cf94d1}}  shows, Recurrent Neural Networks are yet to gain prominence in attack detection in an ICS context - with algorithms such as Naive Bayes, Random Forest, SVM, and J48 being much more widely used. We therefore base our experiments in defending against AML on these methods as the state of the art in ML-driven attack detection methods for ICS.

To conform to other comparable IDSs in ICS systems in Table {{table:1820e2a6-eff0-4238-9881-c0a1e1cf94d1}} , the classifiers were also selected based on their ability to support a high-dimensional feature space. The classifiers included:


Wide Residual Networks

Abstract: Deep residual networks were shown to be able to scale up to thousands of layers and still have improving performance. However, each fraction of a percent of improved accuracy costs nearly doubling the number of layers, and so training very deep residual networks has a problem of diminishing feature reuse, which makes these networks very slow to train. To tackle these problems, in this paper we conduct a detailed experimental study on the architecture of ResNet blocks, based on which we propose a novel architecture where we decrease depth and increase width of residual networks. We call the resulting network structures wide residual networks (WRNs) and show that these are far superior over their commonly used thin and very deep counterparts. For example, we demonstrate that even a simple 16-layer-deep wide residual network outperforms in accuracy and efficiency all previous deep residual networks, including thousand-layer-deep networks, achieving new state-of-the-art results on CIFAR, SVHN, COCO, and significant improvements on ImageNet. Our code and models are available at this https URL

Very Deep Convolutional Networks for Large-Scale Image Recognition

Abstract: In this work we investigate the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting. Our main contribution is a thorough evaluation of networks of increasing depth using an architecture with very small (3x3) convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. These findings were the basis of our ImageNet Challenge 2014 submission, where our team secured the first and the second places in the localisation and classification tracks respectively. We also show that our representations generalise well to other datasets, where they achieve state-of-the-art results. We have made our two best-performing ConvNet models publicly available to facilitate further research on the use of deep visual representations in computer vision.

List of neural networks used in this study.For each model, we list the minimum and maximum number of GPUs used in ourstrong scaling runs. We choose the minimum and maximum GPU counts such thatthe ratio of batch size to number of GPUs is 4 and 1 respectively.

Description of neural networks and hyperparameters

Table {{table:2f35aaeb-13e9-4340-9645-c8951255824a}}  lists the set of neural networks and the
corresponding hyperparameters used in this study. VGG-19 {{cite:89d3606}} and
WideResnet-101 {{cite:59df040}} are two convolutional neural network (CNN) architectures widely used
in computer vision. GPT-3 {{cite:235c6d4}}, a variant of the transformer architecture {{cite:870b917}},
is extremely popular in natural language processing for causal language modeling. For each model, we use the same
hyperparameters (batch size, sequence length, learning rate schedules, gradient clipping, l2
regularization and optimizer hyperparameters) as used by the authors. We use SGD (with momentum {{cite:9494188}}) and the
AdamW {{cite:4408d14}} optimizer for training the CNN and GPT-3 models respectively. We use MegatronLM's highly optimized kernels
to implement the GPT-3 models {{cite:5db5a3f}}. For the convolution neural networks, we use implementations
provided by the torchvision libraryhttps://pytorch.org/vision/stable/index.html.

We profile the neural networks listed in Table {{table:2f35aaeb-13e9-4340-9645-c8951255824a}}  under a
strong scaling setup to demonstrate the efficacy of our optimizations. For every
model, we choose the minimum and maximum GPU counts such that the ratio of batch
size to number of GPUs is 4 and 1 respectively. For a given model, we
fix the batch size irrespective of the GPU count. This is because while
increasing the batch size leads to better performance, it also degrades the
quality of convergence {{cite:5bd4b3e}}. Under a strict definition of strong scaling,
the final answer should be the same irrespective of the number of GPUs. Therefore,
it is important to keep the global batch size fixed. For our approach, we prune the networks to
a sparsity of 90% using You et al.'s “Early Bird Ticket” algorithm {{cite:4373b83}}.

We integrate our optimizations in AxoNN  {{cite:0c44778}} and refer to it
as “AxoNN+SAMO”. We use AxoNN and DeepSpeed-3D {{cite:144ebb0}}, {{cite:aa36c91}} as
baselines for dense computations. DeepSpeed-3D implements a hybrid of data, inter-layer and intra-layer
parallelism for parallel model training. Their data parallel implementation uses
the ZeRO optimizer to shard optimizer state memory across data parallel
ranks {{cite:aa36c91}}. They use MegatronLM's implementation of intra-layer
parallelism of transformers {{cite:5db5a3f}}. DeepSpeed-3D has been used to
train some of the largest neural networks till date such as
Bloom-176B {{cite:515306b}} and Megatron-Turing NLG
530B {{cite:98bd2ca}}. Finally, we integrate Sputnik
 {{cite:f33d259}} in AxoNN to create a sparse matrix multiplication baseline.
Note that Sputnik does not support sparse convolutions, so we do not implement
the convolution architectures in Table {{table:2f35aaeb-13e9-4340-9645-c8951255824a}}  using Sputnik.
We build our baselines using CUDA 11.0, PyTorch 1.12.0, NCCL 2.8.4, GCC 9.1.0
and Spectrum-MPI 10.4.0.3.

Next, we illustrate the results of our strong scaling experiments on WideResnet-101 and VGG-19 in Figure {{figure:8b294469-33c8-4a52-a103-cc8a27c53a01}} ,
GPT-3 XL and GPT-3 2.7B in Figure {{figure:fc027b86-362c-4fe5-86c4-203a000c5340}} , and on GPT-3 6.7B and GPT-3 13B in Figure
{{figure:71040c7a-d406-492f-a19f-93381c97d69c}} . The CNNs used in this study are nearly 10–100{{formula:cb827de5-427e-4d02-995f-6094bbaa2d5b}}  smaller than the GPT-3 based models (see Table {{table:2f35aaeb-13e9-4340-9645-c8951255824a}} ).
Hence, all of AxoNN, DeepSpeed-3D and AxoNN+SAMO are able to run these architectures in a pure data parallel configuration,
with a full copy of the network on each GPU. Thus the only communication here is the all-reduce on the network
gradients. We illustrate these results in Figure {{figure:8b294469-33c8-4a52-a103-cc8a27c53a01}} . We observe similar batch times for
both AxoNN and DeepSpeed-3D. This is explained by the fact that both these frameworks have very similar NCCL-based implementations
of data parallelism. Our approach yields speedups of 7–12% over WideResnet-101 and 18–44% over VGG-19.
While both these speedups are significant, SAMO seems to benefit the latter architecture more than the former.
This is because the WideResnet-101 architecture spends nearly 1.5{{formula:54c96fcf-2484-4171-a036-1ae8138049d4}}  more time in the computation phase as compared to
VGG-19. Also, both these models have similar number of parameters and thus similar communication costs in the data-parallel
all-reduce. Thus the proportion of the batch time spent by the WideResnet-101 architecture in communication is significantly smaller
than VGG-19. Since our approach optimizes communication, the benefits for WideResnet-101 are smaller than that of VGG-19. Note that we do not run Sputnik for the CNNs as
the library does not support sparse convolutions.


Attacks detection on keyless entry systems, exploiting the RF-fingerprint technique;

Provides secure authentication for connected ECUs over CAN;

Hold the Door! Fingerprinting Your Car Key to Prevent Keyless Entry Car Theft

Abstract: Recently, the traditional way to unlock car doors has been replaced with a keyless entry system which proves more convenient for automobile owners. When a driver with a key fob is in the vicinity of the vehicle, doors automatically unlock on user command. However, unfortunately, it has been shown that these keyless entry systems are vulnerable to signal relaying attacks. While it is evident that automobile manufacturers incorporate preventative methods to secure these keyless entry systems, they continue to be vulnerable to a range of attacks. Relayed signals result in valid packets that are verified as legitimate, and this makes it is difficult to distinguish a legitimate door unlock request from a malicious signal. In response to this vulnerability, this paper presents an RF fingerprinting method (coined HOld the DOoR, HODOR) to detect attacks on keyless entry systems the first attempt to exploit the RF fingerprint technique in the automotive domain. HODOR is designed as a sub authentication method that supports existing authentication systems for keyless entry systems and does not require any modification of the main system to perform. Through a series of experiments, the results demonstrate that HODOR competently and reliably detects attacks on keyless entry systems. HODOR achieves both an average false positive rate (FPR) of 0.27 percent with a false negative rate (FNR) of 0 percent for the detection of simulated attacks, corresponding to current research on keyless entry car theft.

Covert Channel-Based Transmitter Authentication in Controller Area Networks

Abstract: In recent years, the security of automotive Cyber-Physical Systems (CPSs) is facing urgent threats due to the widespread use of legacy in-vehicle communication systems. As a representative legacy bus system, the Controller Area Network (CAN) hosts Electronic Control Units (ECUs) that are crucial for the vehicles functioning. In this scenario, malicious actors can exploit the CAN vulnerabilities, such as the lack of built-in authentication and encryption schemes, to launch CAN bus attacks (e.g., suspension, injection, and masquerade attacks) with life-threatening consequences (e.g., disabling brakes). In this article, we present TACAN (Transmitter Authentication in CAN), which provides secure authentication of ECUs on the legacy CAN bus by exploiting the covert channels, without introducing CAN protocol modifications or traffic overheads (no extra bits or CAN messages are used). TACAN turns upside-down the originally malicious concept of covert channels and exploits it to build an effective defensive technique that facilitates transmitter authentication via a centralized, trusted Monitor Node. TACAN consists of three different covert channels for ECU authentication: 1) the Inter-Arrival Time (IAT)-based, leveraging the IATs of CAN messages; 2) the Least Significant Bit (LSB)-based, concealing authentication messages into the LSBs of normal CAN data; and 3) a hybrid covert channel, exploiting the combination of the first two. In order to validate TACAN, we implement the covert channels on the University of Washington (UW) EcoCAR (Chevrolet Camaro 2016) testbed. We further evaluate the bit error, throughput, and detection performance of TACAN through extensive experiments using the EcoCAR testbed and a publicly available dataset collected from Toyota Camry 2010. We demonstrate the feasibility of TACAN and the effectiveness of detecting CAN bus attacks, highlighting no traffic overheads and attesting the regular functionality of ECUs.

Comparison of State-of-the-art Message Authentication-based Protocols among Different Attributes

Table {{table:63b9f21b-7030-4821-96fb-c3a90c03a20d}}  presents a comparative study of state-of-the-art message authentication-based ADPS. This table gives a better overview of relevant ADPS solutions to understand their efficacy in various features.


Graph evolution: Densification and shrinking diameters

Abstract:
How do real graphs evolve over time? What are normal growth patterns in social, technological, and information networks? Many studies have discovered patterns in static graphs, identifying properties in a single snapshot of a large network or in a very small number of snapshots; these include heavy tails for in- and out-degree distributions, communities, small-world phenomena, and others. However, given the lack of information about network evolution over long periods, it has been hard to convert these findings into statements about trends over time.
 Here we study a wide range of real graphs, and we observe some surprising phenomena. First, most of these graphs densify over time with the number of edges growing superlinearly in the number of nodes. Second, the average distance between nodes often shrinks over time in contrast to the conventional wisdom that such distance parameters should increase slowly as a function of the number of nodes (like O(log n) or O(log(log n)).
 Existing graph generation models do not exhibit these types of behavior even at a qualitative level. We provide a new graph generator, based on a forest fire spreading process that has a simple, intuitive justification, requires very few parameters (like the flammability of nodes), and produces graphs exhibiting the full range of properties observed both in prior work and in the present study.
 We also notice that the forest fire model exhibits a sharp transition between sparse graphs and graphs that are densifying. Graphs with decreasing distance between the nodes are generated around this transition point.
 Last, we analyze the connection between the temporal evolution of the degree distribution and densification of a graph. We find that the two are fundamentally related. We also observe that real networks exhibit this type of relation between densification and the degree distribution.

Using Triangles to Improve Community Detection in Directed Networks

Abstract: In a graph, a community may be loosely defined as a group of nodes that are more closely connected to one another than to the rest of the graph. One common theme is many formalizations is that flows should tend to stay within com- munities. Hence, we expect short cycles to play an impor- tant role. For undirected graphs, the importance of trian- gles - an undirected 3-cycle - has been known for a long time and can be used to improve community detection. To identify communities in directed networks, we propose an undirected edge-weighting scheme based on directed trian- gles. We also propose a new metric on quality of the com- munities that is based on the number of 3-cycles that are split across communities. To demonstrate the impact of our new weighting, we use the standard METIS graph partition- ing tool to determine communities and show experimentally that the resulting communities result in fewer 3-cycles be- ing cut. The magnitude of the e!ect varies between a 10% and 50% reduction. We also find that this weighting scheme improves a task where plausible ground-truth communities are known.

Network datasets.

Our experiments are carried out on datasets considered in a state-of-the-art related work {{cite:c6cbee8}}. Although JITuNE can effectively tune the hyperparameters of NE algorithms on massive-scale networks, our counterparts cannot. Therefore, we first evaluate JITuNE against the baselines on small- and moderate-scale networks, wherein the baselines can finish in a reasonable time. To actually demonstrate the benefits of JITuNE, we validate our framework in two large-scale networks that no baselines can work as well in a reasonable time. For the ease of comparison, we select two large-scale networks from the related works {{cite:7b7285f}}, {{cite:c6cbee8}}. We summarize the information of the seven network datasets in our experiments in Table {{table:85f5400a-240c-44b0-a1be-e80d68dd6748}} . The datasets considers connected networks like BlogCatalog and disconnected networks like Arxiv.

We first experiment with networks without attributes. We tune two types of NE algorithms, i.e., factorization-based and sampling-based algorithms. For each type, we test on both connected networks, e.g., BlogCatalog and Wikipedia, and a network with disconnected components, e.g., Arxiv. BlogCatalog is a social network, and Wikipedia is a co-occurrence network of words. Arxiv is a reference network. The network details are given in Table {{table:85f5400a-240c-44b0-a1be-e80d68dd6748}} .

We also experiment with networks with attribute information. We tune the deep neural network-based algorithms on such networks. We test on a connected network, e.g., Pubmed, and a network with disconnected components, e.g., Cora. Pubmed and Cora are both citation networks. The network details are provided in Table {{table:85f5400a-240c-44b0-a1be-e80d68dd6748}} .

JITuNE can excel in hyperparameter tuning for NE algorithms within a time constraint. We now demonstrate the superiority of JITuNE in handling massive-scale networks. The details of the two large-scale datasets are presented in Table {{table:85f5400a-240c-44b0-a1be-e80d68dd6748}} .


Uncertainty Quantification for Sparse Deep Learning

Abstract: Deep learning methods continue to have a decided impact on machine learning, both in theory and in practice. Statistical theoretical developments have been mostly concerned with approximability or rates of estimation when recovering infinite dimensional objects (curves or densities). Despite the impressive array of available theoretical results, the literature has been largely silent about uncertainty quantification for deep learning. This paper takes a step forward in this important direction by taking a Bayesian point of view. We study Gaussian approximability of certain aspects of posterior distributions of sparse deep ReLU architectures in non-parametric regression. Building on tools from Bayesian non-parametrics, we provide semi-parametric Bernstein-von Mises theorems for linear and quadratic functionals, which guarantee that implied Bayesian credible regions have valid frequentist coverage. Our results provide new theoretical justifications for (Bayesian) deep learning with ReLU activation functions, highlighting their inferential potential.

Automated Detection of Presymptomatic Conditions in Spinocerebellar Ataxia Type 2 Using Monte Carlo Dropout and Deep Neural Network Techniques with Electrooculogram Signals

Abstract: Application of deep learning (DL) to the field of healthcare is aiding clinicians to make an accurate diagnosis. DL provides reliable results for image processing and sensor interpretation problems most of the time. However, model uncertainty should also be thoroughly quantified. This paper therefore addresses the employment of Monte Carlo dropout within the DL structure to automatically discriminate presymptomatic signs of spinocerebellar ataxia type 2 in saccadic samples obtained from electrooculograms. The current work goes beyond the common incorporation of this special type of dropout into deep neural networks and uses the uncertainty derived from the validation samples to construct a decision tree at the register level of the patients. The decision tree built from the uncertainty estimates obtained a classification accuracy of 81.18% in automatically discriminating control, presymptomatic and sick classes. This paper proposes a novel method to address both uncertainty quantification and explainability to develop reliable healthcare support systems.

Uncertainty Quantification Using Neural Networks for Molecular Property Prediction

Abstract: Uncertainty quantification (UQ) is an important component of molecular property prediction, particularly for drug discovery applications where model predictions direct experimental design and where unanticipated imprecision wastes valuable time and resources. The need for UQ is especially acute for neural models, which are becoming increasingly standard yet are challenging to interpret. While several approaches to UQ have been proposed in the literature, there is no clear consensus on the comparative performance of these models. In this paper, we study this question in the context of regression tasks. We systematically evaluate several methods on five regression datasets using multiple complementary performance metrics. Our experiments show that none of the methods we tested is unequivocally superior to all others, and none produce a particularly reliable ranking of errors across multiple datasets. While we believe these results show that existing UQ methods are not sufficient for all common use cases and further research is needed, we conclude with a practical recommendation as to which existing techniques seem to perform well relative to others.

Uncertainty Quantification for Deep Context-Aware Mobile Activity Recognition and Unknown Context Discovery

Abstract: Activity recognition in wearable computing faces two key challenges: i) activity characteristics may be context-dependent and change under different contexts or situations; ii) unknown contexts and activities may occur from time to time, requiring flexibility and adaptability of the algorithm. We develop a context-aware mixture of deep models termed the {\alpha}-\b{eta} network coupled with uncertainty quantification (UQ) based upon maximum entropy to enhance human activity recognition performance. We improve accuracy and F score by 10% by identifying high-level contexts in a data-driven way to guide model development. In order to ensure training stability, we have used a clustering-based pre-training in both public and in-house datasets, demonstrating improved accuracy through unknown context discovery.

Uncertainty-Aware Deep Learning-Based Cardiac Arrhythmias Classification Model of Electrocardiogram Signals

Abstract: Deep Learning-based methods have emerged to be one of the most effective and practical solutions in a wide range of medical problems, including the diagnosis of cardiac arrhythmias. A critical step to a precocious diagnosis in many heart dysfunctions diseases starts with the accurate detection and classification of cardiac arrhythmias, which can be achieved via electrocardiograms (ECGs). Motivated by the desire to enhance conventional clinical methods in diagnosing cardiac arrhythmias, we introduce an uncertainty-aware deep learning-based predictive model design for accurate large-scale classification of cardiac arrhythmias successfully trained and evaluated using three benchmark medical datasets. In addition, considering that the quantification of uncertainty estimates is vital for clinical decision-making, our method incorporates a probabilistic approach to capture the model’s uncertainty using a Bayesian-based approximation method without introducing additional parameters or significant changes to the network’s architecture. Although many arrhythmias classification solutions with various ECG feature engineering techniques have been reported in the literature, the introduced AI-based probabilistic-enabled method in this paper outperforms the results of existing methods in outstanding multiclass classification results that manifest F1 scores of 98.62% and 96.73% with (MIT-BIH) dataset of 20 annotations, and 99.23% and 96.94% with (INCART) dataset of eight annotations, and 97.25% and 96.73% with (BIDMC) dataset of six annotations, for the deep ensemble and probabilistic mode, respectively. We demonstrate our method’s high-performing and statistical reliability results in numerical experiments on the language modeling using the gating mechanism of Recurrent Neural Networks.

An Uncertainty-Aware Transfer Learning-Based Framework for COVID-19 Diagnosis

Abstract: The early and reliable detection of COVID-19 infected patients is essential to prevent and limit its outbreak. The PCR tests for COVID-19 detection are not available in many countries, and also, there are genuine concerns about their reliability and performance. Motivated by these shortcomings, this article proposes a deep uncertainty-aware transfer learning framework for COVID-19 detection using medical images. Four popular convolutional neural networks (CNNs), including VGG16, ResNet50, DenseNet121, and InceptionResNetV2, are first applied to extract deep features from chest X-ray and computed tomography (CT) images. Extracted features are then processed by different machine learning and statistical modeling techniques to identify COVID-19 cases. We also calculate and report the epistemic uncertainty of classification results to identify regions where the trained models are not confident about their decisions (out of distribution problem). Comprehensive simulation results for X-ray and CT image data sets indicate that linear support vector machine and neural network models achieve the best results as measured by accuracy, sensitivity, specificity, and area under the receiver operating characteristic (ROC) curve (AUC). Also, it is found that predictive uncertainty estimates are much higher for CT images compared to X-ray images.

Uncertainty Quantification by Ensemble Learning for Computational Optical Form Measurements

Abstract: Uncertainty quantification by ensemble learning is explored in terms of an application from computational optical form measurements. The application requires to solve a large-scale, nonlinear inverse problem. Ensemble learning is used to extend a recently developed deep learning approach for this application in order to provide an uncertainty quantification of its predicted solution to the inverse problem. By systematically inserting out-of-distribution errors as well as noisy data the reliability of the developed uncertainty quantification is explored. Results are encouraging and the proposed application exemplifies the ability of ensemble methods to make trustworthy predictions on high dimensional data in a real-world application.

SleepTransformer: Automatic Sleep Staging with Interpretability and Uncertainty Quantification

Abstract: Background: Black-box skepticism is one of the main hindrances impeding deep-learning-based automatic sleep scoring from being used in clinical environments. Methods: Towards interpretability, this work proposes a sequence-to-sequence sleep-staging model, namely SleepTransformer. It is based on the transformer backbone and offers interpretability of the model's decisions at both the epoch and sequence level. We further propose a simple yet efficient method to quantify uncertainty in the model's decisions. The method, which is based on entropy, can serve as a metric for deferring low-confidence epochs to a human expert for further inspection. Results: Making sense of the transformer's self-attention scores for interpretability, at the epoch level, the attention scores are encoded as a heat map to highlight sleep-relevant features captured from the input EEG signal. At the sequence level, the attention scores are visualized as the influence of different neighboring epochs in an input sequence (i.e. the context) to recognition of a target epoch, mimicking the way manual scoring is done by human experts. Conclusion: Additionally, we demonstrate that SleepTransformer performs on par with existing methods on two databases of different sizes. Significance: Equipped with interpretability and the ability of uncertainty quantification, SleepTransformer holds promise for being integrated into clinical settings.

A brief overview of a few uncertainty quantification (UQ) methods used in the literature.

Fig {{table:22d8e9d6-3255-4903-8f3b-8563860885e2}}  shows the predictive entropy distributions (the uncertainty distribution) of correct (blue) and incorrect (red) predictions for the test set. Also, the mean values of these distributions, their differences, and the prediction accuracy of each method are reported in Table {{table:588ce89d-533a-4317-b625-01260cc2eeea}} . {{formula:77c18b1a-b19f-4e53-9b84-3b56e537241c}}  and {{formula:697e4bd0-66a1-4549-b95a-a976af49538c}}  are representing the center of the approximated Gaussian distribution for correct and incorrect predictions, respectively. Distance is the difference between two distributions. These average values are obtained for 100 runs of each method (train and test sets are randomly generated in each run). According to Fig {{table:22d8e9d6-3255-4903-8f3b-8563860885e2}} , correctly classified cases have a much smaller uncertainty than misclassified ones. Both proposed algorithms lead to a greater distance between means of distributions than the simple MC-Dropout method. This shows that the two algorithms can assign high entropy to missclassified samples and low entropy to correct classifies better than a simple MC-Dropout. The overlap between two densities is minimum for the MC-Dropout with PE-based loss function.


Investigating the Use of Pretrained Convolutional Neural Network on Cross-Subject and Cross-Dataset EEG Emotion Recognition

Abstract: The electroencephalogram (EEG) has great attraction in emotion recognition studies due to its resistance to deceptive actions of humans. This is one of the most significant advantages of brain signals in comparison to visual or speech signals in the emotion recognition context. A major challenge in EEG-based emotion recognition is that EEG recordings exhibit varying distributions for different people as well as for the same person at different time instances. This nonstationary nature of EEG limits the accuracy of it when subject independency is the priority. The aim of this study is to increase the subject-independent recognition accuracy by exploiting pretrained state-of-the-art Convolutional Neural Network (CNN) architectures. Unlike similar studies that extract spectral band power features from the EEG readings, raw EEG data is used in our study after applying windowing, pre-adjustments and normalization. Removing manual feature extraction from the training system overcomes the risk of eliminating hidden features in the raw data and helps leverage the deep neural network’s power in uncovering unknown features. To improve the classification accuracy further, a median filter is used to eliminate the false detections along a prediction interval of emotions. This method yields a mean cross-subject accuracy of 86.56% and 78.34% on the Shanghai Jiao Tong University Emotion EEG Dataset (SEED) for two and three emotion classes, respectively. It also yields a mean cross-subject accuracy of 72.81% on the Database for Emotion Analysis using Physiological Signals (DEAP) and 81.8% on the Loughborough University Multimodal Emotion Dataset (LUMED) for two emotion classes. Furthermore, the recognition model that has been trained using the SEED dataset was tested with the DEAP dataset, which yields a mean prediction accuracy of 58.1% across all subjects and emotion classes. Results show that in terms of classification accuracy, the proposed approach is superior to, or on par with, the reference subject-independent EEG emotion recognition studies identified in literature and has limited complexity due to the elimination of the need for feature extraction.

Machine Learning Strategies to Improve Generalization in EEG-based Emotion Assessment: \\a Systematic Review

Abstract: A systematic review on machine-learning strategies for improving generalizability (cross-subjects and cross-sessions) electroencephalography (EEG) based in emotion classification was realized. In this context, the non-stationarity of EEG signals is a critical issue and can lead to the Dataset Shift problem. Several architectures and methods have been proposed to address this issue, mainly based on transfer learning methods. 418 papers were retrieved from the Scopus, IEEE Xplore and PubMed databases through a search query focusing on modern machine learning techniques for generalization in EEG-based emotion assessment. Among these papers, 75 were found eligible based on their relevance to the problem. Studies lacking a specific cross-subject and cross-session validation strategy and making use of other biosignals as support were excluded. On the basis of the selected papers' analysis, a taxonomy of the studies employing Machine Learning (ML) methods was proposed, together with a brief discussion on the different ML approaches involved. The studies with the best results in terms of average classification accuracy were identified, supporting that transfer learning methods seem to perform better than other approaches. A discussion is proposed on the impact of (i) the emotion theoretical models and (ii) psychological screening of the experimental sample on the classifier performances.

The most representative studies according to their classification accuracy, categorised by EEG dataset (SEED, DEAP, others) and by number and type of classes considered. DIM = Dimensional; DIS = Discrete; sp = self-produced.

As can be seen in Table {{table:b27a7b3b-2457-47a0-afe2-22c88eea1976}} , CSS represents the category of studies exhibiting the best performance in terms of accuracy, at least in four of the cases considered. In {{cite:8e41f3a}} (Deep Domain Confusion, DDC) two identical networks are trained together, the former classifying data from the Source Domain, the latter adapting the distance between Source and Target domains using Target feature data. The combination of both the classification performance and the MMD is used as final loss.

Studies identified as best performers in terms of mean classification accuracy are proposed in Table {{table:b27a7b3b-2457-47a0-afe2-22c88eea1976}} . Only cross-subject studies were considered in this selection, being the statistically most significant percentage among all the generalisation studies surveyed. The ten best results were identified considering as many classification issues. Each issue is defined by considering the number and type of classes identified (binary and ternary on valence and arousal, quaternary on the two-dimensional valence-arousal plane, binary and quaternary on discrete dimensions) and the dataset adopted (considering three possibilities: DEAP, SEED, other). Only studies reporting both mean accuracy and standard deviation were included in the performance assessment.

When aiming for a generalization goal in EEG-based Emotion Recognition, Transfer Learning methods are becoming more and more established in the literature.
Domain Adaptation methods (Deep DA (CSB), Shallow DA (SSB), Source Selection DA, Deep DA (SSS), Shallow DA (TSB) and Supervised DA) exceed 60 % of the total surveyed studies and exhibit very high accuracy performances in the table of best performers (see Table {{table:b27a7b3b-2457-47a0-afe2-22c88eea1976}} ).
In particular, Deep DA (Common Shared Space) is used by five best performers studies. This could be also due to the current massive use of Deep DA (Common Shared Space) in the literature. Indeed, one third of the surveyed studies (Figure {{figure:7ede3c26-3e2b-432e-8085-3a473bdc7b10}} .d) belongs to this category.


Meta Networks

Abstract: Neural networks have been successfully applied in applications with a large amount of labeled data. However, the task of rapid generalization on new concepts with small training data while preserving performances on previously learned ones still presents a significant challenge to neural network models. In this work, we introduce a novel meta learning method, Meta Networks (MetaNet), that learns a meta-level knowledge across tasks and shifts its inductive biases via fast parameterization for rapid generalization. When evaluated on Omniglot and Mini-ImageNet benchmarks, our MetaNet models achieve a near human-level performance and outperform the baseline approaches by up to 6% accuracy. We demonstrate several appealing properties of MetaNet relating to generalization and continual learning.

Metalearning with Hebbian Fast Weights

Abstract: We unify recent neural approaches to one-shot learning with older ideas of associative memory in a model for metalearning. Our model learns jointly to represent data and to bind class labels to representations in a single shot. It builds representations via slow weights, learned across tasks through SGD, while fast weights constructed by a Hebbian learning rule implement one-shot binding for each new task. On the Omniglot, Mini-ImageNet, and Penn Treebank one-shot learning benchmarks, our model achieves state-of-the-art results.

Summary of Learn-to-Adjust FSL approaches.

Several typical L2A approaches include MetaNet {{cite:34d064f}}, CSNs {{cite:2c79550}}, MetaHebb {{cite:fcdacbe}} and FEAT {{cite:b61dbc4}}, which differ in the selection for the parts needing to be adjusted as well as the design for the generated adjustment, as described in Table {{table:e5798dad-dcb2-45c1-858d-49273f541adf}} . MetaNet {{cite:34d064f}} deployed a fast-weight layer attached to each layer of the base learner. The weights of each fast-weight layer are meta-generated by an external meta learner in light of the input sample. These collateral branch layers are used to adjust the intermediate values of the input sample during the feedforward process. CSNs {{cite:2c79550}} selected to adjust the neuron state (i.e., pre-activation) of each hidden node in the base learner. Specifically, it combined a memory module with an attention-based memory read mechanism to generate the condition shift for each neuron pre-activation in the base learner. MetaHebb {{cite:fcdacbe}} added an auxiliary fast-weight matrix in pre-Softmax layer, which is meta-generated via Hebbian learning {{cite:2ba49b9}} and is expected to adjust the input's internal representation fed into final Softmax layer. FEAT {{cite:b61dbc4}} proposed to adapt the embeddings of support samples to render them more discriminative for the task in hand, and four kinds of set-to-set functions including BiLSTM {{cite:5920e0a}}, DeepSets {{cite:7f9093a}}, GCN {{cite:53660f9}} and Transformer {{cite:a2014ec}} that aim to transform the original embedding vector to the adapted vector were investigated by {{cite:b61dbc4}}.

{{cite:fcdacbe}}

{{cite:fcdacbe}}


Self-training with noisy student model and semi-supervised loss function for dcase 2021 challenge task 4

Abstract: This report proposes a polyphonic sound event detection (SED) method for the DCASE 2021 Challenge Task 4. The proposed SED model consists of two stages: a mean-teacher model for providing target labels regarding weakly labeled or unlabeled data and a self-training-based noisy student model for predicting strong labels for sound events. The mean-teacher model, which is based on the residual convolutional recurrent neural network (RCRNN) for the teacher and student model, is first trained using all the training data from a weakly labeled dataset, an unlabeled dataset, and a strongly labeled synthetic dataset. Then, the trained mean-teacher model predicts the strong label to each of the weakly labeled and unlabeled datasets, which is brought to the noisy student model in the second stage of the proposed SED model. Here, the structure of the noisy student model is identical to the RCRNN-based student model of the mean-teacher model in the first stage. Then, it is self-trained by adding feature noises, such as time-frequency shift, mixup, SpecAugment, and dropoutbased model noise. In addition, a semi-supervised loss function is applied to train the noisy student model, which acts as label noise injection. The performance of the proposed SED model is evaluated on the validation set of the DCASE 2021 Challenge Task 4, and then, several ensemble models that combine five-fold validation models with different hyperparameters of the semi-supervised loss function are finally selected as our final models.

Heavily Augmented Sound Event Detection utilizing Weak Predictions

Abstract: The performances of Sound Event Detection (SED) systems are greatly limited by the difficulty in generating large strongly labeled dataset. In this work, we used two main approaches to overcome the lack of strongly labeled data. First, we applied heavy data augmentation on input features. Data augmentation methods used include not only conventional methods used in speech/audio domains but also our proposed method named FilterAugment. Second, we propose two methods to utilize weak predictions to enhance weakly supervised SED performance. As a result, we obtained the best PSDS1 of 0.4336 and best PSDS2 of 0.8161 on the DESED real validation dataset. This work is submitted to DCASE 2021 Task4 and is ranked on the 3rd place. Code availa-ble: https://github.com/frednam93/FilterAugSED.

The impact of non-target events in synthetic soundscapes for sound event detection

Abstract: Detection and Classification Acoustic Scene and Events Challenge 2021 Task 4 uses a heterogeneous dataset that includes both recorded and synthetic soundscapes. Until recently only target sound events were considered when synthesizing the soundscapes. However, recorded soundscapes often contain a substantial amount of non-target events that may affect the performance. In this paper, we focus on the impact of these non-target events in the synthetic soundscapes. Firstly, we investigate to what extent using non-target events alternatively during the training or validation phase (or none of them) helps the system to correctly detect target events. Secondly, we analyze to what extend adjusting the signal-to-noise ratio between target and non-target events at training improves the sound event detection performance. The results show that using both target and non-target events for only one of the phases (validation or training) helps the system to properly detect sound events, outperforming the baseline (which uses non-target events in both phases).The paper also reports the results of a preliminary study on evaluating the system on clips that contain only non-target events. This opens questions for future work on non-target subset and acoustic similarity between target and non-target events which might confuse the system.

PSDS_1 and PSDS_2 of the nine highest-ranked teams based on rankings score plus the baseline.

Table {{table:26c0be71-2595-44fe-b50a-482e9f81b64d}}  presents the PSDS scores obtained on the official evaluation set for both scenarios. In particular, the table reports the results of the 9 highest-ranked teams and the baseline. For each team, the PSDS for the two systems selected for the ranking score are reported.
For three participants, the systems that perform best on scenario 1 and scenario 2 have different behaviors  {{cite:c48bd2a}}, {{cite:61929ac}}, {{cite:f9fdf89}}. Zheng et al. adjust the sigmoid temperature parameter to obtain soft or sharp detection output depending on the scenario {{cite:c48bd2a}}. Lu et al. use a convolutional nerual network (CNN) model on scenario 1 and a conformer on scenario 2 {{cite:61929ac}}. Nam et al. propose to train a weak SED system for scenario 2, primarily focusing on event classification {{cite:f9fdf89}}. For some participants, the best system on scenario 1 is also the best system on scenario 2 {{cite:1259182}}, {{cite:1b6d25a}}, {{cite:3a8cf59}}. In the remainder of the paper we focus on the six submissions listed above, comparing their performance to those obtained with the baseline. Table {{table:26c0be71-2595-44fe-b50a-482e9f81b64d}}  also reports performance obtained by other participants for which the difference between the best system on scenario 1 and the best system on scenario 2 is marginal (the difference is mainly due to a different system adjustment) {{cite:8e89a32}}, {{cite:eb42952}}, {{cite:41f4953}}.


Know Your Master: Driver Profiling-based Anti-theft Method

Abstract: Although many anti-theft technologies are implemented, auto-theft is still increasing. Also, security vulnerabilities of cars can be used for auto-theft by neutralizing anti-theft system. This keyless auto-theft attack will be increased as cars adopt computerized electronic devices more. To detect auto-theft efficiently, we propose the driver verification method that analyzes driving patterns using measurements from the sensor in the vehicle. In our model, we add mechanical features of automotive parts that are excluded in previous works, but can be differentiated by drivers' driving behaviors. We design the model that uses significant features through feature selection to reduce the time cost of feature processing and improve the detection performance. Further, we enrich the feature set by deriving statistical features such as mean, median, and standard deviation. This minimizes the effect of fluctuation of feature values per driver and finally generates the reliable model. We also analyze the effect of the size of sliding window on performance to detect the time point when the detection becomes reliable and to inform owners the theft event as soon as possible. We apply our model with real driving and show the contribution of our work to the literature of driver identification.

Characterizing Driving Styles with Deep Learning

Abstract: Characterizing driving styles of human drivers using vehicle sensor data, e.g., GPS, is an interesting research problem and an important real-world requirement from automotive industries. A good representation of driving features can be highly valuable for autonomous driving, auto insurance, and many other application scenarios. However, traditional methods mainly rely on handcrafted features, which limit machine learning algorithms to achieve a better performance. In this paper, we propose a novel deep learning solution to this problem, which could be the first attempt of extending deep learning to driving behavior analysis based on GPS data. The proposed approach can effectively extract high level and interpretable features describing complex driving patterns. It also requires significantly less human experience and work. The power of the learned driving style representations are validated through the driver identification problem using a large real dataset.

Autoencoder Regularized Network For Driving Style Representation Learning

Abstract: In this paper, we study learning generalized driving style representations from automobile GPS trip data. We propose a novel Autoencoder Regularized deep neural Network (ARNet) and a trip encoding framework trip2vec to learn drivers' driving styles directly from GPS records, by combining supervised and unsupervised feature learning in a unified architecture. Experiments on a challenging driver number estimation problem and the driver identification problem show that ARNet can learn a good generalized driving style representation: It significantly outperforms existing methods and alternative architectures by reaching the least estimation error on average (0.68, less than one driver) and the highest identification accuracy (by at least 3% improvement) compared with traditional supervised learning methods.

Robust Deep Reinforcement Learning for Security and Safety in Autonomous Vehicle Systems

Abstract: To operate effectively in tomorrow's smart cities, autonomous vehicles (AVs) must rely on intra-vehicle sensors such as camera and radar as well as inter-vehicle communication. Such dependence on sensors and communication links exposes AVs to cyber-physical (CP) attacks by adversaries that seek to take control of the AVs by manipulating their data. Thus, to ensure safe and optimal AV dynamics control, the data processing functions at AVs must be robust to such CP attacks. To this end, in this paper, the state estimation process for monitoring AV dynamics, in presence of CP attacks, is analyzed and a novel adversarial deep reinforcement learning (RL) algorithm is proposed to maximize the robustness of AV dynamics control to CP attacks. The attacker's action and the AV's reaction to CP attacks are studied in a game-theoretic framework. In the formulated game, the attacker seeks to inject faulty data to AV sensor readings so as to manipulate the inter-vehicle optimal safe spacing and potentially increase the risk of AV accidents or reduce the vehicle flow on the roads. Meanwhile, the AV, acting as a defender, seeks to minimize the deviations of spacing so as to ensure robustness to the attacker's actions. Since the AV has no information about the attacker's action and due to the infinite possibilities for data value manipulations, the outcome of the players' past interactions are fed to long-short term memory (LSTM) blocks. Each player's LSTM block learns the expected spacing deviation resulting from its own action and feeds it to its RL algorithm. Then, the the attacker's RL algorithm chooses the action which maximizes the spacing deviation, while the AV's RL algorithm tries to find the optimal action that minimizes such deviation.

Crossfire Attack Detection using Deep Learning in Software Defined ITS Networks

Abstract: Recent developments in intelligent transport systems (ITS) based on smart mobility significantly improves safety and security over roads and highways. ITS networks are comprised of the Internet-connected vehicles (mobile nodes), roadside units (RSU), cellular base stations and conventional core network routers to create a complete data transmission platform that provides real-time traffic information and enable prediction of future traffic conditions. However, the heterogeneity and complexity of the underlying ITS networks raise new challenges in intrusion prevention of mobile network nodes and detection of security attacks due to such highly vulnerable mobile nodes. In this paper, we consider a new type of security attack referred to as crossfire attack, which involves a large number of compromised nodes that generate low-intensity traffic in a temporally coordinated fashion such that target links or hosts (victims) are disconnected from the rest of the network. Detection of such attacks is challenging since the attacking traffic flows are indistinguishable from the legitimate flows. With the support of software-defined networking that enables dynamic network monitoring and traffic characteristic extraction, we develop a machine learning model that can learn the temporal correlation among traffic flows traversing in the ITS network, thus differentiating legitimate flows from coordinated attacking flows. We use different deep learning algorithms to train the model and study the performance using Mininet-WiFi emulation platform. The results show that our approach achieves a detection accuracy of at least 80%.

ML techniques applied to vehicular security (Part 02)

In Section IV, we have identified the use of ML-based solutions in five different security aspects including, driver fingerprinting, attack detection, misbehaviour or intrusion detection, trust computation, and privacy protection. Several ML architectures are explored in these areas where we observe supervised learning, reinforcement learning, and deep learning are popular learning algorithms in vehicular security applications, as shown in Table {{table:66d3670a-c2b9-4253-8e2f-7457c8e1c86b}}  and {{table:e84510ef-b078-4c1c-86c1-20faf3ccf9fc}} . In the context of driver fingerprinting, it is of utmost importance to preserve the privacy of driver data and prevent adversaries from tracking the vehicle by linking it to the driver's identification. The older techniques use the idea of hiding location and accessing resources/services without disclosing the true identity (i.e, using pseudonymity) of a driver to ensure privacy. With the advent of autonomous vehicles, true profiling of the driver is important. To enable this, ML techniques are used by researchers to authenticate the true identification and fingerprinting of a user/driver to make vehicles resilient to theft without hiding the true identity of drivers. We also note that supervised learning is the most-widely used ML technique for driver fingerprinting, as shown in Table {{table:66d3670a-c2b9-4253-8e2f-7457c8e1c86b}} . Some works in the literature, use unsupervised ML to cluster driver behaviour before performing classification and achieve better accuracies with the given models. It is observed that the data from smartphone sensors, vehicle sensors, and onboard electronic control units (i.e. using the OBD-II protocol) are the most commonly used data types for driver profiling using ML.


Learning by clustering, reconstruction, and self-supervised classification

Learning by global-local reasoning for 3D objects

Learning by contrasting global and local parts of objects

Learning by contrasting local patches of 3D objects with hard negative sampling

Techniques to improve contrasting scene point cloud views

Learning spatio-temporal data invariance from point cloud sequences

Constructing pseudo scenes with synthetic objects for contrastive learning

Joint contrastive learning with points and voxels

A unified contrastive-learning framework for point cloud pre-training

Pre-training 3D detectors by contrasting region proposals

Learning by discriminating masked object points and sampled noise points

Unsupervised Multi-Task Feature Learning on Point Clouds

Abstract: We introduce an unsupervised multi-task model to jointly learn point and shape features on point clouds. We define three unsupervised tasks including clustering, reconstruction, and self-supervised classification to train a multi-scale graph-based encoder. We evaluate our model on shape classification and segmentation benchmarks. The results suggest that it outperforms prior state-of-the-art unsupervised models: In the ModelNet40 classification task, it achieves an accuracy of 89.1% and in ShapeNet segmentation task, it achieves an mIoU of 68.2 and accuracy of 88.6%.

Global-Local Bidirectional Reasoning for Unsupervised Representation Learning of 3D Point Clouds

Abstract: Local and global patterns of an object are closely related. Although each part of an object is incomplete, the underlying attributes about the object are shared among all parts, which makes reasoning the whole object from a single part possible. We hypothesize that a powerful representation of a 3D object should model the attributes that are shared between parts and the whole object, and distinguishable from other objects. Based on this hypothesis, we propose to learn point cloud representation by bidirectional reasoning between the local structures at different abstraction hierarchies and the global shape without human supervision. Experimental results on various benchmark datasets demonstrate the unsupervisedly learned representation is even better than supervised representation in discriminative power, generalization ability, and robustness. We show that unsupervisedly trained point cloud models can outperform their supervised counterparts on downstream classification tasks. Most notably, by simply increasing the channel width of an SSG PointNet++, our unsupervised model surpasses the state-of-the-art supervised methods on both synthetic and real-world 3D object classification datasets. We expect our observations to offer a new perspective on learning better representation from data structures instead of human annotations for point cloud understanding.

Info3D: Representation Learning on 3D Objects using Mutual Information Maximization and Contrastive Learning

Abstract: A major endeavor of computer vision is to represent, understand and extract structure from 3D data. Towards this goal, unsupervised learning is a powerful and necessary tool. Most current unsupervised methods for 3D shape analysis use datasets that are aligned, require objects to be reconstructed and suffer from deteriorated performance on downstream tasks. To solve these issues, we propose to extend the InfoMax and contrastive learning principles on 3D shapes. We show that we can maximize the mutual information between 3D objects and their "chunks" to improve the representations in aligned datasets. Furthermore, we can achieve rotation invariance in SO${(3)}$ group by maximizing the mutual information between the 3D objects and their geometric transformed versions. Finally, we conduct several experiments such as clustering, transfer learning, shape retrieval, and achieve state of art results.

Self-Supervised Learning of Point Clouds via Orientation Estimation

Abstract: Point clouds provide a compact and efficient representation of 3D shapes. While deep neural networks have achieved impressive results on point cloud learning tasks, they require massive amounts of manually labeled data, which can be costly and time-consuming to collect. In this paper, we leverage 3D self-supervision for learning downstream tasks on point clouds with fewer labels. A point cloud can be rotated in infinitely many ways, which provides a rich label-free source for self-supervision. We consider the auxiliary task of predicting rotations that in turn leads to useful features for other tasks such as shape classification and 3D keypoint prediction. Using experiments on ShapeNet and ModelNet, we demonstrate that our approach outperforms the state-of-the-art. Moreover, features learned by our model are complementary to other self-supervised methods and combining them leads to further performance improvement.

Self-Contrastive Learning with Hard Negative Sampling for Self-supervised Point Cloud Learning

Abstract: Point clouds have attracted increasing attention. Significant progress has been made in methods for point cloud analysis, which often requires costly human annotation as supervision. To address this issue, we propose a novel self-contrastive learning for self-supervised point cloud representation learning, aiming to capture both local geometric patterns and nonlocal semantic primitives based on the nonlocal self-similarity of point clouds. The contributions are two-fold: on the one hand, instead of contrasting among different point clouds as commonly employed in contrastive learning, we exploit self-similar point cloud patches within a single point cloud as positive samples and otherwise negative ones to facilitate the task of contrastive learning. On the other hand, we actively learn hard negative samples that are close to positive samples for discriminative feature learning. Experimental results show that the proposed method achieves state-of-the-art performance on widely used benchmark datasets for self-supervised point cloud segmentation and transfer learning for classification.

Exploring Data-Efficient 3D Scene Understanding with Contrastive Scene Contexts

Abstract: The rapid progress in 3D scene understanding has come with growing demand for data; however, collecting and an-notating 3D scenes (e.g. point clouds) are notoriously hard. For example, the number of scenes (e.g. indoor rooms) that can be accessed and scanned might be limited; even given sufficient data, acquiring 3D labels (e.g. instance masks) requires intensive human labor. In this paper, we explore data-efficient learning for 3D point cloud. As a first step towards this direction, we propose Contrastive Scene Contexts, a 3D pre-training method that makes use of both point-level correspondences and spatial contexts in a scene. Our method achieves state-of-the-art results on a suite of benchmarks where training data or labels are scarce. Our study reveals that exhaustive labelling of 3D point clouds might be unnecessary; and remarkably, on ScanNet, even using 0.1% of point labels, we still achieve 89% (instance segmentation) and 96% (semantic segmentation) of the baseline performance that uses full annotations.

Spatio-temporal Self-Supervised Representation Learning for 3D Point Clouds

Abstract: To date, various 3D scene understanding tasks still lack practical and generalizable pre-trained models, primarily due to the intricate nature of 3D scene understanding tasks and their immense variations introduced by camera views, lighting, occlusions, etc. In this paper, we tackle this challenge by introducing a spatio-temporal representation learning (STRL) framework, capable of learning from unlabeled 3D point clouds in a self-supervised fashion. Inspired by how infants learn from visual data in the wild, we explore the rich spatio-temporal cues derived from the 3D data. Specifically, STRL takes two temporally-correlated frames from a 3D point cloud sequence as the input, transforms it with the spatial data augmentation, and learns the invariant representation self-supervisedly. To corroborate the efficacy of STRL, we conduct extensive experiments on three types (synthetic, indoor, and outdoor) of datasets. Experimental results demonstrate that, compared with supervised learning methods, the learned self-supervised representation facilitates various models to attain comparable or even better performances while capable of generalizing pre-trained models to downstream tasks, including 3D shape classification, 3D object detection, and 3D semantic segmentation. Moreover, the spatio-temporal contextual cues embedded in 3D point clouds significantly improve the learned representations.

RandomRooms: Unsupervised Pre-training from Synthetic Shapes and Randomized Layouts for 3D Object Detection

Abstract: 3D point cloud understanding has made great progress in recent years. However, one major bottleneck is the scarcity of annotated real datasets, especially compared to 2D object detection tasks, since a large amount of labor is involved in annotating the real scans of a scene. A promising solution to this problem is to make better use of the synthetic dataset, which consists of CAD object models, to boost the learning on real datasets. This can be achieved by the pre-training and fine-tuning procedure. However, recent work on 3D pre-training exhibits failure when transfer features learned on synthetic objects to other real-world applications. In this work, we put forward a new method called RandomRooms to accomplish this objective. In particular, we propose to generate random layouts of a scene by making use of the objects in the synthetic CAD dataset and learn the 3D scene representation by applying object-level contrastive learning on two random scenes generated from the same set of synthetic objects. The model pre-trained in this way can serve as a better initialization when later fine-tuning on the 3D object detection task. Empirically, we show consistent improvement in downstream 3D detection tasks on several base models, especially when less training data are used, which strongly demonstrates the effectiveness and generalization of our method. Benefiting from the rich semantic knowledge and diverse objects from synthetic data, our method establishes the new state-of-the-art on widely-used 3D detection benchmarks ScanNetV2 and SUN RGB-D. We expect our attempt to provide a new perspective for bridging object and scene-level 3D understanding.

Self-Supervised Pretraining of 3D Features on any Point-Cloud

Abstract: Pretraining on large labeled datasets is a prerequisite to achieve good performance in many computer vision tasks like 2D object recognition, video classification etc. However, pretraining is not widely used for 3D recognition tasks where state-of-the-art methods train models from scratch. A primary reason is the lack of large annotated datasets because 3D data is both difficult to acquire and time consuming to label. We present a simple self-supervised pertaining method that can work with any 3D data - single or multiview, indoor or outdoor, acquired by varied sensors, without 3D registration. We pretrain standard point cloud and voxel based model architectures, and show that joint pretraining further improves performance. We evaluate our models on 9 benchmarks for object detection, semantic segmentation, and object classification, where they achieve state-of-the-art results and can outperform supervised pretraining. We set a new state-of-the-art for object detection on ScanNet (69.0% mAP) and SUNRGBD (63.5% mAP). Our pretrained models are label efficient and improve performance for classes with few examples.

A Closer Look at Invariances in Self-supervised Pre-training for 3D Vision

Abstract: Self-supervised pre-training for 3D vision has drawn increasing research interest in recent years. In order to learn informative representations, a lot of previous works exploit invariances of 3D features, e.g., perspective-invariance between views of the same scene, modality-invariance between depth and RGB images, format-invariance between point clouds and voxels. Although they have achieved promising results, previous researches lack a systematic and fair comparison of these invariances. To address this issue, our work, for the first time, introduces a unified framework, under which various pre-training methods can be investigated. We conduct extensive experiments and provide a closer look at the contributions of different invariances in 3D pre-training. Also, we propose a simple but effective method that jointly pre-trains a 3D encoder and a depth map encoder using contrastive learning. Models pre-trained with our method gain significant performance boost in downstream tasks. For instance, a pre-trained VoteNet outperforms previous methods on SUN RGB-D and ScanNet object detection benchmarks with a clear margin.

ProposalContrast: Unsupervised Pre-training for LiDAR-based 3D Object Detection

Abstract: Existing approaches for unsupervised point cloud pre-training are constrained to either scene-level or point/voxel-level instance discrimination. Scene-level methods tend to lose local details that are crucial for recognizing the road objects, while point/voxel-level methods inherently suffer from limited receptive field that is incapable of perceiving large objects or context environments. Considering region-level representations are more suitable for 3D object detection, we devise a new unsupervised point cloud pre-training framework, called ProposalContrast, that learns robust 3D representations by contrasting region proposals. Specifically, with an exhaustive set of region proposals sampled from each point cloud, geometric point relations within each proposal are modeled for creating expressive proposal representations. To better accommodate 3D detection properties, ProposalContrast optimizes with both inter-cluster and inter-proposal separation, i.e., sharpening the discriminativeness of proposal representations across semantic classes and object instances. The generalizability and transferability of ProposalContrast are verified on various 3D detectors (i.e., PV-RCNN, CenterPoint, PointPillars and PointRCNN) and datasets (i.e., KITTI, Waymo and ONCE).

Masked Discrimination for Self-Supervised Learning on Point Clouds

Abstract: Masked autoencoding has achieved great success for self-supervised learning in the image and language domains. However, mask based pretraining has yet to show benefits for point cloud understanding, likely due to standard backbones like PointNet being unable to properly handle the training versus testing distribution mismatch introduced by masking during training. In this paper, we bridge this gap by proposing a discriminative mask pretraining Transformer framework, MaskPoint}, for point clouds. Our key idea is to represent the point cloud as discrete occupancy values (1 if part of the point cloud; 0 if not), and perform simple binary classification between masked object points and sampled noise points as the proxy task. In this way, our approach is robust to the point sampling variance in point clouds, and facilitates learning rich representations. We evaluate our pretrained models across several downstream tasks, including 3D shape classification, segmentation, and real-word object detection, and demonstrate state-of-the-art results while achieving a significant pretraining speedup (e.g., 4.1x on ScanNet) compared to the prior state-of-the-art Transformer baseline. Code is available at https://github.com/haotian-liu/MaskPoint.

Summary of context-based methods for unsupervised representation learning of point clouds.

Context-based methods are another important category of URL of point clouds that has attracted increasing attention in recent years. Different from generation-based methods that learn representations in a generative way, these methods employ discriminative pre-text tasks to learn different contexts of point clouds including context similarity, spatial context structures, and temporal context structures. The designed pre-text tasks require no human annotations and Table {{table:33e134cf-4cfc-4ed7-a1d6-fe67fac2f92f}}  lists the recent methods.


MoVi: A large multi-purpose human motion and video dataset

Abstract: Large high-quality datasets of human body shape and kinematics lay the foundation for modelling and simulation approaches in computer vision, computer graphics, and biomechanics. Creating datasets that combine naturalistic recordings with high-accuracy data about ground truth body shape and pose is challenging because different motion recording systems are either optimized for one or the other. We address this issue in our dataset by using different hardware systems to record partially overlapping information and synchronized data that lend themselves to transfer learning. This multimodal dataset contains 9 hours of optical motion capture data, 17 hours of video data from 4 different points of view recorded by stationary and hand-held cameras, and 6.6 hours of inertial measurement units data recorded from 60 female and 30 male actors performing a collection of 21 everyday actions and sports movements. The processed motion capture data is also available as realistic 3D human meshes. We anticipate use of this dataset for research on human pose estimation, action recognition, motion modelling, gait analysis, and body shape reconstruction.

HUMAN4D: A Human-Centric Multimodal Dataset for Motions and Immersive Media

Abstract: We introduce HUMAN4D, a large and multimodal 4D dataset that contains a variety of human activities simultaneously captured by a professional marker-based MoCap, a volumetric capture and an audio recording system. By capturing 2 female and 2 male professional actors performing various full-body movements and expressions, HUMAN4D provides a diverse set of motions and poses encountered as part of single- and multi-person daily, physical and social activities (jumping, dancing, etc.), along with multi-RGBD (mRGBD), volumetric and audio data. Despite the existence of multi-view color datasets captured with the use of hardware (HW) synchronization, to the best of our knowledge, HUMAN4D is the first and only public resource that provides volumetric depth maps with high synchronization precision due to the use of intra- and inter-sensor HW-SYNC. Moreover, a spatio-temporally aligned scanned and rigged 3D character complements HUMAN4D to enable joint research on time-varying and high-quality dynamic meshes. We provide evaluation baselines by benchmarking HUMAN4D with state-of-the-art human pose estimation and 3D compression methods. We apply OpenPose and AlphaPose reaching 70.02% and 82.95% mAPPCKh-0.5 on single- and 68.48% and 73.94% mAPPCKh-0.5 on two-person 2D pose estimation, respectively. In 3D pose, a recent multi-view approach named Learnable Triangulation, achieves 80.26% mAPPCK3D-10cm. For 3D compression, we benchmark Draco, Corto and CWIPC open-source 3D codecs, respecting online encoding and steady bit-rates between 7–155 and 2–90 Mbps for mesh- and point-based volumetric video, respectively. Qualitative and quantitative visual comparison between mesh-based volumetric data reconstructed in different qualities and captured RGB, showcases the available options with respect to 4D representations. HUMAN4D is introduced to enable joint research on spatio-temporally aligned pose, volumetric, mRGBD and audio data cues. The dataset and its code are available online.

Comparison of human motion capture datasets.

Motions.
It is our top priority to include a wider variety of motions that can represent as many daily human activities as possible. Therefore, we first create a new large corpus of motions.
Specifically, we utilize Mixamo motions {{cite:1b72f6c}} and modify them using Unreal Engine {{cite:821b10a}} to enhance their plausibility and diversify the motion data.
We first manually fix some motions that involve self-penetration and then modify the motions in various ways, including the speed of motions, arm movements, foot stances, and head rotations.
For further details, please refer to our supplement.
In total, we created 45,520 natural motions for the 17 human models, i.e., {{formula:63362a33-5858-4f83-ab0e-1ed8b8d0bf47}}  motions per model.
We provide the details of our dataset in Tables {{table:a8cb6c3f-eac8-4084-8457-0d4bb4ec30f4}}  and {{table:bc312523-a3fe-4e90-a124-da599d53df45}} .
Table {{table:a8cb6c3f-eac8-4084-8457-0d4bb4ec30f4}}  compares existing mocap datasets and our motion data.
Note that AMASS {{cite:b41d10a}} is a collection of several existing motion capture datasets {{cite:49ac74d}}, {{cite:f1f495c}}, {{cite:8fc2772}}, {{cite:c0e9268}}, {{cite:5ee5492}}, {{cite:2f54ed8}}, {{cite:0250da6}}, {{cite:24c4dea}}, {{cite:38f8124}}, {{cite:0185ef4}}, {{cite:ac08d87}}, {{cite:24c4dea}}, {{cite:95ffc29}}, {{cite:7cfe3c0}}, {{cite:f8195a4}}, {{cite:a044f54}}, {{cite:e444e82}}, {{cite:55ef5dd}}.
Our dataset contains the largest number of motions with the longest
consecutive 3D human motions.
Table {{table:bc312523-a3fe-4e90-a124-da599d53df45}}  summarises the included motion categories.