Critical percolation in high dimensions.

Abstract: We present Monte Carlo estimates for site and bond percolation thresholds in simple hypercubic lattices with 4-13 dimensions. For d<6 they are preliminary, for d>or =6 they are between 20 and 10(4) times more precise than the best previous estimates. This was achieved by three ingredients: (i) simple and fast hashing that allowed us to simulate clusters of millions of sites on computers with less than 500 Mbytes memory; (ii) a histogram method that allowed us to obtain information for several p values from a single simulation; and (iii) a variance reduction technique that is especially efficient at high dimensions where it reduces error bars by a factor of up to approximately 30 and more. Based on these data we propose a scaling law for finite cluster size corrections.

Spreading with immunization in high dimensions

Abstract: We investigate a model of epidemic spreading with partial immunization which is controlled by two probabilities, namely, for first infections, p0, and for reinfections, p. When the two probabilities are equal, the model reduces to directed percolation, while for perfect immunization one obtains the general epidemic process belonging to the universality class of dynamical percolation. We focus on the critical behaviour in the vicinity of the directed percolation point, especially for high number of dimensions d>2. It is argued that the clusters of immune sites are compact for . This observation implies that a recently introduced scaling argument, suggesting a stretched exponential decay of the survival probability for p = pc, in one spatial dimension, where pc denotes the critical threshold for directed percolation, should apply for any number of dimensions and perhaps for d = 4 as well. Moreover, we show that the phase transition line, connecting the critical points of directed percolation and of dynamical percolation, terminates at the critical point of directed percolation with vanishing slope for d<4 and with finite slope for . Furthermore, an exponent is identified for the temporal correlation length for the case of p = pc and p0 = pc−, , which is different from the exponent of directed percolation. We also improve numerical estimates of several critical parameters and exponents, especially for dynamical percolation for d = 4,5.

Percolation thresholds and Fisher exponents in hypercubic lattices.

Abstract: We use invasion percolation to compute highly accurate numerical values for bond and site percolation thresholds p_{c} on the hypercubic lattice Z^{d} for d=4,...,13. We also compute the Fisher exponent τ governing the cluster size distribution at criticality. Our results support the claim that the mean-field value τ=5/2 holds for d≥6, with logarithmic corrections to power-law scaling at d=6.

Extended-range percolation in five dimensions

Abstract: Percolation on a five-dimensional simple hypercubic (sc(5)) lattice with extended neighborhoods is investigated by means of extensive Monte Carlo simulations, using an effective single-cluster growth algorithm. The critical exponents, including $\tau$ and $\Omega$, the asymptotic behavior of the threshold $p_c$ and its dependence on coordination number $z$ are investigated. Using the bond and site percolation thresholds $p_c = 0.11817145(3)$ and $0.14079633(4)$ respectively given by Mertens and Moore [Phys. Rev. E 98, 022120 (2018)], we find critical exponents of $\tau = 2.4177(3)$, $\Omega = 0.27(2)$ through a self-consistent process. The value of $\tau$ compares favorably with a recent five-loop renormalization predictions $2.4175(2)$ by Borinsky et al. [Phys. Rev. D 103, 116024 (2021)], the value 2.4180(6) that follows from the work of Zhang et al. [Physica A 580, 126124 (2021)], and the measurement of $2.419(1)$ by Mertens and Moore. We also confirmed the bond threshold, finding $p_c = 0.11817150(5)$. sc(5) lattices with extended neighborhoods up to 7th nearest neighbors are studied for both bond and site percolation. Employing the values of $\tau$ and $\Omega$ mentioned above, thresholds are found to high precision. For bond percolation, the asymptotic value of $zp_c$ tends to Bethe-lattice behavior ($z p_c \sim 1$), and the finite-$z$ correction is found to be consistent with both and $zp_{c} - 1 \sim a_1 z^{-0.88}$ and $zp_{c} - 1 \sim a_0(3 + \ln z)/z$. For site percolation, the asymptotic analysis is close to the predicted behavior $zp_c \sim 32\eta_c = 1.742(2)$ for large $z$, where $\eta_c = 0.05443(7)$ is the continuum percolation threshold of five-dimensional hyperspheres given by Torquato and Jiao [J. Chem. Phys 137, 074106 (2015)]; finite-$z$ corrections are accounted for by taking $p_c \approx c/(z + b)$ with $c=1.722(7)$ and $b=1$.

Values of the bond threshold for SC(5). MC = Monte Carlo.

Figure {{figure:0c42c953-2fb0-45cf-87b4-dc46dfe61e47}}  shows the relations of {{formula:95ed30d0-fe5f-44c2-aa69-27f7d6f1187b}}  versus {{formula:801926ed-739a-4626-9e9f-7a39a6f81d06}}  and {{formula:c60df388-ad6a-4ee1-a67c-a045ffc4d3e7}}  versus {{formula:4545a7e5-1930-4886-9a16-c988a81f5203}}  under {{formula:04936d4d-5090-4125-aeb1-59f2a70b7f29}} . To check the value of {{formula:925a0c96-5428-4e29-8237-16ed7a912a94}}  for bond percolation, we also simulated {{formula:419c2e9d-582e-4ffe-8ccd-5dd649f03aba}}  versus {{formula:e2dcc009-832b-49d9-9659-365b70673275}}  under occupying probabilities {{formula:8523714c-7302-4f52-a226-bc28b3cad193}} , as shown in Fig. {{figure:0c42c953-2fb0-45cf-87b4-dc46dfe61e47}} (a). A large number of runs, {{formula:fac1089a-032e-4679-a38e-01fa0764d2b4}} , were required to distinguish the behavior for these very close values of {{formula:720adc0d-b4b6-4ddc-a10d-f9bc9ee86829}} . The threshold is determined by the value of {{formula:c9da2094-6813-4e62-9026-d9d5be6d0d41}}  where the plot is linear for larger {{formula:01bcb74c-321b-43dc-b1ae-23fc51f85ff5}} . The resulting estimate of the threshold {{formula:7f2ef914-874b-4d7e-bd91-38b00b665f0f}}  is consistent with previous results as shown in table {{table:660c9cda-224f-4e2e-8c8f-1337849e11fd}} .
Overall, we found that self-consistent results can be found if we choose the values of {{formula:f11d50d8-1221-462e-96c6-83705542b7c1}} , {{formula:500783e7-fdff-48e4-9f2e-d4a591984624}}  and {{formula:84eab1eb-85e5-4696-9027-15b932a4d49b}} ,
that is, both {{formula:0bcb0fa3-f22c-406e-846f-551a5a455219}}  versus {{formula:36dab572-5d3b-4376-809b-66105383a725}}  and {{formula:598aac3c-4f9f-4981-8cdd-20acf597fa41}}  versus {{formula:bb89eae4-9582-464e-955d-0a5550cab9a2}}  show linear behavior, meanwhile the intercept tends to {{formula:1d7f89e1-ff30-4f82-8518-edb7761e4bcc}}  for ({{formula:38d5d64e-b7ae-4125-8af8-5f62cc9d8dfe}} ), as shown in Fig. {{figure:0c42c953-2fb0-45cf-87b4-dc46dfe61e47}} (b). In addition, the simulation results in Fig. {{figure:0c42c953-2fb0-45cf-87b4-dc46dfe61e47}} (a) also indicate that if {{formula:fe2db74c-96f9-4ca3-9184-dc01fb376074}}  is away from {{formula:113253bd-1401-4e8f-8a3c-5b65d544d368}} , the behavior will be non-linear.


Dynamic Scheduling based on Particle Swarm Optimization for Cloud-based Scientific Experiments

Abstract: Fil: Pacini Naumovich, Elina Rocio. Universidad Nacional de Cuyo. Instituto de Tecnologias de la Informacion y las Comunicaciones; Argentina

Application Scheduling in Mobile Cloud Computing with Load Balancing

Abstract: Mobile cloud computing (MCC) enables the mobile devices to offload their applications to the cloud and thus greatly enriches the types of applications on mobile devices and enhances the quality of service of the applications. Under various circumstances, researchers have put forward several MCC architectures. However, how to reduce the response latency while efficiently utilizing the idle service capacities of the mobile devices still remains a challenge. In this paper, we firstly give a definition of MCC and divide the recently proposed architectures into four categories. Secondly, we present a Hybrid Local Mobile Cloud Model (HLMCM) by extending the Cloudlet architecture. Then, after formulating the application scheduling problems in HLMCM and bringing forward the Hybrid Ant Colony algorithm based Application Scheduling (HACAS) algorithm, we finally validate the efficiency of the HACAS algorithm by simulation experiments.

SIAs for Addressing Issues in CC and EC.

This subsection discusses the works on the use of SIAs for solving issues in CC and EC, as summarized in Table {{table:40ae0649-378f-4edc-977d-d81506087235}} .


2)How can
we detect APTs based on this model?

2)How can we
link different alerts to a single attack?

Correlated events describe an attack.
Detection is based on signatures, profiling, and policy rules.

Recon, initial intrusion, C&C, lateral movement, and exfiltration.

A tag is associated to an event from
the IDS and its propagation underlines the complete chain of the attack.

ATTACK2VEC: Leveraging Temporal Word Embeddings to Understand the Evolution of Cyberattacks

Abstract: Despite the fact that cyberattacks are constantly growing in complexity, the research community still lacks effective tools to easily monitor and understand them. In particular, there is a need for techniques that are able to not only track how prominently certain malicious actions, such as the exploitation of specific vulnerabilities, are exploited in the wild, but also (and more importantly) how these malicious actions factor in as attack steps in more complex cyberattacks. In this paper we present ATTACK2VEC, a system that uses temporal word embeddings to model how attack steps are exploited in the wild, and track how they evolve. We test ATTACK2VEC on a dataset of billions of security events collected from the customers of a commercial Intrusion Prevention System over a period of two years, and show that our approach is effective in monitoring the emergence of new attack strategies in the wild and in flagging which attack steps are often used together by attackers (e.g., vulnerabilities that are frequently exploited together). ATTACK2VEC provides a useful tool for researchers and practitioners to better understand cyberattacks and their evolution, and use this knowledge to improve situational awareness and develop proactive defenses.

UNICORN: Runtime Provenance-Based Detector for Advanced Persistent Threats

Abstract: Advanced Persistent Threats (APTs) are difficult to detect due to their "low-and-slow" attack patterns and frequent use of zero-day exploits. We present UNICORN, an anomaly-based APT detector that effectively leverages data provenance analysis. From modeling to detection, UNICORN tailors its design specifically for the unique characteristics of APTs. Through extensive yet time-efficient graph analysis, UNICORN explores provenance graphs that provide rich contextual and historical information to identify stealthy anomalous activities without pre-defined attack signatures. Using a graph sketching technique, it summarizes long-running system execution with space efficiency to combat slow-acting attacks that take place over a long time span. UNICORN further improves its detection capability using a novel modeling approach to understand long-term behavior as the system evolves. Our evaluation shows that UNICORN outperforms an existing state-of-the-art APT detection system and detects real-life APT scenarios with high accuracy.

Software Updates Strategies: a Quantitative Evaluation against Advanced Persistent Threats

Abstract: Software updates reduce the opportunity for exploitation. However, since updates can also introduce breaking changes, enterprises face the problem of balancing the need to secure software with updates with the need to support operations. We propose a methodology to quantitatively investigate the effectiveness of software updates strategies against attacks of Advanced Persistent Threats (APTs). We consider strategies where the vendor updates are the only limiting factors to cases in which enterprises delay updates from 1 to 7 months based on SANS data. Our manually curated dataset of APT attacks covers 86 APTs and 350 campaigns from 2008 to 2020. It includes information about attack vectors, exploited vulnerabilities (e.g. 0-days vs public vulnerabilities), and affected software and versions. Contrary to common belief, most APT campaigns employed publicly known vulnerabilities. If an enterprise could theoretically update as soon as an update is released, it would face lower odds of being compromised than those waiting one (4.9x) or three (9.1x) months. However, if attacked, it could still be compromised from 14% to 33% of the times. As in practice enterprises must do regression testing before applying an update, our major finding is that one could perform 12% of all possible updates restricting oneself only to versions fixing publicly known vulnerabilities without significant changes to the odds of being compromised compared to a company that updates for all versions.

Research Questions and Answers from the State of the Art about APTs over the years

Tab. {{table:1b291265-194c-432f-979d-70fc5542196a}}  shows the research questions of the SoA and the related
results concerning APTs and TI in the different categories.


BioGRID: a general repository for interaction datasets

Abstract: Access to unified datasets of protein and genetic interactions is critical for interrogation of gene/protein function and analysis of global network properties. BioGRID is a freely accessible database of physical and genetic interactions available at . BioGRID release version 2.0 includes >116 000 interactions from Saccharomyces cerevisiae, Caenorhabditis elegans, Drosophila melanogaster and Homo sapiens. Over 30 000 interactions have recently been added from 5778 sources through exhaustive curation of the Saccharomyces cerevisiae primary literature. An internally hyper-linked web interface allows for rapid search and retrieval of interaction data. Full or user-defined datasets are freely downloadable as tab-delimited text files and PSI-MI XML. Pre-computed graphical layouts of interactions are available in a variety of file formats. User-customized graphs with embedded protein, gene and interaction attributes can be constructed with a visualization system called Osprey that is dynamically linked to the BioGRID.

Characterizing interactions in online social networks during exceptional events

Abstract: Nowadays, millions of people interact on a daily basis on online social media like Facebook and Twitter, where they share and discuss information about a wide variety of topics. In this paper, we focus on a specific online social network, Twitter, and we analyze multiple datasets each one consisting of individuals' online activity before, during and after an exceptional event in terms of volume of the communications registered. We consider important events that occurred in different arenas that range from policy to culture or science. For each dataset, the users' online activities are modeled by a multilayer network in which each layer conveys a different kind of interaction, specifically: retweeting, mentioning and replying. This representation allows us to unveil that these distinct types of interaction produce networks with different statistical properties, in particular concerning the degree distribution and the clustering structure. These results suggests that models of online activity cannot discard the information carried by this multilayer representation of the system, and should account for the different processes generated by the different kinds of interactions. Secondly, our analysis unveils the presence of statistical regularities among the different events, suggesting that the non-trivial topological patterns that we observe may represent universal features of the social dynamics on online social networks during exceptional events.

Combinatorial Analysis of Multiple Networks

Abstract: The study of complex networks has been historically based on simple graph data models representing relationships between individuals. However, often reality cannot be accurately captured by a flat graph model. This has led to the development of multi-layer networks. These models have the potential of becoming the reference tools in network data analysis, but require the parallel development of specific analysis methods explicitly exploiting the information hidden in-between the layers and the availability of a critical mass of reference data to experiment with the tools and investigate the real-world organization of these complex systems. In this work we introduce a real-world layered network combining different kinds of online and offline relationships, and present an innovative methodology and related analysis tools suggesting the existence of hidden motifs traversing and correlating different representation layers. We also introduce a notion of betweenness centrality for multiple networks. While some preliminary experimental evidence is reported, our hypotheses are still largely unverified, and in our opinion this calls for the availability of new analysis methods but also new reference multi-layer social network data.

Navigability of interconnected networks under random failures

Abstract: Significance Network theory has been exploited in the last decades to deepen our comprehension of complex systems. However, real-world complex systems exhibit multiple levels of relationships and require modeling by interconnected networks, characterizing interactions on several levels simultaneously. Questions such as “what is the efficiency of exploration of a city using the multiple transportation layers, like subway and bus?” and “what is its resilience to failures?” have to be answered using the multiplex framework. Here, we introduce fundamental mechanisms to perform such exploration, using random walks on multilayer networks, and we show how the topological structure, together with the navigation strategy, influences the efficiency in exploring the whole structure. Assessing the navigability of interconnected networks (transporting information, people, or goods) under eventual random failures is of utmost importance to design and protect critical infrastructures. Random walks are a good proxy to determine this navigability, specifically the coverage time of random walks, which is a measure of the dynamical functionality of the network. Here, we introduce the theoretical tools required to describe random walks in interconnected networks accounting for structure and dynamics inherent to real systems. We develop an analytical approach for the covering time of random walks in interconnected networks and compare it with extensive Monte Carlo simulations. Generally speaking, interconnected networks are more resilient to random failures than their individual layers per se, and we are able to quantify this effect. As an application––which we illustrate by considering the public transport of London––we show how the efficiency in exploring the multiplex critically depends on layers’ topology, interconnection strengths, and walk strategy. Our findings are corroborated by data-driven simulations, where the empirical distribution of check-ins and checks-out is considered and passengers travel along fastest paths in a network affected by real disruptions. These findings are fundamental for further development of searching and navigability strategies in real interconnected systems.

Real multiplex networks and their properties. The ID in the second column corresponds to Figure 3 (c).

All considered networks are summarized in Table  {{table:32549a71-4ca2-43f3-9cb6-0756bf144481}}The networks are hosted at https://comunelab.fbk.eu/data.php. All considered networks are static. We computed the metrics for all connected components.
The entanglement algorithm was integrated into the Py3plex library {{cite:f9d723b}}https://github.com/SkBlaz/Py3plex/blob/master/examples/example_entanglement.py. The generator is accessible at https://github.com/SkBlaz/Py3plex/blob/master/py3plex/core/random_generators.py. For each network, we computed homogeneity and intensity. For the generation of synthetic networks, we used the following hyperparameter ranges:


Combines lattice alignment and asymmetric cpnf to improve the secure sum rates.

Investigates internet and external eavesdropper scenarios, includes two-hop channel, jammer and improves the secrecy rate.

Defines and derives the secrecy computation-capacity and shows that it can be achieved without sacrificing capacity.

Compute-and-forward can buy secrecy cheap

Abstract: We consider a Gaussian multiple access channel with K transmitters, a (intended) receiver and an external eavesdropper. The transmitters wish to reliably communicate with the receiver while concealing their messages from the eavesdropper. This scenario has been investigated in prior works using two different coding techniques; the random i.i.d. Gaussian coding and the signal alignment coding. Although, the latter offers promising results in a very high SNR regime, extending these results to the finite SNR regime is a challenging task. In this paper, we propose a new lattice alignment scheme based on the compute-and-forward framework which works at any finite SNR. We show that our achievable secure sum rate scales with log(SNR) and hence, in most SNR regimes, our scheme outperforms the random coding scheme in which the secure sum rate does not grow with power. Furthermore, we show that our result matches the prior work in the infinite SNR regime. Additionally, we analyze our result numerically.

Secure Transmission on the Two-hop Relay Channel with Scaled Compute-and-Forward

Abstract: In this paper, we consider communication on a two-hop channel, in which a source wants to send information reliably and securely to the destination via a relay. We consider both the untrusted relay case and the external eavesdropper case. In the untrusted relay case, the relay behaves as an eavesdropper and there is a cooperative node which sends a jamming signal to confuse the relay when the it is receiving from the source. We propose two secure transmission schemes using the scaled compute-and-forward technique. One of the schemes is based on a random binning code and the other one is based on a lattice chain code. It is proved that in either the high Signal-to-Noise-Ratio (SNR) scenario and/or the restricted relay power scenario, if the destination is used as the jammer, both schemes outperform all existing schemes and achieve the upper bound. In particular, if the SNR is large and the source, the relay, and the cooperative jammer have identical power and channels, both schemes achieve the upper bound for secrecy rate, which is merely $1/2$ bit per channel use lower than the channel capacity without secrecy constraints. We also prove that one of our schemes achieves a positive secrecy rate in the external eavesdropper case in which the relay is trusted and there exists an external eavesdropper.

On secure computation over the binary modulo-2 adder multiple-access wiretap channel

Abstract: In this paper, the problem of securely computing a function over the binary modulo-2 adder multiple-access wiretap channel is considered. The problem involves a legitimate receiver that wishes to reliably and efficiently compute a function of distributed binary sources while an eavesdropper has to be kept ignorant of them. In order to characterize the corresponding fundamental limit, the notion of secrecy computation-capacity is introduced. Although determining the secrecy computation-capacity is challenging for arbitrary functions, it surprisingly turns out that if the function perfectly matches the algebraic structure of the channel and the joint source distribution fulfills certain conditions, the secrecy computation-capacity equals the computation capacity, which is the supremum of all achievable computation rates without secrecy constraints. Unlike the case of securely transmitting messages, no additional randomness is needed at the encoders nor does the legitimate receiver need any advantage over the eavesdropper. The results therefore show that the problem of securely computing a function over a multiple-access wiretap channel may significantly differ from the one of securely communicating messages.

An overview of security studies.

Superposition of signals is shown to be beneficial to provide security to wireless networks. Several security studies that use simultaneous transmission are listed in Table {{table:2f697ddc-6fc6-4d6b-ac46-4375a522e7e9}} . In this section, we present the security applications in detail.


SE(3)-equivariant prediction of molecular wavefunctions and electronic densities

Abstract: Machine learning has enabled the prediction of quantum chemical properties with high accuracy and efficiency, allowing to bypass computationally costly ab initio calculations. Instead of training on a fixed set of properties, more recent approaches attempt to learn the electronic wavefunction (or density) as a central quantity of atomistic systems, from which all other observables can be derived. This is complicated by the fact that wavefunctions transform non-trivially under molecular rotations, which makes them a challenging prediction target. To solve this issue, we introduce general SE(3)-equivariant operations and building blocks for constructing deep learning architectures for geometric point cloud data and apply them to reconstruct wavefunctions of atomistic systems with unprecedented accuracy. Our model achieves speedups of over three orders of magnitude compared to ab initio methods and reduces prediction errors by up to two orders of magnitude compared to the previous state-of-the-art. This accuracy makes it possible to derive properties such as energies and forces directly from the wavefunction in an end-to-end manner. We demonstrate the potential of our approach in a transfer learning application, where a model trained on low accuracy reference wavefunctions implicitly learns to correct for electronic many-body interactions from observables computed at a higher level of theory. Such machine-learned wavefunction surrogates pave the way towards novel semi-empirical methods, offering resolution at an electronic level while drastically decreasing computational cost. Additionally, the predicted wavefunctions can serve as initial guess in conventional ab initio methods, decreasing the number of iterations required to arrive at a converged solution, thus leading to significant speedups without any loss of accuracy or robustness.

Efficient and Equivariant Graph Networks for Predicting Quantum Hamiltonian

Abstract: We consider the prediction of the Hamiltonian matrix, which finds use in quantum chemistry and condensed matter physics. Efficiency and equivariance are two important, but conflicting factors. In this work, we propose a SE(3)-equivariant network, named QHNet, that achieves efficiency and equivariance. Our key advance lies at the innovative design of QHNet architecture, which not only obeys the underlying symmetries, but also enables the reduction of number of tensor products by 92\%. In addition, QHNet prevents the exponential growth of channel dimension when more atom types are involved. We perform experiments on MD17 datasets, including four molecular systems. Experimental results show that our QHNet can achieve comparable performance to the state of the art methods at a significantly faster speed. Besides, our QHNet consumes 50\% less memory due to its streamlined architecture. Our code is publicly available as part of the AIRS library (\url{https://github.com/divelab/AIRS}).

Equivariant Quantum Tensor Networks. They develop different modules to apply tensor expansion to obtain equivariant matrix from equivariant features. Currently, batch training can be applied on QHNet with various molecules while other implementations focusing on single same system during training and testing. Meanwhile, the implementation of DeepH-E3, HamGNN and xDeepH can be applied on both molecule and material systems.

Equivariant Networks on Quantum Tensor Predictions:
Due to the intrinsic equivariance nature of Hamiltonian matrix and other quantum tensors, equivariant networks {{cite:dd7e3c2}}, {{cite:7d157ce}}, {{cite:66834ab}}, {{cite:d0e5b55}}, {{cite:841a477}} have become the mainstream methods to obtain these quantum tensors. Especially, tensor expansion stands out as a powerful technique for constructing equivariant matrix from equivariant features.
As listed in Table {{table:4e886daf-b760-4fe5-8d8d-89da8ad8a81c}} , these equivariant networks integrate tensor expansion with various techniques to construct Hamiltonian matrices that satisfy the intrinsic symmetries required for specific tasks.
For the basic Hamiltonian matrix prediction, PhiSNet builds the entire matrix with the atom-orbital quadruple irrep selection. This selection assigns a channel index on the irrep used for tensor expansion to each quadruple {{formula:c94452fd-1899-499b-9f12-9901a502f3b2}} .
While QHNet follows the a fashion of TFN, it introduces learnable parameters in filter operation and maps the channels of output equivariant matrix block to the full orbital matrix.
When considering the spin-orbital coupling, the spin equivariance follows rotation order {{formula:0bb4d600-0032-42b2-bfda-1dfacd08aca1}}  and {{formula:5fa65f02-4982-4efd-837d-24d7472ec3be}} .
Addressing this non-integral equivariance challenge involves employing basis transformation techniques to revert the basis back to an integral basis. Similarly, the basis transformation technique can be used to resolve the time-reversal equivariance issue for Hamiltonian matrix with magnetic momentum.


VINet: Visual-Inertial Odometry as a Sequence-to-Sequence Learning Problem

Abstract: In this paper we present an on-manifold sequence-to-sequence learning approach to motion estimation using visual and inertial sensors. It is to the best of our knowledge the first end-to-end trainable method for visual-inertial odometry which performs fusion of the data at an intermediate feature-representation level. Our method has numerous advantages over traditional approaches. Specifically, it eliminates the need for tedious manual synchronization of the camera and IMU as well as eliminating the need for manual calibration between the IMU and camera. A further advantage is that our model naturally and elegantly incorporates domain specific information which significantly mitigates drift. We show that our approach is competitive with state-of-the-art traditional methods when accurate calibration data is available and can be trained to outperform them in the presence of calibration and synchronization errors.

DeepVO: Towards End-to-End Visual Odometry with Deep Recurrent Convolutional Neural Networks

Abstract: This paper studies monocular visual odometry (VO) problem. Most of existing VO algorithms are developed under a standard pipeline including feature extraction, feature matching, motion estimation, local optimisation, etc. Although some of them have demonstrated superior performance, they usually need to be carefully designed and specifically fine-tuned to work well in different environments. Some prior knowledge is also required to recover an absolute scale for monocular VO. This paper presents a novel end-to-end framework for monocular VO by using deep Recurrent Convolutional Neural Networks (RCNNs). Since it is trained and deployed in an end-to-end manner, it infers poses directly from a sequence of raw RGB images (videos) without adopting any module in the conventional VO pipeline. Based on the RCNNs, it not only automatically learns effective feature representation for the VO problem through Convolutional Neural Networks, but also implicitly models sequential dynamics and relations using deep Recurrent Neural Networks. Extensive experiments on the KITTI VO dataset show competitive performance to state-of-the-art methods, verifying that the end-to-end Deep Learning technique can be a viable complement to the traditional VO systems.

VidLoc: A Deep Spatio-Temporal Model for 6-DoF Video-Clip Relocalization

Abstract: Machine learning techniques, namely convolutional neural networks (CNN) and regression forests, have recently shown great promise in performing 6-DoF localization of monocular images. However, in most cases image-sequences, rather only single images, are readily available. To this extent, none of the proposed learning-based approaches exploit the valuable constraint of temporal smoothness, often leading to situations where the per-frame error is larger than the camera motion. In this paper we propose a recurrent model for performing 6-DoF localization of video-clips. We find that, even by considering only short sequences (20 frames), the pose estimates are smoothed and the localization error can be drastically reduced. Finally, we consider means of obtaining probabilistic pose estimates from our model. We evaluate our method on openly-available real-world autonomous driving and indoor localization datasets.

IONet: Learning to Cure the Curse of Drift in Inertial Odometry

Abstract: Inertial sensors play a pivotal role in indoor localization, which in turn lays the foundation for pervasive personal applications. However, low-cost inertial sensors, as commonly found in smartphones, are plagued by bias and noise, which leads to unbounded growth in error when accelerations are double integrated to obtain displacement. Small errors in state estimation propagate to make odometry virtually unusable in a matter of seconds. We propose to break the cycle of continuous integration, and instead segment inertial data into independent windows. The challenge becomes estimating the latent states of each window, such as velocity and orientation, as these are not directly observable from sensor data. We demonstrate how to formulate this as an optimization problem, and show how deep recurrent neural networks can yield highly accurate trajectories, outperforming state-of-the-art shallow techniques, on a wide range of tests and attachments. In particular, we demonstrate that IONet can generalize to estimate odometry for non-periodic motion, such as a shopping trolley or baby-stroller, an extremely challenging task for existing techniques.

UnDeepVO: Monocular Visual Odometry through Unsupervised Deep Learning

Abstract: We propose a novel monocular visual odometry (VO) system called UnDeepVO in this paper. UnDeepVO is able to estimate the 6-DoF pose of a monocular camera and the depth of its view by using deep neural networks. There are two salient features of the proposed UnDeepVO: one is the unsupervised deep learning scheme, and the other is the absolute scale recovery. Specifically, we train UnDeepVO by using stereo image pairs to recover the scale but test it by using consecutive monocular images. Thus, UnDeepVO is a monocular system. The loss function defined for training the networks is based on spatial and temporal dense information. A system overview is shown in Fig. 1. The experiments on KITTI dataset show our UnDeepVO achieves good performance in terms of pose accuracy.

Deep Auxiliary Learning for Visual Localization and Odometry

Abstract: Localization is an indispensable component of a robot's autonomy stack that enables it to determine where it is in the environment, essentially making it a precursor for any action execution or planning. Although convolutional neural networks have shown promising results for visual localization, they are still grossly outperformed by state-of-the-art local feature-based techniques. In this work, we propose VLocNet, a new convolutional neural network architecture for 6-DoF global pose regression and odometry estimation from consecutive monocular images. Our multitask model incorporates hard parameter sharing, thus being compact and enabling real-time inference, in addition to being end-to-end trainable. We propose a novel loss function that utilizes auxiliary learning to leverage relative pose information during training, thereby constraining the search space to obtain consistent pose estimates. We evaluate our proposed VLocNet on indoor as well as outdoor datasets and show that even our single task model exceeds the performance of state-of-the-art deep architectures for global localization, while achieving competitive performance for visual odometry estimation. Furthermore, we present extensive experimental evaluations utilizing our proposed Geometric Consistency Loss that show the effectiveness of multitask learning and demonstrate that our model is the first deep learning technique to be on par with, and in some cases outperforms state-of-the-art SIFT-based approaches.

RIDI: Robust IMU Double Integration

Abstract: This paper proposes a novel data-driven approach for inertial navigation, which learns to estimate trajectories of natural human motions just from an inertial measurement unit (IMU) in every smartphone. The key observation is that human motions are repetitive and consist of a few major modes (e.g., standing, walking, or turning). Our algorithm regresses a velocity vector from the history of linear accelerations and angular velocities, then corrects low-frequency bias in the linear accelerations, which are integrated twice to estimate positions. We have acquired training data with ground-truth motions across multiple human subjects and multiple phone placements (e.g., in a bag or a hand). The qualitatively and quantitatively evaluations have demonstrated that our algorithm has surprisingly shown comparable results to full Visual Inertial navigation. To our knowledge, this paper is the first to integrate sophisticated machine learning techniques with inertial navigation, potentially opening up a new line of research in the domain of data-driven inertial navigation. We will publicly share our code and data to facilitate further research.

End-to-End Learning Framework for IMU-Based 6-DOF Odometry

Abstract: This paper presents an end-to-end learning framework for performing 6-DOF odometry by using only inertial data obtained from a low-cost IMU. The proposed inertial odometry method allows leveraging inertial sensors that are widely available on mobile platforms for estimating their 3D trajectories. For this purpose, neural networks based on convolutional layers combined with a two-layer stacked bidirectional LSTM are explored from the following three aspects. First, two 6-DOF relative pose representations are investigated: one based on a vector in the spherical coordinate system, and the other based on both a translation vector and an unit quaternion. Second, the loss function in the network is designed with the combination of several 6-DOF pose distance metrics: mean squared error, translation mean absolute error, quaternion multiplicative error and quaternion inner product. Third, a multi-task learning framework is integrated to automatically balance the weights of multiple metrics. In the evaluation, qualitative and quantitative analyses were conducted with publicly-available inertial odometry datasets. The best combination of the relative pose representation and the loss function was the translation and quaternion together with the translation mean absolute error and quaternion multiplicative error, which obtained more accurate results with respect to state-of-the-art inertial odometry techniques.

DeepVIO: Self-supervised Deep Learning of Monocular Visual Inertial Odometry using 3D Geometric Constraints

Abstract: This paper presents an self-supervised deep learning network for monocular visual inertial odometry (named DeepVIO). DeepVIO provides absolute trajectory estimation by directly merging 2D optical flow feature (OFF) and Inertial Measurement Unit (IMU) data. Specifically, it firstly estimates the depth and dense 3D point cloud of each scene by using stereo sequences, and then obtains 3D geometric constraints including 3D optical flow and 6-DoF pose as supervisory signals. Note that such 3D optical flow shows robustness and accuracy to dynamic objects and textureless environments. In DeepVIO training, 2D optical flow network is constrained by the projection of its corresponding 3D optical flow, and LSTM-style IMU preintegration network and the fusion network are learned by minimizing the loss functions from ego-motion constraints. Furthermore, we employ an IMU status update scheme to improve IMU pose estimation through updating the additional gyroscope and accelerometer bias. The experimental results on KITTI and EuRoC datasets show that DeepVIO outperforms state-of-the-art learning based methods in terms of accuracy and data adaptability. Compared to the traditional methods, DeepVIO reduces the impacts of inaccurate Camera-IMU calibrations, unsynchronized and missing data.

RIANN -- A Robust Neural Network Outperforms Attitude Estimation Filters

Abstract: Inertial-sensor-based attitude estimation is a crucial technology in various applications, from human motion tracking to autonomous aerial and ground vehicles. Application scenarios differ in characteristics of the performed motion, presence of disturbances, and environmental conditions. Since state-of-the-art attitude estimators do not generalize well over these characteristics, their parameters must be tuned for the individual motion characteristics and circumstances. We propose RIANN, a ready-to-use, neural network-based, parameter-free, real-time-capable inertial attitude estimator, which generalizes well across different motion dynamics, environments, and sampling rates, without the need for application-specific adaptations. We gather six publicly available datasets of which we exploit two datasets for the method development and the training, and we use four datasets for evaluation of the trained estimator in three different test scenarios with varying practical relevance. Results show that RIANN outperforms state-of-the-art attitude estimation filters in the sense that it generalizes much better across a variety of motions and conditions in different applications, with different sensor hardware and different sampling frequencies. This is true even if the filters are tuned on each individual test dataset, whereas RIANN was trained on completely separate data and has never seen any of these test datasets. RIANN can be applied directly without adaptations or training and is therefore expected to enable plug-and-play solutions in numerous applications, especially when accuracy is crucial but no ground-truth data is available for tuning or when motion and disturbance characteristics are uncertain. We made RIANN publicly available.

CTIN: Robust Contextual Transformer Network for Inertial Navigation

Abstract: Recently, data-driven inertial navigation approaches have demonstrated their capability of using well-trained neural networks to obtain accurate position estimates from inertial measurement units (IMU) measurements. In this paper, we propose a novel robust Contextual Transformer-based network for Inertial Navigation~(CTIN) to accurately predict velocity and trajectory. To this end, we first design a ResNet-based encoder enhanced by local and global multi-head self-attention to capture spatial contextual information from IMU measurements. Then we fuse these spatial representations with temporal knowledge by leveraging multi-head attention in the Transformer decoder. Finally, multi-task learning with uncertainty reduction is leveraged to improve learning efficiency and prediction accuracy of velocity and trajectory. Through extensive experiments over a wide range of inertial datasets~(e.g. RIDI, OxIOD, RoNIN, IDOL, and our own), CTIN is very robust and outperforms state-of-the-art models.

Faster Deep Inertial Pose Estimation with Six Inertial Sensors

Abstract: We propose a novel pose estimation method that can predict the full-body pose from six inertial sensors worn by the user. This method solves problems encountered in vision, such as occlusion or expensive deployment. We address several complex challenges. First, we use the SRU network structure instead of the bidirectional RNN structure used in previous work to reduce the computational effort of the model without losing its accuracy. Second, our model does not require joint position supervision to achieve the best results of the previous work. Finally, since sensor data tend to be noisy, we use SmoothLoss to reduce the impact of inertial sensors on pose estimation. The faster deep inertial poser model proposed in this paper can perform online inference at 90 FPS on the CPU. We reduce the impact of each error by more than 10% and increased the inference speed by 250% compared to the previous state of the art.

Leveraging Self-Attention Mechanism for Attitude Estimation in Smartphones

Abstract: Inertial attitude estimation is a crucial component of many modern systems and applications. Attitude estimation from commercial-grade inertial sensors has been the subject of an abundance of research in recent years due to the proliferation of Inertial Measurement Units (IMUs) in mobile devices, such as the smartphone. Traditional methodologies involve probabilistic, iterative-state estimation; however, these approaches do not generalise well over changing motion dynamics and environmental conditions, as they require context-specific parameter tuning. In this work, we explore novel methods for attitude estimation from low-cost inertial sensors using a self-attention-based neural network, the Attformer. This paper proposes to part ways from the traditional cycle of continuous integration algorithms, and formulate it as an optimisation problem. This approach separates itself by leveraging attention operations to learn the complex patterns and dynamics associated with inertial data, allowing for the linear complexity in the dimension of the feature vector to account for these patterns. Additionally, we look at combining traditional state-of-the-art approaches with our self-attention method. These models were evaluated on entirely unseen sequences, over a range of different activities, users and devices, and compared with a recent alternate deep learning approach, the unscented Kalman filter and the iOS CoreMotion API. The inbuilt iOS had a mean angular distance from the true attitude of 117.31∘, the GRU 21.90∘, the UKF 16.38∘, the Attformer 16.28∘ and, finally, the UKF–Attformer had mean angular distance of 10.86∘. We show that this plug-and-play solution outperforms previous approaches and generalises well across different users, devices and activities.

Deep Learning for Localization.

In Table {{table:a0287021-fcf7-4dbc-ad2c-933744593ee6}} , we have summarized some related works in the navigation field that have utilized deep learning techniques.


Benchmarking Robustness of 3D Point Cloud Recognition Against Common Corruptions

Abstract: Deep neural networks on 3D point cloud data have been widely used in the real world, especially in safety-critical applications. However, their robustness against corruptions is less studied. In this paper, we present ModelNet40-C, the first comprehensive benchmark on 3D point cloud corruption robustness, consisting of 15 common and realistic corruptions. Our evaluation shows a significant gap between the performances on ModelNet40 and ModelNet40-C for state-of-the-art (SOTA) models. To reduce the gap, we propose a simple but effective method by combining PointCutMix-R and TENT after evaluating a wide range of augmentation and test-time adaptation strategies. We identify a number of critical insights for future studies on corruption robustness in point cloud recognition. For instance, we unveil that Transformer-based architectures with proper training recipes achieve the strongest robustness. We hope our in-depth analysis will motivate the development of robust training strategies or architecture designs in the 3D point cloud domain. Our codebase and dataset are included in https://github.com/jiachens/ModelNet40-C

ShapeNet: An Information-Rich 3D Model Repository

Abstract: We present ShapeNet: a richly-annotated, large-scale repository of shapes represented by 3D CAD models of objects. ShapeNet contains 3D models from a multitude of semantic categories and organizes them under the WordNet taxonomy. It is a collection of datasets providing many semantic annotations for each 3D model such as consistent rigid alignments, parts and bilateral symmetry planes, physical sizes, keywords, as well as other planned annotations. Annotations are made available through a public web-based interface to enable data visualization of object attributes, promote data-driven geometric analysis, and provide a large-scale quantitative benchmark for research in computer graphics and vision. At the time of this technical report, ShapeNet has indexed more than 3,000,000 models, 220,000 models out of which are classified into 3,135 categories (WordNet synsets). In this report we describe the ShapeNet effort as a whole, provide details for all currently available datasets, and summarize future plans.

ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenes

Abstract: A key requirement for leveraging supervised deep learning methods is the availability of large, labeled datasets. Unfortunately, in the context of RGB-D scene understanding, very little data is available -- current datasets cover a small range of scene views and have limited semantic annotations. To address this issue, we introduce ScanNet, an RGB-D video dataset containing 2.5M views in 1513 scenes annotated with 3D camera poses, surface reconstructions, and semantic segmentations. To collect this data, we designed an easy-to-use and scalable RGB-D capture system that includes automated surface reconstruction and crowdsourced semantic annotation. We show that using this data helps achieve state-of-the-art performance on several 3D scene understanding tasks, including 3D object classification, semantic voxel labeling, and CAD model retrieval. The dataset is freely available at http://www.scan-net.org.

Revisiting Point Cloud Classification: A New Benchmark Dataset and Classification Model on Real-World Data

Abstract: Deep learning techniques for point cloud data have demonstrated great potentials in solving classical problems in 3D computer vision such as 3D object classification and segmentation. Several recent 3D object classification methods have reported state-of-the-art performance on CAD model datasets such as ModelNet40 with high accuracy (~92%). Despite such impressive results, in this paper, we argue that object classification is still a challenging task when objects are framed with real-world settings. To prove this, we introduce ScanObjectNN, a new real-world point cloud object dataset based on scanned indoor scene data. From our comprehensive benchmark, we show that our dataset poses great challenges to existing point cloud classification techniques as objects from real-world scans are often cluttered with background and/or are partial due to occlusions. We identify three key open problems for point cloud object classification, and propose new point cloud classification neural networks that achieve state-of-the-art performance on classifying objects with cluttered background. Our dataset and code are publicly available in our project page https://hkust-vgd.github.io/scanobjectnn/.

Point cloud classification datasets

Today, there are many point cloud datasets provided by industries and universities. The performance of different methods on these datasets reflects the reliability and accuracy of the methods. These datasets consist of virtual or real scenes, which can provide ground-truth labels for training the network. In this section, we will introduce some commonly used point cloud classification datasets, and the division of each dataset is shown in Table. {{table:7fc8df5c-a85a-4713-9af4-e6479b252ab1}} .


the power consumption patterns can help to reveal how many people are in the home,

what are their sleeping and eating routines, etc.

Virtual Occupancy Sensing: Using Smart Meters to Indicate Your Presence

Abstract: Occupancy detection for buildings is crucial to improving energy efficiency, user comfort, and space utility. However, existing methods require dedicated system setup, continuous calibration, and frequent maintenance. With the instrumentation of electricity meters in millions of homes and offices, however, power measurement presents a unique opportunity for a non-intrusive and cost-effective way to detect occupant presence. This study develops solutions to the problems when no data or limited data is available for training, as motivated by difficulties in ground truth collection. Experimental evaluations on data from both residential and commercial buildings indicate that the proposed methods for binary occupancy detection are nearly as accurate as models learned with sufficient data, with accuracies of approximately 78 to 93 percent for residences and 90 percent for offices. This study shows that power usage contains valuable and sensitive user information, demonstrating a virtual occupancy sensing approach with minimal system calibration and setup.

Smart Meter Privacy: A Utility-Privacy Framework

Abstract: End-user privacy in smart meter measurements is a well-known challenge in the smart grid. The solutions offered thus far have been tied to specific technologies such as batteries or assumptions on data usage. Existing solutions have also not quantified the loss of benefit (utility) that results from any such privacy-preserving approach. Using tools from information theory, a new framework is presented that abstracts both the privacy and the utility requirements of smart meter data. This leads to a novel privacy-utility tradeoff problem with minimal assumptions that is tractable. Specifically for a stationary Gaussian Markov model of the electricity load, it is shown that the optimal utility-and-privacy preserving solution requires filtering out frequency components that are low in power, and this approach appears to encompass most of the proposed privacy approaches.

S$^2$-MLP: Spatial-Shift MLP Architecture for Vision

Abstract: Recently, visual Transformer (ViT) and its following works abandon the convolution and exploit the self-attention operation, attaining a comparable or even higher accuracy than CNNs. More recently, MLP-Mixer abandons both the convolution and the self-attention operation, proposing an architecture containing only MLP layers. To achieve cross-patch communications, it devises an additional token-mixing MLP besides the channel-mixing MLP. It achieves promising results when training on an extremely large-scale dataset. But it cannot achieve as outstanding performance as its CNN and ViT counterparts when training on medium-scale datasets such as ImageNet1K and ImageNet21K. The performance drop of MLP-Mixer motivates us to rethink the token-mixing MLP. We discover that the token-mixing MLP is a variant of the depthwise convolution with a global reception field and spatial-specific configuration. But the global reception field and the spatial-specific property make token-mixing MLP prone to over-fitting. In this paper, we propose a novel pure MLP architecture, spatial-shift MLP (S$^2$-MLP). Different from MLP-Mixer, our S$^2$-MLP only contains channel-mixing MLP. We utilize a spatial-shift operation for communications between patches. It has a local reception field and is spatial-agnostic. It is parameter-free and efficient for computation. The proposed S$^2$-MLP attains higher recognition accuracy than MLP-Mixer when training on ImageNet-1K dataset. Meanwhile, S$^2$-MLP accomplishes as excellent performance as ViT on ImageNet-1K dataset with considerably simpler architecture and fewer FLOPs and parameters.

Privacy and security aspects of smart meters

A summary of most cited papers on privacy and security aspects of smart meters is presented in Table {{table:96d44437-ab7c-4d2e-97bf-2c542f7a6c6a}} .
This research direction is currently the most active one among the mentioned directions, because the privacy and security concerns provide one of the biggest obstacles for the (potential) users of smart meters.
In many cases, data mining and data analytics techniques were applied on the meter reading data, to investigate the above issues questions.
A summary of most cited papers on design of smart meters for smart grid is presented in Table {{table:52733235-0ae3-4be5-9a6c-b221ecb5a8dd}} .


ACE College of Engineering, Trivandrum 695027, Kerala,

Derived features from Fundus and OCT images

Retinal texture and thickness (fundus images by OCT) Demographic data

Decision Tree, Neural network, Random Forest, SVM

Classification of Transgenic Mice by Retinal Imaging Using SVMS

Abstract: Alzheimer's disease is the neuro disorder which characterized by means of Amyloid– β (A  β) in brain. However, accurate detection of this disease is a challenging task since the pathological issues of brain are complex in identification. In this paper, the changes associated with the retinal imaging for Alzheimer's disease are classified into two classes such as wild-type (WT) and transgenic mice model (TMM). For testing, optical coherence tomography (OCT) images are used to classify into two groups. The classification is implemented by support vector machines with the optimum kernel selection using a genetic algorithm. Among several kernel functions of SVM, the radial basis kernel function provides the better classification result. In order to deal with an effective classification using SVM, texture features of retinal images are extracted and selected. The overall accuracy reached 92% and 91% of precision for the classification of transgenic mice.

Retinal texture biomarkers may help to discriminate between Alzheimer’s, Parkinson’s, and healthy controls

Abstract: A top priority in biomarker development for Alzheimer’s disease (AD) and Parkinson’s disease (PD) is the focus on early diagnosis, where the use of the retina is a promising avenue of research. We computed fundus images from optical coherence tomography (OCT) data and analysed the structural arrangement of the retinal tissue using texture metrics. We built clinical class classification models to distinguish between healthy controls (HC), AD, and PD, using machine learning (support vector machines). Median sensitivity is 88.7%, 79.5% and 77.8%, for HC, AD, and PD eyes, respectively. When the same subject has the same classification for both eyes, 94.4% (median) of the classifications are correct. A significant amount of information discriminating between multiple neurodegenerative states is conveyed by OCT imaging of the human retina, even when differences in thickness are not yet present. This technique may allow for simultaneously diagnosing Alzheimer’s and Parkinson’s diseases.

Development details of ML/DL models using OCT and OCTA data. GC-IPL: ganglion cell-inner flexiform layer, RNFL:Retinal Nerve Fibre Layer, IPL: Inner Plexiform Layer, INL: Inner Nuclear Layer, OPL: Outer Plexiform Layer, Sens: Sensitivity, Spec: Specificity, Acc: Accuracy, 3xTg-AD:triple transgenic AD, WD: wild-type

The initial search returned 3073 references. After several steps of elimination (Figure {{figure:5322ce78-927d-404a-adb2-287714fe5e06}} ), only ten were found eligible, including the two research papers already identified by {{cite:e6d2137}}. Table {{table:fd28c749-8fe3-4aef-966d-8bc5846d4552}}  lists all the publications that we have reviewed in detail.


Improving sentence compression by learning to predict gaze

Abstract: We show how eye-tracking corpora can be used to improve sentence compression models, presenting a novel multi-task learning algorithm based on multi-layer LSTMs. We obtain performance competitive with or better than state-of-the-art approaches.

Entity Recognition at First Sight: Improving NER with Eye Movement Information

Abstract: Previous research shows that eye-tracking data contains information about the lexical and syntactic properties of text, which can be used to improve natural language processing models. In this work, we leverage eye movement features from three corpora with recorded gaze information to augment a state-of-the-art neural model for named entity recognition (NER) with gaze embeddings. These corpora were manually annotated with named entity labels. Moreover, we show how gaze features, generalized on word type level, eliminate the need for recorded eye-tracking data at test time. The gaze-augmented models for NER using token-level and type-level features outperform the baselines. We present the benefits of eye-tracking features by evaluating the NER models on both individual datasets as well as in cross-domain settings.

Sample results for different NLP tasks using gaze behaviour learnt at test time. Dataset is the name of one of the datasets that was used. Best Baseline corresponds to the result of the best baseline system, gaze result corresponds to the result of the gaze behaviour system on that dataset, %age increase is the percentage improvement for that NLP task, and {{formula:32199da5-08aa-4e13-9fa7-2c1eea027510}}  says whether or not the improvements were statistically significant. Due to space constraints we are only showing the results for 1 dataset per task. N/R means that the authors haven't reported the results of any statistical significance test.

Table {{table:d14f4e49-e0e6-4b4d-86b2-b175467556ad}}  gives the results of each of the different NLP tasks where we don't record gaze behaviour at run time. Due to space constraints, we report the result of only 1 result and dataset for each NLP task. We report the best baseline result, the corresponding gaze behaviour system result, the improvement in percentage points, and whether or not the improvement in performance was statistically significant.


Adversarially Robust Distillation

Abstract: Knowledge distillation is effective for producing small, high-performance neural networks for classification, but these small networks are vulnerable to adversarial attacks. This paper studies how adversarial robustness transfers from teacher to student during knowledge distillation. We find that a large amount of robustness may be inherited by the student even when distilled on only clean images. Second, we introduce Adversarially Robust Distillation (ARD) for distilling robustness onto student networks. In addition to producing small models with high test accuracy like conventional distillation, ARD also passes the superior robustness of large networks onto the student. In our experiments, we find that ARD student models decisively outperform adversarially trained networks of identical architecture in terms of robust accuracy, surpassing state-of-the-art methods on standard robustness benchmarks. Finally, we adapt recent fast adversarial training methods to ARD for accelerated robust distillation.

Fast is better than free: Revisiting adversarial training

Abstract: Adversarial training, a method for learning robust deep networks, is typically assumed to be more expensive than traditional training due to the necessity of constructing adversarial examples via a first-order method like projected gradient decent (PGD). In this paper, we make the surprising discovery that it is possible to train empirically robust models using a much weaker and cheaper adversary, an approach that was previously believed to be ineffective, rendering the method no more costly than standard training in practice. Specifically, we show that adversarial training with the fast gradient sign method (FGSM), when combined with random initialization, is as effective as PGD-based training but has significantly lower cost. Furthermore we show that FGSM adversarial training can be further accelerated by using standard techniques for efficient training of deep networks, allowing us to learn a robust CIFAR10 classifier with 45% robust accuracy to PGD attacks with $\epsilon=8/255$ in 6 minutes, and a robust ImageNet classifier with 43% robust accuracy at $\epsilon=2/255$ in 12 hours, in comparison to past work based on "free" adversarial training which took 10 and 50 hours to reach the same respective thresholds. Finally, we identify a failure mode referred to as "catastrophic overfitting" which may have caused previous attempts to use FGSM adversarial training to fail. All code for reproducing the experiments in this paper as well as pretrained model weights are at this https URL.

Adversarial Training for Free!

Abstract: Adversarial training, in which a network is trained on adversarial examples, is one of the few defenses against adversarial attacks that withstands strong attacks. Unfortunately, the high cost of generating strong adversarial examples makes standard adversarial training impractical on large-scale problems like ImageNet. We present an algorithm that eliminates the overhead cost of generating adversarial examples by recycling the gradient information computed when updating model parameters. Our "free" adversarial training algorithm achieves comparable robustness to PGD adversarial training on the CIFAR-10 and CIFAR-100 datasets at negligible additional cost compared to natural training, and can be 7 to 30 times faster than other strong adversarial training methods. Using a single workstation with 4 P100 GPUs and 2 days of runtime, we can train a robust model for the large-scale ImageNet classification task that maintains 40% accuracy against PGD attacks. The code is available at this https URL.

Hyperparameters used to train models using prior works on CIFAR-10.

Here we provide the hyperparameters used to train the models in our experiments. Table {{table:d6bca773-c52a-42e9-aea4-358944e051a6}}  provides the hyperparameters used to adversarially train each model. Table {{table:57b63300-3f87-4b36-96ec-d6ceba6d2bd3}}  provides the hyperparameters used when training a student model using RRM. Table {{table:ba14d2f8-49da-41d7-80b8-811ef9b46756}}  provides the hyperparameters used when training a student model using prior works: Robust Data Training (RDT) {{cite:3bc3293}}, Knowledge Distillation (KD) {{cite:a771e6f}}, Fast Adversarial Training (Fast AT) {{cite:98d8cbf}}, and Free Adversarial Training (Free AT) {{cite:89d7bab}}.


Breaching the Human Firewall: Social engineering in Phishing and Spear-Phishing Emails

Abstract: We examined the influence of three social engineering strategies on users' judgments of how safe it is to click on a link in an email. The three strategies examined were authority, scarcity and social proof, and the emails were either genuine, phishing or spear-phishing. Of the three strategies, the use of authority was the most effective strategy in convincing users that a link in an email was safe. When detecting phishing and spear-phishing emails, users performed the worst when the emails used the authority principle and performed best when social proof was present. Overall, users struggled to distinguish between genuine and spear-phishing emails. Finally, users who were less impulsive in making decisions generally were less likely to judge a link as safe in the fraudulent emails. Implications for education and training are discussed.

Social Engineering Attacks: A Survey

Abstract: The advancements in digital communication technology have made communication between humans more accessible and instant. However, personal and sensitive information may be available online through social networks and online services that lack the security measures to protect this information. Communication systems are vulnerable and can easily be penetrated by malicious users through social engineering attacks. These attacks aim at tricking individuals or enterprises into accomplishing actions that benefit attackers or providing them with sensitive data such as social security number, health records, and passwords. Social engineering is one of the biggest challenges facing network security because it exploits the natural human tendency to trust. This paper provides an in-depth survey about the social engineering attacks, their classifications, detection strategies, and prevention procedures.

Detecting and Characterizing Lateral Phishing at Scale

Abstract: Author(s): Ho, G; Cidon, A; Gavish, L; Schweighauser, M; Paxson, V; Savage, S; Voelker, GM; Wagner, D | Abstract: © 2019 by The USENIX Association. All rights reserved. We present the first large-scale characterization of lateral phishing attacks, based on a dataset of 113 million employee-sent emails from 92 enterprise organizations. In a lateral phishing attack, adversaries leverage a compromised enterprise account to send phishing emails to other users, benefit-ting from both the implicit trust and the information in the hijacked user's account. We develop a classifier that finds hundreds of real-world lateral phishing emails, while generating under four false positives per every one-million employee-sent emails. Drawing on the attacks we detect, as well as a corpus of user-reported incidents, we quantify the scale of lateral phishing, identify several thematic content and recipient targeting strategies that attackers follow, illuminate two types of sophisticated behaviors that attackers exhibit, and estimate the success rate of these attacks. Collectively, these results expand our mental models of the 'enterprise attacker' and shed light on the current state of enterprise phishing attacks.

Business Email Compromise (BEC) and Cyberpsychology

Abstract: The paper gives a brief introduction about what BEC (Business Email Compromise) is and why we should be concerned about. In addition, it presents 2 examples, Ubiquity and Peebles Media Group, which have been chosen to analyse the phenomena of BEC and underpin how universal BEC threat is for all companies. The psychology behind this scam has been, then, studied. In particular, the Big Five Framework has been analysed to understand how personality traits play an important role in Social Engineering-based attacks. Furthermore, the 6 basic principles of influence, by Cialdini, have been presented to show which strategies are adopted in such scam. The paper follows with the analysis of the BEC impacts, the incidents evaluation and, finally, with the description of some precautions, that companies should undertake in order to mitigate the likelihood of a Business Email Compromise.

Text Data Augmentation: Towards better detection of spear-phishing emails

Abstract: Text data augmentation, i.e. the creation of synthetic textual data from an original text, is challenging as augmentation transformations should take into account language complexity while being relevant to the target Natural Language Processing (NLP) task (e.g. Machine Translation, Question Answering, Text Classification, etc.). Motivated by a business application of Business Email Compromise (BEC) detection, we propose a corpus and task agnostic text augmentation framework combining different methods, utilizing BERT language model, multi-step back-translation and heuristics. We show that our augmentation framework improves performances on several text classification tasks using publicly available models and corpora (SST2 and TREC) as well as on a BEC detection task. We also provide a comprehensive argumentation about the limitations of our augmentation framework.

Manifolds of absolutely continuous curves and the square root velocity framework

Abstract: A classical result in Riemannian geometry states that the absolutely continuous curves into a (finite-dimensional) Riemannian manifold form an infinite-dimensional manifold. In the present paper this construction and related results are generalised to absolutely continuous curves with values in a strong Riemannian manifolds. As an application we consider extensions of the square root velocity transform (SRVT) framework for shape analysis. Computations in this framework frequently lead to curves which leave the shape space (of smooth curves), and are only contained in a completion. In the vector valued case, this extends the SRVT to a space of absolutely continuous curves. We investigate the situation for shape spaces of manifold valued (absolutely continuous) curves.

Summary of the attack characteristics found in the literature. An asterisk identifies an impersonated indivdual external to the organization. More details about the goals are presented in Table .

Definitions and Contradictions across the Literature

Table {{table:44026321-3ab0-4f5e-a1c9-369b73e8a726}}  presents the terminology used in the literature, as well as the characteristics associated to the definition of each term. Publications are sorted by date, from the oldest to the most recent. Column `Term' contains the term used in a publication (BEC, SP, TA for Targeted Attack, and BEC P for Business Email Compromise Phishing). Column `Attack Vectors' describes the attack vectors used to carry out the attack, whether it is a URL (leading to a fraudulent website), an attachment (containing a malware), or the textual content of the email (to trick the victim into performing an action through social engineering techniques). Column `Attack goals' lists the different attack goals considered in each publication (refer to Table {{table:b8665e7e-3471-466a-8890-529977e58ac4}}  presented in Section  for a description of these goals). Column `Targets' identifies the typical targets of the attack. Finally, Column `Who is impersonated?' gives more details about the different impersonation schemes. It is important to note that the definition of the term used (SP, BEC) is rather vague in most publications, and that some identified characteristics were inferred from the context of the paper.

Regarding the attack vectors, Table {{table:44026321-3ab0-4f5e-a1c9-369b73e8a726}}  shows that there is a growing number of text-based attacks over time. Indeed, we can see from the table that, while URLs and attachments are the main attack vectors of spear phishing, BEC highly relies on a carefully crafted textual content, leveraging social engineering techniques. {{cite:aefe5cd}} explains this evolution of the attack vector over time: What all these email attacks have in common is that they are almost exclusively text-based in an effort to bypass traditional email security tools that look for malicious attachments and links.. There are however inconsistencies across the literature, both for SP and BEC. {{cite:617af08}} mentions that 80% of SP attacks are URL-based, while {{cite:1030777}} reports that 94% of SP attacks are attachment-based. Regarding BEC, some publications consider only text-based attacks {{cite:419d727}}, {{cite:8675a11}}, {{cite:36c418b}}, while others include URL-based and attachment-based attacks {{cite:b8c2ff0}}, {{cite:59cfb7c}}, {{cite:658b4f5}}, {{cite:0432f37}}, {{cite:ca63e75}}. Numbers also vary in the literature. While {{cite:658b4f5}} report that 40.1% of BEC attacks include a malicious link, {{cite:ca63e75}} claim that a link or an attachment is used in only 3% of the cases.

As for the purpose of the attack, the literature once again presents variations, as illustrated in Table {{table:44026321-3ab0-4f5e-a1c9-369b73e8a726}} . Publications on SP focus on the infiltration of organizations (GC3) through a malicious link or attachment, sometimes combined with the theft of confidential documents and critical information (GC2). A notable exception is {{cite:617af08}}, where the goal of SP is only financial (GC1). Regarding BEC, the goal is primarily financial (GC1), via fraudulent wire transfers. In this respect, the term BEC can be misleading because it first evokes the compromise of a business (GC3), as defined in {{cite:57bfe24}}, {{cite:e7c2752}}. However, {{cite:658b4f5}} report that the goal of 46.9% of BEC attacks is a wire transfer (GC1), 12.2% to establish a rapport with the target (GC4), and 0.8% to steal PII (GC2).

A targeted attack typically relies on the impersonation of a trusted party of the victim (see column 'Who is impersonated?' in Table {{table:44026321-3ab0-4f5e-a1c9-369b73e8a726}} ). It may be a family member, a relative, or a friend of the victim. In the context of a corporate attack, it might be a C-suite executive (CEO, CFO), a contractor, a supplier, a customer, or a partner {{cite:e7c2752}}, {{cite:59cfb7c}}. The impersonation can be carried out by compromising directly the means of communication of the impersonated individual. For example, compromising the email account is ideal for the attacker, as existing email conversation threads can be hijacked to carry out targeted attacks – attacks that are very challenging to detect as the email address is authentic. However, in practice, it is very difficult for the attacker to compromise an email account. For this reason, impersonation is usually performed by sending a message containing visual cues to trick the victim, e.g., first name and last name of the impersonated individual in the sender field of the message {{cite:658b4f5}}, {{cite:0432f37}}. In some cases, the attackers imitate the writing style of the impersonated individual to make the attack even more credible {{cite:b08d688}}. In contrast, non-targeted attacks (phishing, spam) appear most of the time out of context to the victim, as there is no relation between the sender and the recipient.

The attack vector (or malicious payload) of the targeted attack can take different forms. Some targeted attacks rely on textual content only, urging the victim to perform an action (wire money, disclose confidential information) by leveraging social engineering techniques. This is more and more the case, as demonstrated by Table {{table:44026321-3ab0-4f5e-a1c9-369b73e8a726}} . We define these attacks as text-based targeted attacks. The attack vector can also take the form of a URL (to lead the victim to a fraudulent website) or the form of an attachment (e.g., malware, ransomware).


False positive was reduced compared to their previous study

A small number of patients were used in the study.

Automated detection of pulmonary nodules in PET/CT images: Ensemble false-positive reduction using a convolutional neural network technique.

Abstract:
PURPOSE
Automated detection of solitary pulmonary nodules using positron emission tomography (PET) and computed tomography (CT) images shows good sensitivity; however, it is difficult to detect nodules in contact with normal organs, and additional efforts are needed so that the number of false positives (FPs) can be further reduced. In this paper, the authors propose an improved FP-reduction method for the detection of pulmonary nodules in PET/CT images by means of convolutional neural networks (CNNs).

METHODS
The overall scheme detects pulmonary nodules using both CT and PET images. In the CT images, a massive region is first detected using an active contour filter, which is a type of contrast enhancement filter that has a deformable kernel shape. Subsequently, high-uptake regions detected by the PET images are merged with the regions detected by the CT images. FP candidates are eliminated using an ensemble method; it consists of two feature extractions, one by shape/metabolic feature analysis and the other by a CNN, followed by a two-step classifier, one step being rule based and the other being based on support vector machines.

RESULTS
The authors evaluated the detection performance using 104 PET/CT images collected by a cancer-screening program. The sensitivity in detecting candidates at an initial stage was 97.2%, with 72.8 FPs/case. After performing the proposed FP-reduction method, the sensitivity of detection was 90.1%, with 4.9 FPs/case; the proposed method eliminated approximately half the FPs existing in the previous study.

CONCLUSIONS
An improved FP-reduction scheme using CNN technique has been developed for the detection of pulmonary nodules in PET/CT images. The authors' ensemble FP-reduction method eliminated 93% of the FPs; their proposed method using CNN technique eliminates approximately half the FPs existing in the previous study. These results indicate that their method may be useful in the computer-aided detection of pulmonary nodules using PET/CT images.

Using Deep Learning for Classification of Lung Nodules on Computed Tomography Images

Abstract: Lung cancer is the most common cancer that cannot be ignored and cause death with late health care. Currently, CT can be used to help doctors detect the lung cancer in the early stages. In many cases, the diagnosis of identifying the lung cancer depends on the experience of doctors, which may ignore some patients and cause some problems. Deep learning has been proved as a popular and powerful method in many medical imaging diagnosis areas. In this paper, three types of deep neural networks (e.g., CNN, DNN, and SAE) are designed for lung cancer calcification. Those networks are applied to the CT image classification task with some modification for the benign and malignant lung nodules. Those networks were evaluated on the LIDC-IDRI database. The experimental results show that the CNN network archived the best performance with an accuracy of 84.15%, sensitivity of 83.96%, and specificity of 84.32%, which has the best result among the three networks.

Deep Learning Approach for Early Stage Lung Cancer Detection

Abstract: Lung cancer is the leading cause of death among different types of cancers. Every year, the lives lost due to lung cancer exceed those lost to pancreatic, breast, and prostate cancer combined. The survival rate for lung cancer patients is very low compared to other cancer patients due to late diagnostics. Thus, early lung cancer diagnostics is crucial for patients to receive early treatments, increasing the survival rate or even becoming cancer-free. This paper proposed a deep-learning model for early lung cancer prediction and diagnosis from Computed Tomography (CT) scans. The proposed mode achieves high accuracy. In addition, it can be a beneficial tool to support radiologists' decisions in predicting and detecting lung cancer and its stage.

A comparison between the state-of-the-art CNN based models for detection lung cancer.

In 2018, {{cite:633ff05}} introduced a 3D CNN model. The study aims to employ a CNN model in predicting a two years survival of NSCLC patients. Their model is gauged in contrast to random forest models. However, the authors pointed out some limitations that the study has, such as the prognostics knowledge refined into the CNN model is based on past treatment options and plans and may not be suitable to predict prognostics for patients treated with the latest technology. {{cite:28985e3}} developed a 3D CNN model to detect cancer regions in CT scans to predict cancer risk. Yunlang She et al. {{cite:2b9b954}} developed a deep learning model (DeepSurv) to predict survival among lung cancer patients. DeepSurv could be used as a useful tool for patients’ treatment recommendations. There are a few advantages for DeepSurv identified in the study: (I) DeepSurv is adaptive to variables such as real-world clinical factors. (II) Flexibility when dealing with complicated elements. (III) Learn and analyze censored features. (IV) Perform better in big data analysis. The study also indicated some limitations: (I) The excessive cost associated with deep learning models training and validating. (II) Interpreting the model’s prediction could be hard because the functionality of deep learning networks is like black boxes. (III) The study lacks external validation. In a study conducted by {{cite:3c78acb}}, three types of deep neural networks were proposed. Convolutional Neural Network (CNN), Deep Neural Network (DNN), and Stacked Autoencoder (SAE). The limitation of this study is pointed out by the authors in that the layers of the models are relatively small. In 2020, {{cite:83db10b}} aims to evaluate the performance of a deep learning model to detect lung cancer. In their study, ResNet34, a 34-layer CNN model is used to classify images. Table {{table:9ba59d45-8164-4d2e-bf63-0f6743f262ee}}  gives a comparison between the state-of-the-art CNN-based models for detecting lung cancer.


Ensemble of multiple models which include 3D CNN-based architectures

Three-pathway multi-scale 2D CNN architecture

Multiple segmentation maps created at different scales

3D U-Net that has been modified with respect to up-sampling pathways, normalization methods, number of filters, and batch size.

Modified nnU-Net {{cite:36aa8b1}} with heavy data augmentations. The winning contributions of the BRATS challenge in 2020.

U-Net-like architecture design with a Swin transformer as the encoder, called Swin UNETR.

Multi-connected adversarial network proposed for unsupervised domain adaptation

GAN-based framewok consisting of a FCN and a adversarial critic network with a multi-scale L1 Loss

CycleGAN-based cross-modality feature learning approach

Ensembles of Multiple Models and Architectures for Robust Brain Tumour Segmentation

Abstract: Deep learning approaches such as convolutional neural nets have consistently outperformed previous methods on challenging tasks such as dense, semantic segmentation. However, the various proposed networks perform differently, with behaviour largely influenced by architectural choices and training settings. This paper explores Ensembles of Multiple Models and Architectures (EMMA) for robust performance through aggregation of predictions from a wide range of methods. The approach reduces the influence of the meta-parameters of individual models and the risk of overfitting the configuration to a particular database. EMMA can be seen as an unbiased, generic deep learning model which is shown to yield excellent performance, winning the first position in the BRATS 2017 competition among 50+ participating teams.

Multiscale CNNs for Brain Tumor Segmentation and Diagnosis

Abstract: Early brain tumor detection and diagnosis are critical to clinics. Thus segmentation of focused tumor area needs to be accurate, efficient, and robust. In this paper, we propose an automatic brain tumor segmentation method based on Convolutional Neural Networks (CNNs). Traditional CNNs focus only on local features and ignore global region features, which are both important for pixel classification and recognition. Besides, brain tumor can appear in any place of the brain and be any size and shape in patients. We design a three-stream framework named as multiscale CNNs which could automatically detect the optimum top-three scales of the image sizes and combine information from different scales of the regions around that pixel. Datasets provided by Multimodal Brain Tumor Image Segmentation Benchmark (BRATS) organized by MICCAI 2013 are utilized for both training and testing. The designed multiscale CNNs framework also combines multimodal features from T1, T1-enhanced, T2, and FLAIR MRI images. By comparison with traditional CNNs and the best two methods in BRATS 2012 and 2013, our framework shows advances in brain tumor segmentation accuracy and robustness.

CNN-based Segmentation of Medical Imaging Data

Abstract: Convolutional neural networks have been applied to a wide variety of computer vision tasks. Recent advances in semantic segmentation have enabled their application to medical image segmentation. While most CNNs use two-dimensional kernels, recent CNN-based publications on medical image segmentation featured three-dimensional kernels, allowing full access to the three-dimensional structure of medical images. Though closely related to semantic segmentation, medical image segmentation includes specific challenges that need to be addressed, such as the scarcity of labelled data, the high class imbalance found in the ground truth and the high memory demand of three-dimensional images. In this work, a CNN-based method with three-dimensional filters is demonstrated and applied to hand and brain MRI. Two modifications to an existing CNN architecture are discussed, along with methods on addressing the aforementioned challenges. While most of the existing literature on medical image segmentation focuses on soft tissue and the major organs, this work is validated on data both from the central nervous system as well as the bones of the hand.

Brain Tumor Segmentation and Radiomics Survival Prediction: Contribution to the BRATS 2017 Challenge

Abstract: Quantitative analysis of brain tumors is critical for clinical decision making. While manual segmentation is tedious, time consuming and subjective, this task is at the same time very challenging to solve for automatic segmentation methods. In this paper we present our most recent effort on developing a robust segmentation algorithm in the form of a convolutional neural network. Our network architecture was inspired by the popular U-Net and has been carefully modified to maximize brain tumor segmentation performance. We use a dice loss function to cope with class imbalances and use extensive data augmentation to successfully prevent overfitting. Our method beats the current state of the art on BraTS 2015, is one of the leading methods on the BraTS 2017 validation set (dice scores of 0.896, 0.797 and 0.732 for whole tumor, tumor core and enhancing tumor, respectively) and achieves very good Dice scores on the test set (0.858 for whole, 0.775 for core and 0.647 for enhancing tumor). We furthermore take part in the survival prediction subchallenge by training an ensemble of a random forest regressor and multilayer perceptrons on shape features describing the tumor subregions. Our approach achieves 52.6% accuracy, a Spearman correlation coefficient of 0.496 and a mean square error of 209607 on the test set.

nnU-Net for Brain Tumor Segmentation

Abstract: We apply nnU-Net to the segmentation task of the BraTS 2020 challenge. The unmodified nnU-Net baseline configuration already achieves a respectable result. By incorporating BraTS-specific modifications regarding postprocessing, region-based training, a more aggressive data augmentation as well as several minor modifications to the nnUNet pipeline we are able to improve its segmentation performance substantially. We furthermore re-implement the BraTS ranking scheme to determine which of our nnU-Net variants best fits the requirements imposed by it. Our final ensemble took the first place in the BraTS 2020 competition with Dice scores of 88.95, 85.06 and 82.03 and HD95 values of 8.498,17.337 and 17.805 for whole tumor, tumor core and enhancing tumor, respectively.

Swin UNETR: Swin Transformers for Semantic Segmentation of Brain Tumors in MRI Images

Abstract: Semantic segmentation of brain tumors is a fundamental medical image analysis task involving multiple MRI imaging modalities that can assist clinicians in diagnosing the patient and successively studying the progression of the malignant entity. In recent years, Fully Convolutional Neural Networks (FCNNs) approaches have become the de facto standard for 3D medical image segmentation. The popular "U-shaped" network architecture has achieved state-of-the-art performance benchmarks on different 2D and 3D semantic segmentation tasks and across various imaging modalities. However, due to the limited kernel size of convolution layers in FCNNs, their performance of modeling long-range information is sub-optimal, and this can lead to deficiencies in the segmentation of tumors with variable sizes. On the other hand, transformer models have demonstrated excellent capabilities in capturing such long-range information in multiple domains, including natural language processing and computer vision. Inspired by the success of vision transformers and their variants, we propose a novel segmentation model termed Swin UNEt TRansformers (Swin UNETR). Specifically, the task of 3D brain tumor semantic segmentation is reformulated as a sequence to sequence prediction problem wherein multi-modal input data is projected into a 1D sequence of embedding and used as an input to a hierarchical Swin transformer as the encoder. The swin transformer encoder extracts features at five different resolutions by utilizing shifted windows for computing self-attention and is connected to an FCNN-based decoder at each resolution via skip connections. We have participated in BraTS 2021 segmentation challenge, and our proposed model ranks among the top-performing approaches in the validation phase. Code: https://monai.io/research/swin-unetr

Unsupervised domain adaptation in brain lesion segmentation with adversarial networks

Abstract: Significant advances have been made towards building accurate automatic segmentation systems for a variety of biomedical applications using machine learning. However, the performance of these systems often degrades when they are applied on new data that differ from the training data, for example, due to variations in imaging protocols. Manually annotating new data for each test domain is not a feasible solution. In this work we investigate unsupervised domain adaptation using adversarial neural networks to train a segmentation method which is more invariant to differences in the input data, and which does not require any annotations on the test domain. Specifically, we learn domain-invariant features by learning to counter an adversarial network, which attempts to classify the domain of the input data by observing the activations of the segmentation network. Furthermore, we propose a multi-connected domain discriminator for improved adversarial training. Our system is evaluated using two MR databases of subjects with traumatic brain injuries, acquired using different scanners and imaging protocols. Using our unsupervised approach, we obtain segmentation accuracies which are close to the upper bound of supervised domain adaptation.

SegAN: Adversarial Network with Multi-scale $L_1$ Loss for Medical Image Segmentation

Abstract: Inspired by classic generative adversarial networks (GAN), we propose a novel end-to-end adversarial neural network, called SegAN, for the task of medical image segmentation. Since image segmentation requires dense, pixel-level labeling, the single scalar real/fake output of a classic GAN's discriminator may be ineffective in producing stable and sufficient gradient feedback to the networks. Instead, we use a fully convolutional neural network as the segmentor to generate segmentation label maps, and propose a novel adversarial critic network with a multi-scale $L_1$ loss function to force the critic and segmentor to learn both global and local features that capture long- and short-range spatial relationships between pixels. In our SegAN framework, the segmentor and critic networks are trained in an alternating fashion in a min-max game: The critic takes as input a pair of images, (original_image $*$ predicted_label_map, original_image $*$ ground_truth_label_map), and then is trained by maximizing a multi-scale loss function; The segmentor is trained with only gradients passed along by the critic, with the aim to minimize the multi-scale loss function. We show that such a SegAN framework is more effective and stable for the segmentation task, and it leads to better performance than the state-of-the-art U-net segmentation method. We tested our SegAN method using datasets from the MICCAI BRATS brain tumor segmentation challenge. Extensive experimental results demonstrate the effectiveness of the proposed SegAN with multi-scale loss: on BRATS 2013 SegAN gives performance comparable to the state-of-the-art for whole tumor and tumor core segmentation while achieves better precision and sensitivity for Gd-enhance tumor core segmentation; on BRATS 2015 SegAN achieves better performance than the state-of-the-art in both dice score and precision.

Cross-Modality Deep Feature Learning for Brain Tumor Segmentation

Abstract: Recent advances in machine learning and prevalence of digital medical images have opened up an opportunity to address the challenging brain tumor segmentation (BTS) task by using deep convolutional neural networks. However, different from the RGB image data that are very widespread, the medical image data used in brain tumor segmentation are relatively scarce in terms of the data scale but contain the richer information in terms of the modality property. To this end, this paper proposes a novel cross-modality deep feature learning framework to segment brain tumors from the multi-modality MRI data. The core idea is to mine rich patterns across the multi-modality data to make up for the insufficient data scale. The proposed cross-modality deep feature learning framework consists of two learning processes: the cross-modality feature transition (CMFT) process and the cross-modality feature fusion (CMFF) process, which aims at learning rich feature representations by transiting knowledge across different modality data and fusing knowledge from different modality data, respectively. Comprehensive experiments are conducted on the BraTS benchmarks, which show that the proposed cross-modality deep feature learning framework can effectively improve the brain tumor segmentation performance when compared with the baseline methods and state-of-the-art methods.

Brain tumor segmentation studies based on deep learning

In this section, we covered different variants of deep learning architectures ranging from CNNs to GANs and their applications in the segmentation of brain tumors, which are summarised in Table {{table:9704f7f0-ec13-4388-9d66-9e7fd291fce1}} . Medical image segmentation has seen a massive influx of deep learning methods. Nevertheless, these methods are associated with certain limitations, such as the demand for a large amount of data, the lack of model interpretability and the generalizability {{cite:2e9cb25}}. Various strategies have been proposed in past literature to optimize the task of segmentation which has led to significant improvements in the development of fully automated diagnosis systems.