Eeg-Based Video Identification Using Graph Signal Modeling and Graph Convolutional Neural Network

Abstract: This paper proposes a novel graph signal-based deep learning method for electroencephalography (EEG) and its application to EEG-based video identification. We present new methods to effectively represent EEG data as signals on graphs, and learn them using graph convolutional neural networks. Experimental results for video identification using EEG responses obtained while watching videos show the effectiveness of the proposed approach in comparison to existing methods. Effective schemes for graph signal representation of EEG are also discussed.

EEG-GCNN: Augmenting Electroencephalogram-based Neurological Disease Diagnosis using a Domain-guided Graph Convolutional Neural Network

Abstract: This paper presents a novel graph convolutional neural network (GCNN)-based approach for improving the diagnosis of neurological diseases using scalp-electroencephalograms (EEGs). Although EEG is one of the main tests used for neurological-disease diagnosis, the sensitivity of EEG-based expert visual diagnosis remains at $\sim$50\%. This indicates a clear need for advanced methodology to reduce the false negative rate in detecting abnormal scalp-EEGs. In that context, we focus on the problem of distinguishing the abnormal scalp EEGs of patients with neurological diseases, which were originally classified as 'normal' by experts, from the scalp EEGs of healthy individuals. The contributions of this paper are three-fold: 1) we present EEG-GCNN, a novel GCNN model for EEG data that captures both the spatial and functional connectivity between the scalp electrodes, 2) using EEG-GCNN, we perform the first large-scale evaluation of the aforementioned hypothesis, and 3) using two large scalp-EEG databases, we demonstrate that EEG-GCNN significantly outperforms the human baseline and classical machine learning (ML) baselines, with an AUC of 0.90.

GCNs-Net: A Graph Convolutional Neural Network Approach for Decoding Time-resolved EEG Motor Imagery Signals

Abstract: Toward the development of effective and efficient brain-computer interface (BCI) systems, precise decoding of brain activity measured by an electroencephalogram (EEG) is highly demanded. Traditional works classify EEG signals without considering the topological relationship among electrodes. However, neuroscience research has increasingly emphasized network patterns of brain dynamics. Thus, the Euclidean structure of electrodes might not adequately reflect the interaction between signals. To fill the gap, a novel deep learning (DL) framework based on the graph convolutional neural networks (GCNs) is presented to enhance the decoding performance of raw EEG signals during different types of motor imagery (MI) tasks while cooperating with the functional topological relationship of electrodes. Based on the absolute Pearson's matrix of overall signals, the graph Laplacian of EEG electrodes is built up. The GCNs-Net constructed by graph convolutional layers learns the generalized features. The followed pooling layers reduce dimensionality, and the fully-connected (FC) softmax layer derives the final prediction. The introduced approach has been shown to converge for both personalized and groupwise predictions. It has achieved the highest averaged accuracy, 93.06% and 88.57% (PhysioNet dataset), 96.24% and 80.89% (high gamma dataset), at the subject and group level, respectively, compared with existing studies, which suggests adaptability and robustness to individual variability. Moreover, the performance is stably reproducible among repetitive experiments for cross-validation. The excellent performance of our method has shown that it is an important step toward better BCI approaches. To conclude, the GCNs-Net filters EEG signals based on the functional topological relationship, which manages to decode relevant features for brain MI. A DL library for EEG task classification including the code for this study is open source at https://github.com/SuperBruceJia/ EEG-DL for scientific research.

Summary of Graph Neural Networks to Model Human Behavior Dynamics

GNNs have demonstrated superior performance while overcoming the limitations of conventional deep learning methods when modeling the heterogeneous sensor data. The core motivation of using GNNs for modeling human behavior dynamics is that graph structured information can capture complex interactions among human behaviors by generating explicit topological representations to enhance the expressive power of the sensory data to further improve the prediction performance. Various works have demonstrated improved performance by applying GNNs in modeling human behaviors using the sensory data compared to other non-GNN approaches, particularly in domains such as social interaction detection, mobility prediction, cognition and physical activity recognition {{cite:5b41aec}} {{cite:cc9bf30}} {{cite:002e84a}} {{cite:9d86fc3}} {{cite:949aad8}} {{cite:edbef19}} {{cite:9f8fd23}} {{cite:bebca9e}} {{cite:edd1e6f}} {{cite:2969892}} {{cite:21eb1c3}} {{cite:002e84a}} {{cite:0459517}} {{cite:ef57a6d}} {{cite:30fbc17}} {{cite:8e13066}} {{cite:02698e1}} {{cite:68db55c}} {{cite:7452427}} {{cite:8fe80e4}} {{cite:9f8fd23}} {{cite:87346e2}} {{cite:565cdd4}} {{cite:0476ef6}} {{cite:ea65b2c}} {{cite:4bc1000}} {{cite:21eb1c3}} {{cite:bde016f}} {{cite:ca2e5d0}} {{cite:8fe80e4}} {{cite:7432dc8}}. We generalize the process of using GNNs to model HSD as a framework and show it in Fig. {{figure:c9092db1-25ba-4694-998d-0e0825aaecba}} . GNN modeling for HSD consists of three steps: 1) human centric sensing; 2) graph representation of human behaviors (sensors); 3) behavior inference. Table {{table:f018e425-f859-4820-9921-f0a8bba873b4}}  summarizes the differences between existing works, including the sensors used, the input data type, the GNN models, the GNN kernels and learning targets.

Human state transitions and sensor interactions can be comprehensively represented based on HSD graph modeling. Edge connections in the graph representation of HSD enable human state information to transmit through the edges. GNNs can produce comprehensive representations of HSD from the graph structure and relevant human state attributes: graph convolutions can aggregate multi-hop human state information to generate high-level embeddings that can capture complex interactions of human state transitions and connected sensors. As shown in Table {{table:f018e425-f859-4820-9921-f0a8bba873b4}} ,
Two types of GNNs have been invented in the surveyed studies. The first type of GNNs for HSD can be characterized as static GNNs, in which the authors consider the spatial features of the topology in human states and sensor networks, as demonstrated in these works {{cite:5b41aec}}, {{cite:cc9bf30}}, {{cite:565cdd4}}, {{cite:0476ef6}}, {{cite:7452427}}, {{cite:87346e2}}, {{cite:2969892}}, {{cite:9f8fd23}}, {{cite:be5ffe9}}. The second type of GNNs for HSD can be characterized as dynamic GNNs, in which the authors consider both spatial and temporal variations of the topology in human states and sensor networks, as demonstrated in these works {{cite:b093322}}, {{cite:7432dc8}}, {{cite:236bdbd}}, {{cite:5580a79}}, {{cite:8fe80e4}}.


Estimating Potential Infection Transmission Routes in Hospital Wards Using Wearable Proximity Sensors

Abstract: Background Contacts between patients, patients and health care workers (HCWs) and among HCWs represent one of the important routes of transmission of hospital-acquired infections (HAI). A detailed description and quantification of contacts in hospitals provides key information for HAIs epidemiology and for the design and validation of control measures. Methods and Findings We used wearable sensors to detect close-range interactions (“contacts”) between individuals in the geriatric unit of a university hospital. Contact events were measured with a spatial resolution of about 1.5 meters and a temporal resolution of 20 seconds. The study included 46 HCWs and 29 patients and lasted for 4 days and 4 nights. 14,037 contacts were recorded overall, 94.1% of which during daytime. The number and duration of contacts varied between mornings, afternoons and nights, and contact matrices describing the mixing patterns between HCW and patients were built for each time period. Contact patterns were qualitatively similar from one day to the next. 38% of the contacts occurred between pairs of HCWs and 6 HCWs accounted for 42% of all the contacts including at least one patient, suggesting a population of individuals who could potentially act as super-spreaders. Conclusions Wearable sensors represent a novel tool for the measurement of contact patterns in hospitals. The collected data can provide information on important aspects that impact the spreading patterns of infectious diseases, such as the strong heterogeneity of contact numbers and durations across individuals, the variability in the number of contacts during a day, and the fraction of repeated contacts across days. This variability is however associated with a marked statistical stability of contact and mixing patterns across days. Our results highlight the need for such measurement efforts in order to correctly inform mathematical models of HAIs and use them to inform the design and evaluation of prevention strategies.

Data on face-to-face contacts in an office building suggest a low-cost vaccination strategy based on community linkers

Abstract: Abstract Empirical data on contacts between individuals in social contexts play an important role in providing information for models describing human behavior and how epidemics spread in populations. Here, we analyze data on face-to-face contacts collected in an office building. The statistical properties of contacts are similar to other social situations, but important differences are observed in the contact network structure. In particular, the contact network is strongly shaped by the organization of the offices in departments, which has consequences in the design of accurate agent-based models of epidemic spread. We consider the contact network as a potential substrate for infectious disease spread and show that its sparsity tends to prevent outbreaks of rapidly spreading epidemics. Moreover, we define three typical behaviors according to the fraction f of links each individual shares outside its own department: residents, wanderers, and linkers. Linkers (f ~ 50%) act as bridges in the network and have large betweenness centralities. Thus, a vaccination strategy targeting linkers efficiently prevents large outbreaks. As such a behavior may be spotted a priori in the offices' organization or from surveys, without the full knowledge of the time-resolved contact network, this result may help the design of efficient, low-cost vaccination or social-distancing strategies.

High-Resolution Measurements of Face-to-Face Contact Patterns in a Primary School

Abstract: Background Little quantitative information is available on the mixing patterns of children in school environments. Describing and understanding contacts between children at school would help quantify the transmission opportunities of respiratory infections and identify situations within schools where the risk of transmission is higher. We report on measurements carried out in a French school (6–12 years children), where we collected data on the time-resolved face-to-face proximity of children and teachers using a proximity-sensing infrastructure based on radio frequency identification devices. Methods and Findings Data on face-to-face interactions were collected on Thursday, October 1st and Friday, October 2nd 2009. We recorded 77,602 contact events between 242 individuals (232 children and 10 teachers). In this setting, each child has on average 323 contacts per day with 47 other children, leading to an average daily interaction time of 176 minutes. Most contacts are brief, but long contacts are also observed. Contacts occur mostly within each class, and each child spends on average three times more time in contact with classmates than with children of other classes. We describe the temporal evolution of the contact network and the trajectories followed by the children in the school, which constrain the contact patterns. We determine an exposure matrix aimed at informing mathematical models. This matrix exhibits a class and age structure which is very different from the homogeneous mixing hypothesis. Conclusions We report on important properties of the contact patterns between school children that are relevant for modeling the propagation of diseases and for evaluating control measures. We discuss public health implications related to the management of schools in case of epidemics and pandemics. Our results can help define a prioritization of control measures based on preventive measures, case isolation, classes and school closures, that could reduce the disruption to education during epidemics.

Contact Patterns in a High School: A Comparison between Data Collected Using Wearable Sensors, Contact Diaries and Friendship Surveys

Abstract: Given their importance in shaping social networks and determining how information or transmissible diseases propagate in a population, interactions between individuals are the subject of many data collection efforts. To this aim, different methods are commonly used, ranging from diaries and surveys to decentralised infrastructures based on wearable sensors. These methods have each advantages and limitations but are rarely compared in a given setting. Moreover, as surveys targeting friendship relations might suffer less from memory biases than contact diaries, it is interesting to explore how actual contact patterns occurring in day-to-day life compare with friendship relations and with online social links. Here we make progresses in these directions by leveraging data collected in a French high school and concerning (i) face-to-face contacts measured by two concurrent methods, namely wearable sensors and contact diaries, (ii) self-reported friendship surveys, and (iii) online social links. We compare the resulting data sets and find that most short contacts are not reported in diaries while long contacts have a large reporting probability, and that the durations of contacts tend to be overestimated in the diaries. Moreover, measured contacts corresponding to reported friendship can have durations of any length but all long contacts do correspond to a reported friendship. On the contrary, online links that are not also reported in the friendship survey correspond to short face-to-face contacts, highlighting the difference of nature between reported friendships and online links. Diaries and surveys suffer moreover from a low sampling rate, as many students did not fill them, showing that the sensor-based platform had a higher acceptability. We also show that, despite the biases of diaries and surveys, the overall structure of the contact network, as quantified by the mixing patterns between classes, is correctly captured by both networks of self-reported contacts and of friendships, and we investigate the correlations between the number of neighbors of individuals in the three networks. Overall, diaries and surveys tend to yield a correct picture of the global structural organization of the contact network, albeit with much less links, and give access to a sort of backbone of the contact network corresponding to the strongest links, i.e., the contacts of longest cumulative durations.

Motifs in Temporal Networks

Abstract: Networks are a fundamental tool for modeling complex systems in a variety of domains including social and communication networks as well as biology and neuroscience. The counts of small subgraph patterns in networks, called network motifs, are crucial to understanding the structure and function of these systems. However, the role of network motifs for temporal networks, which contain many timestamped links between nodes, is not well understood. Here we develop a notion of a temporal network motif as an elementary unit of temporal networks and provide a general methodology for counting such motifs. We define temporal network motifs as induced subgraphs on sequences of edges, design several fast algorithms for counting temporal network motifs, and prove their runtime complexity. We also show that our fast algorithms achieve 1.3x to 56.5x speedups compared to a baseline method. We use our algorithms to count temporal network motifs in a variety of real-world datasets. Results show that networks from different domains have significantly different motif frequencies, whereas networks from the same domain tend to have similar motif frequencies. We also find that measuring motif counts at various time scales reveals different behavior.

Basic statistics of the empirical temporal networks. {{formula:bbe85802-71e3-41aa-a4cb-a86fc3add8dc}}  is the number of individuals; {{formula:a3ef1562-3630-49af-8df7-aab42fc8d3d8}}  is the number of contacts; {{formula:731ae201-09ed-44ed-b69d-95761ec7587f}}  is the total sampling time; {{formula:eab4dc40-f528-418f-b2d5-8d3a4e667dd4}}  is the time resolution of the data set and {{formula:453236a0-2738-41d4-84a5-0bf5baeccb5a}}  is the number of links in the projected static networks.

In addition to the proximity networks, we also study networks from electronic communication. Facebook comes from the wall posts at the social media platform Facebook {{cite:63e6e73}}. College records the network of communication at a Facebook-like service {{cite:2e0f4e8}}. Dating gives the interaction at an early Internet dating website {{cite:5585768}}. Messages and Forum are records of user interaction at a film community {{cite:45014c8}}. Finally we use two data sets of e-mail communication. One, E-mail 1, recording all e-mails to and from a set of sampled accounts {{cite:7333af6}}. The other three, E-mail 2 {{cite:09a26ff}}, 3 {{cite:f7d9bc9}}, and 4 {{cite:00ed002}} recording e-mails within a set of sampled accounts. We list basic statistics—sizes, sampling durations, etc.—of all the data sets in Table {{table:df7d7ac2-4f0d-415e-85f1-41e377f2c88e}} .


The dataset included 400 students from 90 classroom videos in a primary school. The images were collected using single-person images. There are 10,000 images of students raising their hands, walking back and forth, writing on the blackboard, and looking up and down. Additionally, there are 1,000 images of students bending down, standing, and lying on the table.

The dataset that contains five representative student behaviors highly correlated with student engagement (raising hand, standing up, sleeping, yawning, and smiling) and tracks the change trends of these behaviors during the course.

The proposed method utilizes a 10-layer deep convolutional neural network (CNN-10) to recognize student classroom behaviors by extracting key information from human skeleton data, effectively excluding irrelevant information, and achieving higher recognition accuracy and generalization ability.

The proposed StuArt, an innovative automated system that enables instructors to closely monitor the learning progress of each student in the classroom. StuArt also includes user-friendly visualizations to facilitate instructors' understanding of individual and overall learning status.

Classroom Learning Status Assessment Based on Deep Learning

Abstract: Student classroom behavior performance is an important part of classroom teaching evaluation, and conducting student classroom behavior recognition is important for classroom teaching evaluation. The article proposes a deep learning-based student classroom behavior recognition method, which extracts the key information of the human skeleton from student behavior images and combines a 10-layer deep convolutional neural network (CNN-10) to recognize students’ classroom behavior. To verify the effectiveness of this method, the paper conducts a comparison experiment on the student classroom behavior dataset using CNN-10 and the student classroom behavior recognition method. The experimental results show that the student classroom behavior recognition method can effectively exclude the interference of irrelevant information such as students’ physique, dress, and classroom background, highlight the key effective information, and have higher recognition accuracy and generalization ability. Using the human skeleton and a deep learning-based student classroom behavior detection approach to identify students’ typical classroom behaviors might improve intelligent classroom teaching by reflecting students’ learning status in a timely and effective manner.

StuArt: Individualized Classroom Observation of Students with Automatic Behavior Recognition and Tracking

Abstract: Each student matters, but it is hardly for instructors to observe all the students during the courses and provide helps to the needed ones immediately. In this paper, we present StuArt, a novel automatic system designed for the individualized classroom observation, which empowers instructors to concern the learning status of each student. StuArt can recognize five representative student behaviors (hand-raising, standing, sleeping, yawning, and smiling) that are highly related to the engagement and track their variation trends during the course. To protect the privacy of students, all the variation trends are indexed by the seat numbers without any personal identification information. Furthermore, StuArt adopts various user-friendly visualization designs to help instructors quickly understand the individual and whole learning status. Experimental results on real classroom videos have demonstrated the superiority and robustness of the embedded algorithms. We expect our system promoting the development of large-scale individualized guidance of students. More information is in https://github.com/hnuzhy/StuArt.

Datasets and methods of student classroom behavior in recent years

Recently, many researchers have utilized computer vision to detect student classroom behaviors. However, the lack of public student behavior datasets in education restricts the research and application of behavior detection in classroom scenes. Many researchers have also proposed many unpublished datasets. The relevant datasets and methods are shown in Table {{table:f5ffa1f6-b609-4090-9c7c-895676089be2}} .


Don't Decay the Learning Rate, Increase the Batch Size

Abstract: It is common practice to decay the learning rate. Here we show one can usually obtain the same learning curve on both training and test sets by instead increasing the batch size during training. This procedure is successful for stochastic gradient descent (SGD), SGD with momentum, Nesterov momentum, and Adam. It reaches equivalent test accuracies after the same number of training epochs, but with fewer parameter updates, leading to greater parallelism and shorter training times. We can further reduce the number of parameter updates by increasing the learning rate $\epsilon$ and scaling the batch size $B \propto \epsilon$. Finally, one can increase the momentum coefficient $m$ and scale $B \propto 1/(1-m)$, although this tends to slightly reduce the test accuracy. Crucially, our techniques allow us to repurpose existing training schedules for large batch training with no hyper-parameter tuning. We train ResNet-50 on ImageNet to $76.1\%$ validation accuracy in under 30 minutes.

A Stochastic Quasi-Newton Method for Large-Scale Optimization

Abstract: The question of how to incorporate curvature information in stochastic approximation methods is challenging. The direct application of classical quasi- Newton updating techniques for deterministic optimization leads to noisy curvature estimates that have harmful effects on the robustness of the iteration. In this paper, we propose a stochastic quasi-Newton method that is efficient, robust and scalable. It employs the classical BFGS update formula in its limited memory form, and is based on the observation that it is beneficial to collect curvature information pointwise, and at regular intervals, through (sub-sampled) Hessian-vector products. This technique differs from the classical approach that would compute differences of gradients, and where controlling the quality of the curvature estimates can be difficult. We present numerical results on problems arising in machine learning that suggest that the proposed method shows much promise.

Towards Optimal One Pass Large Scale Learning with Averaged Stochastic Gradient Descent

Abstract: For large scale learning problems, it is desirable if we can obtain the optimal model parameters by going through the data in only one pass. Polyak and Juditsky (1992) showed that asymptotically the test performance of the simple average of the parameters obtained by stochastic gradient descent (SGD) is as good as that of the parameters which minimize the empirical cost. However, to our knowledge, despite its optimal asymptotic convergence rate, averaged SGD (ASGD) received little attention in recent research on large scale learning. One possible reason is that it may take a prohibitively large number of training samples for ASGD to reach its asymptotic region for most real problems. In this paper, we present a finite sample analysis for the method of Polyak and Juditsky (1992). Our analysis shows that it indeed usually takes a huge number of samples for ASGD to reach its asymptotic region for improperly chosen learning rate. More importantly, based on our analysis, we propose a simple way to properly set learning rate so that it takes a reasonable amount of data for ASGD to reach its asymptotic region. We compare ASGD using our proposed learning rate with other well known algorithms for training large scale linear classifiers. The experiments clearly show the superiority of ASGD.

A brief lookup table for LML methods based on Optimization Approximation.

Optimization approximation scales up machine learning from the perspective of computational efficiency. In each iteration, these methods only compute the gradients over a few instances or parameters to avoid most useless computations {{cite:5cc8361}}. As a result, they increase the reduction in the optimization error per computation unit and obtain an approximate solution with fewer computations. For a small {{formula:71373014-8312-4520-bbe2-fc3919eac606}} , advanced mathematical techniques must be used to guarantee the effectiveness of approximation. According to targeted scenarios, we further categorize them into mini-batch gradient descent, coordinate gradient descent, and numerical integration based on Markov chain Monte Carlo. For convenience, an overview is provided in Tab.{{table:b14919fd-f315-48c1-966f-9ed81770a510}} .


Validation, comparison, and combination of algorithms for automatic detection of pulmonary nodules in computed tomography images: the LUNA16 challenge

Abstract: Automatic detection of pulmonary nodules in thoracic computed tomography (CT) scans has been an active area of research for the last two decades. However, there have only been few studies that provide a comparative performance evaluation of different systems on a common database. We have therefore set up the LUNA16 challenge, an objective evaluation framework for automatic nodule detection algorithms using the largest publicly available reference database of chest CT scans, the LIDC-IDRI data set. In LUNA16, participants develop their algorithm and upload their predictions on 888 CT scans in one of the two tracks: 1) the complete nodule detection track where a complete CAD system should be developed, or 2) the false positive reduction track where a provided set of nodule candidates should be classified. This paper describes the setup of LUNA16 and presents the results of the challenge so far. Moreover, the impact of combining individual systems on the detection performance was also investigated. It was observed that the leading solutions employed convolutional networks and used the provided set of nodule candidates. The combination of these solutions achieved an excellent sensitivity of over 95% at fewer than 1.0 false positives per scan. This highlights the potential of combining algorithms to improve the detection performance. Our observer study with four expert readers has shown that the best system detects nodules that were missed by expert readers who originally annotated the LIDC-IDRI data. We released this set of additional nodules for further development of CAD systems.

DeepOrgan: Multi-level Deep Convolutional Networks for Automated Pancreas Segmentation

Abstract: Automatic organ segmentation is an important yet challenging problem for medical image analysis. The pancreas is an abdominal organ with very high anatomical variability. This inhibits previous segmentation methods from achieving high accuracies, especially compared to other organs such as the liver, heart or kidneys. In this paper, we present a probabilistic bottom-up approach for pancreas segmentation in abdominal computed tomography (CT) scans, using multi-level deep convolutional networks (ConvNets). We propose and evaluate several variations of deep ConvNets in the context of hierarchical, coarse-to-fine classification on image patches and regions, i.e. superpixels. We first present a dense labeling of local image patches via $P{-}\mathrm{ConvNet}$ and nearest neighbor fusion. Then we describe a regional ConvNet ($R_1{-}\mathrm{ConvNet}$) that samples a set of bounding boxes around each image superpixel at different scales of contexts in a "zoom-out" fashion. Our ConvNets learn to assign class probabilities for each superpixel region of being pancreas. Last, we study a stacked $R_2{-}\mathrm{ConvNet}$ leveraging the joint space of CT intensities and the $P{-}\mathrm{ConvNet}$ dense probability maps. Both 3D Gaussian smoothing and 2D conditional random fields are exploited as structured predictions for post-processing. We evaluate on CT images of 82 patients in 4-fold cross-validation. We achieve a Dice Similarity Coefficient of 83.6$\pm$6.3% in training and 71.8$\pm$10.7% in testing.

Statistics of CT datasets in existing SSL works. Here Cls. denotes classification, Seg. denotes segmentation and Loc. denotes localization. We rank the datasets according to the usage frequency in publications.

We present all the computed tomography (CT) datasets that were used as downstream tasks in previous SSL works in Table {{table:3b509210-ec27-4752-89cc-4f9ef97f35f8}} . According to the rank of published numbers, we selected the top three datasets: LiTS, LUNA2016 and LIDC-ICDR, two of which are for the segmentation task and one for the classification task. Note that the training data in MSD-Liver {{cite:72f7b13}} is from LiTS but its annotations additionally contain the delineation of liver tumour. For simplicity, we use LiTS for liver segmentation and MSD for liver-tumour segmentation, though they are composed of the same CT volumes. Following {{cite:b7bef39}}, we adopt 623 Chest CT scans in LUNA 2016 for pretraining as well and configure the same training and test protocols for the selected target datasets, giving rise to three target tasks called LCS, NCS and NCC. To avoid the test-image leakage between proxy and target tasks, the 623 chest CT scans are the remainder of the LUNA2016 apart from the test set for NCC.


predict requested content & user mobility

Caching as an Image Characterization Problem using Deep Convolutional Neural Networks

Abstract: Caching of popular content closer to the mobile user can significantly increase overall user experience as well as network efficiency by decongesting backbone network segments in the case of congestion episodes. In order to find the optimal caching locations, many conventional approaches rely on solving a complex optimization problem that suffers from the curse of dimensionality, which may fail to support online decision making. In this paper we propose a framework to amalgamate model based optimization with data driven techniques by transforming an optimization problem to a grayscale image and train a convolutional neural network (CNN) to predict optimal caching location policies. The rationale for the proposed modelling comes from CNN’s superiority to capture features in grayscale images reaching human level performance in image recognition problems. The CNN is trained with optimal solutions and numerical investigations reveal that the performance can increase by more than 400% compared to powerful randomized greedy algorithms. To this end, the proposed technique seems as a promising way forward to the holy grail aspect in resource orchestration which is providing high quality decision making in real time.

Network Orchestration in Mobile Networks via a Synergy of Model-driven and AI-based Techniques

Abstract: As data traffic volume continues to increase, caching of popular content at strategic network locations closer to the end user can enhance user experience and ease the utilization of highly congested links in the network. A key challenge in the area of proactive caching is finding the optimal locations to host the popular content items under various optimization criteria. These problems are combinatorial in nature and therefore finding optimal and/or near optimal decisions is computationally expensive. In this paper a framework is proposed to reduce the computational complexity of the underlying integer mathematical program by first predicting decision variables related to optimal locations using a deep convolutional neural network (CNN). The CNN is trained in an offline manner with optimal solutions and is then used to feed a much smaller optimization problems which is amenable for real-time decision making. Numerical investigations reveal that the proposed approach can provide in an online manner high quality decision making; a feature which is crucially important for real-world implementations.

Echo State Networks for Proactive Caching in Cloud-Based Radio Access Networks With Mobile Users

Abstract: In this paper, the problem of proactive caching is studied for cloud radio access networks (CRANs). In the studied model, the baseband units (BBUs) can predict the content request distribution and mobility pattern of each user and determine which content to cache at remote radio heads and the BBUs. This problem is formulated as an optimization problem, which jointly incorporates backhaul and fronthaul loads and content caching. To solve this problem, an algorithm that combines the machine learning framework of echo state networks (ESNs) with sublinear algorithms is proposed. Using ESNs, the BBUs can predict each user’s content request distribution and mobility pattern while having only limited information on the network’s and user’s state. In order to predict each user’s periodic mobility pattern with minimal complexity, the memory capacity of the corresponding ESN is derived for a periodic input. This memory capacity is shown to capture the maximum amount of user information needed for the proposed ESN model. Then, a sublinear algorithm is proposed to determine which content to cache while using limited content request distribution samples. Simulation results using real data from Youku and the Beijing University of Posts and Telecommunications show that the proposed approach yields significant gains, in terms of sum effective capacity, that reach up to 27.8% and 30.7%, respectively, compared with two baseline algorithms: random caching with clustering and random caching without clustering.

Deep Learning Based Proactive Caching for Effective WSN-Enabled Vision Applications

Abstract: Wireless Sensor Networks (WSNs) have a wide range of applications scenarios in computer vision, from pedestrian detection to robotic visual navigation. In response to the growing visual data services in WSNs, we propose a proactive caching strategy based on Stacked Sparse Autoencoder (SSAE) to predict content popularity (PCDS2AW). Firstly, based on Software Defined Network (SDN) and Network Function Virtualization (NFV) technologies, a distributed deep learning network SSAE is constructed in the sink nodes and control nodes of the WSN network. Then, the SSAE network structure parameters and network model parameters are optimized through training. The proactive cache strategy implementation procedure is divided into four steps. (1) The SDN controller is responsible for dynamically collecting user request data package information in the WSNs network. (2) The SSAEs predicts the packet popularity based on the SDN controller obtaining user request data. (3) The SDN controller generates a corresponding proactive cache strategy according to the popularity prediction result. (4) Implement the proactive caching strategy at the WSNs cache node. In the simulation, we compare the influence of spatiotemporal data on the SSAE network structure. Compared with the classic caching strategy Hash + LRU, Betw + LRU, and classic prediction algorithms SVM and BPNN, the proposed PCDS2AW proactive caching strategy can significantly improve WSN performance.

A deep reinforcement learning-based framework for content caching

Abstract: Content caching at the edge nodes is a promising technique to reduce the data traffic in next-generation wireless networks. Inspired by the success of Deep Reinforcement Learning (DRL) in solving complicated control problems, this work presents a DRL-based framework with Wolpertinger architecture for content caching at the base station. The proposed framework is aimed at maximizing the long-term cache hit rate, and it requires no knowledge of the content popularity distribution. To evaluate the proposed framework, we compare the performance with other caching algorithms, including Least Recently Used (LRU), Least Frequently Used (LFU), and First-In First-Out (FIFO) caching strategies. Meanwhile, since the Wolpertinger architecture can effectively limit the action space size, we also compare the performance with Deep Q-Network to identify the impact of dropping a portion of the actions. Our results show that the proposed framework can achieve improved short-term cache hit rate and improved and stable long-term cache hit rate in comparison with LRU, LFU, and FIFO schemes. Additionally, the performance is shown to be competitive in comparison to Deep Q-learning, while the proposed framework can provide significant savings in runtime.

Summary of Deep Learning for Data Caching

We divide the studies regarding deep learning for data caching in edge networks into four categories depending on the DL tools employed: FNN and CNN; RNN; Auto Encoder; DRL. Recently many works utilize more than one DL techniques for jointly considered caching problems. For instance, at the beginning we applies a RNN to predict content popularity, and then a DRL to find suboptimal solutions of content placement for the purpose of reducing time complexity. In such case, we classify the related work into DRL since it represents the caching allocation policy. Unless mention the caching location (such as CRs, MBSs, FBSs, EDs and BBUs) otherwise, the approaches in this section can be utilized for both Layer 1 and Layer 2 caching. Table {{table:adf3b351-4973-4395-a1d3-9645163ed780}}  summarize some studies of DL for caching.


The Design of Arbitrage-Free Data Pricing Schemes

Abstract: Motivated by a growing market that involves buying and selling data over the web, we study pricing schemes that assign value to queries issued over a database. Previous work studied pricing mechanisms that compute the price of a query by extending a data seller's explicit prices on certain queries, or investigated the properties that a pricing function should exhibit without detailing a generic construction. In this work, we present a formal framework for pricing queries over data that allows the construction of general families of pricing functions, with the main goal of avoiding arbitrage. We consider two types of pricing schemes: instance-independent schemes, where the price depends only on the structure of the query, and answer-dependent schemes, where the price also depends on the query output. Our main result is a complete characterization of the structure of pricing functions in both settings, by relating it to properties of a function over a lattice. We use our characterization, together with information-theoretic methods, to construct a variety of arbitrage-free pricing functions. Finally, we discuss various tradeoffs in the design space and present techniques for efficient computation of the proposed pricing functions.

Revenue Maximization for Query Pricing

Abstract:
 Buying and selling of data online has increased substantially over the last few years. Several frameworks have already been proposed that study query pricing in theory and practice. The key guiding principle in these works is the notion of
 arbitrage-freeness
 where the broker can set different prices for different queries made to the dataset, but must ensure that the pricing function does not provide the buyers with opportunities for arbitrage. However, little is known about revenue maximization aspect of query pricing. In this paper, we study the problem faced by a broker selling access to data with the goal of maximizing her revenue. We show that this problem can be formulated as a revenue maximization problem with single-minded buyers and unlimited supply, for which several approximation algorithms are known. We perform an extensive empirical evaluation of the performance of several pricing algorithms for the query pricing problem on real-world instances. In addition to previously known approximation algorithms, we propose several new heuristics and analyze them both theoretically and experimentally. Our experiments show that algorithms with the best theoretical bounds are not necessarily the best empirically. We identify algorithms and heuristics that are both fast and also provide consistently good performance when valuations are drawn from a wide variety of distributions.

Optimized, Direct Sale of Privacy in Personal-Data Marketplaces

Abstract: Very recently, we are witnessing the emergence of a number of start-ups that enables individuals to sell their private data directly to brokers and businesses. While this new paradigm may shift the balance of power between individuals and companies that harvest data, it raises some practical, fundamental questions for users of these services: how they should decide which data must be vended and which data protected, and what a good deal is. In this work, we investigate a mechanism that aims at helping users address these questions. The investigated mechanism relies on a hard-privacy model and allows users to share partial or complete profile data with broker companies in exchange for an economic reward. The theoretical analysis of the trade-off between privacy and money posed by such mechanism is the object of this work. We adopt a generic measure of privacy although part of our analysis focuses on some important examples of Bregman divergences. We find a parametric solution to the problem of optimal exchange of privacy for money, and obtain a closed-form expression and characterize the trade-off between profile-disclosure risk and economic reward for several interesting cases.

Selling Privacy at Auction

Abstract: We initiate the study of markets for private data, though the lens of differential privacy. Although the purchase and sale of private data has already begun on a large scale, a theory of privacy as a commodity is missing. In this paper, we propose to build such a theory. Specifically, we consider a setting in which a data analyst wishes to buy information from a population from which he can estimate some statistic. The analyst wishes to obtain an accurate estimate cheaply. On the other hand, the owners of the private data experience some cost for their loss of privacy, and must be compensated for this loss. Agents are selfish, and wish to maximize their profit, so our goal is to design truthful mechanisms. Our main result is that such auctions can naturally be viewed and optimally solved as variants of multi-unit procurement auctions. Based on this result, we derive auctions for two natural settings which are optimal up to small constant factors: 1. In the setting in which the data analyst has a fixed accuracy goal, we show that an application of the classic Vickrey auction achieves the analyst's accuracy goal while minimizing his total payment. 2. In the setting in which the data analyst has a fixed budget, we give a mechanism which maximizes the accuracy of the resulting estimate while guaranteeing that the resulting sum payments do not exceed the analysts budget. In both cases, our comparison class is the set of envy-free mechanisms, which correspond to the natural class of fixed-price mechanisms in our setting. In both of these results, we ignore the privacy cost due to possible correlations between an individuals private data and his valuation for privacy itself. We then show that generically, no individually rational mechanism can compensate individuals for the privacy loss incurred due to their reported valuations for privacy.

Conducting Truthful Surveys, Cheaply

Abstract: We consider the problem of conducting a survey with the goal of obtaining an unbiased estimator of some population statistic when individuals have unknown costs (drawn from a known prior) for participating in the survey. Individuals must be compensated for their participation and are strategic agents, and so the payment scheme must incentivize truthful behavior. We derive optimal truthful mechanisms for this problem for the two goals of minimizing the variance of the estimator given a fixed budget, and minimizing the expected cost of the survey given a fixed variance goal.

Approximately Optimal Auctions for Selling Privacy when Costs are Correlated with Data

Abstract: We consider a scenario in which a database stores sensitive data of users and an analyst wants to estimate statistics of the data. The users may suffer a cost when their data are used in which case they should be compensated. The analyst wishes to get an accurate estimate, while the users want to maximize their utility. We want to design a mechanism that can estimate statistics accurately without compromising users' privacy. Since users' costs and sensitive data may be correlated, it is important to protect the privacy of both data and cost. We model this correlation by assuming that a user's unknown sensitive data determines a distribution from a set of publicly known distributions and a user's cost is drawn from that distribution. We propose a stronger model of privacy preserving mechanism where users are compensated whenever they reveal information about their data to the mechanism. In this model, we design a Bayesian incentive compatible and privacy preserving mechanism that guarantees accuracy and protects the privacy of both cost and data.

Take it or Leave it: Running a Survey when Privacy Comes at a Cost

Abstract: In this paper, we consider the problem of estimating a potentially sensitive (individually stigmatizing) statistic on a population. In our model, individuals are concerned about their privacy, and experience some cost as a function of their privacy loss. Nevertheless, they would be willing to participate in the survey if they were compensated for their privacy cost. These cost functions are not publicly known, however, nor do we make Bayesian assumptions about their form or distribution. Individuals are rational and will misreport their costs for privacy if doing so is in their best interest. Ghosh and Roth recently showed in this setting, when costs for privacy loss may be correlated with private types, if individuals value differential privacy, no individually rational direct revelation mechanism can compute any non-trivial estimate of the population statistic. In this paper, we circumvent this impossibility result by proposing a modified notion of how individuals experience cost as a function of their privacy loss, and by giving a mechanism which does not operate by direct revelation. Instead, our mechanism has the ability to randomly approach individuals from a population and offer them a take-it-or-leave-it offer. This is intended to model the abilities of a surveyor who may stand on a street corner and approach passers-by.

Redrawing the Boundaries on Purchasing Data from Privacy-Sensitive Individuals

Abstract: We prove new positive and negative results concerning the existence of truthful and individually rational mechanisms for purchasing private data from individuals with unbounded and sensitive privacy preferences. We strengthen the impossibility results of Ghosh and Roth (EC 2011) by extending it to a much wider class of privacy valuations. In particular, these include privacy valuations that are based on ({\epsilon}, {\delta})-differentially private mechanisms for non-zero {\delta}, ones where the privacy costs are measured in a per-database manner (rather than taking the worst case), and ones that do not depend on the payments made to players (which might not be observable to an adversary). To bypass this impossibility result, we study a natural special setting where individuals have mono- tonic privacy valuations, which captures common contexts where certain values for private data are expected to lead to higher valuations for privacy (e.g. having a particular disease). We give new mech- anisms that are individually rational for all players with monotonic privacy valuations, truthful for all players whose privacy valuations are not too large, and accurate if there are not too many players with too-large privacy valuations. We also prove matching lower bounds showing that in some respects our mechanism cannot be improved significantly.

Privacy Auctions for Recommender Systems

Abstract: We study a market for private data in which a data analyst publicly releases a statistic over a database of private information. Individuals that own the data incur a cost for their loss of privacy proportional to the differential privacy guarantee given by the analyst at the time of the release. The analyst incentivizes individuals by compensating them, giving rise to a \emph{privacy auction}. Motivated by recommender systems, the statistic we consider is a linear predictor function with publicly known weights. The statistic can be viewed as a prediction of the unknown data of a new individual, based on the data of individuals in the database. We formalize the trade-off between privacy and accuracy in this setting, and show that a simple class of estimates achieves an order-optimal trade-off. It thus suffices to focus on auction mechanisms that output such estimates. We use this observation to design a truthful, individually rational, proportional-purchase mechanism under a fixed budget constraint. We show that our mechanism is 5-approximate in terms of accuracy compared to the optimal mechanism, and that no truthful mechanism can achieve a $2-\varepsilon$ approximation, for any $\varepsilon > 0$.

Selling Data at an Auction under Privacy Constraints

Abstract: Private data query combines mechanism design with privacy protection to produce aggregated statistics from privately-owned data records. The problem arises in a data marketplace where data owners have personalised privacy requirements and private data valuations. We focus on the case when the data owners are single-minded, i.e., they are willing to release their data only if the data broker guarantees to meet their announced privacy requirements. For a data broker who wants to purchase data from such data owners, we propose the SingleMindedQuery (SMQ) mechanism, which uses a reverse auction to select data owners and determine compensations. SMQ satisfies interim incentive compatibility, individual rationality, and budget feasibility. Moreover, it uses purchased privacy expectation maximisation as a principle to produce accurate outputs for commonly-used queries such as counting, median and linear predictor. The effectiveness of our method is empirically validated by a series of experiments.

Buying Private Data without Verification

Abstract: We consider the problem of designing a survey to aggregate non-verifiable information from a privacy-sensitive population: an analyst wants to compute some aggregate statistic from the private bits held by each member of a population, but cannot verify the correctness of the bits reported by participants in his survey. Individuals in the population are strategic agents with a cost for privacy, \ie, they not only account for the payments they expect to receive from the mechanism, but also their privacy costs from any information revealed about them by the mechanism's outcome---the computed statistic as well as the payments---to determine their utilities. How can the analyst design payments to obtain an accurate estimate of the population statistic when individuals strategically decide both whether to participate and whether to truthfully report their sensitive information? We design a differentially private peer-prediction mechanism that supports accurate estimation of the population statistic as a Bayes-Nash equilibrium in settings where agents have explicit preferences for privacy. The mechanism requires knowledge of the marginal prior distribution on bits $b_i$, but does not need full knowledge of the marginal distribution on the costs $c_i$, instead requiring only an approximate upper bound. Our mechanism guarantees $\epsilon$-differential privacy to each agent $i$ against any adversary who can observe the statistical estimate output by the mechanism, as well as the payments made to the $n-1$ other agents $j\neq i$. Finally, we show that with slightly more structured assumptions on the privacy cost functions of each agent, the cost of running the survey goes to $0$ as the number of agents diverges.

A theory of pricing private data

Abstract:
Personal data has value to both its owner and to institutions who would like to analyze it. Privacy mechanisms protect the owner's data while releasing to analysts noisy versions of aggregate query results. But such strict protections of individual's data have not yet found wide use in practice. Instead, Internet companies, for example, commonly provide free services in return for valuable sensitive information from users, which they exploit and sometimes sell to third parties.
 As the awareness of the value of the personal data increases, so has the drive to compensate the end user for her private information. The idea of monetizing private data can improve over the narrower view of hiding private data, since it empowers individuals to control their data through financial means.
 In this paper we propose a theoretical framework for assigning prices to noisy query answers, as a function of their accuracy, and for dividing the price amongst data owners who deserve compensation for their loss of privacy. Our framework adopts and extends key principles from both differential privacy and query pricing in data markets. We identify essential properties of the price function and micro-payments, and characterize valid solutions.

How to Balance Privacy and Money through Pricing Mechanism in Personal Data Market

Abstract: A personal data market is a platform including three participants: data owners (individuals), data buyers and market maker. Data owners who provide personal data are compensated according to their privacy loss. Data buyers can submit a query and pay for the result according to their desired accuracy. Market maker coordinates between data owner and buyer. This framework has been previously studied based on differential privacy. However, the previous study assumes data owners can accept any level of privacy loss and data buyers can conduct the transaction without regard to the financial budget. In this paper, we propose a practical personal data trading framework that is able to strike a balance between money and privacy. In order to gain insights on user preferences, we first conducted an online survey on human attitude to- ward privacy and interest in personal data trading. Second, we identify the 5 key principles of personal data market, which is important for designing a reasonable trading frame- work and pricing mechanism. Third, we propose a reason- able trading framework for personal data which provides an overview of how the data is traded. Fourth, we propose a balanced pricing mechanism which computes the query price for data buyers and compensation for data owners (whose data are utilized) as a function of their privacy loss. The main goal is to ensure a fair trading for both parties. Finally, we will conduct an experiment to evaluate the output of our proposed pricing mechanism in comparison with other previously proposed mechanism.

Summary of reviewed literature

In the rest of the paper, we first examine work on one-sided markets before moving on to two-sided markets. The reviewed literature is summarised in Table {{table:1797c629-27b5-4ad7-b480-278a87fb8deb}} . Table {{table:1797c629-27b5-4ad7-b480-278a87fb8deb}}  shows that versioning, pay-per-use and tiered pricing are used as pricing methods in the context of complete information, while auction, contract and incentive mechanism are used in the context of asymmetric information. Further, in the context of complete information, pay-per-use, tiered pricing and versioning are often used for null queries, while versioning is used for general queries.