An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale

Abstract: While the Transformer architecture has become the de-facto standard for natural language processing tasks, its applications to computer vision remain limited. In vision, attention is either applied in conjunction with convolutional networks, or used to replace certain components of convolutional networks while keeping their overall structure in place. We show that this reliance on CNNs is not necessary and a pure transformer applied directly to sequences of image patches can perform very well on image classification tasks. When pre-trained on large amounts of data and transferred to multiple mid-sized or small image recognition benchmarks (ImageNet, CIFAR-100, VTAB, etc.), Vision Transformer (ViT) attains excellent results compared to state-of-the-art convolutional networks while requiring substantially fewer computational resources to train.

Training data-efficient image transformers & distillation through attention

Abstract: Recently, neural networks purely based on attention were shown to address image understanding tasks such as image classification. However, these visual transformers are pre-trained with hundreds of millions of images using an expensive infrastructure, thereby limiting their adoption. In this work, we produce a competitive convolution-free transformer by training on Imagenet only. We train them on a single computer in less than 3 days. Our reference vision transformer (86M parameters) achieves top-1 accuracy of 83.1% (single-crop evaluation) on ImageNet with no external data. More importantly, we introduce a teacher-student strategy specific to transformers. It relies on a distillation token ensuring that the student learns from the teacher through attention. We show the interest of this token-based distillation, especially when using a convnet as a teacher. This leads us to report results competitive with convnets for both Imagenet (where we obtain up to 85.2% accuracy) and when transferring to other tasks. We share our code and models.

Comparison of popular ResNet and ViT models. The difference in model robustness can not be blindly attributed to the model architectures. It can be caused by different training settings. WS, GN and WD correspond to Weight Standardization, Group Normalization and Weight Decay, respectively.

We list the state-of-the-art ResNet and ViT models and part of their training settings in Tab. {{table:c2288ad0-b0ca-4048-8c42-087b8f56f79f}} . The techniques applied to boost different models are different, e.g. pretraining. Previous work {{cite:0011e7a}}, {{cite:14b75f6}} have shown that weight decay, data augmentation and pre-training can affect model robustness. In addition, our investigation finds weight standardization and group normalization have a significant impact on model robustness (More in Appendix A). This indicates that the difference in model robustness can not be blindly attributed to the model architectures if models are trained with different settings. {{cite:a190465}} also points out the necessarity of fair setting. Hence, we build fair base models to compare ViT and ResNet as follows.


“Liar, Liar Pants on Fire”: A New Benchmark Dataset for Fake News Detection

Abstract: Automatic fake news detection is a challenging problem in deception detection, and it has tremendous real-world political and social impacts. However, statistical approaches to combating fake news has been dramatically limited by the lack of labeled benchmark datasets. In this paper, we present LIAR: a new, publicly available dataset for fake news detection. We collected a decade-long, 12.8K manually labeled short statements in various contexts from PolitiFact.com, which provides detailed analysis report and links to source documents for each case. This dataset can be used for fact-checking research as well. Notably, this new dataset is an order of magnitude larger than previously largest public fake news datasets of similar type. Empirically, we investigate automatic fake news detection based on surface-level linguistic patterns. We have designed a novel, hybrid convolutional neural network to integrate meta-data with text. We show that this hybrid approach can improve a text-only deep learning model.

Integrating Stance Detection and Fact Checking in a Unified Corpus

Abstract: A reasonable approach for fact checking a claim involves retrieving potentially relevant documents from different sources (e.g., news websites, social media, etc.), determining the stance of each document with respect to the claim, and finally making a prediction about the claim's factuality by aggregating the strength of the stances, while taking the reliability of the source into account. Moreover, a fact checking system should be able to explain its decision by providing relevant extracts (rationales) from the documents. Yet, this setup is not directly supported by existing datasets, which treat fact checking, document retrieval, source credibility, stance detection and rationale extraction as independent tasks. In this paper, we support the interdependencies between these tasks as annotations in the same corpus. We implement this setup on an Arabic fact checking corpus, the first of its kind.

A Richly Annotated Corpus for Different Tasks in Automated Fact-Checking

Abstract: Automated fact-checking based on machine learning is a promising approach to identify false information distributed on the web. In order to achieve satisfactory performance, machine learning methods require a large corpus with reliable annotations for the different tasks in the fact-checking process. Having analyzed existing fact-checking corpora, we found that none of them meets these criteria in full. They are either too small in size, do not provide detailed annotations, or are limited to a single domain. Motivated by this gap, we present a new substantially sized mixed-domain corpus with annotations of good quality for the core fact-checking tasks: document retrieval, evidence extraction, stance detection, and claim validation. To aid future corpus construction, we describe our methodology for corpus creation and annotation, and demonstrate that it results in substantial inter-annotator agreement. As baselines for future research, we perform experiments on our corpus with a number of model architectures that reach high performance in similar problem settings. Finally, to support the development of future models, we provide a detailed error analysis for each of the tasks. Our results show that the realistic, multi-domain setting defined by our data poses new challenges for the existing models, providing opportunities for considerable improvement by future systems.

MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims

Abstract: We contribute the largest publicly available dataset of naturally occurring factual claims for the purpose of automatic claim verification. It is collected from 26 fact checking websites in English, paired with textual sources and rich metadata, and labelled for veracity by human expert journalists. We present an in-depth analysis of the dataset, highlighting characteristics and challenges. Further, we present results for automatic veracity prediction, both with established baselines and with a novel method for joint ranking of evidence pages and predicting veracity that outperforms all baselines. Significant performance increases are achieved by encoding evidence, and by modelling metadata. Our best-performing model achieves a Macro F1 of 49.2%, showing that this is a challenging testbed for claim veracity prediction.

CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims

Abstract: We introduce CLIMATE-FEVER, a new publicly available dataset for verification of climate change-related claims. By providing a dataset for the research community, we aim to facilitate and encourage work on improving algorithms for retrieving evidential support for climate-specific claims, addressing the underlying language understanding challenges, and ultimately help alleviate the impact of misinformation on climate change. We adapt the methodology of FEVER [1], the largest dataset of artificially designed claims, to real-life claims collected from the Internet. While during this process, we could rely on the expertise of renowned climate scientists, it turned out to be no easy task. We discuss the surprising, subtle complexity of modeling real-world climate-related claims within the \textsc{fever} framework, which we believe provides a valuable challenge for general natural language understanding. We hope that our work will mark the beginning of a new exciting long-term joint effort by the climate science and AI community.

Fact or Fiction: Verifying Scientific Claims

Abstract: We introduce scientific claim verification, a new task to select abstracts from the research literature containing evidence that SUPPORTS or REFUTES a given scientific claim, and to identify rationales justifying each decision. To study this task, we construct SciFact, a dataset of 1.4K expert-written scientific claims paired with evidence-containing abstracts annotated with labels and rationales. We develop baseline models for SciFact, and demonstrate that simple domain adaptation techniques substantially improve performance compared to models trained on Wikipedia or political news. We show that our system is able to verify claims related to COVID-19 by identifying evidence from the CORD-19 corpus. Our experiments indicate that SciFact will provide a challenging testbed for the development of new systems designed to retrieve and reason over corpora containing specialized domain knowledge. Data and code for this new task are publicly available at https://github.com/allenai/scifact. A leaderboard and COVID-19 fact-checking demo are available at https://scifact.apps.allenai.org.

Explainable Automated Fact-Checking for Public Health Claims

Abstract: Fact-checking is the task of verifying the veracity of claims by assessing their assertions against credible evidence. The vast majority of fact-checking studies focus exclusively on political claims. Very little research explores fact-checking for other topics, specifically subject matters for which expertise is required. We present the first study of explainable fact-checking for claims which require specific expertise. For our case study we choose the setting of public health. To support this case study we construct a new dataset PUBHEALTH of 11.8K claims accompanied by journalist crafted, gold standard explanations (i.e., judgments) to support the fact-check labels for claims. We explore two tasks: veracity prediction and explanation generation. We also define and evaluate, with humans and computationally, three coherence properties of explanation quality. Our results indicate that, by training on in-domain data, gains can be made in explainable, automated fact-checking for claims which require specific expertise.

X-FACT: A New Benchmark Dataset for Multilingual Fact Checking

Abstract: In this work, we introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims. The dataset contains short statements in 25 languages and is labeled for veracity by expert fact-checkers. The dataset includes a multilingual evaluation benchmark that measures both out-of-domain generalization, and zero-shot capabilities of the multilingual models. Using state-of-the-art multilingual transformer-based models, we develop several automated fact-checking models that, along with textual claims, make use of additional metadata and evidence from news stories retrieved using a search engine. Empirically, our best model attains an F-score of around 40%, suggesting that our dataset is a challenging benchmark for evaluation of multilingual fact-checking models.

Fact Checking in Community Forums

Abstract: Community Question Answering (cQA) forums are very popular nowadays, as they represent effective means for communities around particular topics to share information. Unfortunately, this information is not always factual. Thus, here we explore a new dimension in the context of cQA, which has been ignored so far: checking the veracity of answers to particular questions in cQA forums. As this is a new problem, we create a specialized dataset for it. We further propose a novel multi-faceted model, which captures information from the answer content (what is said and how), from the author profile (who says it), from the rest of the community forum (where it is said), and from external authoritative sources of information (external support). Evaluation results show a MAP value of 86.54, which is 21 points absolute above the baseline.

A Stylometric Inquiry into Hyperpartisan and Fake News

Abstract: This paper reports on a writing style analysis of hyperpartisan (i.e., extremely one-sided) news in connection to fake news. It presents a large corpus of 1,627 articles that were manually fact-checked by professional journalists from BuzzFeed. The articles originated from 9 well-known political publishers, 3 each from the mainstream, the hyperpartisan left-wing, and the hyperpartisan right-wing. In sum, the corpus contains 299 fake news, 97% of which originated from hyperpartisan publishers. We propose and demonstrate a new way of assessing style similarity between text categories via Unmasking---a meta-learning approach originally devised for authorship verification---, revealing that the style of left-wing and right-wing news have a lot more in common than any of the two have with the mainstream. Furthermore, we show that hyperpartisan news can be discriminated well by its style from the mainstream (F1=0.78), as can be satire from both (F1=0.81). Unsurprisingly, style-based fake news detection does not live up to scratch (F1=0.46). Nevertheless, the former results are important to implement pre-screening for fake news detectors.

FakeNewsNet: A Data Repository with News Content, Social Context, and Spatiotemporal Information for Studying Fake News on Social Media

Abstract: Social media has become a popular means for people to consume and share the news. At the same time, however, it has also enabled the wide dissemination of fake news, that is, news with intentionally false information, causing significant negative effects on society. To mitigate this problem, the research of fake news detection has recently received a lot of attention. Despite several existing computational solutions on the detection of fake news, the lack of comprehensive and community-driven fake news data sets has become one of major roadblocks. Not only existing data sets are scarce, they do not contain a myriad of features often required in the study such as news content, social context, and spatiotemporal information. Therefore, in this article, to facilitate fake news-related research, we present a fake news data repository FakeNewsNet, which contains two comprehensive data sets with diverse features in news content, social context, and spatiotemporal information. We present a comprehensive description of the FakeNewsNet, demonstrate an exploratory analysis of two data sets from different perspectives, and discuss the benefits of the FakeNewsNet for potential applications on fake news study on social media.

FakeCovid -- A Multilingual Cross-domain Fact Check News Dataset for COVID-19

Abstract: In this paper, we present a first multilingual cross-domain dataset of 5182 fact-checked news articles for COVID-19, collected from 04/01/2020 to 15/05/2020. We have collected the fact-checked articles from 92 different fact-checking websites after obtaining references from Poynter and Snopes. We have manually annotated articles into 11 different categories of the fact-checked news according to their content. The dataset is in 40 languages from 105 countries. We have built a classifier to detect fake news and present results for the automatic fake news detection and its class. Our model achieves an F1 score of 0.76 to detect the false class and other fact check articles. The FakeCovid dataset is available at Github.

Summary of factual verification datasets with natural inputs. KG denotes knowledge graphs. ChectThat18 has been extended later {{cite:e53442b}}, {{cite:a071eaa}}, {{cite:d7c59bb}}. NELA has been updated by adding more data from more diverse sources {{cite:77e1c22}}, {{cite:593340c}}, {{cite:8a854de}}

Next, we discuss the inputs to factual verification. The most popular type of input to verification is textual claims, which is expected given they are often the output of claim detection. These tend to be sentence-level statements, which is a practice common among fact-checkers in order to include only the context relevant to the claim {{cite:e9f5779}}. Many existing efforts {{cite:74a570c}}, {{cite:a4011f6}}, {{cite:ab73a01}}, {{cite:3052080}} constructed datasets by crawling real-world claims from dedicated websites (e.g. Politifact) due to their availability (see Table {{table:3da68f3f-d027-4f1e-9d3b-6195b2661272}} ). Unlike previous work that focus on English, {{cite:bb3a07c}} collected non-English claims from 25 languages.


Fooling Automated Surveillance Cameras: Adversarial Patches to Attack Person Detection

Abstract: Adversarial attacks on machine learning models have seen increasing interest in the past years. By making only subtle changes to the input of a convolutional neural network, the output of the network can be swayed to output a completely different result. The first attacks did this by changing pixel values of an input image slightly to fool a classifier to output the wrong class. Other approaches have tried to learn "patches" that can be applied to an object to fool detectors and classifiers. Some of these approaches have also shown that these attacks are feasible in the real-world, i.e. by modifying an object and filming it with a video camera. However, all of these approaches target classes that contain almost no intra-class variety (e.g. stop signs). The known structure of the object is then used to generate an adversarial patch on top of it. In this paper, we present an approach to generate adversarial patches to targets with lots of intra-class variety, namely persons. The goal is to generate a patch that is able successfully hide a person from a person detector. An attack that could for instance be used maliciously to circumvent surveillance systems, intruders can sneak around undetected by holding a small cardboard plate in front of their body aimed towards the surveilance camera. From our results we can see that our system is able significantly lower the accuracy of a person detector. Our approach also functions well in real-life scenarios where the patch is filmed by a camera. To the best of our knowledge we are the first to attempt this kind of attack on targets with a high level of intra-class variety like persons.

Invisible Mask: Practical Attacks on Face Recognition with Infrared

Abstract: Accurate face recognition techniques make a series of critical applications possible: policemen could employ it to retrieve criminals' faces from surveillance video streams; cross boarder travelers could pass a face authentication inspection line without the involvement of officers. Nonetheless, when public security heavily relies on such intelligent systems, the designers should deliberately consider the emerging attacks aiming at misleading those systems employing face recognition. We propose a kind of brand new attack against face recognition systems, which is realized by illuminating the subject using infrared according to the adversarial examples worked out by our algorithm, thus face recognition systems can be bypassed or misled while simultaneously the infrared perturbations cannot be observed by raw eyes. Through launching this kind of attack, an attacker not only can dodge surveillance cameras. More importantly, he can impersonate his target victim and pass the face authentication system, if only the victim's photo is acquired by the attacker. Again, the attack is totally unobservable by nearby people, because not only the light is invisible, but also the device we made to launch the attack is small enough. According to our study on a large dataset, attackers have a very high success rate with a over 70\% success rate for finding such an adversarial example that can be implemented by infrared. To the best of our knowledge, our work is the first one to shed light on the severity of threat resulted from infrared adversarial examples against face recognition.

AdvHat: Real-World Adversarial Attack on ArcFace Face ID System

Abstract: In this paper we propose a novel easily reproducible technique to attack the best public Face ID system ArcFace in different shooting conditions. To create an attack, we print the rectangular paper sticker on a common color printer and put it on the hat. The adversarial sticker is prepared with a novel algorithm for off-plane transformations of the image which imitates sticker location on the hat. Such an approach confuses the state-of-the-art public Face ID model LResNet100E-IR, ArcFace@ms1m-refine-v2 and is transferable to other Face ID models.

A General Framework for Adversarial Examples with Objectives

Abstract: Images perturbed subtly to be misclassified by neural networks, called adversarial examples, have emerged as a technically deep challenge and an important concern for several application domains. Most research on adversarial examples takes as its only constraint that the perturbed images are similar to the originals. However, real-world application of these ideas often requires the examples to satisfy additional objectives, which are typically enforced through custom modifications of the perturbation process. In this article, we propose adversarial generative nets (AGNs), a general methodology to train a generator neural network to emit adversarial examples satisfying desired objectives. We demonstrate the ability of AGNs to accommodate a wide range of objectives, including imprecise ones difficult to model, in two application domains. In particular, we demonstrate physical adversarial examples—eyeglass frames designed to fool face recognition—with better robustness, inconspicuousness, and scalability than previous approaches, as well as a new attack to fool a handwritten-digit classifier.

List of Papers Analyzed

Critique 3: Small Sample Size, Low Sample Diversity, and Other Sampling Problems

Finally, all of the papers we reviewed had noticeably small sample sizes. Even when there is a wide variation in testing environments, the studies only included data from one or two people {{cite:1e18e12}}, {{cite:487704f}}. With many of these technologies, the characteristics of the test population, including their gender, race, and age could inform the generalizability of the results. Many papers do not even report this information (see Table {{table:b2f51166-35a1-4106-85e7-f5443a9da398}} ).


Named Entity Recognition, Relation Classification

SocialDial: A Benchmark for Socially-Aware Dialogue Systems

Abstract: Content Warning: this paper may contain content that is offensive or upsetting. Dialogue systems have been widely applied in many scenarios and are now more powerful and ubiquitous than ever before. With large neural models and massive available data, current dialogue systems have access to more knowledge than any people in their life. However, current dialogue systems still do not perform at a human level. One major gap between conversational agents and humans lies in their abilities to be aware of social norms. The development of socially-aware dialogue systems is impeded due to the lack of resources. In this paper, we present the first socially-aware dialogue corpus -- SocialDial based on Chinese social culture. SocialDial consists of two parts: 1,563 multi-turn dialogues between two human speakers with fine-grained labels, and 4,870 synthetic conversations generated by ChatGPT. The human corpus covers five categories of social norms, which have 14 sub-categories in total. Specifically, it contains social factor annotations including social relation, context, social distance, and social norms. However, collecting sufficient socially-aware dialogues is costly. Thus, we harness the power of ChatGPT and devise an ontology-based synthetic data generation framework. This framework is able to generate synthetic data at scale. To ensure the quality of synthetic dialogues, we design several mechanisms for quality control during data collection. Finally, we evaluate our dataset using several pre-trained models, such as BERT and RoBERTa. Comprehensive empirical results based on state-of-the-art neural models demonstrate that modeling of social norms for dialogue systems is a promising research direction. To the best of our knowledge, SocialDial is the first socially-aware dialogue dataset that covers multiple social factors and has fine-grained labels.

UMASS_BioNLP at MEDIQA-Chat 2023: Can LLMs generate high-quality synthetic note-oriented doctor-patient conversations?

Abstract: This paper presents UMASS_BioNLP team participation in the MEDIQA-Chat 2023 shared task for Task-A and Task-C. We focus especially on Task-C and propose a novel LLMs cooperation system named a doctor-patient loop to generate high-quality conversation data sets. The experiment results demonstrate that our approaches yield reasonable performance as evaluated by automatic metrics such as ROUGE, medical concept recall, BLEU, and Self-BLEU. Furthermore, we conducted a comparative analysis between our proposed method and ChatGPT and GPT-4. This analysis also investigates the potential of utilizing cooperation LLMs to generate high-quality datasets.

Textbooks Are All You Need

Abstract: We introduce phi-1, a new large language model for code, with significantly smaller size than competing models: phi-1 is a Transformer-based model with 1.3B parameters, trained for 4 days on 8 A100s, using a selection of ``textbook quality"data from the web (6B tokens) and synthetically generated textbooks and exercises with GPT-3.5 (1B tokens). Despite this small scale, phi-1 attains pass@1 accuracy 50.6% on HumanEval and 55.5% on MBPP. It also displays surprising emergent properties compared to phi-1-base, our model before our finetuning stage on a dataset of coding exercises, and phi-1-small, a smaller model with 350M parameters trained with the same pipeline as phi-1 that still achieves 45% on HumanEval.

LLM-powered Data Augmentation for Enhanced Crosslingual Performance

Abstract: This paper explores the potential of leveraging Large Language Models (LLMs) for data augmentation in multilingual commonsense reasoning datasets where the available training data is extremely limited. To achieve this, we utilise several LLMs, namely Dolly-v2, StableVicuna, ChatGPT, and GPT-4, to augment three datasets: XCOPA, XWinograd, and XStoryCloze. Subsequently, we evaluate the effectiveness of fine-tuning smaller multilingual models, mBERT and XLMR, using the synthesised data. We compare the performance of training with data generated in English and target languages, as well as translated English-generated data, revealing the overall advantages of incorporating data generated by LLMs, e.g. a notable 13.4 accuracy score improvement for the best case. Furthermore, we conduct a human evaluation by asking native speakers to assess the naturalness and logical coherence of the generated examples across different languages. The results of the evaluation indicate that LLMs such as ChatGPT and GPT-4 excel at producing natural and coherent text in most languages, however, they struggle to generate meaningful text in certain languages like Tamil. We also observe that ChatGPT falls short in generating plausible alternatives compared to the original dataset, whereas examples from GPT-4 exhibit competitive logical consistency.

ToxiGen: A Large-Scale Machine-Generated Dataset for Adversarial and Implicit Hate Speech Detection

Abstract: Toxic language detection systems often falsely flag text that contains minority group mentions as toxic, as those groups are often the targets of online hate. Such over-reliance on spurious correlations also causes systems to struggle with detecting implicitly toxic language.To help mitigate these issues, we create ToxiGen, a new large-scale and machine-generated dataset of 274k toxic and benign statements about 13 minority groups. We develop a demonstration-based prompting framework and an adversarial classifier-in-the-loop decoding method to generate subtly toxic and benign text with a massive pretrained language model. Controlling machine generation in this way allows ToxiGen to cover implicitly toxic text at a larger scale, and about more demographic groups, than previous resources of human-written text. We conduct a human evaluation on a challenging subset of ToxiGen and find that annotators struggle to distinguish machine-generated text from human-written language. We also find that 94.5% of toxic examples are labeled as hate speech by human annotators. Using three publicly-available datasets, we show that finetuning a toxicity classifier on our data improves its performance on human-written data substantially. We also demonstrate that ToxiGen can be used to fight machine-generated toxicity as finetuning improves the classifier significantly on our evaluation subset.

A Holistic Approach to Undesired Content Detection in the Real World

Abstract: We present a holistic approach to building a robust and useful natural language classification system for real-world content moderation. The success of such a system relies on a chain of carefully designed and executed steps, including the design of content taxonomies and labeling instructions, data quality control, an active learning pipeline to capture rare events, and a variety of methods to make the model robust and to avoid overfitting. Our moderation system is trained to detect a broad set of categories of undesired content, including sexual content, hateful content, violence, self-harm, and harassment. This approach generalizes to a wide range of different content taxonomies and can be used to create high-quality content classifiers that outperform off-the-shelf models.

Dr. LLaMA: Improving Small Language Models on PubMedQA via Generative Data Augmentation

Abstract: Large Language Models (LLMs) have made remarkable advancements in the field of natural language processing. However, their increasing size poses challenges in terms of computational cost. On the other hand, Small Language Models (SLMs) are known for their efficiency, but they often struggle with limited capacity and training data, especially in specific domains. In this paper, we introduce a novel method aimed at improving SLMs in the medical domain using LLM-based generative data augmentation. The objective of our approach is to develop more efficient and capable models that are specifically tailored for specialized applications. Through experiments conducted on the PubMedQA dataset, we demonstrate the effectiveness of LLMs in refining and diversifying existing question-answer pairs. This refinement process leads to improved performance in a significantly smaller model after fine-tuning. Notably, our best SLM, with under 1.6 billion parameters, outperforms the few-shot GPT-4 on the PubMedQA dataset. Our code and generated data are publicly available to facilitate further explorations.

Exploring Zero and Few-shot Techniques for Intent Classification

Abstract: Conversational NLU providers often need to scale to thousands of intent-classification models where new customers often face the cold-start problem. Scaling to so many customers puts a constraint on storage space as well. In this paper, we explore four different zero and few-shot intent classification approaches with this low-resource constraint: 1) domain adaptation, 2) data augmentation, 3) zero-shot intent classification using descriptions large language models (LLMs), and 4) parameter-efficient fine-tuning of instruction-finetuned language models. Our results show that all these approaches are effective to different degrees in low-resource settings. Parameter-efficient fine-tuning using T-few recipe (Liu et al., 2022) on Flan-T5 (Chang et al., 2022) yields the best performance even with just one sample per intent. We also show that the zero-shot method of prompting LLMs using intent descriptions

TinyStories: How Small Can Language Models Be and Still Speak Coherent English?

Abstract: Language models (LMs) are powerful tools for natural language processing, but they often struggle to produce coherent and fluent text when they are small. Models with around 125M parameters such as GPT-Neo (small) or GPT-2 (small) can rarely generate coherent and consistent English text beyond a few words even after extensive training. This raises the question of whether the emergence of the ability to produce coherent English text only occurs at larger scales (with hundreds of millions of parameters or more) and complex architectures (with many layers of global attention). In this work, we introduce TinyStories, a synthetic dataset of short stories that only contain words that a typical 3 to 4-year-olds usually understand, generated by GPT-3.5 and GPT-4. We show that TinyStories can be used to train and evaluate LMs that are much smaller than the state-of-the-art models (below 10 million total parameters), or have much simpler architectures (with only one transformer block), yet still produce fluent and consistent stories with several paragraphs that are diverse and have almost perfect grammar, and demonstrate reasoning capabilities. We also introduce a new paradigm for the evaluation of language models: We suggest a framework which uses GPT-4 to grade the content generated by these models as if those were stories written by students and graded by a (human) teacher. This new paradigm overcomes the flaws of standard benchmarks which often requires the model's output to be very structures, and moreover provides a multidimensional score for the model, providing scores for different capabilities such as grammar, creativity and consistency. We hope that TinyStories can facilitate the development, analysis and research of LMs, especially for low-resource or specialized domains, and shed light on the emergence of language capabilities in LMs.

How to Unleash the Power of Large Language Models for Few-shot Relation Extraction?

Abstract: Scaling language models have revolutionized widespread NLP tasks, yet little comprehensively explored few-shot relation extraction with large language models. In this paper, we investigate principal methodologies, in-context learning and data generation, for few-shot relation extraction via GPT-3.5 through exhaustive experiments. To enhance few-shot performance, we further propose task-related instructions and schema-constrained data generation. We observe that in-context learning can achieve performance on par with previous prompt learning approaches, and data generation with the large language model can boost previous solutions to obtain new state-of-the-art few-shot results on four widely-studied relation extraction datasets. We hope our work can inspire future research for the capabilities of large language models in few-shot relation extraction. Code is available in https://github.com/zjunlp/DeepKE/tree/main/example/llm.

LogiCoT: Logical Chain-of-Thought Instruction-Tuning Data Collection with GPT-4

Abstract: Generative Pre-trained Transformer 4 (GPT-4) demonstrates impressive chain-of-thought reasoning ability. Recent work on self-instruction tuning, such as Alpaca, has focused on enhancing the general proficiency of models. These instructions enable the model to achieve performance comparable to GPT-3.5 on general tasks like open-domain text generation and paraphrasing. However, they fall short of helping the model handle complex reasoning tasks. To bridge the gap, this paper presents LogiCoT, a new instruction-tuning dataset for Logical Chain-of-Thought reasoning with GPT-4. We elaborate on the process of harvesting instructions for prompting GPT-4 to generate chain-of-thought rationales. LogiCoT serves as an instruction set for teaching models of logical reasoning and elicits general reasoning skills.

Instruction Tuning with GPT-4

Abstract: Prior work has shown that finetuning large language models (LLMs) using machine-generated instruction-following data enables such models to achieve remarkable zero-shot capabilities on new tasks, and no human-written instructions are needed. In this paper, we present the first attempt to use GPT-4 to generate instruction-following data for LLM finetuning. Our early experiments on instruction-tuned LLaMA models show that the 52K English and Chinese instruction-following data generated by GPT-4 leads to superior zero-shot performance on new tasks to the instruction-following data generated by previous state-of-the-art models. We also collect feedback and comparison data from GPT-4 to enable a comprehensive evaluation and reward model training. We make our data generated using GPT-4 as well as our codebase publicly available.

GPT-Calls: Enhancing Call Segmentation and Tagging by Generating Synthetic Conversations via Large Language Models

Abstract: Transcriptions of phone calls are of significant value across diverse fields, such as sales, customer service, healthcare, and law enforcement. Nevertheless, the analysis of these recorded conversations can be an arduous and time-intensive process, especially when dealing with extended or multifaceted dialogues. In this work, we propose a novel method, GPT-distilled Calls Segmentation and Tagging (GPT-Calls), for efficient and accurate call segmentation and topic extraction. GPT-Calls is composed of offline and online phases. The offline phase is applied once to a given list of topics and involves generating a distribution of synthetic sentences for each topic using a GPT model and extracting anchor vectors. The online phase is applied to every call separately and scores the similarity between the transcripted conversation and the topic anchors found in the offline phase. Then, time domain analysis is applied to the similarity scores to group utterances into segments and tag them with topics. The proposed paradigm provides an accurate and efficient method for call segmentation and topic extraction that does not require labeled data, thus making it a versatile approach applicable to various domains. Our algorithm operates in production under Dynamics 365 Sales Conversation Intelligence, and our research is based on real sales conversations gathered from various Dynamics 365 Sales tenants.

How Large Language Models are Transforming Machine-Paraphrase Plagiarism

Abstract: The recent success of large language models for text generation poses a severe threat to academic integrity, as plagiarists can generate realistic paraphrases indistinguishable from original work.However, the role of large autoregressive models in generating machine-paraphrased plagiarism and their detection is still incipient in the literature.This work explores T5 and GPT3 for machine-paraphrase generation on scientific articles from arXiv, student theses, and Wikipedia.We evaluate the detection performance of six automated solutions and one commercial plagiarism detection software and perform a human study with 105 participants regarding their detection performance and the quality of generated examples.Our results suggest that large language models can rewrite text humans have difficulty identifying as machine-paraphrased (53% mean acc.).Human experts rate the quality of paraphrases generated by GPT-3 as high as original texts (clarity 4.0/5, fluency 4.2/5, coherence 3.8/5).The best-performing detection model (GPT-3) achieves 66% F1-score in detecting paraphrases.We make our code, data, and findings publicly available to facilitate the development of detection solutions.

UZH_CLyp at SemEval-2023 Task 9: Head-First Fine-Tuning and ChatGPT Data Generation for Cross-Lingual Learning in Tweet Intimacy Prediction

Abstract: This paper describes the submission of UZH_CLyp for the SemEval 2023 Task 9 “Multilingual Tweet Intimacy Analysis. We achieved second-best results in all 10 languages according to the official Pearson’s correlation regression evaluation measure. Our cross-lingual transfer learning approach explores the benefits of using a Head-First Fine-Tuning method (HeFiT) that first updates only the regression head parameters and then also updates the pre-trained transformer encoder parameters at a reduced learning rate. Additionally, we study the impact of using a small set of automatically generated examples (in our case, from ChatGPT) for low-resource settings where no human-labeled data is available. Our study shows that HeFiT stabilizes training and consistently improves results for pre-trained models that lack domain adaptation to tweets. Our study also shows a noticeable performance increase in cross-lingual learning when synthetic data is used, confirming the usefulness of current text generation systems to improve zeroshot baseline results. Finally, we examine how possible inconsistencies in the annotated data contribute to cross-lingual interference issues.

Does Synthetic Data Generation of LLMs Help Clinical Text Mining?

Abstract: Recent advancements in large language models (LLMs) have led to the development of highly potent models like OpenAI's ChatGPT. These models have exhibited exceptional performance in a variety of tasks, such as question answering, essay composition, and code generation. However, their effectiveness in the healthcare sector remains uncertain. In this study, we seek to investigate the potential of ChatGPT to aid in clinical text mining by examining its ability to extract structured information from unstructured healthcare texts, with a focus on biological named entity recognition and relation extraction. However, our preliminary results indicate that employing ChatGPT directly for these tasks resulted in poor performance and raised privacy concerns associated with uploading patients' information to the ChatGPT API. To overcome these limitations, we propose a new training paradigm that involves generating a vast quantity of high-quality synthetic data with labels utilizing ChatGPT and fine-tuning a local model for the downstream task. Our method has resulted in significant improvements in the performance of downstream tasks, improving the F1-score from 23.37% to 63.99% for the named entity recognition task and from 75.86% to 83.59% for the relation extraction task. Furthermore, generating data using ChatGPT can significantly reduce the time and effort required for data collection and labeling, as well as mitigate data privacy concerns. In summary, the proposed framework presents a promising solution to enhance the applicability of LLM models to clinical text mining.

Large Language Model as Attributed Training Data Generator: A Tale of Diversity and Bias

Abstract: Large language models (LLMs) have been recently leveraged as training data generators for various natural language processing (NLP) tasks. While previous research has explored different approaches to training models using generated data, they generally rely on simple class-conditional prompts, which may limit the diversity of the generated data and inherit systematic biases of LLM. Thus, we investigate training data generation with diversely attributed prompts (e.g., specifying attributes like length and style), which have the potential to yield diverse and attributed generated data. Our investigation focuses on datasets with high cardinality and diverse domains, wherein we demonstrate that attributed prompts outperform simple class-conditional prompts in terms of the resulting model's performance. Additionally, we present a comprehensive empirical study on data generation encompassing vital aspects like bias, diversity, and efficiency, and highlight three key observations: firstly, synthetic datasets generated by simple prompts exhibit significant biases, such as regional bias; secondly, attribute diversity plays a pivotal role in enhancing model performance; lastly, attributed prompts achieve the performance of simple class-conditional prompts while utilizing only 5\% of the querying cost of ChatGPT associated with the latter. The data and code are available on \url{https://github.com/yueyu1030/AttrPrompt}.

Neural Machine Translation Data Generation and Augmentation using ChatGPT

Abstract: Neural models have revolutionized the field of machine translation, but creating parallel corpora is expensive and time-consuming. We investigate an alternative to manual parallel corpora - hallucinated parallel corpora created by generative language models. Although these models are themselves trained on parallel data, they can leverage a multilingual vector space to create data, and may be able to supplement small manually-procured corpora. Our experiments highlight two key findings - despite a lack of diversity in their output, the hallucinated data improves the translation signal, even when the domain clashes with the original dataset.

RobuT: A Systematic Study of Table QA Robustness Against Human-Annotated Adversarial Perturbations

Abstract: Despite significant progress having been made in question answering on tabular data (Table QA), it’s unclear whether, and to what extent existing Table QA models are robust to task-specific perturbations, e.g., replacing key question entities or shuffling table columns. To systematically study the robustness of Table QA models, we propose a benchmark called RobuT, which builds upon existing Table QA datasets (WTQ, WikiSQL-Weak, and SQA) and includes human-annotated adversarial perturbations in terms of table header, table content, and question. Our results indicate that both state-of-the-art Table QA models and large language models (e.g., GPT-3) with few-shot learning falter in these adversarial sets. We propose to address this problem by using large language models to generate adversarial examples to enhance training, which significantly improves the robustness of Table QA models.

Summary of research works exploring GLLMs for data generation-based data augmentation. Here ZS represents zero-shot and FS represents few-shot.

Research works exploring GLLMs for data generation-based data augmentation. The research community explored GLLMs for data generation-based data augmentation in various NLP tasks like dialogue generation {{cite:3895970}}, training smaller LLMs {{cite:584da40}}, {{cite:36d8bc2}}, common sense reasoning {{cite:1d68eff}}, hate speech detection {{cite:6fd43aa}}, undesired content detection {{cite:7350a03}}, question answering {{cite:dc873e2}}, {{cite:f6f9f24}}, intent classification {{cite:f52cfb7}}, relation extraction {{cite:1499661}}, {{cite:bf38564}}, instruction tuning {{cite:eec2981}}, {{cite:2dcf6b6}}, paraphrase detection {{cite:7eff40c}}, tweet intimacy prediction {{cite:57767b1}}, named entity recognition {{cite:bf38564}}, machine translation {{cite:c5c3543}} etc. GLLM-based data generation for data augmentation is explored in multiple domains like general {{cite:1d68eff}}, {{cite:f52cfb7}}, {{cite:36d8bc2}}, {{cite:1499661}}, {{cite:eec2981}}, {{cite:2dcf6b6}}, {{cite:7eff40c}}, {{cite:c5c3543}}, {{cite:f6f9f24}}, {{cite:38b76fb}}, social media {{cite:bf10498}}, {{cite:6fd43aa}}, {{cite:7350a03}}, {{cite:57767b1}}, {{cite:f951769}}, news {{cite:f951769}}, scientific literature {{cite:1499661}}, {{cite:7eff40c}}, healthcare {{cite:3895970}}, {{cite:dc873e2}}, {{cite:bf38564}}, dialogue {{cite:653cb0c}}, programming {{cite:584da40}} etc. Table {{table:79cb5cb9-ca1c-49bb-9009-635a0621a69f}}  presents a summary of research works exploring GLLMs for data generation-based data augmentation.


137 statistical features extracted from GPS data

Decision Tree, Random Forest, KNN and MLP

Know Your Master: Driver Profiling-based Anti-theft Method

Abstract: Although many anti-theft technologies are implemented, auto-theft is still increasing. Also, security vulnerabilities of cars can be used for auto-theft by neutralizing anti-theft system. This keyless auto-theft attack will be increased as cars adopt computerized electronic devices more. To detect auto-theft efficiently, we propose the driver verification method that analyzes driving patterns using measurements from the sensor in the vehicle. In our model, we add mechanical features of automotive parts that are excluded in previous works, but can be differentiated by drivers' driving behaviors. We design the model that uses significant features through feature selection to reduce the time cost of feature processing and improve the detection performance. Further, we enrich the feature set by deriving statistical features such as mean, median, and standard deviation. This minimizes the effect of fluctuation of feature values per driver and finally generates the reliable model. We also analyze the effect of the size of sliding window on performance to detect the time point when the detection becomes reliable and to inform owners the theft event as soon as possible. We apply our model with real driving and show the contribution of our work to the literature of driver identification.

Characterizing Driving Styles with Deep Learning

Abstract: Characterizing driving styles of human drivers using vehicle sensor data, e.g., GPS, is an interesting research problem and an important real-world requirement from automotive industries. A good representation of driving features can be highly valuable for autonomous driving, auto insurance, and many other application scenarios. However, traditional methods mainly rely on handcrafted features, which limit machine learning algorithms to achieve a better performance. In this paper, we propose a novel deep learning solution to this problem, which could be the first attempt of extending deep learning to driving behavior analysis based on GPS data. The proposed approach can effectively extract high level and interpretable features describing complex driving patterns. It also requires significantly less human experience and work. The power of the learned driving style representations are validated through the driver identification problem using a large real dataset.

Autoencoder Regularized Network For Driving Style Representation Learning

Abstract: In this paper, we study learning generalized driving style representations from automobile GPS trip data. We propose a novel Autoencoder Regularized deep neural Network (ARNet) and a trip encoding framework trip2vec to learn drivers' driving styles directly from GPS records, by combining supervised and unsupervised feature learning in a unified architecture. Experiments on a challenging driver number estimation problem and the driver identification problem show that ARNet can learn a good generalized driving style representation: It significantly outperforms existing methods and alternative architectures by reaching the least estimation error on average (0.68, less than one driver) and the highest identification accuracy (by at least 3% improvement) compared with traditional supervised learning methods.

Investigations on Driver Unique Identification from Smartphone’s GPS Data Alone

Abstract: Driver identification is an emerging area of interest in vehicle telematics, automobile control, and insurance. Recent body of works indicates that it may be possible to uniquely identify a driver using multiple dedicated sensors. In this paper, we present an approach for driver identification using smartphone GPS data alone. For our experiments, we collected data from 38 drivers for two months. We quantified the driver’s natural style by extracting a set of 137 statistical features from data generated for each completed trip. The analysis shows that, for the “driver identification” problem, an average accuracy of 82.3% is achieved for driver groups of 4-5 drivers. This is comparable to the state of the arts where mostly a multisensor approach has been taken. Further, it is shown that certain behavioral attributes like high driving skill impact identification accuracy. We observe that Random Forest classifier offers the best results. These results have great implications for various stakeholders since the proposed method can identify a driver based on his/her naturalistic driving style which is quantified in terms of statistical parameters extracted from only GPS data.

Summary of Literature on Driver Identification or Fingerprintings using ML

The autonomous vehicles should maintain correct identification and true profiling of the driver to prevent hacking and theft of the vehicle. To enable this, ML techniques are used by researchers to authenticate the true identification and fingerprinting of a user/driver to make vehicles resilience to the theft without hiding the true identity of drivers. Recently, there has been an immense investigation on the driver behavioural data and driving patterns to identify the true user. The data from smartphone sensors and on-board electronic control units (i.e. using the on-board diagnostic (OBD)-II protocol) are most-commonly used for driver profiling. Few studies in the literature focus on biometric and key-based authentication techniques for driver identification {{cite:52ccd8f}}, {{cite:d987c5d}}, {{cite:17a1540}}. However, these techniques are difficult to achieve higher accuracy and implement in the low-powered vehicle security systems. Therefore, recent works deal with driver identification by applying ML algorithms to behavioural data. The summarized tabular comparison of solutions using ML for driver fingerprinting is presented in Table. {{table:a19faddf-8874-4875-b437-550518d6f89a}} .


Graphonomy: Universal Human Parsing via Graph Transfer Learning

Abstract: Prior highly-tuned human parsing models tend to fit towards each dataset in a specific domain or with discrepant label granularity, and can hardly be adapted to other human parsing tasks without extensive re-training. In this paper, we aim to learn a single universal human parsing model that can tackle all kinds of human parsing needs by unifying label annotations from different domains or at various levels of granularity. This poses many fundamental learning challenges, e.g. discovering underlying semantic structures among different label granularity, performing proper transfer learning across different image domains, and identifying and utilizing label redundancies across related tasks. To address these challenges, we propose a new universal human parsing agent, named "Graphonomy", which incorporates hierarchical graph transfer learning upon the conventional parsing network to encode the underlying label semantic structures and propagate relevant semantic information. In particular, Graphonomy first learns and propagates compact high-level graph representation among the labels within one dataset via Intra-Graph Reasoning, and then transfers semantic information across multiple datasets via Inter-Graph Transfer. Various graph transfer dependencies (\eg, similarity, linguistic knowledge) between different datasets are analyzed and encoded to enhance graph transfer capability. By distilling universal semantic graph representation to each specific task, Graphonomy is able to predict all levels of parsing labels in one system without piling up the complexity. Experimental results show Graphonomy effectively achieves the state-of-the-art results on three human parsing benchmarks as well as advantageous universal human parsing performance.

Devil in the Details: Towards Accurate Single and Multiple Human Parsing

Abstract: Human parsing has received considerable interest due to its wide application potentials. Nevertheless, it is still unclear how to develop an accurate human parsing system in an efficient and elegant way. In this paper, we identify several useful properties, including feature resolution, global context information and edge details, and perform rigorous analyses to reveal how to leverage them to benefit the human parsing task. The advantages of these useful properties finally result in a simple yet effective Context Embedding with Edge Perceiving (CE2P) framework for single human parsing. Our CE2P is end-to-end trainable and can be easily adopted for conducting multiple human parsing. Benefiting the superiority of CE2P, we achieved the 1st places on all three human parsing benchmarks. Without any bells and whistles, we achieved 56.50\% (mIoU), 45.31\% (mean $AP^r$) and 33.34\% ($AP^p_{0.5}$) in LIP, CIHP and MHP v2.0, which outperform the state-of-the-arts more than 2.06\%, 3.81\% and 1.87\%, respectively. We hope our CE2P will serve as a solid baseline and help ease future research in single/multiple human parsing. Code has been made available at \url{https://github.com/liutinglt/CE2P}.

Correlating Edge, Pose With Parsing

Abstract: According to existing studies, human body edge and pose are two beneficial factors to human parsing. The effectiveness of each of the high-level features (edge and pose) is confirmed through the concatenation of their features with the parsing features. Driven by the insights, this paper studies how human semantic boundaries and keypoint locations can jointly improve human parsing. Compared with the existing practice of feature concatenation, we find that uncovering the correlation among the three factors is a superior way of leveraging the pivotal contextual cues provided by edges and poses. To capture such correlations, we propose a Correlation Parsing Machine (CorrPM) employing a heterogeneous non-local block to discover the spatial affinity among feature maps from the edge, pose and parsing. The proposed CorrPM allows us to report new state-of-the-art accuracy on three human parsing datasets. Importantly, comparative studies confirm the advantages of feature correlation over the concatenation.

CDGNet: Class Distribution Guided Network for Human Parsing

Abstract: The objective of human parsing is to partition a human in an image into constituent parts. This task involves labeling each pixel of the human image according to the classes. Since the human body comprises hierarchically structured parts, each body part of an image can have its sole position distribution characteristic. Probably, a human head is less likely to be under the feet, and arms are more likely to be near the torso. Inspired by this observation, we make instance class distributions by accumulating the original human parsing label in the horizontal and vertical directions, which can be utilized as supervision signals. Using these horizontal and vertical class distribution labels, the network is guided to exploit the intrinsic position distribution of each class. We combine two guided features to form a spatial guidance map, which is then superimposed onto the baseline network by multiplication and concatenation to distinguish the human parts precisely. We conducted extensive experiments to demonstrate the effectiveness and superiority of our method on three well-known benchmarks: LIP, ATR, and CIHP databases.††Our code is available at https://github.com/tjpulkl/CDGNet.

Quantitative results on CIHP test in terms of mean pixel Intersection-over-Union (mIoU) (%)

In this section, we outline the performance of several of the previous methods on popular segmentation benchmarks. Since some models are not reported their performance on standard datasets, it is hard to make a full comparison. The following tables summarize the performances of several of the prominent models on different datasets. Table {{table:f7de7341-a0e6-4f62-938b-a63c45ab7878}}  focuses on the PASCAL-Person-Part test set. Since the introduction of FCN-based models, the performance has been greatly improved. Table {{table:631a904f-0766-4e4b-82b1-4c78fb58d104}}  focuses on the LIP validation. On this dataset, the latest model improved by 32.01% in terms of mIoU over the original model. Table {{table:d936b018-501f-4a00-aeff-680c54e314e1}}  focuses on the CIHP test set. This dataset is more challenging than the PASCAL-Person-Part dataset, both two datasets are multi-person. Table {{table:ab1df25e-0982-4318-abfe-1fb26582083c}}  focuses on the ATR test set. The latest model achieves 87.16 in terms of F-1. Finally, Table {{table:af4d1e0a-dcc5-4a1d-9ad2-5141d7748284}}  summarizes the performance of several models for the Fashion Clothing dataset.


Self-Supervised Equivariant Attention Mechanism for Weakly Supervised Semantic Segmentation

Abstract: Image-level weakly supervised semantic segmentation is a challenging problem that has been deeply studied in recent years. Most of advanced solutions exploit class activation map (CAM). However, CAMs can hardly serve as the object mask due to the gap between full and weak supervisions. In this paper, we propose a self-supervised equivariant attention mechanism (SEAM) to discover additional supervision and narrow the gap. Our method is based on the observation that equivariance is an implicit constraint in fully supervised semantic segmentation, whose pixel-level labels take the same spatial transformation as the input images during data augmentation. However, this constraint is lost on the CAMs trained by image-level supervision. Therefore, we propose consistency regularization on predicted CAMs from various transformed images to provide self-supervision for network learning. Moreover, we propose a pixel correlation module (PCM), which exploits context appearance information and refines the prediction of current pixel by its similar neighbors, leading to further improvement on CAMs consistency. Extensive experiments on PASCAL VOC 2012 dataset demonstrate our method outperforms state-of-the-art methods using the same level of supervision. The code is released online.

Self-supervised Image-specific Prototype Exploration for Weakly Supervised Semantic Segmentation

Abstract: Weakly Supervised Semantic Segmentation (WSSS) based on image-level labels has attracted much attention due to low annotation costs. Existing methods often rely on Class Activation Mapping (CAM) that measures the correlation between image pixels and classifier weight. However, the classifier focuses only on the discriminative regions while ignoring other useful information in each image, resulting in incomplete localization maps. To address this issue, we propose a Self-supervised Image-specific Prototype Exploration (SIPE) that consists of an Image-specific Prototype Exploration (IPE) and a General-Specific Consistency (GSC) loss. Specifically, IPE tailors prototypes for every image to capture complete regions, formed our Image-Specific CAM (IS-CAM), which is realized by two sequential steps. In addition, GSC is proposed to construct the consistency of general CAM and our specific IS-CAM, which further optimizes the feature representation and empowers a self-correction ability of prototype exploration. Extensive experiments are conducted on PASCAL VOC 2012 and MS COCO 2014 segmentation benchmark and results show our SIPE achieves new state-of-the-art performance using only image-level labels. The code is available at https://github.com/chenqi1126/SIPE.

Pseudo-mask Matters in Weakly-supervised Semantic Segmentation

Abstract: Most weakly supervised semantic segmentation (WSSS) methods follow the pipeline that generates pseudo-masks initially and trains the segmentation model with the pseudo-masks in fully supervised manner after. However, we find some matters related to the pseudo-masks, including high quality pseudo-masks generation from class activation maps (CAMs), and training with noisy pseudo-mask supervision. For these matters, we propose the following designs to push the performance to new state-of-art: (i) Coefficient of Variation Smoothing to smooth the CAMs adaptively; (ii) Proportional Pseudo-mask Generation to project the expanded CAMs to pseudo-mask based on a new metric indicating the importance of each class on each location, instead of the scores trained from binary classifiers. (iii) Pretended Under-Fitting strategy to suppress the influence of noise in pseudo-mask; (iv) Cyclic Pseudo-mask to boost the pseudo-masks during training of fully supervised semantic segmentation (FSSS). Experiments based on our methods achieve new state-of-art results on two changeling weakly supervised semantic segmentation datasets, pushing the mIoU to 70.0% and 40.2% on PAS-CAL VOC 2012 and MS COCO 2014 respectively. Codes including segmentation framework are released at https://github.com/Eli-YiLi/PMM

Multi-class Token Transformer for Weakly Supervised Semantic Segmentation

Abstract: This paper proposes a new transformer-based framework to learn class-specific object localization maps as pseudo labels for weakly supervised semantic segmentation (WSSS). Inspired by the fact that the attended regions of the one-class token in the standard vision transformer can be leveraged to form a class-agnostic localization map, we investigate if the transformer model can also effectively capture class-specific attention for more discriminative object localization by learning multiple class tokens within the transformer. To this end, we propose a Multi-class Token Transformer, termed as MCTformer, which uses multiple class tokens to learn interactions between the class tokens and the patch tokens. The proposed MCTformer can successfully produce class-discriminative object localization maps from the class-to-patch attentions corresponding to different class tokens. We also propose to use a patch-level pairwise affinity, which is extracted from the patch-to-patch transformer attention, to further refine the localization maps. Moreover, the proposed framework is shown to fully complement the Class Activation Mapping (CAM) method, leading to remarkably superior WSSS results on the PASCAL VOC and MS COCO datasets. These results underline the importance of the class token for WSSS. 11https://github.com/xulianuwa/MCTformer

List of code repositories used for reproducing the results. See Section  for what each step involves.

Discussion on the limits of reproducibility

This convoluted pipeline becomes especially tricky to reproduce, due to the fact that the steps are usually split across multiple code repositories. This introduces additional possibilities for misaligned implementation details, especially if they are not fully listed. Commonly, the authors of WSSS papers provide code for steps 1-3, and the subsequent steps are typically implemented in a different code repository, and in some cases even maintained by different authors. See Table {{table:872ee137-4460-45f9-ba7b-83a8eb13aa91}}  for the repositories that we used in the different steps for each method, which includes a total of 9 unique code repositories. A further argument that speaks for this being the main reason, is that we did manage to reproduce the CAM results. The region similarity of our reproduced CAMs matched the reported results within the margin of error, in most cases. See Table {{table:62eb13a3-cb18-4ba3-b41d-e0681f92377d}}  for a comparison between reported and reproduced CAM results. In all cases, the CAM results were more closely reproduced than the final segmentation results. This means that the subsequent steps introduce differences in the implementation, which is reasonable as this is typically not the main focus of WSSS papers.


Controlling for openness in the male-dominated collaborative networks of the global film industry

Abstract: Studies of gender inequality in film industries have noted the persistence of male domination in creative roles (usually defined as director, producer, writer) and the slow pace of reform. Typical policy remedies are premised on aggregate counts of women as a proportion of overall industry participation. Network science offers an alternative way of identifying and proposing change mechanisms, as it puts emphasis on relationships instead of individuals. Preliminary work on applying network analysis to understand inequality in the film industry has been undertaken. However, in this study we offer a comprehensive approach that enables us to not only understand what inequality in the film industry looks like through the lens of network science but also how we can attempt to address this issue. We offer a data-driven simulation framework that investigates various what-if scenarios when it comes to network evolution. We then assess each of these scenarios with respect to its potential to address gender inequality in the film industry. As suggested by previous studies, inequality is exacerbated when industry networks are most closed. We review evidence from three different national film industries on network relationships in creative teams and identify a high proportion of men who only work with other men. In response to this observation, we test several mechanisms through which industry structures may generate higher levels of openness. Our results reveal that the most critical factor for improving network openness is not simply the statistical improvement of the number of women in a network, nor the removal of men who do not work with women. The most likely behavioural changes to a network will involve the production of connections between women and powerful men.

Exhaustive generation of $k$-critical $\mathcal H$-free graphs

Abstract: We describe an algorithm for generating all $k$-critical $\mathcal H$-free graphs, based on a method of Ho\`{a}ng et al. Using this algorithm, we prove that there are only finitely many $4$-critical $(P_7,C_k)$-free graphs, for both $k=4$ and $k=5$. We also show that there are only finitely many $4$-critical graphs $(P_8,C_4)$-free graphs. For each case of these cases we also give the complete lists of critical graphs and vertex-critical graphs. These results generalize previous work by Hell and Huang, and yield certifying algorithms for the $3$-colorability problem in the respective classes. Moreover, we prove that for every $t$, the class of 4-critical planar $P_t$-free graphs is finite. We also determine all 27 4-critical planar $(P_7,C_6)$-free graphs. We also prove that every $P_{10}$-free graph of girth at least five is 3-colorable, and determine the smallest 4-chromatic $P_{12}$-free graph of girth five. Moreover, we show that every $P_{13}$-free graph of girth at least six and every $P_{16}$-free graph of girth at least seven is 3-colorable. This strengthens results of Golovach et al.

Graphs with few Hamiltonian Cycles

Abstract:
We describe an algorithm for the exhaustive generation of non-isomorphic graphs with a given number k ≥ 0 of hamiltonian cycles, which is especially efficient for small k. Our main findings, combining applications of this algorithm and existing algorithms with new theoretical results, revolve around graphs containing exactly one hamiltonian cycle (1H) or exactly three hamiltonian cycles (3H). Motivated by a classic result of Smith and recent work of Royle, we show that there exist nearly cubic 1H graphs of order n iff n ≥ 18 is even. This gives the strongest form of a theorem of Entringer and Swart, and sheds light on a question of Fleischner originally settled by Seamone. We prove equivalent formulations of the conjecture of Bondy and Jackson that every planar 1H graph contains two vertices of degree 2, verify
it up to order 16, and show that its toric analogue does not hold. We treat Thomassen’s conjecture that every hamiltonian graph of minimum degree at least 3 contains an edge such that both its removal and its contraction yield
hamiltonian graphs. We also verify up to order 21 the conjecture of Sheehan that there is no 4-regular 1H graph. Extending work of Schwenk, we describe all orders for which cubic 3H triangle-free graphs exist. We verify up to order 48 Cantoni’s conjecture that every planar cubic 3H graph contains a triangle, and show that there exist infinitely many planar cyclically 4-edge-connected cubic graphs with exactly four hamiltonian cycles, thereby answering a question of
Chia and Thomassen. Finally, complementing work of Sheehan on 1H graphs of maximum size, we determine the maximum size of graphs containing exactly one hamiltonian path and give, for every order n, the exact number of such
graphs on n vertices and of maximum size.

Threshold effects for two pathogens spreading on a network

Abstract: Diseases spread through host populations over the networks of contacts between individuals, and a number of results about this process have been derived in recent years by exploiting connections between epidemic processes and bond percolation on networks. Here we investigate the case of two pathogens in a single population, which has been the subject of recent interest among epidemiologists. We demonstrate that two pathogens competing for the same hosts can both spread through a population only for intermediate values of the bond occupation probability that lie above the classic epidemic threshold and below a second higher value, which we call the coexistence threshold, corresponding to a distinct topological phase transition in networked systems.

Current SNSs and their generations

We review and discuss the above mentioned three elements in the context of the modelling generations while considering two types of heterogeneity (i) the existence of a given element (represented with a ✩) and in addition to its existence, (ii) the capability of an element to change over time (represented with a ★) (See Table {{table:f87ea6bd-5530-4873-9de4-fc0615558eac}} ).

Generation 1 (G1) of models focuses on dynamic process on static networks (see Table {{table:f87ea6bd-5530-4873-9de4-fc0615558eac}}  and Fig. {{figure:ef558998-70f4-4ff7-8728-1b7838f33f7f}} ). They simulate networks that are "frozen" in time, with a dynamic process taking place on the networks where parameters of this process do not change during the simulation (e.g. epidemic spreading process on static social networks with a fixed infection rate {{cite:8caa4b6}}, {{cite:976c0ad}}). Most studies focus on SNSs in G1 {{cite:358eef2}}, {{cite:7b51318}}, {{cite:0f633de}}, {{cite:0daf4b1}}, {{cite:176afdf}}, {{cite:33ed649}}, {{cite:6f4403d}}, {{cite:0dae261}}, {{cite:a24bf7a}}, {{cite:8f85d2b}}, {{cite:896759e}}, {{cite:230c9de}}, {{cite:b9424e8}}, {{cite:9860059}}, {{cite:b748813}}, {{cite:d2bb79b}}, {{cite:ace7206}}, {{cite:587bf0b}}, {{cite:cc8b638}}, {{cite:05b1b0c}}, {{cite:735aab3}}, {{cite:43451fa}}, {{cite:b0ca28f}}, {{cite:0206e26}}, {{cite:8caa4b6}}, {{cite:ce42ef3}}, {{cite:6fc0729}}, {{cite:1c20682}}, {{cite:c9abca6}}, {{cite:601fb77}}, {{cite:18a04b8}}, {{cite:ddaa61f}}, {{cite:1232f92}}, {{cite:e0ae13b}}, {{cite:ff708e5}}, {{cite:598f1a9}}, {{cite:bc8afb6}}, {{cite:1ac60e1}}, {{cite:f8f3f70}}. Also, many of the simulators generate static social networks without any consideration of dynamic processes {{cite:358eef2}}, {{cite:7b51318}}, {{cite:0f633de}}, {{cite:0daf4b1}}, {{cite:176afdf}}, {{cite:33ed649}}, {{cite:6f4403d}}, {{cite:0dae261}}, {{cite:a24bf7a}}, {{cite:8f85d2b}}, {{cite:896759e}}, {{cite:230c9de}}, {{cite:b9424e8}}, {{cite:9860059}}, {{cite:b748813}}, {{cite:d2bb79b}}, {{cite:ace7206}}, {{cite:587bf0b}}, {{cite:cc8b638}}.

SNSs in Generstion 2 (G2) have two variations, one termed as G2a and the other as G2b. SNSs in G2a focus on evolving dynamic process on static networks (see Table {{table:f87ea6bd-5530-4873-9de4-fc0615558eac}}  and Fig. {{figure:d164bb9d-e2d6-4f16-bee6-b9743df38365}} ). They simulate a static network where dynamic process changes its parameters over time and gets captured in snapshots (e.g. epidemic spreading processes on static social networks with a changeable infection rate {{cite:90fb295}}). There are only few studies on SNSs in G2a {{cite:bb5ad2b}}, {{cite:90fb295}}, {{cite:593c245}}.

SNSs in G2b focus on dynamic process on evolving networks (see Table {{table:f87ea6bd-5530-4873-9de4-fc0615558eac}}  and Fig. {{figure:40b84678-1c7f-44c7-b80f-bff984613898}} ). They simulate network snapshots that describe the network topology changes over time, where the dynamic process takes place without changing its parameters (e.g. epidemic spreading process with a non-changeable infection rate on social network that evolves over time {{cite:22fe119}}). There are many studies on SNSs that model only the dynamics of network structure in this space {{cite:eb4efff}}, {{cite:5acd847}}, {{cite:f4d3333}}, {{cite:57713ec}}, {{cite:d098826}}, {{cite:4b99c31}}, {{cite:a93994a}}, {{cite:eb5a75b}}, {{cite:e7fd9d2}}, {{cite:b9f6137}}, {{cite:531f8ca}}, {{cite:7ab27c6}}, {{cite:b953092}}, {{cite:ea73ea3}}, {{cite:83ef14d}}, {{cite:a276fe2}}, {{cite:22fe119}}, {{cite:587bf0b}}, {{cite:6b84e36}} while few of them additionally consider the dynamic processes {{cite:a276fe2}}, {{cite:22fe119}}.


Predict real-time steering wheel angle for autonomous driving.

Provide efficient communication and computation services for ground mobile devices in outdoor events.

Improve the performance of the ML models on mobile robots with low communication costs.

Address the data island problem in IIoT for dynamic perception and intelligent decision.

Speed up model convergence for detecting and dealing with the concept drift on edge.

Real-time End-to-End Federated Learning: An Automotive Case Study

Abstract: With the development and the increasing interests in ML/DL fields, companies are eager to apply Machine Learning/Deep Learning approaches to increase service quality and customer experience. Federated Learning was implemented as an effective model training method for distributing and accelerating time-consuming model training while protecting user data privacy. However, common Federated Learning approaches, on the other hand, use a synchronous protocol to conduct model aggregation, which is inflexible and unable to adapt to rapidly changing environments and heterogeneous hardware settings in real-world scenarios. In this paper, we present an approach to real-time end-to-end Federated Learning combined with a novel asynchronous model aggregation protocol. Our method is validated in an industrial use case in the automotive domain, focusing on steering wheel angle prediction for autonomous driving. Our findings show that asynchronous Federated Learning can significantly improve the prediction performance of local edge models while maintaining the same level of accuracy as centralized machine learning. Furthermore, by using a sliding training window, the approach can minimize communication overhead, accelerate model training speed and consume real-time streaming data, proving high efficiency when deploying ML/DL components to heterogeneous real-world embedded systems.

Privacy-Preserving Federated Learning for UAV-Enabled Networks: Learning-Based Joint Scheduling and Resource Management

Abstract: Unmanned aerial vehicles (UAVs) are capable of serving as flying base stations (BSs) for supporting data collection, artificial intelligence (AI) model training, and wireless communications. However, due to the privacy concerns of devices and limited computation or communication resource of UAVs, it is impractical to send raw data of devices to UAV servers for model training. Moreover, due to the dynamic channel condition and heterogeneous computing capacity of devices in UAV-enabled networks, the reliability and efficiency of data sharing require to be further improved. In this paper, we develop an asynchronous federated learning (AFL) framework for multi-UAV-enabled networks, which can provide asynchronous distributed computing by enabling model training locally without transmitting raw sensitive data to UAV servers. The device selection strategy is also introduced into the AFL framework to keep the low-quality devices from affecting the learning efficiency and accuracy. Moreover, we propose an asynchronous advantage actor-critic (A3C) based joint device selection, UAVs placement, and resource management algorithm to enhance the federated convergence speed and accuracy. Simulation results demonstrate that our proposed framework and algorithm achieve higher learning accuracy and faster federated execution time compared to other existing solutions.

FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots

Abstract: Smartphones, autonomous vehicles, and the Internet-of-things (IoT) devices are considered the primary data source for a distributed network. Due to a revolutionary breakthrough in internet availability and continuous improvement of the IoT devices capabilities, it is desirable to store data locally and perform computation at the edge, as opposed to share all local information with a centralized computation agent. A recently proposed Machine Learning (ML) algorithm called Federated Learning (FL) paves the path towards preserving data privacy, performing distributed learning, and reducing communication overhead in large-scale machine learning (ML) problems. This paper proposes an FL model by monitoring client activities and leveraging available local computing resources, particularly for resource-constrained IoT devices (e.g., mobile robots), to accelerate the learning process. We assign a trust score to each FL client, which is updated based on the client's activities. We consider a distributed mobile robot as an FL client with resource limitations either in memory, bandwidth, processor, or battery life. We consider such mobile robots as FL clients to understand their resource-constrained behavior in a real-world setting. We consider an FL client to be untrustworthy if the client infuses incorrect models or repeatedly gives slow responses during the FL process. After disregarding the ineffective and unreliable client, we perform local training on the selected FL clients. To further reduce the straggler issue, we enable an asynchronous FL mechanism by performing aggregation on the FL server without waiting for a long period to receive a particular client's response.

Adaptive Federated Learning and Digital Twin for Industrial Internet of Things

Abstract: Industrial Internet of Things (IoT) enables distributed intelligent services varying with the dynamic and realtime industrial environment to achieve Industry 4.0 benefits. In this article, we consider a new architecture of digital twin (DT) empowered Industrial IoT, where DTs capture the characteristics of industrial devices to assist federated learning. Noticing that DTs may bring estimation deviations from the actual value of device state, a trusted-based aggregation is proposed in federated learning to alleviate the effects of such deviation. We adaptively adjust the aggregation frequency of federated learning based on Lyapunov dynamic deficit queue and deep reinforcement learning (DRL), to improve the learning performance under the resource constraints. To further adapt to the heterogeneity of industrial IoT, a clustering-based asynchronous federated learning framework is proposed. Numerical results show that the proposed framework is superior to the benchmark in terms of learning accuracy, convergence, and energy saving.

Asynchronous Federated Learning for Sensor Data with Concept Drift

Abstract: Federated learning (FL) involves multiple distributed devices jointly training a shared model without any of the participants having to reveal their local data to a centralized server. Most of previous FL approaches assume that data on devices are fixed and stationary during the training process. However, this assumption is unrealistic because these devices usually have varying sampling rates and different system configurations. In addition, the underlying distribution of the device data can change dynamically over time, which is known as concept drift. Concept drift makes the learning process complicated because of the inconsistency between existing and upcoming data. Traditional concept drift handling techniques such as chunk based and ensemble learning-based methods are not suitable in the federated learning frameworks due to the heterogeneity of local devices. We propose a novel approach, FedConD, to detect and deal with the concept drift on local devices and minimize the effect on the performance of models in asynchronous FL. The drift detection strategy is based on an adaptive mechanism which uses the historical performance of the local models. The drift adaptation is realized by adjusting the regularization parameter of objective function on each local device. Additionally, we design a communication strategy on the server side to select local updates in a prudent fashion and speed up model convergence. Experimental evaluations on three evolving data streams and two image datasets show that \model~detects and handles concept drift, and also reduces the overall communication cost compared to other baseline methods.

The Applications of Asynchronous Federated Learning on Heterogeneous Devices

AFL is adopted in various application scenarios to provide an efficient and flexible training process on heterogeneous devices while maintaining the privacy of local training data. The application scenarios of AFL and related work are summarized and compared in Table {{table:2dcf7875-c289-4a91-891b-66f59384f73a}} .


Elucidating the Design Space of Diffusion-Based Generative Models

Abstract: We argue that the theory and practice of diffusion-based generative models are currently unnecessarily convoluted and seek to remedy the situation by presenting a design space that clearly separates the concrete design choices. This lets us identify several changes to both the sampling and training processes, as well as preconditioning of the score networks. Together, our improvements yield new state-of-the-art FID of 1.79 for CIFAR-10 in a class-conditional setting and 1.97 in an unconditional setting, with much faster sampling (35 network evaluations per image) than prior designs. To further demonstrate their modular nature, we show that our design changes dramatically improve both the efficiency and quality obtainable with pre-trained score networks from previous work, including improving the FID of a previously trained ImageNet-64 model from 2.07 to near-SOTA 1.55, and after re-training with our proposed improvements to a new SOTA of 1.36.

Diffusion Models Beat GANs on Image Synthesis

Abstract: We show that diffusion models can achieve image sample quality superior to the current state-of-the-art generative models. We achieve this on unconditional image synthesis by finding a better architecture through a series of ablations. For conditional image synthesis, we further improve sample quality with classifier guidance: a simple, compute-efficient method for trading off diversity for fidelity using gradients from a classifier. We achieve an FID of 2.97 on ImageNet 128$\times$128, 4.59 on ImageNet 256$\times$256, and 7.72 on ImageNet 512$\times$512, and we match BigGAN-deep even with as few as 25 forward passes per sample, all while maintaining better coverage of the distribution. Finally, we find that classifier guidance combines well with upsampling diffusion models, further improving FID to 3.94 on ImageNet 256$\times$256 and 3.85 on ImageNet 512$\times$512. We release our code at https://github.com/openai/guided-diffusion

Additional experimental details

Training: We stick with hyperparameters introduced in the section.. We use AdamW{{cite:a233df3}} as our optimizer and Exponential Moving Averaging with the exponential decay rate of 0.9999. We use 8 {{formula:80b9939c-b9bf-4dc0-b903-eaf55a7b9d92}}  Nvidia A100 GPU for all experiments. For further, training setup, please refer to Table.{{table:4d20f2ba-4221-4bd8-8040-45865dd86c61}} .


Exploring Adversarial Examples in Malware Detection

Abstract: The convolutional neural network (CNN) architecture is increasingly being applied to new domains, such as malware detection, where it is able to learn malicious behavior from raw bytes extracted from executables. These architectures reach impressive performance with no feature engineering effort involved, but their robustness against active attackers is yet to be understood. Such malware detectors could face a new attack vector in the form of adversarial interference with the classification model. Existing evasion attacks intended to cause misclassification on test-time instances, which have been extensively studied for image classifiers, are not applicable because of the input semantics that prevents arbitrary changes to the binaries. This paper explores the area of adversarial examples for malware detection. By training an existing model on a production-scale dataset, we show that some previous attacks are less effective than initially reported, while simultaneously highlighting architectural weaknesses that facilitate new attack strategies for malware classification. Finally, we explore how generalizable different attack strategies are, the trade-offs when aiming to increase their effectiveness, and the transferability of single-step attacks.

Improving Network Intrusion Detection Classifiers by Non-payload-Based Exploit-Independent Obfuscations: An Adversarial Approach

Abstract: Machine-learning based intrusion detection classifiers are able to detect unknown attacks, but at the same time, they may be susceptible to evasion by obfuscation techniques. An adversary intruder which possesses a crucial knowledge about a protection system can easily bypass the detection module. The main objective of our work is to improve the performance capabilities of intrusion detection classifiers against such adversaries. To this end, we firstly propose several obfuscation techniques of remote attacks that are based on the modification of various properties of network connections; then we conduct a set of comprehensive experiments to evaluate the effectiveness of intrusion detection classifiers against obfuscated attacks. We instantiate our approach by means of a tool, based on NetEm and Metasploit, which implements our obfuscation operators on any TCP communication. This allows us to generate modified network traffic for machine learning experiments employing features for assessing network statistics and behavior of TCP connections. We perform the evaluation of five classifiers: Gaussian Naive Bayes, Gaussian Naive Bayes with kernel density estimation, Logistic Regression, Decision Tree, and Support Vector Machines. Our experiments confirm the assumption that it is possible to evade the intrusion detection capability of all classifiers trained without prior knowledge about obfuscated attacks, causing an exacerbation of the TPR ranging from 7.8% to 66.8%. Further, when widening the training knowledge of the classifiers by a subset of obfuscated attacks, we achieve a significant improvement of the TPR by 4.21% - 73.3%, while the FPR is deteriorated only slightly (0.1% - 1.48%). Finally, we test the capability of an obfuscations-aware classifier to detect unknown obfuscated attacks, where we achieve over 90% detection rate on average for most of the obfuscations.

Summary of Feature Space Adversarial Attacks in Network Security and Their Effectiveness

Feature Space and Problem Space - Dimensional Space Classification

Feature space and problem space dimensional classification of various adversarial attacks are shown in Tables {{table:6c3e42ca-94e9-43b6-91b4-8e23670afbc1}}  and {{table:f910fb26-4a7b-4c7c-b0f2-92cd3e165b00}}  respectively. From our observation, adversarial attacks in problem space are more difficult to generate and also more difficult to defend against.


Decentralized User-Centric Access Control using PubSub over Blockchain

Abstract: We present a mechanism that puts users in the center of control and empowers them to dictate the access to their collections of data. Revisiting the fundamental mechanisms in security for providing protection, our solution uses capabilities, access lists, and access rights following well-understood formal notions for reasoning about access. This contribution presents a practical, correct, auditable, transparent, distributed, and decentralized mechanism that is well-matched to the current emerging environments including Internet of Things, smart city, precision medicine, and autonomous cars. It is based on well-tested principles and practices used in a distributed authorization, cryptocurrencies, and scalable computing.

A Novel Medical Blockchain Model for Drug Supply Chain Integrity Management in a Smart Hospital

Abstract: At present, in pharmacology one of the most serious problems is counterfeit drugs. The Health Research Funding organization reported that in developing countries, nearly 10–30% of the drugs are fake. Counterfeiting is not the main issue itself, but, rather, the fact that, as compared to traditional drugs, these counterfeit drugs produce different side effects to human health. According to WHO, around 30% of the total medicine sold in Africa, Asia, and Latin America is counterfeit. This is the major worldwide problem, and the situation is worse in developing countries, where one out of every 10 medicines are either fake or do not follow drug regulations. The rise of Internet pharmacies has made it more difficult to standardize drug safety. It is difficult to detect counterfeits because these drugs pass through different complex distributed networks, thus forming opportunities for counterfeits to enter the authentic supply chain. The safety of the pharmaceutical supply chain has become a major concern for public health, which is a collective process. In this paper, we propose a novel drug supply chain management using Hyperledger Fabric based on blockchain technology to handle secure drug supply chain records. The proposed system solves this problem by conducting drug record transactions on a blockchain to create a smart healthcare ecosystem with a drug supply chain. A smart contract is launched to give time-limited access to electronic drug records and also patient electronic health records. We also carried out a number of experiments in order to demonstrate the usability and efficiency of the designed platform. Finally, we used Hyperledger Caliper as a benchmarking tool to conduct the performance of the designed system in terms of transactions per second, transaction latency, and resource utilization.

Existing research on DLT applications in Smart Cities, applicable to user data and anonymity.

Smart Cities, DLTs and Current Use Cases: London is acknowledged as one the largest Smart Cities in the world, with investment specified in the Mayor of London's “Smarter London Together” roadmap {{cite:56173a8}}. Recently, there has been a noticeable interest in the application of DLTs within the context of Smart Cities (see Table {{table:6bb35826-b420-433f-ae0a-3905e39e0ce9}} ). A Data Verification System for CCTV Surveillance Cameras Using Blockchain Technology in Smart Cities {{cite:f8f7544}} outlines a Blockchain-based system to guarantee trustworthiness of CCTV recordings. This is relevant as it discusses privacy, immutability, and how to ensure that personal data cannot be manipulated, which will be a focus point in the proposed female safety system. Overall, the importance of authentication and verification of data for Smart Cities will be a critical consideration for this project. Current research has proven that DLTs and IoT can work together to solve many of the existing security issues and concerns.