Visual Classification via Description from Large Language Models

Abstract: Vision-language models (VLMs) such as CLIP have shown promising performance on a variety of recognition tasks using the standard zero-shot classification procedure -- computing similarity between the query image and the embedded words for each category. By only using the category name, they neglect to make use of the rich context of additional information that language affords. The procedure gives no intermediate understanding of why a category is chosen, and furthermore provides no mechanism for adjusting the criteria used towards this decision. We present an alternative framework for classification with VLMs, which we call classification by description. We ask VLMs to check for descriptive features rather than broad categories: to find a tiger, look for its stripes; its claws; and more. By basing decisions on these descriptors, we can provide additional cues that encourage using the features we want to be used. In the process, we can get a clear idea of what features the model uses to construct its decision; it gains some level of inherent explainability. We query large language models (e.g., GPT-3) for these descriptors to obtain them in a scalable way. Extensive experiments show our framework has numerous advantages past interpretability. We show improvements in accuracy on ImageNet across distribution shifts; demonstrate the ability to adapt VLMs to recognize concepts unseen during training; and illustrate how descriptors can be edited to effectively mitigate bias compared to the baseline.

What does a platypus look like? Generating customized prompts for zero-shot image classification

Abstract: Open-vocabulary models are a promising new paradigm for image classification. Unlike traditional classification models, open-vocabulary models classify among any arbitrary set of categories specified with natural language during inference. This natural language, called "prompts", typically consists of a set of hand-written templates (e.g., "a photo of a {}") which are completed with each of the category names. This work introduces a simple method to generate higher accuracy prompts, without relying on any explicit knowledge of the task domain and with far fewer hand-constructed sentences. To achieve this, we combine open-vocabulary models with large language models (LLMs) to create Customized Prompts via Language models (CuPL, pronounced "couple"). In particular, we leverage the knowledge contained in LLMs in order to generate many descriptive sentences that contain important discriminating characteristics of the image categories. This allows the model to place a greater importance on these regions in the image when making predictions. We find that this straightforward and general approach improves accuracy on a range of zero-shot image classification benchmarks, including over one percentage point gain on ImageNet. Finally, this simple baseline requires no additional training and remains completely zero-shot. Code available at https://github.com/sarahpratt/CuPL.

Statistics for measuring attribute sensitivity in classification via description. CWD=Classification With (Multiple) Descriptors; SS=Single Sentence. Note that there are more adjectives/description in the single-sentence case, potentially explaining its increased sensitivity to “Change ADJ" in Fig. 5 (main paper).

In Table {{table:b4f45f1f-a604-4de6-ba3b-5016bace90aa}} , we provide key statistics regarding the descriptors generated for this analysis.


Deciphering Malware's use of TLS (without Decryption)

Abstract: The use of TLS by malware poses new challenges to network threat detection because traditional pattern-matching techniques can no longer be applied to its messages. However, TLS also introduces a complex set of observable data features that allow many inferences to be made about both the client and the server. We show that these features can be used to detect and understand malware communication, while at the same time preserving the privacy of benign uses of encryption. These data features also allow for accurate malware family attribution of network communication, even when restricted to a single, encrypted flow. To demonstrate this, we performed a detailed study of how TLS is used by malware and enterprise applications. We provide a general analysis on millions of TLS encrypted flows, and a targeted study on 18 malware families composed of thousands of unique malware samples and ten-of-thousands of malicious TLS flows. Importantly, we identify and accommodate the bias introduced by the use of a malware sandbox. The performance of a malware classifier is correlated with a malware family's use of TLS, i.e., malware families that actively evolve their use of cryptography are more difficult to classify. We conclude that malware's usage of TLS is distinct from benign usage in an enterprise setting, and that these differences can be effectively used in rules and machine learning classifiers.

Unsupervised Detection and Clustering of Malicious TLS Flows

Abstract: Malware abuses TLS to encrypt its malicious traffic, preventing examination by content signatures and deep packet inspection. Network detection of malicious TLS flows is an important, but challenging, problem. Prior works have proposed supervised machine learning detectors using TLS features. However, by trying to represent all malicious traffic, supervised binary detectors produce models that are too loose, thus introducing errors. Furthermore, they do not distinguish flows generated by different malware. On the other hand, supervised multi-class detectors produce tighter models and can classify flows by malware family, but require family labels, which are not available for many samples. To address these limitations, this work proposes a novel unsupervised approach to detect and cluster malicious TLS flows. Our approach takes as input network traces from sandboxes. It clusters similar TLS flows using 90 features that capture properties of the TLS client, TLS server, certificate, and encrypted payload; and uses the clusters to build an unsupervised detector that can assign a malicious flow to the cluster it belongs to, or determine it is benign. We evaluate our approach using 972K traces from a commercial sandbox and 35M TLS flows from a research network. Our clustering shows very high precision and recall with an F1 score of 0.993. We compare our unsupervised detector with two state-of-the-art approaches, showing that it outperforms both. The false detection rate of our detector is 0.032% measured over four months of traffic.

Client features. (R)=Reduced. (C)=Changed. (E)=Could be Encrypted

Client features.
These 11 features, summarized in Table {{table:2dd1a17d-69c8-41f4-bb1a-984b964d8fa8}} ,
are extracted from the Client Hello message.
They capture the functionality supported by the TLS client software.
Programs either use the default
configuration of a cryptographic library or OS API, or
configure them with their preferences.
Client features identify programs whose TLS functionality is
configured similarly.
The features correspond to
the highest supported TLS and record versions,
the list of supported TLS versions (extension added in TLS 1.3),
the list of supported ciphers, compression methods,
elliptic curves and point formats,
the list of extensions included in the message,
the domain name in the SNI extension, and
the list of supported application protocols in the
Application Layer Protocol Negotiation (ALPN) extension
(e.g., HTTP/0.9, SPDY/1).
The fake resumption feature is explained later in the resumed sessions
paragraph.

TLS 1.3.
Most features presented so far are available up to TLS 1.2.
However, TLS 1.3 changes the protocol to reduce the
information available in the TLS handshake.
In particular, some fields become encrypted and other fields have
fewer values to choose from.
These changes are captured in the TLS 1.3 columns in
Tables {{table:2dd1a17d-69c8-41f4-bb1a-984b964d8fa8}} –{{table:ddd8ed01-79de-4fed-bca7-7c353363a65d}} .
In particular, client features such as the list of ciphers, elliptic curves,
and point formats provide less information in TLS 1.3,
as many values have been removed for increased protection,
e.g., against downgrade attacks.
However, as long as clients still support TLS 1.2, we expect
the removal of those values to happen slowly over time.
Furthermore, certificates are encrypted so their features cannot be
extracted directly from the network trace.
On the other hand, payload features are not affected.
Despite these changes, our approach is able to produce
accurate TLS 1.3 clusters.


Fine-Grained Visual Classification of Aircraft

Abstract: This paper introduces FGVC-Aircraft, a new dataset containing 10,000 images of aircraft spanning 100 aircraft models, organised in a three-level hierarchy. At the finer level, differences between models are often subtle but always visually measurable, making visual recognition challenging but possible. A benchmark is obtained by defining corresponding classification tasks and evaluation protocols, and baseline results are presented. The construction of this dataset was made possible by the work of aircraft enthusiasts, a strategy that can extend to the study of number of other object classes. Compared to the domains usually considered in fine-grained visual classification (FGVC), for example animals, aircraft are rigid and hence less deformable. They, however, present other interesting modes of variation, including purpose, size, designation, structure, historical style, and branding.

Describing Textures in the Wild

Abstract: Patterns and textures are key characteristics of many natural objects: a shirt can be striped, the wings of a butterfly can be veined, and the skin of an animal can be scaly. Aiming at supporting this dimension in image understanding, we address the problem of describing textures with semantic attributes. We identify a vocabulary of forty-seven texture terms and use them to describe a large dataset of patterns collected "in the wild". The resulting Describable Textures Dataset (DTD) is a basis to seek the best representation for recognizing describable texture attributes in images. We port from object recognition to texture recognition the Improved Fisher Vector (IFV) and Deep Convolutional-network Activation Features (DeCAF), and show that surprisingly, they both outperform specialized texture descriptors not only on our problem, but also in established material recognition datasets. We also show that our describable attributes are excellent texture descriptors, transferring between datasets and tasks, in particular, combined with IFV and DeCAF, they significantly outperform the state-of-the-art by more than 10% on both FMD and KTH-TIPS-2b benchmarks. We also demonstrate that they produce intuitive descriptions of materials and Internet images.

Details of the fine-grained linear classification datasets.

The details about the fine-grained classification datasets are presented in  {{table:2728a41f-f696-4903-a699-f3086024d77e}}


A Scientific Information Extraction Dataset for Nature Inspired Engineering

Abstract: Nature has inspired various ground-breaking technological developments in applications ranging from robotics to aerospace engineering and the manufacturing of medical devices. However, accessing the information captured in scientific biology texts is a time-consuming and hard task that requires domain-specific knowledge. Improving access for outsiders can help interdisciplinary research like Nature Inspired Engineering. This paper describes a dataset of 1,500 manually-annotated sentences that express domain-independent relations between central concepts in a scientific biology text, such as trade-offs and correlations. The arguments of these relations can be Multi Word Expressions and have been annotated with modifying phrases to form non-projective graphs. The dataset allows for training and evaluating Relation Extraction algorithms that aim for coarse-grained typing of scientific biological documents, enabling a high-level filter for engineers.

DESIGN KNOWLEDGE REPRESENTATION WITH TECHNOLOGY SEMANTIC NETWORK

Abstract: Abstract Engineers often need to discover and learn designs from unfamiliar domains for inspiration or other particular uses. However, the complexity of the technical design descriptions and the unfamiliarity to the domain make it hard for engineers to comprehend the function, behavior, and structure of a design. To help engineers quickly understand a complex technical design description new to them, one approach is to represent it as a network graph of the design-related entities and their relations as an abstract summary of the design. While graph or network visualizations are widely adopted in the engineering design literature, the challenge remains in retrieving the design entities and deriving their relations. In this paper, we propose a network mapping method that is powered by Technology Semantic Network (TechNet). Through a case study, we showcase how TechNet’s unique characteristic of being trained on a large technology-related data source advantages itself over common-sense knowledge bases, such as WordNet and ConceptNet, for design knowledge representation.

Function Based Design-by-Analogy: A Functional Vector Approach to Analogical Search

Abstract: Design-by-analogy is a powerful approach to augment traditional concept generation methods by expanding the set of generated ideas using similarity relationships from solutions to analogous problems. While the concept of design-by-analogy has been known for some time, few actual methods and tools exist to assist designers in systematically seeking and identifying analogies from general data sources, databases, or repositories, such as patent databases. A new method for extracting functional analogies from data sources has been developed to provide this capability, here based on a functional basis rather than form or conﬂict descriptions. Building on past research, we utilize a functional vector space model (VSM) to quantify analogous similarity of an idea’s functionality. We quantitatively evaluate the functional similarity between represented design problems and, in this case, patent descriptions of products. We also develop document parsing algorithms to reduce text descriptions of the data sources down to the key functions, for use in the functional similarity analysis and functional vector space modeling. To do this, we apply Zipf’s law on word count order reduction to reduce the words within the documents down to the applicable functionally critical terms, thus providing a mapping process for function based search. The reduction of a document into functional analogous words enables the matching to novel ideas that are functionally similar, which can be customized various ways. This approach thereby provides relevant sources of design-by-analogy inspiration. As a veriﬁcation of the approach, two original design problem case studies illustrate the distance range of analogical solutions that can be extracted. This range extends from very near-ﬁeld, literal solutions to far-ﬁeld cross-domain analogies. [DOI:

Existing AI-based methods and algorithms that have been adopted or developed for DbA

The AI-based techniques that have already been used in DbA studies are summarized in Table {{table:289807a2-e50e-4a27-b190-7175aa4479db}} . As shown in Fig. {{figure:a7ecbea3-c1de-46d1-bfd4-9745145fd7d5}} , the broadly defined NLP methods, including text mining, topic modeling, semantic networks, and pre-trained word embeddings, have played essential roles in supporting all stages of DbA. These methods aim at understanding and mining the textual nuances of natural language. However, some of them tackled the rough source data using only traditional textual pre-processing methods. They developed some syntactic rules for the processed dataset, which were still limited to the human experience. These pre-defined rules cannot be updated automatically to match the rapid advances in AI technologies. Some researchers that adopted knowledge graph techniques generally relied on the commonsense semantic networks trained on non-engineering data sources such as WordNet and ConceptNet. However, engineering designers have perceptions of technical terms that are distinct from the commonsense knowledge {{cite:4d2a7fe}}. In this case, the technological and engineering semantic networks (e.g., TechNet and B-link) that were recently developed based on engineering data can serve as infrastructure to support broad DbA studies.


Your Voice Assistant is Mine: How to Abuse Speakers to Steal Information and Control Your Phone

Abstract: Previous research about sensor based attacks on Android platform focused mainly on accessing or controlling over sensitive components, such as camera, microphone and GPS. These approaches obtain data from sensors directly and need corresponding sensor invoking permissions. This paper presents a novel approach (GVS-Attack) to launch permission bypassing attacks from a zero-permission Android application (VoicEmployer) through the phone speaker. The idea of GVS-Attack is to utilize an Android system built-in voice assistant module -- Google Voice Search. With Android Intent mechanism, VoicEmployer can bring Google Voice Search to foreground, and then plays prepared audio files (like "call number 1234 5678") in the background. Google Voice Search can recognize this voice command and perform corresponding operations. With ingenious design, our GVS-Attack can forge SMS/Email, access privacy information, transmit sensitive data and achieve remote control without any permission. Moreover, we found a vulnerability of status checking in Google Search app, which can be utilized by GVS-Attack to dial arbitrary numbers even when the phone is securely locked with password. A prototype of VoicEmployer has been implemented to demonstrate the feasibility of GVS-Attack. In theory, nearly all Android (4.1+) devices equipped with Google Services Framework can be affected by GVS-Attack. This study may inspire application developers and researchers to rethink that zero permission doesn't mean safety and the speaker can be treated as a new attack surface.

Houdini: Fooling Deep Structured Prediction Models

Abstract: Generating adversarial examples is a critical step for evaluating and improving the robustness of learning machines. So far, most existing methods only work for classification and are not designed to alter the true performance measure of the problem at hand. We introduce a novel flexible approach named Houdini for generating adversarial examples specifically tailored for the final performance measure of the task considered, be it combinatorial and non-decomposable. We successfully apply Houdini to a range of applications such as speech recognition, pose estimation and semantic segmentation. In all cases, the attacks based on Houdini achieve higher success rate than those based on the traditional surrogates used to train the models while using a less perceptible adversarial perturbation.

Crafting Adversarial Examples For Speech Paralinguistics Applications

Abstract: Computational paralinguistic analysis is increasingly being used in a wide range of applications, including security-sensitive applications such as speaker verification, deceptive speech detection, and medical diagnosis. While state-of-the-art machine learning techniques, such as deep neural networks, can provide robust and accurate speech analysis, they are susceptible to adversarial attacks. In this work, we propose a novel end-to-end scheme to generate adversarial examples by perturbing directly the raw waveform of an audio recording rather than specific acoustic features. Our experiments show that the proposed adversarial perturbation can lead to a significant performance drop of state-of-the-art deep neural networks, while only minimally impairing the audio quality.

Audio Adversarial Examples: Targeted Attacks on Speech-to-Text

Abstract: We construct targeted audio adversarial examples on automatic speech recognition. Given any audio waveform, we can produce another that is over 99.9% similar, but transcribes as any phrase we choose (recognizing up to 50 characters per second of audio). We apply our white-box iterative optimization-based attack to Mozilla's implementation DeepSpeech end-to-end, and show it has a 100% success rate. The feasibility of this attack introduce a new domain to study adversarial examples.

Representative voice attack techniques

One important factor of attacks is the adversary's knowledge. White-box attacks assume that the adversary knows all details (e.g., design details and characteristics) of the target, while black-box attacks assume that the adversary does not have such information. Table {{table:aeb79481-fd70-44e3-bd40-9ab251ccd4e5}}  summarizes several attack schemes discussed in this paper, including their type and the required adversary's knowledge.
Hardware level attacks are usually white-box attacks, because the device can be easily dis-assembled and tested, e.g., in {{cite:48f7d50}}, the authors first test the frequency response of the MEMS microphone and then take advantage of its nonlinearity to conduct the attack. All OS level attacks are white-box attacks. Note that all discussed schemes {{cite:53a7340}}, {{cite:32b6924}}, {{cite:6430b3b}} are targeting Android devices, which is not surprising since Android is open-source and its inner workings (e.g., authority management, inter-process communication) are well known. In contrast, the OS of Amazon Echo is closed, which prevents it from being attacked. In fact, it is difficult to perform black-box hardware and OS level attacks.