What Can We Do With Embeddings? In part I of the series, we will explore the following applications of embeddings: Use Text Embeddings To Find Similar TextsUse Visual Embeddings To Find Similar ImagesUse Different Types of Embeddings To Find Similar ItemsUse Different Weights on Embeddings To Find Similar ItemsPut Image & Text Embeddings In the Same Space (Multimodal) Each point builds on the previous point, causing a multiplicative effect. The code snippets are self-sufficient and beginner-friendly to play around with. They can be used as building blocks for creating more complex systems. Now, let us dive deeper into each use case below. I. Use Text Embeddings To Find Similar Texts Text embeddings = representing text (words or sentences) as real-valued vectors in a lower dimensional space. Now, suppose we want to know how similar or dissimilar the above sentences are. We can do that using a text encoder. Here, we use a pretrained DistilBERT by Hugging Face to calculate text embeddings and cosine similarity to compute similarity! We observe that texts containing similar and matching words will give a higher similarity score. On the other hand, texts that are different from each other in terms of word choices, meaning, and context (depending on the model!) will result in a lower similarity score as shown below. Thus, text embeddings aim at using the semantic closeness of words to establish a meaningful relationship. Real-World Applications: Document Search, Information Retrieval, Sentiment Analysis, Search Engine [3] II. Use Visual Embeddings To Find Similar Images Visual embeddings = representing images (that is, pixel values) as real-valued vectors in a lower dimensional space. Instead of text, now we want to know how visually similar the two images are. We can do that using an image encoder. Here, we use a pretrained ResNet-18 to calculate visual embeddings. You can download the images here [4]. Drawing parallels with text, embeddings learned from images find similarity based on similar visual features. For instance, the visual embeddings might learn to recognize a cat's ear, nose, and whiskers giving a score of 0.81. On the other hand, a dog has different visual features than a cat resulting in a lower similarity score of 0.50. Note that there might be some bias based on the images chosen, however, this is the general underlying principle. Thus, visual embeddings aim at using pixel closeness of images to establish a meaningful relationship. Real-World Applications: Recommender Systems, Image Retrieval, Similarity Search III. Use Different Types of Embeddings To Find Similar Items Earlier, we talked about how embeddings are a way of representing input information in a compressed form. This information can be of any type, ranging from various text attributes to visual attributes, and more. Naturally, the more attributes we can use to describe an item, the stronger the signal for similarity. Let us take a simple example and walk through this concept together. Below, we have a dataset of five food items. I have created the attributes myself and generated the swatch colors using the Pillow library (explore the script here). You can download the data here (product images by [5]). For each item, we have 5 textual attributes and 2 visual attributes. For each of these attributes, we can extract an embedding vector that represents that specific attribute. A natural question that crops up now: How do we decide what is similar? For instance, we saw that two sentences can be similar but if we were to visualize them, would they look the same? In contrast, two animals might have a similar score because they look similar, but if we were to characterize them using features, are they still similar? Therein lies the opportunity for leveraging different types of embeddings available to us. We start by creating a single text feature called 'TEXTUAL_ATTR' by simply joining the words. Note that we take a simple example here. In real-world use cases, we would need to apply advanced NLP techniques to clean the text data before concatenating. The GIFs below demonstrate what happens when we take into account either textual embeddings or visual embeddings. Both approaches have their limitations, as outlined above. Therefore, we can turn to the ensemble approach in machine learning by combining the above embeddings to provide as much information about the item as possible. This helps us get stronger similarity signals. For example, we can find similar items by using textual, product image, and product color embeddings. Let us see what happens when we combine all the embeddings! Real-World Applications: Recommender Systems, Substitution Search IV. Use Different Weights on Embeddings To Find Similar Items This concept builds upon point III and is a more flexible extension. Let us take 3 different scenarios as follows: You are looking for a replacement toaster at Costco. You want the same functionality as your previous toaster. In this case, you do not care if your toaster looks the same as your old one, but you do care about the features it provides (that is, textual features). In this case, you would want to use text embeddings only I tore my favorite pants and because I love them so much, I want a replacement that is as close as it can get. I am willing to sacrifice some of the features that my old pants had, but I do care about the aesthetics and how they looked on me. In this scenario, we can place more weight on the visual embeddings and slightly lower weight on the text embeddings.If the given data has certain background elements like a human model, we risk learning the human model's features instead of the actual product we are interested in. In the example below, we want to learn the features of the basketball and not the model holding it. In cases such as these, instead of using image segmentation, we can simply place lesser weight on the full body images, and more on the textual attributes as well as the color of the product to capture the desired embeddings. In each of these cases, using different weights gives us control over which sets of embeddings are important to us, and how to manipulate them. This idea can be used to build flexible systems to solve different types of business problems. Real-World Applications: Bias Reduction, Adjustable Recommender Systems, Adjustable Similarity Search [7][8] V. Put Image & Text Embeddings In the Same Space (Multimodal) So far, we have been looking at text and visual data as entirely separate inputs. The closest we have come is combining the effects at the inference stage in point III. But what if we thought of text and images as objects? How do we collectively represent these objects? The answer is multi-modality and CLIP (Contrastive LanguageImage Pre-training [9]). If you are unsure of what multi-modality means, it simply combines different types of data (text, image, audio, numeric) to make more accurate predictions. Using CLIP, we can represent objects (in this case, text or image) as numeric vectors (embeddings) to help project them to the same embedding space. The true value of the multi-modal nature of CLIP lies in translating similar concepts in text and images into similar vectors. These vectors then get placed close to each other depending on how similar they are. This means that if a text captures the concepts of an image well, it is placed closer to that image. Similarly, if an image is described well by a sentence, it lies close to that sentence. In the example below, the text "dog playing at the beach" outputs a similar embedding to that of an image of a dog playing at the beach. As seen in the illustration above, CLIP unlocks the ability to move across the domain of text and image by combining the semantics of text with the visuals of an image (Image-to-Text, Text-to-Image). We can also stay within a specific domain like Text-to-Text or Image-to-Image. This opens up the possibility for powerful applications ranging from recommendation systems to self-driving cars, document classification, and more! Let us demonstrate a text-to-image example using CLIP by HuggingFace. We will use the images of food items [5], as shown in point III. The idea is to traverse from text to image directly. Watch the video below as I was walk you through a live demo or try it yourself here! We see some cool results in the demo above! The dataset contains two images of the food item cake. Upon looking up "cake", we first get an image containing a chocolate cake. However, when I add more descriptive keywords like "red velvet", it gives me a slice of red velvet. Although both of these images portrayed a cake in a sense, only one of them was a red velvet cake that the model recognized. Similarly, we get a plate of fish and chips when we look up "fish and chips" but only "chips" gives us a box of fries. The model is also able to differentiate between fries and churros even though they look quite similar. Now, let us do an image-to-text in a similar fashion. Here, we take examples [4] from point II to classify whether an image is of a dog or a cat. For each image, we get the probability of the text matching the image. In the case of cat images, the label cat has a high probability. On the other hand, the dog image is classified as a dog. Real-World Applications: Text-to-Image (Lookup), Image-to-Image (Similarity), Image-to-Text (Classification) In part two of the series, we will look at more applications ranging from art to cool visualizations. Stay tuned!


10 Cool Things You Can Do With Embeddings! [Part 1]

References [1]: Definition of embeddings [2]: Embeddings explained [3]: Google's search engine using BERT text encoder [4]: Images from Dogs vs. Cats (Kaggle) [5]: Images from Food Images (Food-101) [6]: Photo by Malik Skydsgaard on Unsplash [7]: Combining different types of embeddings [8]: Embeddings for recommender systems [9]: OpenAI's CLIP [10]: Images by Oscar Sutton, Hossein Azarbad on Unsplash [11]: CLIP blog


This AI newsletter is all you need #62

What happened this week in AI by Louie This week we have been watching development in coding models at META as well as new fine-tuning capabilities at OpenAI. Meta has introduced Code LLaMA, a large language model with the capability to both generate code based on prompts and describe codes. They unveiled three iterations featuring varying parameter counts (7B, 13B, and 34B), each of which has undergone training on an additional 500 billion code-related tokens. The models have compatibility with widely recognized programming languages such as Python, C++, Java, PHP, and others. Furthermore, two specialized models are built on top of it. Code LLaMA - Instruct is the tuned iteration of Code LLaMA, designed to follow instructions. Additionally, there is Code LLaMA - Python, a dedicated model tailored for the Python programming language. The models have been made available under licenses for both research and commercial use. The open source release of this model enables rapid iteration and already we have seen further models built on top of it including WizardCoder, which demonstrated superior performance compared to the majority of existing language models, coming close to but not quite reaching the level of GPT-4. In other big news, OpenAI has introduced the capability to fine-tune GPT-3.5-turbo as a service. It's important to highlight that utilizing the fine-tuned model for inference comes with a significant increase in cost. OpenAI has also unveiled ChatGPT Enterprise, offering unrestricted usage, enhanced speed, and an extended context window for organizations. We were happy to see the release of Code LLaMA and think there is huge potential for models fine-tuned and optimized for coding to make significant improvements - both as co-pilot tools for developers and for opening up software development to non-developers. The GPT Turbo fine-tuning release is also exciting, and we expect will lead to some high-quality coding and fine-tuned models also, but we are particularly excited by the prospect of fine-tuning for GPT-4 opening up later this year. We think a GPT-4 fine-tuned for coding has the potential to be incredibly powerful. - Louie Peters - Towards AI Co-founder and CEO


This AI newsletter is all you need #62

Hottest News GPT-3.5 Turbo Fine-Tuning Released OpenAI has introduced fine-tuning for GPT-3.5 Turbo, which delivers enhanced performance on specific tasks. This refined version can potentially equal or even surpass the capabilities of the base GPT-4 model. Initial testers have managed to substantially reduce prompt length through the fine-tuning process. The costs for training and usage input/output are provided at $0.008, $0.012, and $0.016 per 1K tokens, respectively. 2. Introducing Code Llama, a State-of-the-Art Large Language Model for Coding Meta has released Code Llama, an advanced LLM for coding that can generate code and natural language related to code. It is available in three models and various sizes to meet different needs. It supports many of the most popular languages used today, including Python, C++, Java, PHP, TypeScript (JavaScript), C#, and Bash. 3. Introducing a Foundational Multimodal Model for Speech Translation Meta has developed a robust foundational model known as SeamlessM4T, which is capable of managing diverse text and speech tasks across 100 languages. It encompasses automatic speech recognition, speech-to-text translation, speech-to-speech translation, text-to-text translation, and text-to-speech translation, supporting a wide range of input and output languages. 4. Introducing ChatGPT Enterprise OpenAI has launched ChatGPT Enterprise, providing security and privacy features suitable for enterprise use. This version offers unlimited access to GPT-4 at higher speeds, extended context windows (32k) for handling longer inputs, advanced data analysis capabilities, customization options, and additional features. 5. Alibaba Releases a New Chatbot That Can 'Read' Images Alibaba Cloud has unveiled two open-source AI models: Qwen-VL and Qwen-VL-Chat. These models are trained using the company's Tongyi Qianwen (Qwen) LLM. They can interpret visual data, like text in images, and respond to location-based queries, like offering directions by interpreting images of signs.


This AI newsletter is all you need #62

Five 5-minute reads/videos to keep you learning Making LLMs Lighter With AutoGPTQ and Transformers Hugging Face has introduced AutoGPTQ integration in Transformers, facilitating streamlined 2, 3, 4, and 8-bit quantization with negligible accuracy reduction. This integration is compatible with Nvidia GPUs as well as RoCm-powered AMD GPUs. 2. Teaching Language Models To Reason Algorithmically This paper explores the effectiveness of teaching algorithmic reasoning to LLMs, focusing on overcoming challenges such as overfitting and spurious correlations. It proposes a four-step approach that includes formulating algorithms as skills, teaching multiple skills simultaneously, teaching skill composition, and teaching the use of skills as tools. 3. Code Llama on Hugging Face Code Llama is now accessible via Hugging Face, offering the capability to execute code infilling utilizing the 7B and 13B models. It has been made available under the same permissive community license as Llama 2 and is open for commercial utilization. 4. Language to Rewards for Robotic Skill Synthesis Language-to-reward systems, fueled by LLMs, empower robots to learn directly from language. These systems translate natural language instructions into reward-specifying codes, compute rewards based on robot actions, and facilitate learning through reinforcement learning (RL). 5. What Is MetaGPT? LLM Agents Collaborating To Solve Complex Tasks MetaGPT represents a novel approach to enhancing collaborations among AI agents. This video reveals the inner workings of MetaGPT's innovative design, delves into the role of SOPs (Standard Operating Procedures), and explores how multiple AI agents collaborate seamlessly.


This AI newsletter is all you need #62

Papers & Repositories Reinforced Self-Training (ReST) for Language Modeling This paper presents a straightforward algorithm for aligning LLMs with human preferences, drawing inspiration from growing batch reinforcement learning. Reinforced Self-Training (ReST), developed by DeepMind, offers a more economical alternative to RLHF. It employs a two-step process, Grow and Improve, to enhance the training dataset and fine-tune the LLM. 2. Giraffe - Long Context LLMs Giraffe is a new series of models derived from LLaMA and LLaMA2, encompassing variants with context window sizes of 4k, 16k, and 32k tokens. These models have undergone fine-tuning based on LLaMA and LLaMA2, and they feature experiments involving the expansion of the context window through positional encoding modifications. 3. New LLM Platypus Tops the Hugging Face LLM Leaderboard Platypus, the latest LLM featured on HuggingFace's Open LLM Leaderboard, leverages the Open-Platypus dataset to attain impressive performance in STEM and logic. It effectively addresses bias during training by utilizing LoRA modules and the PEFT library. However, its challenge with languages beyond English is attributed to its underlying model, LLaMa-2. 4. Graph of Thoughts: Solving Elaborate Problems With Large Language Models Graph of Thoughts (GoT) represents a framework that enhances the prompting capabilities of large language models (LLMs) beyond what paradigms like Chain-of-Thought or Tree of Thoughts (ToT) provide. GoT has showcased enhanced performance in comparison to alternative methods, notably enhancing sorting quality (62%) while concurrently reducing costs (31%). 5. QuIP: 2-Bit Quantization of Large Language Models With Guarantees This paper introduces quantization with incoherence processing (QuIP), a new approach that achieves 2-bit quantization of language model models using adaptive rounding. It stands as the first algorithm of its type to be accompanied by a theoretical analysis, demonstrating its potential influence on other quantization methods, such as OPTQ. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #62

Meme of the week! Meme shared by neon8052


This AI newsletter is all you need #62

Featured Community post from the Discord DrDub has initiated a remarkable project named "Tell-and-Show," which serves as an experiment in community-owned machine learning. The project builds recommendation profiles that are exclusively yours. It also provides tools and models available for adoption by other Free Software projects, to enhance the utility of these recommendation profiles. Check it out here and support a fellow community member! You can help this project by sharing your individual preferences over key items or joining as a volunteer. Share your questions and feedback in the thread here.


This AI newsletter is all you need #62

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #62

Article of the week A Framework For Efficiently Serving Your Large Language Models by Zoumana Keita Deploying large language models is undoubtedly one of the most challenging tasks, not because the deployment teams are incompetent, but simply due to the complexity of deploying these types of models. This is where the vLLM library comes in handy, an open-source library developed by UC Berkeley under the Apache license. The philosophy behind vLLM is to make serving and inference of large language models affordable for both industry and small research teams.


This AI newsletter is all you need #62

Our must-read articles Building Intuition on the Concepts behind LLMs like ChatGPT - Part 1- Neural Networks, Transformers, Pretraining, and Fine Tuning by Stephen Bonifacio Generative Adversarial Networks (GANs) for Image Augmentation by Tan Pengshi Alvin Top Important Computer Vision for the Week from 14/8 to 20/8 by Youssef Hosni If you want to publish with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #62

Job offers Waitlist: Mentor - Data Engineering Career Track @Springboard (Part-time/Remote) Intermediate Software Engineer @Datacom (Remote) Lead Software Developer @TherapyNotes.com (Remote) Software Engineer @Sonovate (Remote) Lead Machine Learning Engineer, Recommendation Algorithm @Multi Media LLC (Remote) Senior Software Engineer, Actions Deployment @GitHub (Remote) Software Engineer - Intern @Bayut  dubizzle (Dubai, UAE) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Inside Code Llama: Meta AI's Entrance in the Code LLM Space

Inside Code Llama The release of Code Llama does not include a single model but three different variants, characterized by their parameter sizes of 7B, 13B, and 34B. Each of these models has been trained on an extensive pool of 500B tokens encompassing code and code-related information. Notably, the 7B and 13B base and instruct models have been endowed with fill-in-the-middle (FIM) competence, empowering them to seamlessly insert code into existing code structures. This attribute equips them to handle tasks like code completion right from the outset.The trio of models caters to distinct requisites concerning serving and latency. For instance, the 7B model boasts the ability to operate on a single GPU. While the 34B model stands out for yielding optimal outcomes and elevating coding assistance, the smaller 7B and 13B versions excel in speed, making them fitting for low-latency tasks such as real-time code completion. Meta AI's innovations further extend to two nuanced adaptations of Code Llama: Code Llama - Python and Code Llama - Instruct. Code Llama - Python is a specialized derivation, meticulously honed on a substantial volume of Python code spanning 100B tokens. Given Python's central role in code generation benchmarks and its significance within the AI community, this focused model augments utility.Code Llama - Instruct represents an alignment and refinement of Code Llama through instructional fine-tuning. This novel training approach entails furnishing the model with "natural language instruction" inputs paired with anticipated outputs. This strategic methodology enhances the model's capacity to grasp human expectations in prompts. For endeavors involving code generation, it is advised to opt for Code Llama - Instruct versions, as they have been calibrated to yield useful and secure natural language responses. Deep diving into the Code Llama training and fine-tuning, there are a few aspects that are worth highlighting 1) DatasetLlama's training rests on a meticulously curated dataset enriched with publicly available code, offering a near-duplicate-free landscape. The dataset consists of 500B tokens during the initial phase, starting from the 7B, 13B, and 34B versions. A supplementary 8% of sample data is garnered from natural language datasets linked to code domains. 2) InfillingWithin the realm of Code Infilling, a pivotal task revolves around predicting missing segments within a program while being guided by contextual surroundings. Pragmatic applications encompass code completion within Integrated Development Environments (IDEs), type inference, and even the generation of in-code documentation such as docstrings. Operating in alignment with the concept of causal masking, a framework expounded by Aghajanyan et al. (2022) and Fried et al. (2023), Meta AI molds infilling models. The training process entails shifting parts of training sequences to the conclusion, paving the path for autoregressive predictions. In this endeavor, both the versatile 7B and 13B models undergo infilling-oriented training, echoing the strategies advised by Bavarian et al. (2022). 3) Long Context Fine-Tuning:Unraveling the intricacies of handling extensive sequences is a formidable pursuit in the realm of transformer-based language models. The pivotal challenges orbit around extrapolation - delving into sequence lengths beyond those encountered during training - and the quadratic complexity of attention passes that tilts the balance towards short-to-medium inputs for effective training. Meta AI steps forward with a unique solution, introducing the dedicated domain of long context fine-tuning (LCFT). Embracing sequences encompassing 16,384 tokens, a substantial leap from the 4,096 tokens featured in Llama 2's initial code training stages, LCFT empowers models with extended-range capabilities. This strategic shift occurs within a fine-tuning phase, circumventing undue escalation in training costs. 4) Instruction Fine-Tuning:Code Llama's prowess extends to instruction fine-tuning, witnessed in the refined Code Llama - Instruct models. This iteration leverages Code Llama as its foundation, sculpted to aptly respond to queries. Merging Supervised Fine-Tuning with an expansive pool of Rejection Sampling examples yields this instructive competence. 5) Self-InstructIn the realm of datasets, Meta AI embarks on a proprietary journey, curating instances tethered to code-related tasks. In recognition of the resource-intensive nature of acquiring data from human annotators or through human feedback, a particular emphasis on self-instruction is embraced. The domain of coding tasks, steeped in the insights of professional developers, forms the canvas on which this innovative approach is painted.


The Results The evaluate Code Llama, Meta AI engaged two widely acknowledged coding benchmarks: HumanEval and Mostly Basic Python Programming (MBPP). The HumanEval benchmark systematically assesses the model's prowess in code completion via docstrings, while the MBPP benchmark scrutinizes the model's capacity to translate descriptions into executable code.The meticulous benchmarking endeavor unfolded illuminating results: Code Llama outshone open-source, code-centric Large Language Models (LLMs) and even outperformed its predecessor, Llama 2. For instance, in the case of Code Llama 34B, remarkable scores emerged - an impressive 53.7% on the HumanEval benchmark and a formidable 56.2% on the MBPP benchmark. These scores stood as the highest amongst comparable state-of-the-art solutions, positioning Code Llama 34B on par with the notable capabilities of ChatGPT. Code Llama promises to be one of the most important code LLMs in the near future. It certainly contributes to reaffirm the value of open-source foundation models across different domains.


'Chat With Company Documents' Using Azure OpenAI

Deployment Options The solution can be deployed as a web app or as a power virtual agent (PVA) bot directly from the Azure OpenAI Chat playground. The data source needs to be set up and then deployed based on the two available deployment options.


Web App The web app is a basic chat interface that's accessible via browsers on laptops or mobile devices. The web app has been released as open source by Microsoft with an MIT license, which permits commercial use. The GitHub repository can be found here. Customers are free to customize the application as required. The web app can be deployed 'as-is' in one click as an app service hosted in Azure. This will be deployed with the Azure logo and default color schemes.


Power Virtual Agent The PVA bots can be deployed on company websites. It also has native integration and can be easily embedded into other MS products such as Sharepoint sites and MS Teams.


Loading company documents to the Chatbot's knowledge database The following document types can be loaded into the bot's database: txt, PDF, Word files, PowerPoint files, and HTML. It's recommended to use text-rich documents like company policies, contracts, how-to instructions, etc., for this chatbot. Tabular data, such as those stored in Excel/CSV or relational databases, is not recommended. The uploaded documents will be stored in a storage blob within the customer's Azure environment and will be kept private as well. The chatbot queries data from an index in the Azure Cognitive Search service. It has an intuitive indexer that can load all supported document types from a single location (e.g., a local folder or an Azure Blob) and into the index. The search service (which compares the user query/input versus the relevant document from the index that will be used as the basis for the chatbot's response) supports lexical or keyword comparison based on the BM25 search algorithm as well as embedding/vector-based search algorithms like cosine similarity or dot product. It also supports a hybrid of these two - a combination of full-text search and vector search. References: Azure OpenAI on your data (preview)


Talk to your documents as PDFs, txts, and even web pages

Content Table · Introduction· How does it work· Steps (part 1) · Break: Let's recap (part 1) · Steps (part 2) · Break: Let's recap (part 2) · Web App· For the impatient one (code)· Conclusions· References


Introduction We all have to read eternal documents to get two sentences containing the value/information we need. Look no further! Welcome to the project "Talking to Documents Using LLM". It's like recovering treasures from PDFs, TXT files or web pages without brain fatigue. And don't worry, you don't need a tech wizarding degree to have fun. We've created a user-friendly interface with Streamlit, so even your non-tech-savvy friends - marketing assistants - can join in. This article will dive deeper into the theory and code behind application intelligence, shedding light on how web applications work. To give you a quick overview of the technologies we'll be using, check out the following image that shows the four main tools in action. And of course, you can find the code in my GitHub repository, or if you want to test the code directly, you can go to the "For the impatient (code)" section.


How does it work Let's take a peek through the veil of "Talking to Documents Using LLM".This project is like an interesting duo: a smart player (back-end) and a user-friendly website (front-end). Smart reader: The AI brain reads and understands documents like a super smart friend, excellently handling PDFs, TXTs and web content.User-friendly website: User-friendly web interface to configure and interact with the model. It consists of two main pages, each with a specific purpose: Step 1⃣ Create database and Step 2⃣ Request documents. Our project's outline looks something like this: Although it may seem complicated, we will simplify every essential step to make it functional. By focusing on the workings of intelligence, we will discover how it works. These steps are supported by the Python TalkDocument class and we will demonstrate each step with the corresponding code to bring it to life.


Steps (part 1)  Get ready to explore every step of the way as we discover what makes the magic behind the scenes!


1- Import documents This step is obvious, right? This is where you decide what type of material you provide. Whether it's PDF, plain text, web URL or even raw string format, it's all on the menu. Depending on the type of file you upload, the document will be read by different methods. An interesting improvement could involve adding automatic format detection to the mix.


2- Split Type Now, you may be wondering why the term "type" exists? Well, in the app, you can choose the split method. But before diving into this topic, let's explain what document splitting is. Think of it like this: Just as humans need chapters, paragraphs, and sentences to structure information (imagine reading a book with an endless paragraph - yes!), machines need structure too. We need to cut the document into several smaller parts to understand it better. This parsing can be done by character or by token.


3- Embedding type We humans can understand words and pictures easily, but machines need a little more guidance. This becomes obvious when: We are trying to convert categorical variables in a dataset into numbersImage management in neural networks. For example, before an image is fed into a neural network model, it undergoes transformations to become a numerical tensor. As we can see, mathematical models have the language of numbers. This phenomenon is also seen in the field of NLP, where the concept of word embedding is promoted. This conversion or encoding is done using specialized algorithms. It is important to note that this process converts sentences into digital vectors and that this encoding is not random; it follows a structured method. This code is very simple:We instantiate the object responsible for the integration. It's important to note that at this point, we're just creating an Integration Object. In the next steps, we will do the actual conversion.


Break: Let's recap (part 1)  To grasp the initial three steps, let's consider the following example: We start with an input text. We do a distribution based on the number of characters (about 50 characters in this example).We do integrations, turning text snippets into digital vectors. How wonderful!  Now, let's start the thrilling journey through the remaining levels. Fasten your seat belts because we're about to unlock some real tech magic!


Steps (part 2)  We will continue to see the steps to follow.


4- Model VectorerStore Type Now that we've turned our text into code (embedded), we need a place to store them. This is where the concept of "vector store" comes in. It's like a smart library of these codes, making it easy to find and retrieve similar codes when asking a question. The creation of this type of database is managed by specialized algorithms designed for this purpose, such as FAISS (Facebook AI Similarity Search). There are other options as well, and currently, this class supports CHROMA and SVM. This step and the next step share code. In this step, you choose the type of vector repository you want to create, while the next step is where the actual creation takes place.


5. Model VectoreStore (Creation) This type of database handles two main aspects: Vector storage: It stores the vectors generated by the integration.Similar calculation: It calculates the similarity between vectors. But what exactly is the similarity between these vectors and why does it matter? Well, remember I mentioned that the integration is not random? It is designed so that words or phrases with similar meanings have similar vectors. This way we can calculate the distance between the vectors (like using Euclidean distance) and this gives us a measure of their "similarity". To visualize this with an example, imagine we have three sentences. Two are related to recipes, while the third is related to motorcycles. By representing them as vectors (thanks to integration), we can calculate the distance between these points or sentences. This distance serves as a measure of their similarity. In terms of code, let's take a look at the required requirements: The split textThe embedding typeThe vector store model To illustrate what happens in this step, we can visualize the following image. It shows how encoded text fragments are stored in the vector store, allowing us to calculate the distance/similarity between vectors/points.


Break: Let's recap (part 2)  Great! We now have a database that is capable of storing our documents and calculating the similarity between encrypted text chunks. Imagine a situation in which we want to encode an external sentence and store it in our vector store. This will allow us to calculate the distance between the new vector and the document split. (Note that here the same integration should be used to create the vector store.) We can visualize this through the following image. Remember the previous picture... we have two questions, and we turn them into numbers using integration. We then measure the distance and find the closest sentences to our question. These sentences are like perfect combinations! It's like finding the best puzzle piece in an instant!


6- Question Remember that our goal is to ask a question about a document and get an answer. In this step, we collect the questions as user-supplied input.


7 & 8- Relevant Splits This is where we insert our question into the vector repository, embedding it to turn the sentence into a digital vector. We then calculate the distance between our question and the parts of the document, determining which parts are closest to our question. The code is: Try it with the code below. Learn how to answer as a list of 4 items. And when we look inside, it's actually separate parts of the document you entered. It's like the app that presents you with the best 4 puzzle pieces that fit your question like a glove!


9- Response (Nature Language) Great, now we have the most relevant text for our question. But we can't just hand those parts over to the user and terminate it. We need a concise and precise answer to their question. And that's where our Language Model (LLM) comes into play! There are many LLM flavors. In the code, we set "flan-alpaca-large" as default. Don't hesitate to choose the person you are most touched by!  Here is the plan: We restore the most important parts (splits) related to the question.We prepare a prompt that includes the question, how we want the answer, and those text elements.We pass this prompt to our Intelligent Language Model (LLM). He knows the question and the information contained in these elements and gives us natural answers. This last part is shown in the following image:  Sure enough, this exact flow is what gets executed in the code. You'll notice in the code there's an extra step involved in creating a "prompt". Indeed, we can come up with a final answer that is a combination of the answers found by the LLM and other choices. For simplicity, let's do it the simplest way: "stuff". This means that the answer is the first solution found by the LLM. Here, you don't get lost in theory!


Web App This interface is designed for people without technical knowledge, so you can get the most out of this technology without any problems.  Using the site is extremely simple and powerful. Basically, the user just needs to provide the document. If you are not ready to benefit from additional parameters, in the next step, you may already start asking questions.  Let's see the steps to follow: Provide the document or web link. By page Step 1⃣ Create Data Base.Set up the configuration (optional). By page Step 1⃣ Create Data Base.Ask your questions and get answers!  By page Step 2⃣ Ask to the document.


1. Provide the document or web link. In this step, you add the document by uploading it to the web. Remember that if you don't have the Face Hugging API key as an environment variable, the "home" tab will ask you to enter it.  Once you've attached the document and set up your preferences, a summary table of your configuration is displayed. The "Create Database" button is then unlocked. When you click the button, the database or vector store will be created, and you'll be redirected to the question section.


2. Set up the configuration (optional). As I mentioned earlier, we can configure the vector store according to our preferences. There are different methods for splitting documents, embedding, and more. This tab allows you to customize it as you like. It comes with a default configuration for faster setup.


3. Ask your questions and get answers! At this point, you can start making all the queries you want. Now it's time to enjoy and interact with the tool to your heart's content!


For the impatient one (code) For those who are eager to dive in, you can directly grab the TalkDocument class and paste it into a Jupyter notebook to start tinkering. You might need to install a few dependencies, but I'm sure it won't be a challenge for you. Have a great time exploring and experimenting! Happy coding!!


Conclusions Congratulations on making it this far in our exciting journey! We've delved into how the intelligence behind this web application works. From uploading documents to obtaining answers, you've covered a lot of ground! If you're feeling inspired, the code is available in my GitHub repository. Feel free to collaborate and reach out to me on LinkedIn for any questions or suggestions! Have fun exploring and experimenting with this powerful tool! If you want, you can check my GitHub


References Langchain documentationHugging Face DocumentationIntroduction to Facebook AI Similarity Search (Faiss)Steamlit Documentation


Transformer Architectures and the Rise of BERT, GPT, and T5: A Beginner's Guide

The Magic Behind Transformers In our daily lives, we're constantly bombarded with information. From the hum of traffic outside our windows to the buzz of conversations in a cafe, our senses pick up a myriad of stimuli. Yet, amidst this cacophony, our brains possess a remarkable ability: the power of selective attention. If you've ever found yourself engrossed in a book while a party rages around you, or if you've managed to pick out a familiar voice in a crowded room, you've experienced this firsthand. This innate human ability to focus on what's crucial and filter out the noise is the essence of the magic behind the Transformer architecture in AI. At a fundamental level, the Transformer is designed to handle sequences of data, much like a series of events or a string of thoughts. Traditional models, when faced with sequences like sentences or paragraphs, would process them much like reading a book word by word, linearly and in order. While effective to a degree, this method often missed the broader context, the intricate dance of meaning between words spaced far apart. It's akin to understanding the plot of a novel by only reading every tenth page; you'd get some of the story, but miss out on the depth and nuance. Enter the Transformer. Instead of being bound by this linear approach, it can, metaphorically speaking, read multiple parts of a book simultaneously. It can focus on the introduction while also considering the climax, drawing connections and understanding relationships that a linear read might miss. This is achieved through what's known as the 'attention mechanism'. Just as our brains weigh the importance of stimuli, deciding what to focus on, the Transformer weighs the significance of different parts of a sequence. Let's consider a practical example. Imagine the sentence: "Jane, who grew up in Canada, is fluent in both English and French." A traditional model might first focus on "Jane" and then move to "Canada", taking time to understand the relationship between the two. The Transformer, however, can instantly recognize the connection between "Jane" and "Canada", while simultaneously understanding the significance of her fluency in "English and French". It grasps the entire context, the full story behind Jane's linguistic abilities, in a holistic manner. This capability becomes even more crucial in complex scenarios. Consider a mystery novel where a clue in the first chapter is only resolved in the last. While a linear approach might forget the initial hint by the time the conclusion rolls around, the Transformer retains and connects these distant pieces of information, much like an astute detective linking disparate clues to solve a case. Moreover, the Transformer's magic isn't limited to just text. It's been applied to a range of data types, from images to sounds. Think of watching a movie and understanding the significance of a character's gesture based on a flashback scene, or listening to a symphony and recalling a recurring motif. The Transformer can do this with data, drawing connections, recognizing patterns, and providing a depth of understanding previously unattainable. In essence, the Transformer has redefined the rules of the game in AI. It doesn't just process information; it understands context, relationships, and nuances, bridging gaps and illuminating connections. It's a leap forward, a shift from mere computation to genuine comprehension.


BERT: The Context Whisperer Language, in its essence, is a tapestry of words woven together by the threads of context. Every word we utter or write carries weight and meaning, often shaped by the words that surround it. This intricate dance of words and meanings is what BERT, an acronym for Bidirectional Encoder Representations from Transformers, is designed to understand and interpret. Imagine reading a novel where a character says, "I'm feeling blue today." Without context, one might visualize the color blue. However, with an understanding of language nuances, it's clear the character is expressing sadness. This is the kind of contextual understanding that BERT brings to the table. Instead of analyzing words in isolation, BERT looks at them in relation to their neighbors, both preceding and following. It's like reading both the left and the right page of a book simultaneously to grasp the full story. Let's delve into another example. Consider the sentence: "I went to the bank to withdraw money." Now, compare it with: "I sat by the river bank and watched the sunset." The word 'bank' appears in both sentences, but its meaning shifts dramatically based on context. Traditional models might struggle with such nuances, but BERT shines. It recognizes the different implications of 'bank' in each scenario, ensuring accurate interpretation. This bidirectional approach of BERT is akin to having two flashlights in a dark room, one shining from the start of a sentence and the other from the end, illuminating the words from both directions. The result? A well-lit room where the meaning of each word, influenced by its neighbors, becomes crystal clear. BERT's prowess in understanding context has made it a cornerstone in numerous AI applications. From search engines that better grasp user queries to chatbots that respond with uncanny accuracy, BERT is reshaping our digital interactions. It's not just about recognizing words; it's about understanding the stories they tell when strung together.


GPT: The Storyteller In the annals of human history, storytelling has been a powerful tool. From ancient campfires to modern-day cinemas, stories shape our understanding, evoke emotions, and bridge cultures. In the realm of AI, GPT, which stands for Generative Pre-trained Transformer, emerges as a digital storyteller, weaving tales and crafting narratives with a finesse that often feels eerily human. Imagine sitting around a campfire, starting a tale, and then passing the torch to someone else to continue. GPT operates on a similar principle, but in the vast landscape of language. Feed it a sentence or a phrase, and it takes the baton, continuing the narrative in ways that are coherent, contextually relevant, and often creatively surprising. It's like having a co-author that never tires, always ready to pick up where you left off. Let's consider a practical scenario. If you were to give GPT the beginning of a story, such as "In a town where magic was forbidden, a young girl discovered a mysterious book in her attic," GPT could spin a tale of adventure, intrigue, and suspense, detailing the girl's journey, the challenges she faces, and the secrets the book unveils. It doesn't just add sentences; it builds a world, populates it with characters, and charts a narrative arc. This ability of GPT to generate text isn't limited to just stories. It can craft poems, answer questions, write essays, and even generate technical content. Its versatility stems from its training on vast amounts of diverse text, enabling it to don multiple hats - from a novelist to a poet, from a journalist to a tutor. In essence, GPT is not just a model; it's a digital bard. In its strings of code and algorithms, it carries the legacy of ancient storytellers, blending it with modern-day AI capabilities. It's a testament to how far we've come in the journey of AI, where machines don't just compute but also create.


T5: The Swiss Army Knife In the world of tools, the Swiss Army knife stands out, not because of its size or its singular function, but due to its incredible versatility. It's compact, yet packed with tools ready to tackle a myriad of tasks. Similarly, in the digital realm of AI, T5, short for Text-to-Text Transfer Transformer, emerges as the versatile multi-tool, adept at handling a diverse range of linguistic challenges. Imagine having a single tool that could seamlessly translate languages, summarize lengthy articles, answer intricate questions, and even rewrite content in a different tone. That's T5 for you. Instead of being designed for one specific task, T5 approaches challenges with a unique perspective: it views every problem as a text-to-text task. Whether it's converting a question into an answer or translating English to Mandarin, T5 perceives it as transforming one sequence of text into another. For instance, give T5 a complex scientific article and ask it for a summary. It reads the detailed content and distills it into a concise, understandable version. Or pose a question about a historical event, and T5 sifts through its knowledge to craft a relevant answer. Its adaptability and wide-ranging capabilities make T5 a standout, much like the Swiss Army knife in a world of specialized tools.


Why Does All This Matter? The rise of Transformer-based models like BERT, GPT, and T5 has significantly impacted our daily lives. From the chatbots that assist us on websites to the voice assistants that answer our queries, these models play a pivotal role. Their ability to understand and generate human language has opened doors to countless applications. Businesses can offer better customer support, content creators can get AI-driven suggestions, and researchers can analyze vast amounts of text quickly. The Transformer architecture, with its unique approach to data and attention, has reshaped the landscape of AI. These models have set new standards in understanding and generating human language. As we continue to innovate and refine these models, the line between human and machine understanding of language might become even blurrier, heralding a future where AI truly understands us.


Deep Learning Applied to Physics and Fluids

Computational Fluid Dynamics simulations A common numerical simulation can describe physical systems by solving a set of Partial Differential Equations (PDE), which typically have the form: 𝐹 represent s the differential operator over the domain Ω ∈ ℝ , bounds 𝜕Ω and parameters 𝑞 . The solution 𝑠(𝑥, 𝑡) of the system relies on spatial coordinates and time, with subscripts denoting partial derivatives. The set of equations can be solved by discretizing the physical domain into small parts (finite elements or finite volumes) to get a linearized system. This approach finds particular application in fluid dynamics. In fluid dynamics, the system is represented mainly by the Navier-Stokes equations, a set of laws with no analytical resolution to describe the behavior of every fluid based on the mass and force balances. In a simpler 2D form, they can be described as: Where 𝑢 is the velocity along x-axis, 𝑣 is the velocity in y, 𝑝 is the pressure, 𝜌 is the density and 𝜈 is the viscosity. Computational Fluid Dynamics (CFD) simulations consist of resolving the discretized linearized system along with its boundary conditions, such as the pressure and velocity at the limits of the domain, by iterative multigrid solution methods. Direct methods are impractical for real-world applications, where the inversion of the matrix for a 3D Cartesian and equally spaced grid (i³ elements) achieves i⁷ complexity. Even with efficient solvers working in an HPC parallel environment, the computational cost of such operations can achieve long hours and become detrimental for a dynamic engineering process. The solution? As we see more and more, AI!


Surrogate models When a possible input-output relationship is present, AI arises as a candidate to model such behavior. This scenario aligns perfectly with CFD, where the geometric setup, parametrized as a grid and its elements, in addition to the boundary conditions, can be linked to the output: the physical fields (pressure, velocity, etc.) in each point of the grid. The models built can act as meshless solvers which can replace traditional simulators with a lower computational cost. In a general manner, we want to learn the mapping between the PDE parameters (𝑥, 𝑞, 𝑡): 𝑋 ∈ ℝⁿ and its solution 𝑠(𝑥, 𝑡): 𝑌 ∈ ℝⁿ . In other words, we aim to find the predictive function 𝐺: (𝑥, 𝑞, 𝑡) -> 𝑠(𝑥, 𝑡) where 𝑡 is sometimes found to be constant (steady state analysis). Thus, we can imagine different ways to do that.


Simplified models The easiest approach to model the relationship is to simplify it by reducing the dimensionality of the data. This method can be applied both to the input and the output. For instance, instead of using the full coordinates of grid points, we can represent the previously described geometry with a reduced set of parameters, denoted as 𝑘 ∈ N  𝑘 < n. A gear has a number of teeth, a primitive radius, a width, etc. For the output, a viable choice is a global performance metric denoted as s(𝑘) ∈ N. Examples are the forces acting on the prototype, the drag and lift coefficient, etc. The upside of the simplifications is that it allows us to apply more basic and faster AI models. Even a linear/polynomial regression could be used with no bigger issues to learn the function 𝐺: k -> 𝑠(k). The downside is that, when reducing the dimensionality this way, there's an intrinsic loss of information, and the models become less generalizable when facing data outside of the design space.


Volumetric models Instead of reducing the dimensionality of the data, we can opt to work with the original volumetric grid (𝑌 ∈ ℝⁿ), which introduces greater complexity and requires the utilization of Deep Learning techniques. When dealing with an unstructured grid, a common approach involves interpolating it onto a uniform structured mesh. Essential features like freestream velocity and pressure can be embedded within each voxel, enabling corresponding predictions. Consequently, regions devoid of fluid are represented as null values, thereby encoding the geometry. This voxel-based representation facilitates the use of a technique commonly employed in image recognition tasks: convolution. Convolutional Neural Networks (CNNs) can thus extract local and global features via their filtering approach. Varying scales of feature extraction can be achieved by integrating different stages, leading to increasingly intricate models, including u-nets and auto-encoders/decoders. Instead of transforming the unordered data to the ordered shape of voxels/pictures, another solution involves actively encoding the coordinates. This permits data description in tabular format, where each point is associated with the corresponding simulation/example index. As we are going to see in the next section, this is what pytorch geometric does! While it is theoretically plausible to train and apply a model to each point, this approach normally fails to capture inter-node relationships critical for determining local information. Enter Graph Neural Networks, constituting another class of models tailored to address this limitation.


Geometric models In CFD we are often interested in determining the physical fields not in the volume but in the surface. This prompts a consideration of model-building strategies. While the voxelization method outlined earlier has shown promising results, its application to represent sparse point clouds corresponding to the geometry requires the creation of a uniform structured grid. However, this becomes impractical for intricate geometries, as it leads to either unnecessary computational costs due to encompassing irrelevant information around the shape, or a loss of valuable information due to a coarse grid. A more efficient solution is offered by the field of Geometric Deep Learning, which is mostly known for its success in object recognition and semantic segmentation. This approach is closely linked to Graph Neural Networks and directly treats a Point Cloud - an unordered set of points - describing the data as a table of coordinates and indexes as we mentioned in the last section. Luckily, that's exactly what we have with an unstructured mesh! In order to do describe the data in such a format, we can use pytorch geometric, an extension of the state-of-the-art pythorch framework tailored for GNNS and geometric models. It also contains a series of graph datasets we can use, like AirfRANS⁴, a 2D NACA wings dataset for RANS simulations. Let's briefly explore how this translates into actual code: With a grasp of the dataset's nature, the question arises: How can we train models using it? We transform it to the shape described above with a little help of Pytorch Geometric. The DataLoader allows us to loop through the dataset in mini-batchs thanks to the batch attribute, a vector that maps each node to its respective graph in the batch. . This proves crucial for employing aggregation functions in each simulation, as we'll delve into shortly. Having established the data description, what types of models are suitable? Pytorch geometric offers a range of pre-designed neural layers and operations that can be very useful. One of the most traditional ones is, perhaps, PointNet¹. One of the pioneers of the field, this architecture proposed in 2017 introduced two major advantages that are until these days, very important. The key to Pointnet's success lies in its pooling operation. By condensing information of an entire simulation into a single vector, it captures global insights and enables the transition from multiple points to a singular global value. This effectively addresses the issue, for example, of applying a model to a single point and not being able to change the dimensions of the model. In other words, we'll consider the dataset 𝐷 = {Dᵢ  i ∈ N, 1 ≤ i ≤ m}, m being the total number of simulations. We can define for each simulation Dᵢ the set of points X = {Xⱼ(pⱼ, fⱼ)  j ∈ N , 1 ≤ i ≤ n}. The concatenated feature vector (pⱼ, fⱼ) ∈ ℝʰ  h ∈ N. In this way, pooling performs the following transformation: This is necessary for classification tasks where the Point Cloud should be reduced to a single value or vector. Another advantage of pooling is infusing global information into each point, useful in segmentation tasks. The global information vector can be concatenated to the feature vectors of each point, increasing its dimension. In this way, the model applied to each point will not only have its punctual information, but a general idea of the entire example. Formally, considering z as the total number of points in the unstructured dataset, the series of operations Pointnet architecture performs can be described, in a simplified way: Where the same MLP is applied to each point/row and columns represent the features. More complex models were proposed in the sequence. Pointnet++², for example, proposed the idea of using Pointnet at different hierarchy levels. This approach not only concatenates global features but also incorporates vectors representing smaller-scale phenomena. This proves particularly valuable when tackling CFD problems. For instance, in simulating an entire aircraft, distinct behaviors arise across different geometrical components. Wings exhibit different phenomena compared to turbines. Even on a finer scale, variations exist between the back and front of a wing, leading to characteristic pressure distributions. Graph Neural Networks generalize the aforementioned concepts by dby depicting data as edges and vertices, reflecting relationships between nodes. In this way, it is equivalent to CNNs in a way that it aggregates local features but, instead of using a regular grid, it can be directly applied to an unordered set of points. A typical GNN encodes a node as vertices (vⱼ = Xⱼ), having its coordinates as features. The edge represents the connectivity between nodes and encodes, for example, the distance between a specific one and its neighbors (or more complex formulas), e.g., E = {(vᵢ, vⱼ)  ‖pᵢ, pⱼ‖₂ < r} being p the coordinates, v the neighbor's vertices i and j within a distance r. Once this graph is computed, it undergoes iterative updates to propagate local information, generating non-linearity and increasingly complex embeddings. The more update steps are done, the more nodes acquire information from further reaches of the graph. Different graph architectures can be built by defining the update function, the number of updates, the way the graph is initially built, how the neighbors are computed, and many other possible intermediate procedures (for instance residual connections or even an attention mechanism). A simple updating procedure employs aggregating functions in a way: Where 𝑓 is the aggregating function of a vertex in a neighborhood, updated by the non linear function 𝜎 , and 𝑔 the function to update the edges, like the Euclidian distance we defined above. A generalizable model should also have the inductive characteristics of being invariant to rotation, translation, and permutation. A classic graph architecture is GraphSage³. Its light computational cost allows it to efficiently train on geometrical data with great success. It uses the formula above by specifying f as simple concatenating operation followed by a weight matrix multiplication with Wᵏ. Suggested aggregators, in its turn, are average or max pooling; while neighbors are uniformly sampled with fixed-size from the whole set. Besides from the simpler models described, more complex and modern graph architectures have been developed in later years and are an promising field of research.


Represent the physics Now that we have identified suitable models, the question arises: How can we effectively incorporate the unique characteristics of Physical Systems? One commonly used approach involves enforcing physical laws through soft constraints or penalization methods. Specifically, Physically Informed Neural Networks (PINNs) implement this strategy by penalizing the model using the residual of PDEs, thereby ensuring their accurate adherence. When referring to CFD, those are the Navier-Stokes equations. The partial derivatives in each equation can be computed from the model's prediction by AD using the Deep Learning framework. Similarly, both Dirichlet and Neumann boundary conditions can be incorporated using the same methodology. Consequently, this approach leads to a refined format for the loss function: Where LG and LB represent, respectively, the loss of the data and the boundary conditions, constituting a supervised function with two arguments; Lᵣₑₛ is the residue function of the PDE; Xb and xb are the input data of the model and the PDE on boundary conditions points; B retrieves the boundary conditions values themselves; G𝜃 is the forward function of the network with parameters 𝜃. Consequently, the model learns to respect both the pressure and velocity balance, and generalizes better to unseen data. The loss expression highlights that PINNs can be used in a supervised approach, being data and physics driven. Notwithstanding, the LG term is optional, and the network can be trained in an unsupervised manner. This will have a much large space of solutions. As initial predictions are inherently random, optimizing the network proves challenging, and identifying an appropriate search objective remains an open question in Machine Learning. Various solutions can be imagined, from the alternative loss functions and optimization strategies to enforcing the boundary conditions by hard constraint, where a component of the network must directly meet the specified values.


High-frequency signals Geometric Deep Learning models have encountered a challenge in terms of slow convergence and the struggle to learn high-frequency functions. This phenomenon is particularly prominent in fluid dynamics, especially when dealing with turbulent flows. To address this concern, the field of Implicit Neural Representations offers strategies that prove effective. These transformations are rooted by the fact that deep networks are biased toward learning lower-frequency functions. Consequently, those techniques can be useful to improve the representation of higher frequency functions. An example is the expansion by Fourier features: Employing this technique with Neural Networks facilitates learning high-frequency functions within low-dimensional problem domains like geometric contexts. Adjusting the frequency parameter allows for the manipulation of the spectrum of frequencies the model can grasp. In practical terms, superior outcomes are achieved by setting aⱼ = 1 and selecting bⱼ from a random distribution. Fine-tuning can be performed on the standard deviation of this distribution, usually Gaussian. A broader distribution expedites convergence for high-frequency components, leading to improved results (notably in image-related tasks, resulting in higher definition). Conversely, an excessively wide distribution can introduce artifacts into the output (yielding a noisy image), presenting a trade-off between underfitting and overfitting. A more modern approach can be achieved by using sinusoidal Representation Networks (SIRENs). They propose periodic activation functions in the form: Beyond improving the high-frequency representation, SIRENs are also differentiable. That happens because the derivative of a SIREN is itself a SIREN, similar to how the derivative of the sine is a cosine: a phase-shifted sine. Not all common activation functions possess this quality; for instance, ReLU has a discontinuous derivative and zero second derivative throughout. Some other functions do present this desired capability, such as Softplus, Tanh, or ELU; however, its derivatives can be not well-behaved and fail to represent the fine details searched. Therefore, they are well-suited to represent inverse problems, like the PDEs we are so interested in. Furthermore, SIRENs were proved to converge faster than other architectures. To achieve the intended outcomes, an appropriate initialization scheme is essential. This initialization preserves the activation distribution throughout the network to ensure the final output remains independent of the number of layers. The solution lies in adopting a uniform initialization scheme in the following manner: , so that the input to each unit is normally distributed with a standard deviation of 1. Moreover, the first layer of the network should span multiple periods over [-1, 1], which can be achieved using ω₀ = 30 in sin(ω₀ ⋅ 𝑊𝑥 + 𝑏) ; this value should be changed depending on the modeled function frequency and the number of observations. The limitation of using a single frequency was later treated with the called Modulated SIRENs.


Conclusion Hopefully, you now have a clearer understanding of how Deep Learning can create surrogate models for numerical simulations, even when dealing with unstructured and noisy data. Improving the network's ability to generalize can be achieved through various techniques. These range from incorporating the physics of the underlying partial differential equations (PDEs) to using Implicit Neural Representation, among others we haven't had a chance to explore. This dynamic research field is poised to expand significantly in the coming years as it becomes more reliable. Despite its name, this approach doesn't seek to replace numerical simulations. Instead, it offers a quicker alternative that leverages simulations themselves and even experimental data. If we can combine fluid dynamics simulations, physical equations, and Deep Learning, why restrain to just one of them?


References [1]: PointNet C. R. Qi, H. Su, K. Mo, L. J. Guibas, PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation, arXiv:1612.00593 [cs]ArXiv: 1612.00593 (Apr. 2017). URL http://arxiv.org/abs/1612.00593 [2]: PointNet++ C. R. Qi, L. Yi, H. Su, L. J. Guibas, PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, arXiv:1706.02413[cs] ArXiv: 1706.02413 (Jun. 2017). URL http://arxiv.org/abs/1706.02413 [3]: GraphSage W. L. Hamilton, R. Ying, J. Leskovec, Inductive Representation Learning on Large Graphs, arXiv:1706.02216 [cs, stat]ArXiv: 1706.02216 (Sep. 2018). URL http://arxiv.org/abs/1706.02216 [4]: AirfRANS F. Bonnet, A. J. Mazari, P. Cinnella, P. Gallinari, Airfrans: High fidelity computational fluid dynamics dataset for approximating reynolds averaged navier-stokes solutions (2023). arXiv:2212.07564. [5]: Point-GNN W. Shi, R. R. Rajkumar, Point-GNN: Graph Neural Network for 3D Object Detection in a Point Cloud (2020) IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020): 17081716. [6]: Scientific Machine Learning through Physics-Informed Neural Networks: Where we are and What's next S. Cuomo, V.S. Di Cola, F. Giampaolo, Scientific Machine Learning Through PhysicsInformed Neural Networks: Where we are and What's Next. J Sci Comput 92, 88 (2022). https://doi.org/10.1007/s10915-022-01939-z [7]: Learning differentiable solvers for systems with hard constraints G. Negiar, M.W. Mahoney, A.S. Krishnapriyan, Learning differentiable solvers for systems with hard constraints (2022). ArXiv, abs/2207.08675.


OpenAI Blows Open the Gates to Enterprise AI

The Rise of Customizable AI Enterprise Tools The game changed when OpenAI released Fine-Tuning for GPT-3.5 Turbo. If you thought AI was already making waves in the enterprise space, think again. This new capability brought something to the table that was only possible with smaller and open-source models - customization. But what does that mean for your enterprise, and why should you care? Well, because customization has proven to be the defining factor in terms of performance & scalability when aiming for a specific use case.


GPT-3.5 Turbo and Fine-Tuning GPT-3.5 Turbo isn't just another pre-trained language model; it's already prepared to follow instructions (work well as a chat agent). But now it comes with the capability to be fine-tuned to a degree of specificity that previous iterations couldn't touch. This essentially allows you to make the model "your own" teaching it the nuances of your domain, the intricacies of your brand's voice, or even the specific problem-solving strategies you require. Recent publications from OpenAI show that a fine-tuning GPT-3.5 can render better results than GPT-4 for the designed use case. Furthermore, the cost per token of interacting with fine-tuned 3.5 is 6080% cheaper than GPT-4 (8k context window) calls. Interestingly, the input/output cost is different in rate, which makes it even more interesting to use fine-tuned GPT-3.5 for use cases where low-input large-output is expected.


What is Fine-Tuning Anyway? Alright, you're pumped about the customizable superpowers that fine-tuning brings to GPT-3.5 Turbo. But let's take a step back and dig into what fine-tuning actually is. Because, let's be honest, "fine-tuning" sounds cool but might seem a tad technical for some.


The Essence of Fine-Tuning Imagine you've got a sports car. It's already blazing fast right off the production line. But now you've decided you want it to be the best on a specific track. You won't just replace the whole engine; instead, you tweak the existing machinery - perhaps you modify the gear ratios, or maybe you invest in better aerodynamics. You're fine-tuning it. In a similar vein, fine-tuning takes an already-trained AI model and customizes it for specific tasks or domains. You're not building a new model from scratch; you're just giving the existing one a "minor" makeover to improve its performance in your desired niche.


Steps in Fine-Tuning Data Preparation: Gather domain-specific data.Preprocessing: Clean and format this data to make it compatible.Training: The fine-tuning itself - train the model on your tailored data.Evaluation: Assess the model's performance in the specific domain.Deployment: Integrate the model into your application.


So, Why Does It Matter? Okay, that sounds interesting, but should you care? Yes!, Here's why: Reduced API Costs When you fine-tune a model, it becomes more efficient at generating relevant outputs. This means fewer API calls and, by extension, lower costs. Companies have reported a decrease in API costs by up to 40% after fine-tuning their models. This can even be larger if you break up your use case to use the least smart (cheapest) model for each step that still accomplishes your goals. Performance Uplift A generic model tries to be a jack-of-all-trades. But a fine-tuned model becomes a master of one - your specific domain. The result? Improved accuracy and less need for human intervention. Differentiated Solutions A model fine-tuned with your selection of data will perform differently from any other version out there. This enables companies to truly create unique outputs, opening a door for those startups building LLM wrappers to step up and make their solution less easily replicable.


The Precedence of Prompt Engineering While fine-tuning is akin to turning your sports car into a track champion, there are performance boosts you can achieve without heading into the garage for a full overhaul. This is, of course, our well-known discipline of Prompt Engineering - your low-hanging fruit in the AI optimization garden.


Fine-Tuning Isn't Always Step One Don't get me wrong; fine-tuning is a powerful tool. However, it's resource-intensive, requiring both computational power and domain-specific data. So before you dive deep into customization, ask yourself, "Can I get good enough results without changing the underlying model?"


Prompt Engineering: Your Quick Win If you're new to the term, prompt engineering is basically the art of crafting queries or instructions in a way that guides the AI model to generate desired outputs. No model adjustments, no added costs. Simple text tweaks can result in dramatically different and highly targeted outputs. Here's a basic example: General Query: "Tell me about coffee."Engineered Prompt: "Summarize the history, health benefits, and brewing methods of coffee." The second prompt is designed to elicit a more structured and comprehensive answer, allowing you to maximize the utility of the base model.


Advanced Techniques: Prompt Chaining & Function Calling Let's level up. Prompt chaining involves breaking down a complex task into smaller, digestible queries and feeding the output of one as the input to another. For instance, asking the model to first summarize a document and then translate that summary into another language. Function calling is a more recent innovation that enables the use of pre-built functions within a chat session to accomplish specific tasks. This is like giving your model a set of utilities to pull from, streamlining processes like data extraction or text summarization even further.


Why Prioritize Prompt Engineering? Three reasons: Speed: Real-time results, no waiting for model training.Cost-Efficiency: Zero extra costs, especially useful if you're still experimenting with AI capabilities.Flexibility: Easy to adjust and refine, enabling agile iterations to match your evolving needs. Before you invest in a fine-tuned model, test the waters with prompt engineering. Sometimes, the difference between a good solution and the best solution is just a well-engineered question.


Control & Compliance: The Enterprise Edition The cornerstone of an enterprise AI deployment: Security and Compliance. Let's get real; all the bells and whistles of a technology solution mean nothing if it can't stand up to the rigorous security standards your organization requires. This is where OpenAI's Enterprise Offering, announced on Aug 28th, comes into play. Although the announcement was just made this week, OpenAI shows that a long list of very respectable clients have already been using their Enterprise offering without it being yet public. Details are still scarce, and implementation today requires dedicated interaction with OpenAI's sales teams. Still, the solution promises to fulfill the four C's that the industry has been looking for - Control, Compliance, Customization & Collaboration.


Control: Admin Console The Enterprise version will ship with an intuitive Admin Console. This is your control center, providing a dashboard to manage API keys, monitor usage, and set permissions at a granular level. It's where governance meets agility - right from setting quotas to delegating roles, the console puts you in the driver's seat.


Compliance: Enterprise-Grade Security This is the bedrock on which the enterprise edition is built. Expect state-of-the-art encryption both at rest and in transit. Compliance standards such as GDPR and SOC 2 are met (no word on HIPAA, which Anthropic provides), ensuring you're never on the wrong side of legal obligations.


Customization: Tailored Retention & Fine-Tune In addition to the fine-tuning capabilities we have discussed, OpenAI ensures that you will have control over how data is handled. The enterprise version comes with options to set custom data retention policies, ensuring you can comply with any regulations or internal guidelines about how long to hold onto your data. Furthermore, and although I haven't been able to validate this, the company communication states that the addition of company knowledge will have an easy-to-use interface to add data sources for the model that can take effect company-wide.


Collaboration: Shareable Chat Templates Working in a silo has been an issue since the start of ChatGPT. First with shareable conversations and now with templates, OpenAI is acknowledging that the modern workspace is all about collaboration. With chat templates, teams will be able to distribute, test, and improve conversational designs without risking unauthorized access or modification. And yes, these are secure links that adhere to best practices in privacy & protection.


Specific Use-Cases Reaping the Rewards Fine-tuning shines across diverse sectors, and its potential is immense: Customer Service: Forget canned responses. Businesses are now fine-tuning GPT-3.5 Turbo to understand their products inside out, providing customer support that's both automated and astonishingly adept.Content Writing: Something our clients have often asked about is how to ensure that content written by AI will follow their brand style and guidelines. This is usually achieved through red-teaming, but still is not very accurate. With these changes, your brand can have a clear message that will be honored across your generated content.Formatted Outputs: Current Apps built on LLMs commonly try to extract JSON, Markdown, or other specific formatting out of the responses. This usually comes with risks, as the model may not always reply in perfect format. So, Devs have had to build test & retry flows. Now, you can educate your model to better follow your format needs.Legal Sector: Models tuned to the documents that your company has already processed can sift through legal jargon to produce summaries or even draft initial versions of legal documents that match the legal arguments made and use the latest jurisprudence. If you don't want to be in the news for citing imagined cases, this is the way to go.Healthcare & Research: Custom-tailored models assist with medical diagnoses, offering preliminary suggestions based on years of journals and studies.


Early Adopters and Their Triumphs From global giants to sprightly startups, many have jumped on the fine-tuning bandwagon and are reaping tangible benefits. A notable case is Salesforce, which has integrated fine-tuned GPT-3.5 Turbo into its customer relationship management (CRM) software. The result? A 35% reduction in average handle time for customer support queries.


A Note on the Horizon While we've covered the plethora of capabilities OpenAI's newly minted Enterprise edition promises, it's worth mentioning that full details are still behind the veil of OpenAI's sales engagement. As a result, keep an eye on this space - this article may undergo updates to reflect new insights as they come to light. Moreover, as the enterprise landscape begins to dabble more intensely with fine-tuning options, expect additional articles to delve deeper into emerging use cases, performance metrics, and best practices. In a rapidly evolving domain, staying in the know is not just advantageous - it's essential for a competitive edge.


Sources OpenAI Developer Guide: Fine-TuningOpenAI Blog: ChatGPT EnterpriseOpenAI Blog: Function Calling and Other API UpdatesOpenAI Pricing: API Cost by 1k TokensDeeplearning.ai - Fine-Tuning Large Language ModelsHow Salesforce Research Uses Generative AI as a Force for Good


LLM Chat Applications with Declarai, FastAPI, and Streamlit - Part 2

Streaming in Action  As demonstrated, streaming offers enhanced responsiveness and has become a standard in recent AI chat applications. Are you eager to see how we achieved this? Let's dive into the code:


Declarai  To utilize streaming, ensure the chatbot is declared with the streaming feature enabled by passing streaming=True . When interacting with the chatbot using streaming: What's happening behind the scenes? By enabling streaming, OpenAI returns a generator, which produces items one at a time and only when required. Thus, as soon as a new chunk is available from OpenAI, it's immediately processed.


FastAPI  For the streaming capability, we've introduced a distinct REST endpoint. This endpoint dispatches a StreamingResponse - essentially a generator where each chunk is a JSON containing the chunk's details. It's essential to specify the media_type as media_type="text/event-stream" which states the response type is of type stream.


Streamlit App On the front end, our goal is to display each new response chunk to the user as it's received. This requires accessing the delta in response between each chunk of the response. Here's how it's accomplished: Start with an HTTP POST request with stream=True.Iterate through each chunk and decode its JSON.Extract the deltavalue and add it to the full response. This is basically the newly generated values.Update the UI with the accumulated response. Ready to deploy your streaming chatbot ? Dive into the complete code in this repository - Stay in touch with Declarai developments . Connect with us on Linkedin Page , and give us a star ⭐ on GitHub if you find our tools valuable! You can also dive deeper into Declarai's capabilities by exploring our documentation


Inside DSPy: The New Language Model Programming Framework You Need to Know About

DSPy DSPy prioritizes programming over prompting tasks, unifying strategies for both prompting and fine-tuning language models (LMs). The framework enhances these techniques with reasoning and tool/retrieval augmentation, all conveyed through a concise collection of Pythonic operations that are composable and capable of learning. Within DSPy's framework, there exist composable and declarative modules designed to guide LLMs, following a familiar Pythonic structure. Notably, DSPy introduces an automatic compiler that educates LLMs on executing declarative stages within a program. This compiler internally traces the program's execution and subsequently devises high-quality prompts suitable for larger LMs, or even trains automatic finetuning for smaller LMs, imparting knowledge of the task's intricacies. DSPy functions as a comprehensive solution for intricate tasks involving language models (LMs) and retrieval models (RMs), DSPy harmonizes approaches for both prompting and fine-tuning LMs, while also accommodating methods for reasoning and tool/retrieval augmentation. Through DSPy, these methodologies are encapsulated by a compact ensemble of Pythonic modules, which facilitate composability and learning. DSPy's framework is enabled by its provision of composable and declarative modules, structured in a manner familiar to Python users. This advancement elevates "prompting techniques" such as chain-of-thought and self-reflection from manual string manipulation tactics to versatile modular procedures capable of adapting to diverse tasks. Notably, DSPy introduces an automatic compiler, imparting LMs with the capability to execute declarative steps outlined in a program. The DSPy compiler methodically traces the program's execution, yielding well-crafted prompts suitable for large-scale LMs, and it can also train automatic finetuning for smaller LMs. This empowers these models to internalize the task's procedural nuances. Importantly, the DSPy compiler operates without the need for manual labels for intermediary steps within a program. This stands in contrast to the conventional practice of "prompt engineering" relying on ad hoc string manipulation techniques. Instead, DSPy opens the door to exploring a structured realm of modular and trainable components, providing a systematic approach that replaces brittle manual processes. The DSPy programming experience is based on two fundamental constructs: Signatures and Teleprompters.


I. Signatures: Crafting LLM Behavior Within the realm of DSPy, when tasks are allocated to LMs, specificity in behavior is expressed through Signatures. A Signature encapsulates a declarative representation of an input/output behavioral pattern associated with a DSPy module. Unlike expending effort on guiding an LLM through sub-tasks, Signatures empower DSPy users to elucidate the nature of the sub-task. Subsequently, the DSPy compiler takes on the responsibility of devising intricate prompts tailored for a large LM or fine-tuning a smaller LLM, all in line with the specified Signature, the provided data, and the existing pipeline. A Signature comprises three fundamental constituents: 1) A concise delineation of the sub-task addressed by the LM. 2) Elaboration on one or more input fields (e.g., input question) that will serve as inputs for the LM. 3) Explanation of one or more output fields (e.g., question's answer) expected to be generated by the LM. The following code illustrates a signature based on the ChainOfThought module with quite a few controls:


II. Enabling Program Optimization via dspy.teleprompt. Compilation is at the center of the DSPy experience. Compiling hinges on three pivotal factors: a potentially compact training dataset, a validation metric, and the selection of a teleprompter from DSPy's repertoire. These teleprompters, embedded within DSPy, are potent optimizers capable of mastering the art of formulating impactful prompts tailored to any program's modules. (The "tele-" prefix in "teleprompter" implies "at a distance," referencing the automatic nature of prompting.) Remarkably, DSPy's demands for labeling are notably modest. Consider an example retrieve augmented generation(RAG) pipeline: even a handful of instances containing questions and their human-annotated answers could suffice. Even when encompassing multiple intricate stages, as exemplified in the basic RAG model comprising context retrieval, a chain of thought, and the ultimate answer, labels are only mandated for the initial query and the final response. DSPy ingeniously extrapolates intermediate labels necessary to support the entire pipeline. Should modifications be made to the pipeline structure, the bootstrapped data will dynamically evolve to align with the altered setup. An instance of the RAG training set could be structured as follows: following that, articulating validation criteria is imperative. This logic imposes constraints on program behavior or module performance. For instance, a validation function for RAG might involve a check as depicted: Distinct teleprompter options come with varied trade-offs related to cost optimization versus quality enhancement. In the context of RAG, a choice could be the straightforward BootstrapFewShot teleprompter. This involves initializing the teleprompter itself with a custom validation function (such as my_rag_validation_logic) and subsequently conducting compilation against a designated training set (my_rag_trainset). When utilizing the compiled RAG instance, it ushers in a phase of invoking the LM with intricate prompts. These prompts incorporate concise demonstrations of chain-of-thought retrieval-augmented question answering, all tailored to the unique dataset in use. DSPy is still in super early stages but already highlights quite a few promising ideas. A simple and consistent programming model that resembles the PyTorch experience and a modular experience allows LMP developers to build quite sophisticated experiences. I hope that DSPy evolves from this initial stage to a stack that can match LangChain and LlamaIndex because a lot of the ideas of the framework are definitely needed in the LMP space.


Exploring Large Language Models -Part 3

Fine Tuning on Custom Domain Data All the popular models like GPT3/3.4/4 and LLAMA2 are trained primarily on the data scraped from the internet. Common Crawl, WebText, GitHub, StackOverflow etc: These are massive datasets of text and code that are crawled from the public web and a few curated like the QA dataset SQAD. The worldview and information the model has learned are also based on this data. However, this means that if we have some domain-specific data that the model has not seen, then it won't be able on its own to answer questions related to such data in case of Closed Book QA use-case or any other use case that depends on the specific domain data. For example, most online portals are adding virtual assistants for their customers, banks, e-commerce, customer support etc. And a huge if not the majority of data in the world still lives outside of the internet in enterprises. We have seen in Part 2 how LLMs can help address information retrieval use cases based on Vector space embeddings. But what if our use case is more high level? It needs domain "understanding", maybe some higher level reasoning tasks. This is where fine-tuning with custom data comes into play. I am not able to provide a use case where higher-level reasoning can be used. There are a few simpler ones, like training on custom issues and then asking it to reason on similar issues and possible solutions, but these are as of now not tested. So let's stick with a simpler use-case Closed-Book QA - the model answers questions based on the knowledge it internally has for now. The above is from a 2021 paper Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA? This is already outdated in the sense of the number and size of models and training released. The authors with 2021 models could not achieve great results and the great results they found in some studies described could be attributed to the high train and test overlap in datasets. There are also a lot of tutorials on the internet that try to portray this concept with toy datasets. The real trouble is making the model 'understand' the data first and not just parrot it out. Without understanding, it will parrot out the answer based on the similarity of the question in the training set, or both the question and answer. To prevent this, the authors have an intermediate step called 'Recite' where the model is made to recite/output the relevant passages and, after that, output the answer. Just to be clear, there is no doubt now (2023), especially with GPT3/4, LLAMA2 and similar models about the feasibility of this use case, that a model can understand the question, has some ability for causal reasoning, and can generalize to learn a world model from its training data, and to use both to create a well-formed answer to the question. Let's see the difficulties one by one however, of training a large model. First is the importance of the model size. This GIF from the Google AI blog illustrates this beautifully. It is relatively easy and cost-efficient to train or fine-tune a small model with our custom data, as the GPU and infrastructure requirements are very less. On the contrary, it needs huge fleets of GPUs and training infrastructure to load very large language models and fine-tune them (without quantisation) in a distributed way (e.g. see libraries like DeepSpeed) LLMs come in various sizes, based on the number of trainable parameters or weights. The smaller ones, which have less than 1 billion parameters (GPT2 124 M, Bloom 560M, Flan-T5 783 M ) etc can be trained on a laptop GPU with 8 to 15 GB GPU RAM ) For quite some time, this is what I tried. I tried to overfit a small test data set on decoder models like GPP2-small, GPT-Medium, and Bloom and encoder-decoder models like Flan-T5, thinking somehow that the understanding we see in ChatGPT ( see- unsupervised learning Part 1) may come in some form if we train on these smaller models. ( less than one billion parameters). As per the paper, I tried both Causal training, where the model is presented with only previous tokens, and Masked LM-based training, where the model is presented with full tokens, but a certain percentage of tokens are masked in random, and the model has to predict it. The next option was to fine-tune a large model with the data. However, this is extremely difficult to do, and even if cloud-based solutions are used, it would be pretty expensive. (What OpenAI provides now is Instruct Fine-Tuning, which we will cover later) It takes months of GPU fleet time and a specialized library and infrastructure to distribute training across multiple GPUs needed to train LLMs. For example, even a relatively small model like the BigScience Bloom 3 Billion model, even when the weights are loaded in 16 Bit cannot be trained with A100 on ColabPro with 40GB GPU RAM ( the highest you can get) as it goes out of memory. Solution - Fine-Tuning Large Models via Qunaitsation and Parmeter Efficient Tuning The solution to this is to reduce the size of the models so that they can fit a commodity GPU and then fine-tune them. There are two parts to this- Quantisation and Parameter Efficient Tuning. The real magic of this is that a laptop with a sufficient recent GPU (having Tensor Cores), can run the 7 billion Lamma2 pre-trained model open-sourced recently by Meta Research. Imagine the compressed knowledge and an NLU (Natural Language Understanding) model running on your local laptop. This is still a smallish model, but it's still capable of understanding and has sufficient world knowledge embedded in it to be quite useful. Imagine what a model like this or better models in the future could do if it could run in small servers or in cars, and leverage its causal reasoning and world model knowledge to supervise lower-level/specialist AI/ML systems. So we have now a way to fit reasonably large models (7B or more) in a single GPU, via Quantisation and then train them in a parameter-efficient way via LoRa/QLoRa. Take 1: Un-supervised Training Fine-tuning with QLoRa Using the small training data and QLoRA, I first tried to train a large 7B Lamma2 model by feeding in the training text as is (Causal LM model training via UnSupervised learning). Note that this model was loaded in 4-bit, making it runnable on a single T4 GPU and trained with QLoRa. With QLoRA, only a fraction of the adapter weights are trained and summed with the existing frozen pre-trained weights of the model during inference. Here is an illustrative Colab notebook. You can see that training the model with just the text as is, does not result in proper output to questions. The answers are not affected by the training data. Take 2: Instruct Fine-tuning with QLoRa Instruction Tuning concept is a higher-level training concept introduced by this paper FineTuned Language Models Are Zero shot Learners (FLAN) We leverage the intuition that NLP tasks can be described via natural language instructions, such as "Is the sentiment of this movie review positive or negative?" or "Translate 'how are you' into Chinese." We take a pre-trained language model of 137B parameters and perform instruction tuning ... Since we use QLoRa we are effectively closely following this paper - QLORA: Efficient Finetuning of Quantized LLMs concerning the training data set, the format that the authors used to train their Gauanco model This is the format for the Llama2 model and will be different for others. One of the hardest problems of training is finding or creating a good quality data set to train. In our case, converting the available training data set to the instruction data set. Since our use case is Closed Book QA, we need to convert this to a QA format. Using older NLP methods like NER (Named Entity Recognition) and then using that to create a QA dataset was not effective. This is where the Self-instruct concept could be used However previous to Llama2, the best-performing model was the GPT 3/4 model via ChatGPT or its API and using these models to do the same was expensive. The 7 billion model of Llama2 has sufficient NLU (Natural Language Understanding) to create output based on a particular format. Running this in 4-bit mode via Quantisation makes it feasible compute-wise to run this on a large data set and convert it to a QA dataset. This was the prompt used. The context was a sliding window from the text dataset. Some minimal parsing and finetuning were done on the output of the model, and we could generate a QA dataset of the format below. This was fed to the QLoRA-based fine-tuning (Colab Notebook). We can see that the output from a fine-tuned 4-bit quantized llama2 7 B model is pretty good. Colab Notebook Trying to reduce hallucination via fine-tuning In the generated dataset, I added a specific tag `Source:8989REF`. The idea was that via attention, this token will be somehow associated with the text that we were training on. And then to use this hash somehow to tweak the prompt to control hallucination. Something like "[INST] <<SYS>>\nYou are a helpful Question Answering Assistant. Please only answer from this reference Source:8989REF" However, that turned out to be a very naive attempt. Also, note that the generated QA missed transforming training data related to Professor Thiersch's method to a proper QA dataset. These and other improvements need to be experimented with, as well as to train with some completely new data that the model has not seen to test more effectively. Update: Training with new data was done by writing an imaginary story with ChatGPT help and then creating an instruction tuning data set (colab notebook). The model was then trained and tested (colab notebook) with this generated instruct dataset. The results confirm that the model learns via Instruct tuning, not only the fed questions but other details and relations of the domain. Problems with hallucinations remain (Bordor, Lila characters who are not in the story). The LLama2 13B 4-bit fine-tuned model has better output than the 7B model. A lot more needs to be explored in Fine-tuning. One observation is that slight changes in prompts give different answers. Since the output is not deterministic (that is, with even the same prompt, it varies over time), it is all the more difficult to fine-tune prompts to give the most effective output. This needs to be studied more. Also to be updated are higher level use-cases that should be possible with the fine-tuned models.