This AI newsletter is all you need #65

Hottest News Stable Audio London-based startup Stability AI, renowned for its AI model Stable Diffusion, has introduced Stable Audio, an AI model capable of generating high-quality commercial music with greater control over synthesized audio. 2. Google Nears Release of AI Software Gemini, the Information Reports Google nears the release of its conversational AI Software called Gemini. It is an advanced language model that is intended to compete with OpenAI's GPT-4 model. It is currently in the early testing phase and offers a range of functionalities, including chatbots, text summarization, and code-writing assistance. 3. Microsoft Releases Prompt Flow Microsoft has introduced Prompt Flow, a development suite for LLM-based apps. It offers a range of functionalities, including creating executable workflows, debugging and iterating flows, assessing flow quality and performance with larger datasets, integrating testing and evaluation into CI/CD systems, and easily deploying flows to selected serving platforms or app code bases. 4. IBM Releases MoE LLMs IBM has recently launched MoE LLMs, including models with 4B and 8B parameters. These models provide computational efficiency comparable to dense models with fewer parameters. They have been trained on a large dataset and employ the ModuleFormer architecture. 5. Pulitzer Prize Winner and Others Sue OpenAI Pulitzer Prize-winning US novelist Michael Chabon and several other writers have filed a proposed class action accusing OpenAI of copyright infringement, alleging it pulled their work into the datasets used to train the models behind ChatGPT. OpenAI argues that its language learning models are protected by "fair use," igniting discussions on AI and copyright law in the field.


This AI newsletter is all you need #65

Five 5-minute reads/videos to keep you learning LLM Training: RLHF and Its Alternatives. This article breaks down RLHF in a step-by-step manner to provide a reference for understanding its central idea and importance. It presents five different approaches with corresponding research papers, such as Constitutional AI, The Wisdom of Hindsight, Direct Preference Optimization, and more. 2. New AI Usage Data Shows Who's Using AI - And Uncovers a Population of 'Super-Users'. Salesforce has released Generative AI Snapshot Research: "The AI Divide," which is a survey of more than 4,000 people across the United States, UK, Australia, and India. The survey indicates that nearly half of the population utilizes it, with a third using it daily. Younger generations, particularly Gen Z and Millennials, are the "super users" of generative AI. 3. Overview of Natively Supported Quantization Schemes in  Transformers. Quantization schemes in Transformers, such as BitsandBytes and Auto-GPTQ, offer methods for running large models on smaller devices. This article aims to provide a clear overview of the pros and cons of each quantization scheme supported in transformers to help you decide which one you should opt for. 4. Why Open Source AI Will Win. Open source is likely to have a more significant impact on the future of LLMs and image models than the broader public believes. This article presents the current arguments against open source and its limitations and delves into its future and importance. 5. Validating Large Language Model Outputs. LLMs Are Powerful but Can Produce Inconsistent Results. Validating outputs is essential for reliable and accurate applications. This article discusses LLM output validation and provides examples of how to implement it using an open-source package called Guardrails AI.


This AI newsletter is all you need #65

Papers & Repositories From Sparse to Dense: GPT-4 Summarization With Chain of Density Prompting A recent study introduced the "Chain of Density" (CoD) prompting technique, which generates dense summaries using GPT-4. By iteratively adding important entities without increasing the length, the resulting abstract summaries outperformed standard prompt summaries in terms of abstractive quality and reduced lead bias. 2. Large Language Models for Compiler Optimization This paper introduces a 7B-parameter transformer model trained from scratch to optimize LLVM assembly for code size. The model surpasses baselines and exhibits exceptional code reasoning capabilities, resulting in a 3% reduction in instruction counts. It generates compilable code 91% of the time and perfectly emulates the compiler's output 70% of the time. 3. When Less Is More: Investigating Data Pruning for Pretraining LLMs at Scale In this work, researchers take a wider view and explore scalable estimates of data quality that can be used to systematically measure the quality of pretraining data. They found that perplexity is a more effective method than complex scoring techniques for pruning pretraining data for language models. 4. NExT-GPT: Any-to-Any Multimodal LLM NExT-GPT is an end-to-end general-purpose any-to-any MM-LLM system. It can process and generate content in various modalities such as text, images, videos, and audio. It achieves this by utilizing already-trained encoders and decoders, with minimal parameter tuning required. 5. Clinical Text Summarization: Adapting Large Language Models Can Outperform Human Experts This work employs domain adaptation methods on eight LLMs, covering six datasets and four distinct summarization tasks: radiology reports, patient questions, progress notes, and doctor-patient dialogue. This research is the first to demonstrate that LLMs outperform humans in multiple clinical summarization tasks. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #65

Weekly AI Podcast In this week's episode of the "What's AI" podcast, Louis Bouchard interviews Petar Veličković, a research scientist at DeepMind. They discuss his academic background and his journey from competitive programming to machine learning. Petar also shares insights on the value of a Ph.D., emphasizing its role as an entry ticket into research and the opportunity it provides to build connections and adaptability. He highlights the evolving landscape of AI research, where diverse backgrounds and contributions are essential. Overall, the interview offers valuable perspectives on academia, industry, and the importance of curiosity in driving impactful research. Listen to the full episode on Spotify or Apple podcasts!


This AI newsletter is all you need #65

Upcoming Community Events The Learn AI Together Discord community hosts weekly AI seminars to help the community learn from industry experts, ask questions, and get a deeper insight into the latest research in AI. Join us for free, interactive video sessions hosted live on Discord weekly by attending our upcoming events. Explaining AI Alignment In the Webinar, Ruiqi Zhong will give a talk on AI Alignment, hosted on the server as part of the Prompt Hackathon series of events. Learn more about Ruiqi and AI alignment before the talk in his post "Explaining AI Alignment as an NLPer and Why I am Working on It." Join the event here! Date & Time: 28th September 2023, 12:00 pm EST Add our Google calendar to see all our free AI events!


This AI newsletter is all you need #65

Meme of the week! Meme shared by rucha8062


This AI newsletter is all you need #65

Featured Community post from the Discord Penguin is working on a website that aims to make it easier to discover recent research papers in AI, ML, NLP, Computer Vision, and Robotics. This website is a valuable resource for AI enthusiasts and professionals who wish to stay updated on the latest research in the field. Check it out here and support a fellow community member! Share your feedback and join the conversation.


This AI newsletter is all you need #65

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #65

Article of the week Top Important Computer Vision Papers for the Week from 4/9 to 10/9 by Youssef Hosni This article will provide a comprehensive overview of the most significant papers published in the first week of September 2023, highlighting the latest research and advancements in computer vision. Whether you're a researcher, practitioner, or enthusiast, this article will provide valuable insights into the state-of-the-art techniques and tools in computer vision.


This AI newsletter is all you need #65

Our must-read articles Towards 3D Deep Learning: Artificial Neural Networks with Python by Florent Poux, Ph.D. PyTorch LSTM - Shapes of Input, Hidden State, Cell State, And Output by Sujeeth Kumaravel Walkthrough of Graph Attention Network (GAT) with Visualized Implementation by David Winer If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #65

Job offers Senior Deep Learning Algorithm Engineer @NVIDIA (Santa Clara, CA, USA) Senior Software Engineer - Python Backend @Teramind (Remote) Staff Deep Learning NLP Engineer @H1 (Remote) Sr. Machine Learning Researcher @Casetext (Remote) Machine Learning Success Manager @Snorkel AI (Remote) Software Engineer @Sonera (Berkeley, CA, USA) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Beyond GPT-4: What's New?

LLM Variants and Meta's Open Source Before shedding light on four major trends, I'd share the latest Meta's Llama 2 and Code Llama. Meta's Llama 2 represents a sophisticated evolution in LLMs. This suite spans models pretrained and fine-tuned across a parameter spectrum of 7 billion to 70 billion. A specialized derivative, Llama 2-Chat, has been engineered explicitly for dialogue-centric applications. Benchmarking revealed Llama 2's superior performance over most extant open-source chat models. Human-centric evaluations, focusing on safety and utility metrics, positioned Llama 2-Chat as a potential contender against proprietary, closed-source counterparts. The development trajectory of Llama 2 emphasized rigorous fine-tuning methodologies. Meta's transparent delineation of these processes aims to catalyze community-driven advancements in LLMs, underscoring a commitment to collaborative and responsible AI development. Code Llama is built on top of Llama 2 and is available in three models: Code Llama, the foundational code model;Codel Llama - Python specialized for Python;and Code Llama - Instruct, which is fine-tuned for understanding natural language instructions. Based on its benchmark testing, Code Llama outperformed state-of-the-art publicly available LLMs (except GPT-4) on code tasks. Llama 2, Llama 2-Chat, and Code Llama are key steps in LLM development but still have a way to go compared to GPT-4. Meta's open access and commitment to improving these models promise transparent and faster LLM progress in the future. Please refer to the LLM and Llama variants below:


Beyond GPT-4: What's New?

From LLMs to Multimodal LLMs, like OpenAI's ChatGPT (GPT-3.5), primarily focus on understanding and generating human language. They've been instrumental in tasks like text generation, translation, and even creative writing. However, their scope is limited to text. Enter multimodal models like GPT-4. These are a new breed of AI models that can understand and generate not just text, but also images, sounds, and potentially other types of data. The term "multimodal" refers to their ability to process multiple modes or types of data simultaneously. This is a game-changer. Imagine an AI that can not only read a description of a dress but also visualize it or even design it! Multimodal AI models are moving us towards more holistic AI systems. These systems can potentially understand our world in a more comprehensive manner, bridging the gap between different forms of data and providing richer, more integrated solutions. As we stand on the cusp of this new era, it's exciting to envision the myriad of applications and innovations that Multimodal models will bring to the table. The future of AI looks more integrated and versatile than ever before.


Beyond GPT-4: What's New?

From Connections to Vector DB The AI landscape is witnessing a fascinating transition: from Language Model (LLM) connections or integrations, e.g., LangChain and LlamaIndex, to the rise of Vector Databases (Vector DB) such as Weaviate, Milvus, Pinecone, Chroma, and Vespa.ai. But what's driving this shift, and why does it matter? LLM connections, like the LlamaIndex, primarily focus on linking and understanding vast amounts of external data. They've been pivotal in creating semantic connections, enabling more intuitive search experiences, and enhancing data accessibility. However, as the volume and variety of data grow, the need for more advanced storage and retrieval mechanisms becomes evident. This is where Vector DBs come into play. Unlike traditional databases that store data in rows and columns, Vector DBs store data in high-dimensional space, allowing for more efficient and accurate similarity searches. Tools like Weaviate and Milvus are designed to handle massive datasets, making them ideal for tasks like image recognition, recommendation systems, and more. The rise of Vector DBs represents a broader trend in AI: the quest for more efficient, scalable, and versatile data handling solutions. As we navigate this evolution, it's clear that the combination of LLMs and Vector DBs will redefine how we store, access, and understand data in the AI-driven future.


Beyond GPT-4: What's New?

From Agents to OS The AI realm is abuzz with innovations, and one of the most intriguing shifts we're witnessing is the transition from LLM agents to using LLMs as Operating Systems (OS). Let's delve into this evolution and its implications. LLM agents, like AutoGPT, AgentGPT, BabyAGI, and HuggingGPT, have been groundbreaking in automating tasks based on user requests. These agents leverage the power of Language Models (LLMs) to understand and execute commands, making them invaluable in tasks ranging from content generation to data analysis. Their adaptability and intelligence have made them a staple in many AI toolkits. However, the vision for AI doesn't stop there. The concept of LLM as an OS is emerging as the next big thing. Imagine an operating system where the core is a language model, orchestrating everything around it. Such a system would not just execute tasks but would understand context, anticipate needs, and offer solutions in real time. It's like turning the LLM into the brain of the digital ecosystem, making devices and applications more intuitive and responsive than ever. The move towards LLM as OS signifies a paradigm shift in how we perceive and utilize AI. It's not just about automation anymore; it's about creating a seamless, intelligent interface between humans and technology. As we stand on the brink of this transformation, the potential for LLM-driven OS to revolutionize our digital interactions is immense.


Beyond GPT-4: What's New?

From Fine-tuning to Plugins The world of LLMs is undergoing a transformative shift, moving from intricate fine-tuning processes to the more dynamic realm of plugins. Let's unpack this evolution. Historically, fine-tuning has been the cornerstone of LLM optimization. There are two primary ways to fine-tune LLMs: feeding data into the LLM in real-time and directly fine-tuning on the LLM. From a technical standpoint, this involves three methods: Transfer Learning: Adapting a pre-trained model to new tasks.Sequential Fine-tuning: Refining models in stages for specific tasks.Task-specific Fine-tuning: Tailoring models for a particular function. Moreover, LLM techniques like In-context learning, Few-shot learning, and Zero-shot learning have further enhanced the model's adaptability, allowing them to understand and generate content with minimal data. However, the future of LLMs is leaning towards plugins. With the introduction of tools like GPT-4 Plugins, the focus is on extending LLMs seamlessly. Instead of running LLMs as a service, they're envisioned as platforms. This means integrating LLMs with various tools, enhancing their capabilities, and offering a more modular and scalable approach to AI applications. The journey from fine-tuning to plugins represents a move from static optimization to dynamic adaptability, ensuring that LLMs remain at the forefront of AI innovation.


Beyond GPT-4: What's New?

In a Nutshell The AI domain is witnessing rapid shifts, with LLMs playing a central role. Initially, the move was from LLMs to Multimodal models, expanding from text to include images and sounds. Simultaneously, the trend shifted from LLM connections, which linked external data, to Vector Databases for efficient high-dimensional storage. Another evolution saw LLM agents, which automated tasks, transitioning towards LLMs as Operating Systems. This change aims for more intuitive, context-aware devices and applications. Furthermore, the traditional fine-tuning processes of LLMs are now being replaced by dynamic plugins, turning LLMs into platforms integrated with various tools. Leading this LLM revolution are OpenAI's GPT-4 and Meta's LLaMA2. Their pioneering efforts are setting the stage for an AI future that's more integrated, responsive, and attuned to human interactions.


Beyond GPT-4: What's New?

More Readings Harnessing the Power of LLMs in Practice: A Survey on ChatGPT and Beyond: https://arxiv.org/abs/2304.13712Sparks of Artificial General Intelligence: Early experiments with GPT-4: https://arxiv.org/abs/2303.12712GPT4All-J: https://huggingface.co/nomic-ai/gpt4all-jIntroducing Code Llama, a state-of-the-art large language model for coding: https://ai.meta.com/blog/code-llama-large-language-model-coding/Llama 2: Open Foundation and Fine-Tuned Chat Models: https://ai.meta.com/research/publications/llama-2-open-foundation-and-fine-tuned-chat-models/


Walkthrough of Graph Attention Network (GAT) with Visualized Implementation

GNN Layer The general formula for a GNN Layer is that for each node, we take all the neighbors of each node and sum over the features multiplied by a weight matrix, and then pass through an activation function. I've created a blank block with this formula as the title and passed it in the Adj matrix and the node features, and I will implement that formula inside the block. When we implement this formula, we don't want to actually run a loop. If we can completely vectorize this, then any training and inference with GPUs will be much faster because multiplication can be a single computing step. So, instead, we tile (i.e., broadcast) the node features into a 3D shape, so we had [5, 8] shape of node features, now we'll have a [5, 5, 8] shape where each cell in the 0th dimension is a repeat of the node features. We can think of the last dimension as "neighbor" features now. Each node has a set of 5 possible neighbors. We cannot directly broadcast the node features from [5, 8] to a [5, 5, 8] shape. Instead, we have to first broadcast to [25, 8] because when broadcasting, every dimension in shape has to be greater than or equal to the original dimension. So that's why we get the 5 and 8 parts of the shape (get_sub_arrays) and then multiply the first to get 25, then concatenate them all together. Finally, we reshape the resulting [25, 8] back to [5, 5, 8], and we can indeed verify right in Graphbook that each set of node features in the final 2 dimensions is identical. Next, we want to also broadcast the adjacency matrix to the same shape. This means that for every 1 in the adjacency matrix on row i and col j, there's a row of 1.0s of num_feats at dimension [i, j] . So in this adjacency, row 0 has a 1 in the 1st, 2nd, and 3rd cols, so there is a row of num_feats 1.0s in rows 1, 2, and 3 in the 0th cell (i.e., [0, 1:3, :]). The implementation is quite simple here, just parse the adjacency matrix to decimal and broadcast from [5, 5] shape to [5, 5, 8]. Now we can element wise multiply this adjacency mask by our tiled node neighbor features. We also want to include a self-loop to our adjacency matrix, so that when we sum over the neighbor features, we also include that node's own node feature. After doing an element-wise multiply (and including the self-loop) we get the neighbor features for each node and zeros for the nodes that aren't connected by an edge (are not neighbors). For the 0th node, that includes features for nodes 0 through 3. For the 3rd node, that includes the 3rd and 4th nodes. Next, we'll reshape to [25, 8] so that every neighbor feature is its own row, and pass that through a parameterized linear layer with your desired hidden size. Here I chose 32 and saved as a global constant so it can be reused. The output of the linear layer will be [25, hidden_size]. Simply reshape that output, create shape [5, 5, hidden_size] and now we're finally ready for the sum part of the formula! We sum over the middle dimension (dimension index 1) so that we are summing the neighbor features for each node. The result is a [5, hidden_size] set of node embeddings that have gone through 1 layer. Simply chain these layers together and you have a GNN network, and follow guides from https://www.youtube.com/@Graphbook for how to train.


Graph Attention Layer From the paper, the secret sauce behind the graph attention layer is the attention coefficient, given in the above formula. In essence, we're concatenating the node embeddings that are in an edge and running through another linear layer, before applying softmax. These attention coefficients are then used to compute a linear combination of the features corresponding to the original node features. What we need to do is tile each node's features for each neighbor, and then concatenate that with the node's neighbor features. The secret sauce is to get the node's features tiled for each neighbor. To do that, we swap the 0 and 1 dimensions of the tiled node features prior to mask. The result is still a [5, 5, 8] shaped array, but now every row in [i, :, :] is the same, and corresponds to node i's feature. Now, we can use the element-wide multiply to create the node features repeating only when they contain a neighbor. Finally, we concatenate that with the neighbor features as we created them for GNN and produce the concatenated features. We're almost there! Now that we have the concatenated features, we can put these through a linear layer. We need to reshape back to [5, 5, hidden_size] so that we can softmax over the middle dimension and produce our attention coefficients. Now that we have our attention coefficients with shape [5, 5, hidden_size], which is essentially one embedding per graph edge for our n node graph. The paper says these should be transposed (dimensions swapped) so I've gone back and done that prior to ReLU, and now I softmax over the last dimension so that they're normalized per dimension index along the hidden size dimension. We're going to multiply these coefficients by the original node embeddings. Recall, the original node embeddings were shaped [5, 5, 8], where 8 came arbitrarily from slicing the first 8 features off BERT's encodings of our text documents. Multiply [5, hidden_size, 5] shape by [5, 5, 8] shape produces [5, hidden_size, 8] shape. Then we sum over the hidden_size dimension to finally output [5, 8], matching our input shape. We could also now put this through a non-linearity like another ReLU, and then chain this layer multiple times.


Conclusion Thus far, we've gone over a visual implementation of single GNN layers and GAT layers. You can find the project in this github repo. In the paper, they also explain how they extend the method for multi-head attention. Let me know in the comments if you'd like me to cover this part as well, or if there's anything else you'd like me to cover using Graphbook.


WizardCoder: Why It's the Best Coding Model Out There

What Sets WizardCoder Apart One might wonder what makes WizardCoder's performance on HumanEval so exceptional, especially considering its relatively compact size. To put it into perspective, let's compare WizardCoder-python-34B with CoderLlama-Python-34B: The unique and most important factor of such large difference in HumanEval benchmark performance is the dataset the model trained on.


The Power of Data: WizardCoder's Unique Dataset One of the key factors contributing to WizardCoder's remarkable performance is its training dataset. Most models rely on a dataset structure that typically includes: Solid base with a lot of simple instructionsReduced amount of complex instructionsAnd minimal amount of really complex instructions To train a model for peak performance on evaluation benchmarks, training dataset should have a balance between simple instructions, complex instructions and really complex instructions. This is where WizardCoder's dataset shines. It boasts: Good amount of really complex instructionsGood amount of complex instructionsSolid base with a lot of simple instructions But there's a challenge: creating a dataset with complex instructions is inherently difficult, while simple instructions are readily available.


Evol Instruct Evol-Instruct is an evolutionary algorithm for generating diverse and complex instruction datasets using LLMs(GPT-4). It is designed to enhance the performance of LLMs by providing them with high-quality instructions that are difficult to create manually. In simple teams, Evol-Instruct is a complexity cascade of synthetically genearted (GPT-4) instruction dataset.


Instruction Evolution LLMs can make given instructions more complex and difficult using specific prompts. Additionally, they can generate entirely new instructions that are equally complex but completely different. Using this, we can iteratively evolve an initial instruction dataset, improving the difficulty level and expanding its richness and diversity.


A. Instruction Evolver The Instruction Evolver is an LLM that uses prompts to evolve (develop) instructions, with two types: In-depth evolving.In-breadth evolving A base dataset is given (e.g., Alpaca: generated using self-instruct, or 70k ShareGPT (shared by real users)) and using this base dataset, we can create a more complex and diverse dataset. a) In-depth Evolving In-Depth Evolving enhances instructions by making them more complex and difficult through five types of prompts: Prompt of In-depth Evolving In-depth Evolving aims to: (i) Add constraints (ii) Deepening (iii) Concretizing (more specific) (iv) Increased Reasoning Steps (v) Complicating Inputs The core part of In-Depth Evolving's prompt is The example prompt of add constraints is: These prompts help generate a complex instruction dataset, with similar templates for the other types of In-depth Evolving. b) In-breadth Evolving In-breadth Evolving addresses the limitation of open-domain instruction finetune datasets (e.g., Alpaca, ShareGPT, etc.), which are often small in scale, and lacking topic and skill diversity. In-breadth Evolving solves this problem by designing a prompt to generate a completely new instruction based on the given instruction, requiring the new instruction to be more long-tailed. Prompt of In-breadth Evolving In-breadth Evolving aims to 1. Enhance topic coverage 2. skill coverage 3. Overall dataset diversity The in-breadth prompt is as follows:


B. Response Generation The same LLM used to generate the corresponding responses for the evolved instructions using the prompt: <Here is instruction>


C. Elimination Evolving(Instruction Eliminator) The evolved instruction may challenge the LLM to generate a response. Sometimes, when the generated response contains "sorry' and is relatively short in length (i.e., less than 80 words), it often indicates that the LLM struggles to respond to the evolved instruction. So, we can use this rule to make a judgment. The response generated by the LLM only contains punctuation and stop words.


D. Finetuning the LLM on the Evolved Instructions Once all evolutions are done, the initial instruction dataset (the 52K instruction dataset of Alpaca) merges with evolved instruction data from all epochs and randomly shuffled the samples to create the final fine-tuning dataset. This processing ensures an even distribution of instructions of varying difficulty levels in the dataset, maximizing model fine-tuning smoothness. Wizardlm validates Evol-Instruct by fine-tuning open-source LLaMA 7B with evolved instructions and evaluating its performance and name the model WizardLM. Evol-Instruct works by generating a pool of initial instructions(52k instruction dataset of Alpaca), which are then evolved through a series of steps to create more complex and diverse instructions. Once the instruction pool is generated, it is used to fine-tune an LLM, resulting in a new model called WizardCoder. The fine-tuning process involves training the LLM on the instruction data to improve its ability to generate coherent and fluent text in response to various inputs.


Prompt Format For WizardCoder, the Prompt should be as follows:


Best Use Cases WizardCoder can be used for a variety of code-related tasks, including code generation, code completion, and code summarization. Here are some examples of input prompts that can be used with the model: Code generation: Given a description of a programming task, generate the corresponding code. Example input: "Write a Python function that takes a list of integers as input and returns the sum of all even numbers in the list."Code completion: Given an incomplete code snippet, complete the code. Example input: "def multiply(a, b): \n return a * b _"Code summarization: Given a long code snippet, generate a summary of the code. Example input: "Write a Python program that reads a CSV file and calculates the average of a specific column." The 34B model is not just a coding assistant; it's a powerhouse capable of: Automating DevOps Scripts: Generate shell scripts or Python scripts for automating tasks.Data Analysis: Generate Python code for data preprocessing, analysis, and visualization.Machine Learning Pipelines: Generate end-to-end ML pipelines, from data collection to model deployment.Web Scraping: Generate code for web scraping tasks.API Development: Generate boilerplate code for RESTful APIs.Blockchain: Generate smart contracts for Ethereum or other blockchain platforms


Evaluation WizardCoder beats all other open-source Code LLMs, attaining state-of-the-art (SOTA) performance, according to experimental findings from four code-generating benchmarks, including HumanEval, HumanEval+, MBPP, and DS-100. WizardCoder-Python-34B has demonstrated exceptional performance on code-related tasks. The model has outperformed other open-source and closed LLMs on prominent code generation benchmarks, including HumanEval (73.2%), HumanEval+, and MBPP(61.2%). WizardCoder-Python-34B-V1.0 attains the second position in HumanEval Benchmarks, surpassing GPT4 (2023/03/15, 73.2 vs. 67.0), ChatGPT-3.5 (73.2 vs. 72.5) and Claude2 (73.2 vs. 71.2). WizardCoder-15B-v1.0 model achieves the 57.3 pass@1 on the HumanEval Benchmarks, which is 22.3 points higher than the SOTA open-source Code LLMs, including StarCoder, CodeGen, CodeGee, and CodeT5+. Additionally, WizardCoder significantly outperforms all the open-source Code LLMs with instructions fine-tuning, including InstructCodeT5+, StarCoder-GPTeacher, and Instruct-Codegen-16B. In conclusion, WizardCoder's success is attributed to its unique dataset and the innovative use of Evol-Instruct to enhance instruction complexity, leading to its outstanding performance across various code-related tasks and benchmarks.


References YouTube: WizardCoder 34B: Complex Fine-Tuning Explained GitHub Paper: WizardLM- Empowering Large Language Models to Follow Complex Instructions Paper: WizardCoder: Empowering Code Large Language Models with Evol-Instruct


LangChain 101: Part 2ab. All You Need to Know About (Large Language) Models

Overview LangChain supports a variety of different models, so you can choose the one that suits your needs best: LLMs utilize APIs that take input text and generate text outputsChat Models employ models that process chat messages and produce responsesText Embedding Models convert text into numerical representations


Tokenization Tokenization breaks down human language into smaller units like words or subwords to convert text into machine-readable numerical representations. Further on, when we talk about text- or code- generation, we'll understand it as different kinds of token generation.


Text generation Say we want to generate the continuation of the phrase: The Encoder sends logits for all the tokens we have (if you don't know what logits are - consider them as scores) that can be converted, using the softmax function, to probabilities of the token being selected for generation. Read more about text generation:


.from_pretrained(<model>) In many pre-trained language models, the tokenizer and the model architecture are designed and trained together. The reason for this coupling is that the tokenizer and the model architecture must be compatible to ensure consistent tokenization and decoding. If the tokenizer and the model architecture were different or not synchronized, it could lead to tokenization errors, mismatched embeddings, and incorrect predictions.


API keys You will first need an API key for the LLM provider you want to use. We must choose between proprietary or open-source foundation models based on a trade-off between performance and cost. Many open-source models are organized and hosted on Hugging Face as a community hub. Open-source models are usually smaller with lower capabilities than proprietary ones, but they are more cost-effective. Examples: BLOOM (BigScience), LLaMA (Meta), Flan-T5 (Google), etc. Proprietary models are closed-source foundation models owned by companies. They usually are larger than open-source models and thus have better performance, but they may have expensive APIs. Examples: OpenAI, co:here, AI21 Labs, Anthropic, etc.


Embeddings Embeddings are compact numerical representations of words or entities that help computers understand and process language more effectively. These representations encode the meaning and context of words, allowing machines to work with language more meaningfully and efficiently. OpenAIEmbeddingsHuggingFaceEmbeddingsGPT4AllEmbeddingsSpacyEmbeddingsFakeEmbeddings Below, you can see the embeddings for the nine sentences of several topics. The heatmap values were presorted for better visualization. Best travel neck pillow for long flightsLightweight backpack for hiking and travelWaterproof duffel bag for outdoor adventuresStainless steel cookware set for induction cooktopsHigh-quality chef's knife setHigh-performance stand mixer for bakingNew releases in fiction literatureInspirational biographies and memoirsTop self-help books for personal growth As one can notice, the similarity of sentence 1 is close to 2 and 3, with relatively low similarity with other sentences. One can see that all the sentences can be aggregated into three groups.


Large Language and Chat Models A language model is a probabilistic model of a natural language that can generate probabilities of a series of words based on text corpora in one or multiple languages it was trained on. A large language model is an advanced language model trained using deep learning techniques on massive amounts of text data. A language model usually backs chat models, but their APIs are more structured. These models take a list of Chat Messages as input and return a Chat Message.


Asking GPT a question There is no need to connect LangChain to ask the model a question. In the example below, we're preparing a prompt, connecting to openai package, and returning the extracted response. As one can see, simple questions can be answered by the model correctly. But for complicated requests, LangChain may be the way to go.


How Much RAM? To determine the model size in bytes, you multiply the parameters by the chosen precision size. Let's say the precision we've chosen is bfloat16 (16 bits = 2 bytes). We want to use the BLOOM-176B model. This means we need 176 billion parameters * 2 bytes = 352GB! A mixed precision approach is used in machine learning, where training and inference are ideally done in FP32, but computation is done in FP16/BF16 for faster training. The weights are held in FP32, and FP16/BF16 gradients are used to update them. During inference, half-precision weights often provide a similar quality as FP32, allowing for faster processing with fewer GPUs. Quantization is a compression technique that reduces model memory usage and enhances inference efficiency. Post-training quantization is about converting the weights of a pre-trained model into lower precision without retraining.


Running Model CPU-only One may be surprised, but it is possible to run a LLM CPU-only: The response will take several minutes, though, but it shows the possibility of running llms even on Rasberry devices! This is the end of Part 1ab. The next part (cde) will be dedicated to finetuning models.


This AI newsletter is all you need #66

What happened this week in AI by Louie This week in AI, OpenAI again dominated the headlines as it announced the imminent rollout of new voice and image capabilities into ChatGPT. The LLM race is also continuing to heat up, with Amazon announcing significant investment into Anthropic AI. Open AI is rolling out new multi-modal versions of GPT Turbo and GPT-4, together with ChatGPT integrations with a new text-to-speech model, Whisper (a speech-to-text model), and Dalle-3. These now give ChatGPT the ability to see, hear, and speak, together with artistic capability. These new capabilities of ChatGPT will allow users to engage in voice conversations and use images to enhance the interaction. Users can now snap a picture to have live conversations about it, helping in various activities such as identifying landmarks, planning meals based on the contents of a fridge, or assisting with math problems. These functionalities are initially rolling out to Plus and Enterprise users, with voice available on iOS and Android and images on all platforms. We are also excited by the new image generation model Dalle 3, which has a less prompt-reliant approach to image generation. We are hopeful to see some great art produced by this new iterative design via chat. OpenAI was keen to beat Google Deepmind's Gemini to release a full multi-modal model (although Bard already has some more limited image capabilities). It also looks set to beat Amazon's Alexa to market with a LLM-powered text-to-speech chatbot. Amazon announced the upcoming launch of its updated Alexa powered by a multimodal LLM, capable of processing voice, video, and text embeddings. While the demo video for Alexa's LLM primarily showcases text generation tasks, Amazon reveals that the Alexa LLM is connected to thousands of APIs and can execute complex sequences of tasks. In other Amazon news, it announced an up to $4bn investment in Anthropic AI to develop reliable, high-performing foundation models. The AI race is heating up further, and now it looks like the leading cloud providers have lined up their key LLM labs partners with Google Cloud behind Deepmind, Microsoft Azure backing OpenAI, and Amazon AWS backing Anthropic. - Louie Peters - Towards AI Co-founder and CEO


This AI newsletter is all you need #66

Hottest News 1.OpenAI Announces DALL·E 3 OpenAI is launching DALL·E 3, an improved version that excels in following instructions, requires less prompt engineering, and can communicate with ChatGPT. This integration enables users to refine DALL·E 3 prompts by describing their ideas to ChatGPT. Starting in October, DALL·E 3 will be available to ChatGPT Plus and Enterprise customers. 2. Amazon Invests $4B Into Anthropic to Competing Against Microsoft (OpenAI) Amazon plans to invest up to $4 billion in the AI startup Anthropic. The agreement is part of a broader collaboration to develop the industry's most reliable and high-performing foundation models. Amazon Web Services (AWS) will be the primary cloud provider for Anthropic. The investment includes AWS's Trainium and Inferentia chips for model training and deployment. 3. DeepMind Announces AlphaMissense, a Catalog of Genetic Mutations To Help Pinpoint the Cause of Diseases DeepMind has released AlphaMissense, a model that uses AlphaFold's protein structure prediction to categorize missense genetic mutations as benign or malign. It surpasses human efforts by classifying 89% of 71 million variants. 4. Announcing Microsoft Copilot, Your Everyday AI Companion Microsoft rolled out a Copilot to provide tailored assistance based on workplace data and web context. It enhances productivity and creativity in Windows 11, Microsoft 365, Edge, and Bing while prioritizing privacy. Additionally, Bing and Edge users will enjoy a personalized experience with OpenAI's DALL.E 3 model, including AI shopping and image creation. 5. Meta Is Bringing Out 'Sassy Chatbots' for Younger Users Meta is preparing to announce a generative AI chatbot, internally called "Gen AI Personas," aimed at younger users. Meta also has plans to develop 'dozens' of chatbot personas, including ones for celebrities to interact with their fans.


This AI newsletter is all you need #66

Five 5-minute reads/videos to keep you learning 1.Adept.ai Shares Curious Errors Occurring During Their Large Training Runs Adept.ai, an AI company, shares insights into errors that can occur during large training runs. These errors can lead to learning curve issues, where models may appear fine but must be fixed due to accumulating minor errors over time. 2. Prompt Engineering for Claude's Long Context Window This article provides prompting guidance to leverage Claude's 100,000 token-long context window. It shares a quantitative case study on two techniques that can improve Claude's recall over extended contexts by extracting reference quotes and providing examples increases. 3. DeepMind's Cofounder: Generative AI Is Just a Phase. What's Next Is Interactive AI Mustafa Suleyman, co-founder of DeepMind, emphasizes the positive impact of technology on healthcare and leads an AI policy development team at Google. Backed by influential figures and companies, Suleyman introduces Pi, a friendly AI that advocates for interactive AI to connect technology with societal impact. 4. Object Detection Leaderboard Hugging Face has introduced the Object Detection Leaderboard, featuring top-performing models based on the DETA and DETR architectures. This blog demonstrates how the models were evaluated and demystifies the popular metrics used in Object Detection, from Intersection over Union (IoU) to Average Precision (AP) and Average Recall (AR). 5. GPT 3.5 vs. Llama 2 Fine-tuning: A Comprehensive Comparison In this post, the author documents their experiments benchmarking the fine-tuning of GPT 3.5 against Llama 2 in an SQL task and a functional representation task. GPT 3.5 slightly outperforms Llama 2. However, training and deploying GPT 3.5 costs 46 times higher than Llama 2.


This AI newsletter is all you need #66

Papers & Repositories 1.LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models LongLoRA is a method for efficiently extending the context size of pre-trained language models (LLMs) in artificial intelligence. By utilizing sparse local attention during training and dense global attention during inference, this approach allows for cost-effective fine-tuning and maintains performance. It demonstrates impressive results on various tasks and enables context extension of up to 100k tokens in LLMs. 2. Scaling Laws for Sparsely-Connected Foundation Models Researchers have discovered a unique scaling law that shows the relationship between weight sparsity, non-zero parameters, and training data volume in foundation models. They also found that the optimal sparsity level for performance increases with more data. 3. PDFTriage: Question Answering over Long, Structured Documents Researchers have developed PDFTriage, a solution that enhances the performance of Language model-based question-answering systems on structured documents like PDFs. By incorporating document structure and content, PDFTriage outperforms existing models in answering complex questions across various categories. 4. Chain-of-Verification Reduces Hallucination in Large Language Models Chain-of-Verification (CoVe) is a straightforward approach that minimizes hallucinations in Language Model-based systems. CoVe has successfully reduced hallucinations across various tasks, including question-answering and text generation, through its systematic process of generating, verifying, and delivering responses. 5. Connecting Large Language Models with Evolutionary Algorithms Yields Powerful Prompt Optimizers EvoPrompt, a new evolutionary algorithm framework, optimizes prompt generation for language models like GPT-3.5 and Alpaca. It surpasses human-engineered prompts and current methods, demonstrating its effectiveness for language tasks. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #66

Upcoming Community Events The Learn AI Together Discord community hosts weekly AI seminars to help the community learn from industry experts, ask questions, and get a deeper insight into the latest research in AI. Join us for free, interactive video sessions hosted live on Discord weekly by attending our upcoming events. Explaining AI Alignment In the Webinar, Ruiqi Zhong will give a talk on AI Alignment, hosted on the server as part of the Prompt Hackathon series of events. Learn more about Ruiqi and AI alignment before the talk in his post "Explaining AI Alignment as an NLPer and Why I am Working on It." Date & Time: 28th September 2023, 12:00 pm EST Join the event here or Add our Google calendar to see all our free AI events!


This AI newsletter is all you need #66

Meme of the week! Meme shared by archiesnake


This AI newsletter is all you need #66

Featured Community post from the Discord Kylejmichel is building Glimpse, a browser extension that pulls context from real-time web data and any website you browse. It highlights key points, suggests questions and topics, and filters and rephrases the text of any website, document, article, or video. It'll be most beneficial for students, developers, and content consumers. Check it out here and support a fellow community member! Share your questions and feedback in the thread.


This AI newsletter is all you need #66

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #66

Article of the week How to Automatically Explore and Clean Your Data with PandasAI by Cornellius Yudha Wijaya With the GPT model around, one of the exciting applications is combining the power of LLM with Pandas. PandasAI is a Python package that provides LLM implementation in Pandas. It is aimed at complementing Pandas, not replacing them. By using PandasAI, we can turn the Pandas package into a conversational tool that would automatically explore and clean our data.


This AI newsletter is all you need #66

Our must-read articles Generative AI for time series by Fabiana Clemente How Does an LLM Generate Text? by Ivan Reznikov If you want to publish with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #66

Job offers Machine Learning Platform Engineer @Mercari (Remote) Machine Learning Engineer @OXMAN (New York, NY, USA) Intelligent Automation / AI-ML Engineer @Node.Digital (Washington, DC, USA) Senior Data Scientist @DNSFilter (Remote) Junior Software Engineer (Vaga Exclusiva para PCDs) @Loggi (Remote) Machine Learning Engineer @Blackbird.AI (Remote) Senior Data Scientist / Machine Learning Engineer @Zenni Optical (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Walkthrough of LoRA Fine-tuning on GPT and BERT with Visualized Implementation

Visualization Details To implement LoRA and do the surgery, we will work with a node graph visualization tool. Each block is an operation that takes the inputs on the left side and produces the data for the output variables on the right side. Links denote the passing of data from outputs to inputs, and circles on inputs mean the data is specified in place and is static. Operations are either composite containing an "unbox" icon, which then decomposes into a sub-graph whose inputs are the parent's inputs and whose outputs are the parent's outputs, or they are primitive, meaning they cannot be decomposed further and correspond to low-level tensor operations like from NumPy or TensorFlow. Colors indicate data type and patterns indicate the data shape. Blue means the dat type is an integer, whereas purple/pink means it's a decimal data type. Solid links indicate that the data shape is scalar, whereas dots in the link indicate the number of dimensions of the array (the number of dots between the dashes). At the bottom of each graph is a table that characterizes the shape, type, and operation name of each variable that is carrying data in the model.


BERT LoRA First, I'll show LoRA in the BERT implementation, and then I'll do the same for GPT.


GPT LoRA I also added another composite around the LoRA layer and the "add" operation so that I can drop it as one single modifier.


How to Fit Large Language Models in Small Memory: Quantization

Quantization Let's start with a simple example. We'll need to transform 2023 to binary: As you can see, the process is relatively straightforward. In order to store the number 2023, we'll need 12+ bits (1 bit for the + or - sign). For the number, we might use the int16 type. There is a big difference between storing int as binary and float as such. Let's try converting 20.23 to binary: As one can see, the floating part (mantissa) is calculated as the combination of 1/2^n, and cannot be calculated very precisely, even with 10 bits dedicated to the floating part. The whole number part (exponent) is set to 5 bits, covering all numbers up to 32. In total, we're using 16 bits (FP16) for storing the closest we can to 20.23, but is it the most effective way to keep floats? What if the whole part number is much larger, say 202.3? If we look at the standard float types, we'll notice that to store 202.3, we'll need to use FP32, which, from a computational perspective, is far from reasonable. Instead, we can use a bfloat16 to save the range (exponent) as 8 bits and 7 bits for precision (mantissa). This allows us to widen the scope of possible decimals without losing much precision. To be clear, while training occurs, we need all the precision we can take. But prioritizing speed and size over the 6th decimal point makes sense for inference. Can we decrease the memory usage from bfloat16 to, say, int8?


Zero-point and abs-max quantization In fact, we can, and there are several approaches for such quantization: Zero-point quantization saves half the memory by converting a fixed range (-1, 1) to int8 (-127, 127), followed by converting int8 back to bfloat16. Abs-max quantization is similar to zero-point, but instead of setting a custom range (-1,1), we set it as (-abs(max), abs(max)). Let's take a look at how these practices are used in an example of matrix multiplication: Zero-point quantization: Abs-max quantization: As one can notice, the score for large values [-1579, -1780] is quite low ([-1579, -1752] for zero-point and [-1565,-1786] for abs-max). In order to overcome such issues, we can separate outlier multiplication: As you can see, the results are much closer to true values. But is there a way we can use even less space without losing much quality? To my very surprise, there is a way! What if we, instead of independently converting each number to a lower-type, would account for the error and use it for adjustments? This tecnique is called GPTQ. Like previous quantization, we find the closest match for the decimals we can, keeping the total conversion error as close to zero as possible. We fill the matrix row-wise in such a manner. The result, in combination with anomaly-separate calculations, provides quite okay results: We can now compare all the methods: LLM.int8() methods perform pretty well! The GPTQ approach loses quality but allows the use of twice as much GPU memory as the int8 method. In the code, you might find something similar to the following: The load_in_4bit flag specifies that the model should be loaded in 4-bit precision. The bnb_4bit_use_double_quant flag specifies that double quantization should be used. The bnb_4bit_quant_type flag specifies the quantization type. The bnb_4bit_compute_dtype flag specifies the compute dtype. To sum up, we've learned how decimals are stored in memory, how to reduce memory footprint with some precision loss, and how to run selected models with 4-bit quantization. The original article was published on my LinkedIn page.


Program Synthesis - Making Code Write Itself

Program Basically, programming is the act of telling a computer what to do. Scheduled a reminder on your phone, customize your social media photos, optimize traffic, or set up this Medium article, all of them are programming. To ensure we're in sync, let's look at the Programming Triplet: A programmer turns a task into a program.A program is a thing that the programmer gives, that the interpreter understandsAn interpreter takes in the program and does stuff. So why do we spend time learning to program? Because computers offer abilities, we can't match. We do not need to manually stitch together a panoramic photo from a dozen individual pics, I certainly can't remember all my friends' birthdays without a little nudge from my phone. I cannot deliver the blog to all of you in person.


Program Synthesis Program synthesis is a system that makes programming easier. GCC frees you from having to write assembly, your smartphone operation system takes a dozen individual pics and produces a panoramic photograph. The small green 'publish' button on the top of this page will send you all this article. With program synthesis, you can work in an enhanced version of a programming language - let's call it "Program++". This is more straightforward than the original Program. The synthesizer acts as a middleman, converting your Program++ back into the original program. When you pair this synthesizer with an interpreter, you get an advanced interpreter - let's call it "Interpreter++". This upgraded version takes in your Program++ (commonly referred to as a spec or specification) and carries out the tasks. The process is recursive - you can keep adding layers of synthesizers and interpreters to make programming easier and easier. Thus, program synthesis is simply "easier programming".


Nowadays Program Synthesis The art of program synthesis is making program++/spec as simple as possible. Nowadays, program synthesis techniques have become increasingly sophisticated, like ChatGPT or CoPilot, which generates code from high-level requirements, reducing the need for "hand-written" code and thereby minimizing human error. After we define what a Program and Program Synthesis are, we need to define precisely what a problem is.


Problem Let's start by asking: How does a programmer write a program so that the interpreter does the right thing for a task?


Task as Spec Tasks can be specified in various ways - imagine the different recipes you could use to make a salad. Verifying a program's correctness also has multiple angles - consider the different criteria to judge whether the salad is tasty, healthy, or visually appealing. However, the nuances of many real-world tasks often escape computational understanding.


Spec as Problem A specification or spec is a way of stating the task so that both humans and computers can agree on what needs to be done. One of the rare and special forms of communication is Input-Output (or just I/O) which is readily understood by both humans and computers. There are other forms of specifications, for instance, it could be an objective function to be maximized, a set of constraints on the computer's behaviors, or a natural language sentence. But there's a lot of latitude in how you might write that spec. Different specs can describe the same problem from different angles, syntax, or levels of granularity. For instance, if you want to sort a list(L) of n numbers, we could provide: A series of I/O pairs: In₁[3,2,1] Out₁[1,2,3]  In₂[0,-1,2] Out₂[-1,0,2]  In₃[5,8,100] Out₃[5,8,100]  In₄[9] Out₄[9] ...A single sentence: "Sort a list of integers in ascending order."A mathematical function: f( X ) = x₁≤ x₂≤ x₃ ... ≤ xₙA recursive definition: Sorted(L) ⇔ L[1] ≤ min( L[2:n] ) ∧ Sorted( L[2:n] ) Similarly, for making a salad, the spec could look like: I/O Pairs: In₁[lettuce, tomatoes, cucumber] Out₁[salad]  In₂[cucumber, caesar, tomatoes] Out₂[not salad]Natural Language: Combine lettuce, tomatoes, and cucumberConstraint-based: Must include[ at least one green vegetable and one source of protein.]


correctness With a spec, the computer can automatically decide if a program "did the right stuff" by checking if executing the program on the inputs produces the corresponding outputs. we name this process a Verifier. Now, a simple Verifier gives you a straightforward answer: True or False. is_sorted( [1,2,4] ) = True. Are the combined elements [tomatoes, caesar] are salad ingredients? False. But we can dial it up a notch - some Verifiers go beyond simple binary answers, diving into metrics or scales that measure how well the program did.


From Logic to Learning Traditionally, program synthesis was a field dominated by strict logical rules and algorithmic reasoning. However, the landscape has been changing rapidly, thanks to the infusion of machine learning techniques. This shift toward learning-based approaches is revolutionizing the speed, accuracy, and efficiency of program synthesis. Diverse Approaches: The Taxonomy of Program Synthesis Program synthesis is not a one-size-fits-all solution. Depending on the needs of the task, different approaches or types of synthesis can be more advantageous. Let's delve into some of these types. Deductive Synthesis - The Traditionalist. Relies on formal methods, logical frameworks, and algorithmic principles. Highly accurate but may require substantial computational resources. Ideal for critical systems where reliability is paramount.Inductive Synthesis - The Modernist. Primarily driven by machine learning techniques and heuristic methods.Excels in speed and adaptability but may lack the stringent accuracy of deductive methods. Best suited for rapid development cycles and less mission-critical applications.Stochastic Synthesis - The Realist. Utilizes probabilistic models to deal with the uncertainties in specifications or environments. Offers a balanced approach but may require fine-tuning to achieve desired reliability. It is well-suited for applications where some level of uncertainty or variation is expected.


Program Synthesis - Beyond Human Capabilities We do not need to limit ourselves. We aim to build a synthesizer that can go even further beyond!! The aim is not just to replicate human capabilities but to exceed them since Program Synthesis discovers new solutions that humanity currently cannot tackle. We're talking about potentially breaking new ground in medicine, finance, and maybe even decoding the cosmos...


Further Reading & Reference Armando's lecture notes on deductive synthesis ↩A 2021 survey on neuro-symbolic program synthesis ↩Competitive programming with AlphaCode ↩Evaluating Large Language Models Trained on Code ↩Bringing machine learning and compositional semantics together by Percy Liang ↩ACL 2018 tutorial on neural semantic parsing ↩The Child as Hacker ↩Probabilistic Models of Cognition ↩Armando's lecture notes on definition, history of program synthesis ↩a minimalist guide to program synthesisA 2017 survey on program synthesis by Microsoft folks ↩


Inside AGENTS: The New Open Source Framework for Building Semi-Autonomous LLM Agents

The AGENTS Principles The AGENT's framework stands out as an open-source platform tailored for language agents powered by LLMs. Its core tenet is to streamline the process of customizing, deploying, and fine-tuning language agents. Designed to be user-friendly for beginners, AGENTS is based on a series of core principles: 1. Long-short-term memory: AGENTS recognize the importance of memory in autonomous agents. While conventional machine learning models react to single inputs, autonomous agents interact continuously with environments or other agents. To address this, AGENTS has incorporated memory components, as mentioned. It boasts capabilities like storing long-term memories using VectorDB, enabling semantic searches, and updating short-term memories using a dedicated scratchpad. 2. Tool usage & Web navigation: Language agents often need to move beyond mere linguistic interactions. They need the capacity to utilize external tools and explore the internet. AGENTS provides integration with popular external APIs and an adaptable class for the addition of more tools. It further empowers agents to search and browse the web through specialized API interfaces. 3. Multi-agent communication: AGENTS isn't just about individual agent capabilities. It ventures into the realm of multi-agent systems, useful for various domains like gaming, social experiments, and software development. An innovative feature within this realm is the "dynamic scheduling" approach. Rather than relying on static rules for agent activities, AGENTS allows for a controller agent - a "moderator" of sorts - to decide the subsequent actions of agents, keeping in mind their roles and past activities. 4. Human-agent interaction: A shortcoming in many agent frameworks is their limited scope of interaction with humans, especially in multi-agent setups. AGENTS effortlessly bridges this gap. It champions interactions between humans and agents, regardless of whether it's a singular or multi-agent environment. 5. Controllability: Conventional frameworks often restrict agent behavior to system prompts. AGENTS introduces the concept of standard operating procedures (SOPs). These SOPs, similar to real-world applications, are thorough step-by-step guides dictating agent tasks and actions. Such detailed plans can be produced by an LLM and subsequently modified by users. These are quite a few concepts! However, the initial implementation of AGENTS makes building intelligent agents that use these capabilities quite simple.


Programming with AGENTS The AGENTS framework elegantly structures itself around three principal classes: Agent, SOP (Standard Operating Procedure), and Environment. These classes are conveniently initialized via a config file crafted in simple plain text. Let's delve deeper into the architecture of AGENTS and its coding foundation. Agent Class: Serving as the essence of the AGENTS framework, the Agent class encapsulates the features and behaviors of a language agent. As visualized in Figure 1's UML diagram, the agent manages its intricate long-short term memory. Within this class, methods enable the agent to: Engage with its surroundings (agent._observe(environment))Take actions based on its prevailing state (agent._act())Revise its memory data (agent._update_memory()). For a simplified experience, the above functionalities are integrated into the agent.step() method. 2. SOP (Standard Operating Procedure) Class: This class paints a broader picture, charting out the agent's state progression. Each state in the SOP class maps to a specific sub-goal or sub-task that agents need to achieve. Every state, constructed as a 'State' class object, houses specialized prompts aiding the agent to harness the capabilities of an LLM. Additionally, it provides a toolkit of APIs to be utilized within that state. 3. Environment Class: Acting as the backdrop for the agents, this class offers a representation of the external conditions that agents operate within. The class primarily splits into two functional aspects: The environment._observed() function illustrates the environment's effect on the agent's actions, outlining the transferable information upon observation.The environment.update() function defines the repercussions of the agent's actions on the environment. The AGENTS framework not only embodies the foundational principles discussed, such as tool incorporation and multi-agent communication, but it also gives prominence to the human-agent interface. Notably, by merely adjusting the "is_human" attribute in the config file to "True", AGENTS offers the flexibility for users to step into an agent's shoes. Such an arrangement allows for dynamic interaction between human users and other language agents within the given environment, all via a dedicated console interface. In terms of deployment, AGENTS champions FastAPI as its preferred route. Moreover, the framework is versatile enough to cater to both individual and multi-agent configurations.


The AI Boom: Practical Guide to Generative AI for Small Businesses

Getting the basics right To understand the AI boom, we need to start by defining some key terms:


History and development of AI Before we explore the current AI boom, let's take a brief historical detour. The term "Artificial Intelligence" was coined in the mid-20th century by John McCarthy, often regarded as the father of AI. He described AI as "the science and engineering of making intelligent machines". However, AI research experienced periods of rapid progress as well as "AI winters" when expectations exceeded technological capabilities. In the 21st century, we've witnessed the rapid growth of AI, primarily due to advancements in machine learning, access to vast datasets, and powerful computing resources. This resurgence has led to AI being applied across various domains, from voice assistants such as Siri, and Alexa and recommendation systems found in websites such as Netflix, and Amazon to self-driving cars such as Tesla autopilot and language translation such as Google Translate.


Explosive Growth of Generative AI One of the most exciting aspects of the current AI boom is the explosive growth of Generative AI. In just the past few months, we've seen remarkable innovations, including new iterations of models like GPT (Generative Pre-trained Transformer) and groundbreaking projects like Midjourney, DALL·E 2, LLMA, AutoGPT pushing the boundaries of what's possible. Last November, ChatGPT from OpenAI wowed the world with its ability to generate content, from emails to legal documents. It even passed the Bar Exam, hinting at AI's potential in the legal domain. Walmart is exploring AI for contract negotiations, while Google's Med-PaLM 2 excels in medical diagnostics. In the coding world, Code Llama, an advanced Language Model System (LLM), is set to accelerate coding tasks and democratize access for all developers, seasoned or novices, reshaping the coding landscape. What sets Generative AI apart is its ability to create content. By providing a simple prompt, you can generate text, images, videos, and more. Prompt engineering, the art of crafting prompts, plays a crucial role in controlling the output of these models (more on this in a future article).


Practical Impact on Small Businesses In recent years, we've seen slower productivity growth, but McKinsey's estimates that automation of work tasks enabled by technology could lead to an annual global productivity increase of 0.2% to 3.3% between 2023 and 2040. Here are some real-world examples of how small businesses can leverage AI: Content Creation: AI can assist in generating captivating content for websites, blogs, social media, etc. It helps produce relevant content quickly, freeing up time for other tasks.Visual Design: AI can generate graphics, logos, and social media posts that align seamlessly with a brand's identity, saving time and ensuring consistency.Music and Audio Creation: AI can compose original jingles, soundtracks, and background music for videos, adding a unique sonic layer to content.Personalized Marketing Campaigns: Based on specific customer information, you can leverage Gen AI to generate personalized marketing materials (e.g., emails, product recommendations, ads), enhancing customer engagement and relevance.Product and Prototype Design: AI can contribute innovative ideas and prototypes during brainstorming sessions in creative fields, such as fashion and product design.Language Translation and Localisation: AI provides accurate translation and localization services, expanding into new markets effectively.Text Summarisation and Content Curation: AI simplifies the process of summarising articles and curating relevant content for newsletters or industry updates.Storytelling and Narrative Generation: AI can craft compelling narratives for presentations, sales pitches and other speeches, helping businesses connect with their audience.Customer Support: AI can help you curate frequently asked questions and reply to customer queries. Customized AI-powered chatbots can also provide 24/7 customer support, enhancing customer satisfaction and reducing the workload on support teams.Human Resources: AI can streamline HR processes, from writing up job adverts to preparing employee onboarding material, making the hiring process more efficient and reducing administrative burdens.


Myths About AI Before diving into AI, it's essential to dispel some misconceptions: AI Is Only for Big Businesses: AI tools are accessible to businesses of all sizes, including small enterprises. Hopefully, you are convinced about this one after reading this blog post.AI Is Always Perfect: AI can make errors, and human oversight is often required to ensure quality. These errors are called hallucinations. An AI hallucination is "where a large language model (LLM) like OpenAI's GPT4 makes up false information or facts that aren't based on real data or events".AI Is Too Expensive: Cost-effective AI tools and solutions are available with a significant return on investment. The cost also depends on the application, scale and customization needed for your company to use AI. Some low-cost tools are mentioned later in this blog post.AI Knows Everything About Your Business: AI's knowledge is based on data and requires proper training and customization to understand specific business nuances. If you want your AI model to provide precise information about your private data, then you should apply techniques like fine-tuning and Retrieval Augmented Generation (RAG) systems. RAG combines a knowledge retrieval system (like a librarian with access to your data) with a language model (like a GPT) to provide precise information when needed. Fine-tuning involves training an existing AI model (like a GPT) on specific data to adapt it to a particular domain or to change its response style, but it requires significant computational resources and updates every time you have new data.


Accessing Generative AI for Small Businesses Small business owners can access Generative AI through various means: Educate Yourself: Gain a basic understanding of AI concepts and terminology to make informed decisions. Platforms like AWS, Google Cloud and LinkedIn offer some non-technical courses tailored to business users. Such as "What is Generative AI", "Generative AI for Executives" and "Introduction to Generative AI Learning Path".Consult Experts: Partner with AI consultants who can guide you on tool selection, implementation, and strategy.Choose User-Friendly Tools: Opt for pre-built Generative AI tools designed for non-tech-savvy users. You can find some in the next section.Partner with AI Vendors: Collaborate with AI vendors to develop custom solutions tailored to your business.Hire AI Experts: Employ AI experts to create personalized AI solutions for your specific needs. Depending on the size, budget and needs of your company, you can choose which of these options are most suitable for you.


Affordable or Free AI Tools While I can't vouch for their quality, several AI tools are available for small businesses: Website Creation: Tools like 10Web and Wix ADI offer user-friendly website building with AI assistance.Image Generation: OpenAI's DALL·E 2, Midjourney and Stable Diffusion are excellent for generating images based on prompts.Content Creation: Tools like chatGPT, Grammarly, Plus AI, and Jasper.ai assist in writing and editing content.Sales and Marketing: Tools like Seamless.ai, Ocoya and Salesforce (Einstein) focus on lead generation and social media automation.Financial Management: Booke AI, Truewind.ai and other tools use AI for accounting and insights.Customer Interactions: Companies such as Boundless AI and Caffeinated CX offer personalised customer interactions through AI-powered chatbots. If you have the internal expertise, it's also possible to fine-tune an LLM or connect your private data to an LLM using platforms such as Google Vertex AI, Pinecone and Single Store.


Preparing for the Future The AI boom is here to stay, and its influence on small businesses will only grow. To prepare for the future: Stay Informed: Keep up with AI developments, trends, and applications in your industry.Experiment and Learn: Start small, experiment with AI tools, and learn from the experience.Collaborate: Consider partnerships with AI experts and vendors to tailor solutions for your business.Up-skill Your Workforce: Invest in training and up-skilling to make the most of AI's potential.Embrace Change: Be open to change and adapt your business strategies as AI continues to evolve. In conclusion, Generative AI is transforming the ways of executing work and, therefore, productivity not just for big organizations but also for small and medium businesses. While challenges and misconceptions exist, AI offers immense potential for innovation, efficiency, and growth. Small business owners who embrace AI with a clear strategy and a commitment to adapt will be better positioned to thrive in this AI-powered era.


The Difference Between Open Source Models and Commercial AI/ML APIs

1. Before we get started, let's get some definitions right! An Open Source ML model is the combination of a model architecture (design of the model) and weights (the knowledge the model has). You can go on a public hub of models like Hugging Face, Pytorch Hub, Tensorflow Hub and download it. A Commercial ML API (ex: OpenAI API) is a service accessed through an API endpoint, encapsulating, among other things we'll discuss shortly, an ML Model. These ML APIs can be found among the services of cloud providers (AWS, Azure, GCP, etc.) or smaller companies specialising in a subset of domains/ML tasks (Mindee, Lettria, Gladia, etc.) On the surface, they look very similar: you input your data and receive the desired output, such as text generation. But let's dig deeper to see how they are different.


2. An ML API is not just an "ML model wrapped in an API" Let's look at the blocks around the model and what are they used for: 2.1 Infrastructure for hosting: The first thing you do when you pick an ML model is to decide where to run it. If you're experimenting and it's a fairly small model, you can run it on your machine. But this not always possible or scalable so you would go through a cloud provider (AWS, GCP, Azure) and rent servers. 2.2 ML serving framework: A model is not just a 'python function'. To serve predictions in an optimal way, you end up going through a serving framework (TorcheServe, TFServing and more LLM-focused ones here). These frameworks are a necessary evil for achieving optimised latency and handling high concurrency but they do come with a learning curve. 2.3 Engineering logic: This is very case dependent and sometimes can consume a huge amount of time from ML engineers (the devil is in the details). Let's take sentiment analysis, for example, a simple use case. Below is an extract from hugging face: If all you need is generating the sentiment of a few lines of text, then that's the end of it. However, if you have, say, 10k documents (different formats: pdf, word, text), and you want to detect their sentiment. You'll need to : First, you run the PDFs through an OCR (Optical character recognition) model.Deal with any edge cases (Multiple pages, pages that are not oriented well, etc..)Potentially use a different model based on the document type and build an orchestration workflow to get all the data to the right formatYou pass everything to the sentiment analysis model. But you notice you can't do it sequentially as it will take foreverYou implement a parallelisation pipeline that takes X documents, splits them into multiple processes (maybe multiple servers), run predictions and return the resultsCreate tests and and automated processes to ensure stability of the above solution These components is the devil in the details that's abstracted by a Commercial ML API. The API provider will hire engineers in their team to do the work above, maintain it, and update it, so you can use something like: 2.4 API and infrastructure scaling: This includes two aspects. First, package all the above components into an API to make your model predictions accessible to the rest of your tech stack and product. Second, make sure it can handle concurrency and sudden increases of traffic. To illustrate: To package your model, you'll need to combine your inference code (the inference.py script), any other pre/post-processing files, create a docker container, and use something like FastAPI, Amazon API gateway, Apigee API management to create the API. Then comes scaling, the more users or predictions you make the more infrastructure you will need behind the API you just created. You will start with a small server (~30$/month) and maybe have to scale to (~200$/month) based on the traffic of your users. Then, scale down again when no one is using your API. 2.5 Customer support and SLA: Last but not least, when things don't work as expected or something breaks, who will fix it? When you build the solution in-house using open-source ML models, you also should account for at least 1 engineer in the team to dedicate time for support and bug fixing. When you use a Commercial ML API, it's part of the service to get support under service level agreements (SLAs). An SLA would, for example be, "if our service breaks, we'll assign an engineer to fix it in under 6 hours". Finally, note that just because the Commercial ML API does more things by design doesn't mean you can't do them yourself. What matters is that you know that these are the elements you'll need to build eventually if you want to achieve the same level of service as a Commercial ML API.


3. When comparing costs, use Total Cost of Ownership (TCO) Most stop at the "ML model is free" or "API calls are too expensive", let's see how much of that is true. Starting with the cost of generating the predictions: 1/Using an open source model: Hosting a ResNet50 (for image classification) on a g4dn.xlarge instance costs $0.526/hour and can achieve ~75 Predictions Per Second for this model (table at the end here). Assume you need to ensure a real-time response to your users, so you keep the instance up 24/7, it will cost you ~$400, and you'll be able to make ~200 million (75*60*60*24*31) predictions if you can the API nonstop. 2/Using the Amazon Rekognition API: It will cost you $0.00025 per prediction, and if you want to achieve 200 million predictions in a month, that's a whopping $50, 000 So clearly, you should go for option 1 right? Well, it depends on how many calls you're making a month: Note that the graph doesn't take into account "bursts" in traffic. The more users you have calling your ML models at once, the more servers you'll need to spin up to handle the load. So, the price of GPU is not completely fixed, but for simplification purposes, let's focus on steady workloads and conclude the following: -> If you're making less than 400,000 predictions a month, you're better off paying for Commercial ML API. If your volume is higher than that, each month, yes, go for the open-source version and host it yourself. Note: A similar study on LLMs has been done here. - So far, we have only looked at how much it costs to make predictions. However, when looking at cost, you should look at TCO (Total Cost of Ownership). It's basically answering the question: The engineering logic, the configuration of the frameworks, the scalability of your infrastructure, maintenance and support etc. You pay for these things with human capital. It's the hardest part to estimate because it depends on what ML task you want to achieve, what level of service you want to provide, and what resources you have. Assuming you hire two people, one with expertise in ML and another in backend/cloud. The average annual salary for Software Engineers is $120k in US (and 60k in Europe) so that's a total of $240k (~120k in Europe) minimum* (indeed, glassdoor)


4. Fine-tuning "Yes, but I like to fine-tune my models to my data and my use case". You can do that with both open-source models and commercial ML APIs. The difference is in terms of the degree of customization and the effort it will take you to customize. Keep in mind that in both cases, the hardest part is getting the data and making it ready to be ingested for fine-tuning. For open-source models, you definitely have more flexibility. As you have access to the model, you can go deep and even change the model's architecture. However, this requires you to have someone with ML knowledge, not only to understand how to properly fine-tune the model but also the architecture of the model and its parameters. For Commercial ML APIs, you'll get less flexibility but also less work on your side. The best example of a fine-tuning commercial API is the recently released GPT-3.5 fine-tuning. Other examples are custom labels with Amazon Rekognition, which is very useful if you use specific images or Mindee's OCR fine-tuning API for custom documents. The Commercial ML API will ask you to provide the data and the labels through their API/Interface, and they take care of the rest: Training, optimizing and deploying a custom model for you with the same stack as the generalist models.


5. Security & Privacy Security is about how secure your system is against attacks. Unless you have security experts on your team, the harsh truth is that you're more likely to build less secure systems than those of a Commercial ML API. They have a dedicated security team, implement industry standards, and run continuous checks on their systems. Privacy, on the other hand, is different; once your data goes out of your data center/servers and through an API, you take the risk of your data being read by someone else besides you. When this is tied to regulatory rules (i.e.: If your company data goes through service x then you need to do y) then it's just a matter of respecting those rules and jumping through approval hoops. When privacy is related to a concern of "the provider will look at my data", keep in my mind that their business is selling you API calls, and they can't do that if they can't be trusted. Check out who's providing the API before you use it, who they are exactly, and what are their privacy rules. Established vendors are clear about what they do with your data, encrypt your data in transit and at rest and sometimes even propose options to opt out of the use of your data for training purposes etc.


Summary and takeaways Below is a table that provides guidelines on when to choose what Most importantly keep in mind that both Open Source ML models and Commercial ML APIs are needed, they just serve different purposes. Which one is better depends on the criteria we mentioned above and your situation. A summary of recommendations: The higher the monthly volume of predictions (>~400500k) the more chances you're better off using Open Source Models and hosting them.Always try the Commercial ML APIs first. If they get the job done and the price fits your budget, stick to that. If the price is slightly higher, consider how much it will cost you to hire people to build and maintain a similar service.Try fine-tuning features of the Commercial ML APIs if the results are bad, work on your data, and/or contact the provider for help. If it's still bad, consider investing in an ML specialist, and if the ML performance is great then invest in an ML engineer/DevOps to help put the model in production. - Thanks for reading! If we haven't met, I'm Othmane and I spent 5 years at AWS as an ML engineer, helping over 30 companies embed ML in their products/businesses. If you have any questions, don't hesitate to reach out over Twitter (@OHamzaoui1) or Linkedin (hamzaouiothmane).


Exploring Large Language Models -Part 3

Fine Tuning on Custom Domain Data All the popular models like GPT3/3.4/4 and LLAMA2 are trained primarily on the data scraped from the internet. Common Crawl, WebText, GitHub, StackOverflow etc: These are massive datasets of text and code that are crawled from the public web and a few curated like the QA dataset SQAD. The worldview and information the model has learned are also based on this data. However, this means that if we have some domain-specific data that the model has not seen, then it won't be able on its own to answer questions related to such data in case of Closed Book QA use-case or any other use case that depends on the specific domain data. For example, most online portals are adding virtual assistants for their customers, banks, e-commerce, customer support etc. And a huge if not the majority of data in the world still lives outside of the internet in enterprises. We have seen in Part 2 how LLMs can help address information retrieval use cases based on Vector space embeddings. But what if our use case is more high level? It needs domain "understanding", maybe some higher level reasoning tasks. This is where fine-tuning with custom data comes into play. I am not able to provide a use case where higher-level reasoning can be used. There are a few simpler ones, like training on custom issues and then asking it to reason on similar issues and possible solutions, but these are as of now not tested. So let's stick with a simpler use-case Closed-Book QA - the model answers questions based on the knowledge it internally has for now. The above is from a 2021 paper Can Generative Pre-trained Language Models Serve as Knowledge Bases for Closed-book QA? This is already outdated in the sense of the number and size of models and training released. The authors with 2021 models could not achieve great results and the great results they found in some studies described could be attributed to the high train and test overlap in datasets. There are also a lot of tutorials on the internet that try to portray this concept with toy datasets. The real trouble is making the model 'understand' the data first and not just parrot it out. Without understanding, it will parrot out the answer based on the similarity of the question in the training set, or both the question and answer. To prevent this, the authors have an intermediate step called 'Recite' where the model is made to recite/output the relevant passages and, after that, output the answer. Just to be clear, there is no doubt now (2023), especially with GPT3/4, LLAMA2 and similar models about the feasibility of this use case, that a model can understand the question, has some ability for causal reasoning, and can generalize to learn a world model from its training data, and to use both to create a well-formed answer to the question. Let's see the difficulties one by one however, of training a large model. First is the importance of the model size. This GIF from the Google AI blog illustrates this beautifully. It is relatively easy and cost-efficient to train or fine-tune a small model with our custom data, as the GPU and infrastructure requirements are very less. On the contrary, it needs huge fleets of GPUs and training infrastructure to load very large language models and fine-tune them (without quantisation) in a distributed way (e.g. see libraries like DeepSpeed) LLMs come in various sizes, based on the number of trainable parameters or weights. The smaller ones, which have less than 1 billion parameters (GPT2 124 M, Bloom 560M, Flan-T5 783 M ) etc can be trained on a laptop GPU with 8 to 15 GB GPU RAM ) For quite some time, this is what I tried. I tried to overfit a small test data set on decoder models like GPP2-small, GPT-Medium, and Bloom and encoder-decoder models like Flan-T5, thinking somehow that the understanding we see in ChatGPT ( see- unsupervised learning Part 1) may come in some form if we train on these smaller models. ( less than one billion parameters). As per the paper, I tried both Causal training, where the model is presented with only previous tokens, and Masked LM-based training, where the model is presented with full tokens, but a certain percentage of tokens are masked in random, and the model has to predict it. The next option was to fine-tune a large model with the data. However, this is extremely difficult to do, and even if cloud-based solutions are used, it would be pretty expensive. (What OpenAI provides now is Instruct Fine-Tuning, which we will cover later) It takes months of GPU fleet time and a specialized library and infrastructure to distribute training across multiple GPUs needed to train LLMs. For example, even a relatively small model like the BigScience Bloom 3 Billion model, even when the weights are loaded in 16 Bit cannot be trained with A100 on ColabPro with 40GB GPU RAM ( the highest you can get) as it goes out of memory. Solution - Fine-Tuning Large Models via Qunaitsation and Parmeter Efficient Tuning The solution to this is to reduce the size of the models so that they can fit a commodity GPU and then fine-tune them. There are two parts to this- Quantisation and Parameter Efficient Tuning. The real magic of this is that a laptop with a sufficient recent GPU (having Tensor Cores), can run the 7 billion Lamma2 pre-trained model open-sourced recently by Meta Research. Imagine the compressed knowledge and an NLU (Natural Language Understanding) model running on your local laptop. This is still a smallish model, but it's still capable of understanding and has sufficient world knowledge embedded in it to be quite useful. Imagine what a model like this or better models in the future could do if it could run in small servers or in cars, and leverage its causal reasoning and world model knowledge to supervise lower-level/specialist AI/ML systems. So we have now a way to fit reasonably large models (7B or more) in a single GPU, via Quantisation and then train them in a parameter-efficient way via LoRa/QLoRa. Take 1: Un-supervised Training Fine-tuning with QLoRa Using the small training data and QLoRA, I first tried to train a large 7B Lamma2 model by feeding in the training text as is (Causal LM model training via UnSupervised learning). Note that this model was loaded in 4-bit, making it runnable on a single T4 GPU and trained with QLoRa. With QLoRA, only a fraction of the adapter weights are trained and summed with the existing frozen pre-trained weights of the model during inference. Here is an illustrative Colab notebook. You can see that training the model with just the text as is, does not result in proper output to questions. The answers are not affected by the training data. Take 2: Instruct Fine-tuning with QLoRa Instruction Tuning concept is a higher-level training concept introduced by this paper FineTuned Language Models Are Zero shot Learners (FLAN) We leverage the intuition that NLP tasks can be described via natural language instructions, such as "Is the sentiment of this movie review positive or negative?" or "Translate 'how are you' into Chinese." We take a pre-trained language model of 137B parameters and perform instruction tuning ... Since we use QLoRa we are effectively closely following this paper - QLORA: Efficient Finetuning of Quantized LLMs concerning the training data set, the format that the authors used to train their Gauanco model This is the format for the Llama2 model and will be different for others. One of the hardest problems of training is finding or creating a good quality data set to train. In our case, converting the available training data set to the instruction data set. Since our use case is Closed Book QA, we need to convert this to a QA format. Using older NLP methods like NER (Named Entity Recognition) and then using that to create a QA dataset was not effective. This is where the Self-instruct concept could be used However previous to Llama2, the best-performing model was the GPT 3/4 model via ChatGPT or its API and using these models to do the same was expensive. The 7 billion model of Llama2 has sufficient NLU (Natural Language Understanding) to create output based on a particular format. Running this in 4-bit mode via Quantisation makes it feasible compute-wise to run this on a large data set and convert it to a QA dataset. This was the prompt used. The context was a sliding window from the text dataset. Some minimal parsing and finetuning were done on the output of the model, and we could generate a QA dataset of the format below. This was fed to the QLoRA-based fine-tuning (Colab Notebook). We can see that the output from a fine-tuned 4-bit quantized llama2 7 B model is pretty good. Colab Notebook Trying to reduce hallucination via fine-tuning In the generated dataset, I added a specific tag `Source:8989REF`. The idea was that via attention, this token will be somehow associated with the text that we were training on. And then to use this hash somehow to tweak the prompt to control hallucination. Something like "[INST] <<SYS>>\nYou are a helpful Question Answering Assistant. Please only answer from this reference Source:8989REF" However, that turned out to be a very naive attempt. Also, note that the generated QA missed transforming training data related to Professor Thiersch's method to a proper QA dataset. These and other improvements need to be experimented with, as well as to train with some completely new data that the model has not seen to test more effectively. Update: Training with new data was done by writing an imaginary story with ChatGPT help and then creating an instruction tuning data set (colab notebook). The model was then trained and tested (colab notebook) with this generated instruct dataset. The results confirm that the model learns via Instruct tuning, not only the fed questions but other details and relations of the domain. Problems with hallucinations remain (Bordor, Lila characters who are not in the story). The LLama2 13B 4-bit fine-tuned model has better output than the 7B model. A lot more needs to be explored in Fine-tuning. One observation is that slight changes in prompts give different answers. Since the output is not deterministic (that is, with even the same prompt, it varies over time), it is all the more difficult to fine-tune prompts to give the most effective output. This needs to be studied more. Also to be updated are higher level use-cases that should be possible with the fine-tuned models.


Exploring Large Language Models -Part 1

Introduction Large Language Models (LLMs) present us with two obvious capabilities - a natural language interface for communicating with the model and a vast amount of knowledge stored very efficiently in the models- the whole of the internet textual data. The larger the models are, the better they get at both of these capabilities. There is another capability that is not that obvious, but that could be the most powerful. It is implicit in the first capability and termed technically as NLU, Natural Language Understanding. To understand something, you need a model of that. For humans - a mental model. To understand language you need a model of language syntax and semantics. To understand a user question and to be able to answer that effectively the model needs an internal world model. There is a debate between thought leaders in this field on whether there is some internal world model learned by these LLMs, or if it seems to us as if it is present. However, their natural language understanding is so good, and their internal world model has enough information that they pass the Turing test as it was envisioned (https://www.nature.com/articles/d41586-023-02361-7) and also the Mini Turing Test based on Causal Reasoning proposed by current AI's most famous critic Judea Pearl. We will come back to this topic later. Still, this does not mean that they are intelligent in the real sense of the term, but we could more safely say, intelligent in understanding natural language. This is the key property that sets these models apart. LLMs start as dumb automata, but somewhere in their training, they become smart enough to generalize their training to tasks for which they are not explicitly trained. This is what we can more freely term as "understanding". As I mentioned before, this is a highly debated subject and our aim is not to go deep into the debate but to try to learn.