Copying Classmates: Using LLMs to Bootstrap One Another As RLHF's prohibitive price deters many, some researchers are cutting costs by replacing human feedback with the output of other language models. Stanford's Human-Centered AI group demonstrated this shortcut. Seeking to retool Meta's 7 billion parameters LLaMA into a more capable assistant like ChatGPT, they wrote 175 seed tasks and used OpenAI's DaVinci model to generate a 52,000 example dataset for just $500. Rather than painstaking human annotation, davinci produced labeled data automatically. Fine-tuning LLaMA on this synthetic dataset created Alpaca-7B, an adept assistant, for just $600 total. Researchers at UC Berkeley collaborated to train the 13 billion parameters Vicuna model using 70,000 conversations scraped from ShareGPT (a website where people share their ChatGPT threads). Vicuna reached 90% of ChatGPT's performance, trained on data at a fraction of the cost. Meanwhile, Microsoft and Peking University developed Evol-Instruct, a novel method to exponentially grow seed data using an LLM. Their resulting WizardLM even surpasses GPT-4 in certain skills, demonstrating the power of synthetic datasets. Open-source datasets like RedPajama's 1.2 trillion token replicas of LLaMA also aim to democratize model training.


We Train Our Machines, Then They Retrain Us: The Recursive Nature of Building AI

We become what we behold. We shape our tools, and then our tools shape us Churchill grasped centuries ago how constructed spaces shape collective norms. His insight rings true once more as LLMs proliferate through synthetic training loops. The architectural choices made in those initial models propagate as their output trains new generations. Early flaws and biases subtly ripple down the line, shaping discourse itself. When LLMs generate text read by millions daily, their idiosyncrasies permeate human expression. As models optimized for engaging chat advise writers, their tendencies infuse published content. Our tools reflect our values, and in time, remake us in their image. But this dance between creator and creation is not abnormal - it's the story of human progress. From the printing press to the television, inventions have always circled back to influence their makers. LLMs are just the latest invention to shape society as it is shaped by us.


Metas New Text-to-Image Model - CM3leon Paper explained

Results But, okay, first, let's have a closer look at the results and why CM3Leon is so special! We already looked at nicely generated images, but when we look at the numbers in the figure above, we can really see how much better CM3leon is than other models, such as DALLE and Parti. Now, one can argue how well the FID score actually captures the realism of the generated image, but nevertheless, CM3Leon sets a new state-of-the-art FID score on the MS COCO dataset. Where I do have to say, choosing the autoregressive version of the DALLE model is a bit unfair.


How does DALLE work? Just on a small tangent, DALLE has an intermediate model, called the prior, that maps the text embedding generated by CLIP to a corresponding image encoding, which is then used to generate the final image using diffusion. This prior can again be a diffusion process or an autoregressive one. Both appear to yield similar results, but the diffusion process is more efficient, so the authors of DALLE went with that one.So yeah, choosing the less efficient version for the comparison here might be a bit unfair for DALLE.


Back to image generation results Now, okay. The nice FID score alone is great, but even better is that CM3Leon achieves this performance while being a lot more efficient!! With its largest 7 billion parameter model, it is way smaller than the largest Parti model, and it uses a fraction of the training data and time!Also, the authors introduce a new metric for "responsible" training. The images used for training CM3Leon are all licensed from Shutterstock, so (hopefully) no more fear of lawsuits!Then there is the column "Nr. of retrieved documents" which is one of the main features of the CM3Leon model that makes it so great, and we'll get to know more about it in a second. But in short, given a text prompt, the model can somehow retrieve relevant images or even text from a memory bank and use it as further context for the image generation process.


What is considered zero-shot generation? Now, the thing that makes me wonder is that if the model retrieves further images for the generation process, is it really zero-shot generation? I guess the model itself retrieves the extra images and they are not provided by the person prompting the model, but yeah, not 100% sure here. On that note, in the abstract, the authors mention their top-performing model that achieves a "zero-shot MS-COCO FID [score] of 4.88" But this score is achieved with two retrieved images, i.e. they refer to the retrieval as zero-shot. But in their caption of Figure 1, they say the following: So, what is a zero-shot generation? With retrieval or without?


Multimodal benchmarks Okay, up until now, it was all about image generation, but since this model is an autoregressive, decoder-only model, pretty much like all big LLMs, it can also interpret images as normal tokens and use them as context for text generation! In other words, after applying Supervised Fine-tuning (or, for short SFT), our model can also do more complex multimodal tasks. It is very good at interleaved text and image tasks such as "Text guided editing", and "Image-to-Image Grounded Generation" where you can provide a segmentation map, sketch with only outlines, or even a depth map, and based on those and a text prompt the new image is generated. Or, with "Spatially Grounded Image Generation", you can even provide the coordinates of an object in the text prompt, and the generated image will place that object at those coordinates. On top of that, the model is quite good at actually generating text in images, which was not the case for a while.And by a while, I mean literally less than one or two years ago :))Finally, with the correct Supervised Fine-tuning, CM3Leon can also take images as input and do tasks like Image captioning with short or long answers, Visual Question Answering, and Reasoning. It still is not as good as dedicated image captioning models like Flamingo [4], but since this is more or less just a side effect of the model design, the results are still very impressive!


How does CM3Leon work? Okay, sick, but how does CM3Leon work, and what does retrieval-augmented, auto-regressive, decoder-only model mean!? At this point, we all more or less know how diffusion works. A model is trained to predict noise in an image so that when we start off with completely random noise, we can apply this model and remove the noise step by step. This noise removal process can now also be conditioned on the text so that we can guide the generation process with our prompt. Autoregressive models work a bit differently.Let's have a look at how Parti implements this idea of an autoregressive image generation model.


What is Autoregressive? Remember how Autoencoders work? We have an Encoder network that maps an image to some embedding so that the decoder can then generate the same image only from a latent vector representation. In the image below, this idea is illustrated by the green module. Now, what if those embeddings the decoder (in the case of Parti called detokenizer) uses for generating the image come in the form of tokens predicted by a Language Model? Think of how a GPT model starts with a simple Start-of-Sequence token and predicts the next token or rather token-embedding out of a given vocabulary in the know autoregressive fashion.A vision transformer also generates embeddings for each patch token of the image and those can also be constrained to be from a certain vocabulary. That means that our autoregressive text-decoder (blue module) can also generate each image embedding token and then let the image generator (or again, here called detokenizer) generate the image.Now, to condition the image token generation process, Parti decides to go with an encoder-decoder approach. It uses the full Transformer architecture for what it was originally designed, translating text. In this case simply translating the language of text into the language of images, or, text tokens into image tokens. In other words, they use Cross-attention where the text-encoder (yellow module) embeddings are used as conditioning for the text-decoder that predicts one image token after the other. This autoregressive approach solves some issues that diffusion models had. E.g. autoregressive models can much better deal with long text prompts and can generate text in images very well!But it can do so only with scale. The case of "simply go bigger and it will work" is quite extreme here. We can look at this example with the kangaroo and the text "Welcome Friends!".


What is decoder-only? Okay, we now know how autoregressive image generation works, but CM3Leon is an autoregressive decoder-only model and not an encoder-decoder model like Parti. What that pretty much means, is that the text conditioning for the generation process happens not via an encoder and cross-attention, but as simple text tokens in the decoder context. That means that the vocabulary now has to include the image token vocabulary (where the authors use an already existing tokenizer) and the text token vocabulary (where the authors train a new tokenizer themselves). Additionally, the authors introduce a new <break> token that indicates a transition in modalities. Now our decoder input and output can look something like this where we start with the prompt "A Photo of a cat shown on a dslr" followed by the <break> token and the decoder then predicts the next image tokens one by one. Once we reach another <break> or <EOS> token, the image decoder can take over and generate the image! When it comes to training data, the model can handle multimodal cases such as "Image of a chameleon:" followed by the image tokens, where the model is simply trained on the standard next token prediction loss. But this also means that the model can handle image captioning tasks by simply reformatting the same example to mask a certain part of the sample and expect the model the predict the masked part after the <infill> request. In other words, the model sees the image and has to predict what this image contains. Our model can now generate both text and images. Autoregressive, check.Decoder-only, check.Finally, how does this "retrieval-augmentation" thing work?


What is retrieval-augmentation? I already mentioned, that this pretty much means, that given the initial prompt, the model can retrieve either images or text, or both, and add it to its context. Since we now know how the input to the decoder-only model works, we can pretty easily understand how simple it is to just add more images and text to the context. We can simply add as much text and images sitting in between <break> tokens as the maximum context length allows! And with a sequence length of 4096, we should be okay to add one or two retrieved documents.Documents hereby refer to elements that can be single images or text, or an image and caption pair. CM3Leon builds a lot on the RA-CM3 paper, [6], which proposed to add this retrieval augmentation feature to the method proposed in the CM3 paper, [7]. Now that's research, one paper building on the other! In the RA-CM3 paper, we can very nicely see the effects of retrieving an image for a given text prompt. Here e.g. we can see some outputs to the prompt "French Flag waving on the moon's surface" where the vanilla CM3 without retrieval and Stable Diffusion simply place the American flag on the moon, but if we now add a retrieved image of the French flag to the context, RA-CM3 generates a proper result. Similar effects are seen when retrieving two images, and so on. This, of course, also allows you to specify an image manually to control the style of the generated image, as we can see here in this example. Providing an image of a person with a red jacket will also lead to the model inpainting a person with a red jacket. Cool, so how does this retrieval work? The idea is actually really simple.The authors use an off-the-shelf, frozen CLIP model to encode the input query, e.g., a simple text prompt, and sort similar candidates from a memory bank by a relevance score. The individual text and image examples in the memory bank simply pass through the CLIP model once, but for the image and caption documents, the authors split the text and image, encode them separately, and then average the two as a vector representation of the whole document. The authors don't simply select the most similar document but use different heuristics to make the retrieval more informative. E.g., a document consisting of an image and text is more informative than only text or only an image. Or they skip a candidate document if it is too similar to the query or if pretty much the same document has already been retrieved. And a few other tricks. There are other minor (but important!) details, to make the model output itself work even better, but those are no fundamentally new ideas, just add-on tricks that generally appear to increase performance. E.g., randomly sampling different temperature values, or something called TopP sampling, applying classifier-free guidance during inference, or their own adaptation of Contrastive Decoding. So, in the end, we now know how the retrieval-augmented, autoregressive, decoder-only CM3Leon model works, and what it can do. This retrieval idea is a key contributor to the fact that the model is so parameter and thus data efficient! The model does not have to memorize the whole world's information, like, what is a cherry blossom tree or Mount Rushmore and what they look like. And using a decoder-only architecture makes the training and fine-tuning to new tasks much easier! After potentially thinking diffusion models are the best way to go for image generation, this paper just again shows how powerful autoregressive models such as the famous transformer models are for a broad range of text and image tasks. And if you enjoyed reading this post, don't forget to leave a clap and follow me for more exciting AI paper explanation posts!


References [1] Scaling Autoregressive Multi-Modal Models: Pretraining and Instruction Tuning, L. Yu, B Shi, R Pasunuru, et al., 2023, Link to paper [2] High-Resolution Image Synthesis with Latent Diffusion Models, R. Rombach, A. Blattmann, D. Lorenz, P. Esser, B., 2021, https://arxiv.org/abs/2112.10752 [3] Zero-Shot Text-to-Image Generation, Ramesh et al., 2021, https://arxiv.org/abs/2102.12092 [4] Flamingo: a Visual Language Model for Few-Shot Learning, B. Alayrac, J. Donahue, P. Luc, A. Miech et al., 2022, https://arxiv.org/abs/2204.14198 [5] Scaling Autoregressive Models for Content-Rich Text-to-Image Generation, Yu et al., 2022, https://arxiv.org/abs/2206.10789 [6] Retrieval-Augmented Multimodal Language Modeling, Yasunaga et al., 2022, https://arxiv.org/abs/2211.12561 [7] CM3: A Causal Masked Multimodal Model of the Internet, A. Aghajanyan et al., 2022, https://arxiv.org/abs/2201.07520


Deploying LLM Chat Applications with Declarai, FastAPI, and Streamlit

Declarai in Action  Declarai's ethos is empowering developers to declare their intended task. For our demonstration, we'll create a SQL Chatbot that fields SQL-related queries. To begin, we design our system prompt - the guiding message that sets the boundaries for our chatbot's capabilities. We are using gpt3.5 to create a SQL Chatbot that is helping us with any SQL-related questions. In Declarai, this guiding message is seamlessly embedded within the class's docstring, ensuring clarity and readability. We can also initiate the chat with a friendly greeting: Now, let's interact with our chatbot: With the SQLChat example laid out, you've glimpsed the power of a well-structured conversational model. Note : Remember, the SQLChat is just one example. You can easily tailor the chatbot to your specific needs by adjusting the system message. The next step is setting up our backend. Using FastAPI and Streamlit, we'll bring our chatbot to life and make it accessible to users.


The API Backend  To bring our chatbot to a wider audience, we'll utilize FastAPI as our RESTful API gateway and Streamlit for frontend integration. Our FastAPI configuration establishes two key routes: one for submitting new messages and another for retrieving chat histories. Declarai's FileMessageHistory ensures continuity in our chat threads across sessions by saving the conversation state at every interaction. If you prefer to save message history into a database (Postgres/redis/mongo, you can do so by simply replacing the FileMessageHistory class)


The Frontend Experience  Our user interface is a chat GUI developed using Streamlit. The streamlit setup is fairly concise as well: Upon providing a chat session name, users can initiate a conversation. Every message interaction calls the backend, with a spinner providing visual feedback during processing. Ready to deploy your chatbot ? Dive into the complete code in this repository - Stay in touch with Declarai developments . Connect with us on Linkedin Page , and give us a star ⭐ on GitHub if you find our tools valuable! Dive deeper into Declarai's capabilities by exploring our documentation


ROUGE Metrics: Evaluating Summaries in Large Language Models.

What is ROUGE? ROUGE consists of a set of metrics that are used to compare the quality of a generated text against a reference text. It can be used whenever we have a reference text available. Some of the most common applications are: translations, text summarization, or entity extraction between others. It's important to note that ROUGE is only as reliable as the quality of the reference text. If they have poor quality, the results provided by ROUGE may not reflect correctly the quality of the generated text. Another utility of ROUGE is measuring how different the outputs produced by two different models are. Maybe we want to see if there are any significant differences between the texts produced by the two models. Additionally, we can use ROUGE to estimate whether processes like pruning or quantization (weight reduction of models) have significantly altered the results produced by our model for the specific task we want to use it for. The metrics provided are: ROUGE-1: This indicates the match between the generated text and the reference text using 1-grams or individual words.ROUGE-2: The same than ROUGE-1, but It considers sets of 2-grams.ROUGE-L: This metric evaluates the match of the longest common subsequence of words between the two texts. The words do not need to be in the exact same order. More information about the ROUGE metrics can be found at: https://pypi.org/project/rouge-score/


What are we going to measure in our Notebook? We will measure a base T5 model against a T5 model fine-tuned for summary generation. Initially, we will analyze the differences between the two models to determine if the fine-tuning process has produced a model with distinct behavior. Later, we will evaluate both models using the reference texts provided in the CNN-DailyMail Dataset, to see which one produces better summaries. The source code for this article, as well as the entire course on Large Language Models, can be found in my GitHub repository along with the corresponding English articles. The notebook used for this article can be accessed at: https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/rouge-evaluation-untrained-vs-trained-llm.ipynb The models used can be found on Hugging Face. Fine-Tuned: t5-Base: https://huggingface.co/flax-community/t5-base-cnn-dmBase: t5-Base: https://huggingface.co/t5-base


Loading the data. In the first part of the article, we will use a dataset available on Kaggle containing MIT news that does not have generated summaries. This dataset will be used to verify that both loaded models generate summaries different enough. Dataset Link: https://www.kaggle.com/datasets/deepanshudalal09/mit-ai-news-published-till-2023 Let's begin by importing the typical Python libraries: All of these are well-known libraries in the Python universe. Importing torch is necessary because ROUGE requires it. Let's load the dataset. The complete article content is contained in the column Article Body. This is the only field from the Dataset that we need. We are selecting only three articles to speed up the execution of the notebook. We've stored the complete text of the three loaded articles in the variable named articles. If we want to work with more or fewer news articles, it is as simple as changing the value of the variable MAX_NEWS.


Load the models and create the summaries. The two models used are from the T5 family. One of them is the foundational T5-Base model, and the other is a model derived from T5-Base but fine-tuned with the CNN-DailyMail dataset. Since both models are available on Hugging Face, we'll work with the transformers library. To obtain the tokenizers and models, I created the following function: This function receives the model_id as input, and returns the loaded model jointly with the tokenizer. The tokenizer is used to convert the text into sequences of tokens that can be processed by the model. Using the get_model function, we can obtain the two tokenizers and models: To generate the summaries, we will create a function that takes four parameters: A list of texts to summarize.The tokenizer.The model.The maximum length for the obtained summary. The function should format, a bit, the text of the articles to be summarized. This involves introducing a prefix before each article. This prefix will consist of instructions for the model. In our case, we want the model to make a summary of the articles, so the prefix to add will be: "Summarize this news: ". Using the tokenizer, each article will be transformed into encodings, that will be sent to the model. The model will return a summary, that needs to be decoded and converted from encodings to text. Let's look into a specific part of the function's code: the model invocation for generating the summaries: As you can see, we are starting the block with the line: with torch.no_grad() which indicates that we don't want to calculate gradients within the block. This improves the speed of the code. Then we call model.generate with the following parameters: input_ids: The encodings representing the tokenized input text.attention_mask: The attention mask, provided by the tokenizer.max_length: The maximum length of the model's response.beams: The greater the number of beams, the greater the diversity in the model's generated response. We're keeping it at 2 to introduce some variety.early_stopping: this parameter allows the model to stop generating text before the response reaches the maximum length. It is a good practice to set his value to True. Let's see how to call the create_summaries function to obtain the summaries: As you can see, it's simple. You just need to pass the texts to be summarized along with the tokenizers and the models. Let's take a look at the generated summaries from the two models: summaries_small: ['MIT and MIT-Watson AI Lab have developed a unified framework. the system can simultaneously predict molecular properties and generate new molecules. it uses this grammar to construct viable molecules and predict their properties.', '\'BioAutoMATED\' is an automated machine-learning system that can select and build an appropriate model for a given dataset. it can even take care of the laborious task of data preprocessing, whittling down a months-long process to just a few hours. \'"We want to lower these barriers for a lot of folks that want to use machine learning or biology," says first co-author Jacqueline Valeri.', "MIT and IBM research scientists have made a computer vision model more robust by training it to work like a part of the brain that humans and other primates rely on for object recognition. 'we asked the artificial neural network to make the function of one of your inside simulated "neural" layers as similar as possible to the corresponding biological neural layer,' says MIT professor."] summaries_reference: ['Researchers created a machine-learning system that automatically learns the "language" of molecules using only a small, domain-specific dataset. The system learns to construct viable molecules and predict their properties. Computational design and Fabrication Group will be presented at the International Conference for Machine Learning.', "Automated machine-learning system can select and build an appropriate model for a given dataset. 'BioAutoMATED' is an automated machine-learning system. The tool includes binary classification models, multi-class classification models, and more complex neural networks.", "MIT and IBM researchers have found that artificial neural networks resemble the multilayered brain circuits that process visual information in humans and other primates. 'We asked it to do both of those things as well as the standard, computer vision approach,' said one expert. The network found to be more robust by training it to work like a part of the brain that humans rely on for object recognition."] At first look, you can already observe that the generated summaries are different. However, it is difficult to determine which one of them is better for our needs simply by looking at the summaries. To determine whether the summaries are significantly different, we'll use ROUGE. When comparing two generated summaries, where neither of them serves as the reference summary, we aren't gaining an understanding of the quality of the generated summaries per se. Instead, we're obtaining an indication of how different the results are from each other. Allowing us to identify if the fine-tuning process has had some impact on the outcomes of the summaries.


Calculating ROUGE Let's install and load the necessary libraries so that we can run the ROUGE metrics. While there are several libraries that implement ROUGE calculation, I've decided to use the evaluate library. Sometimes, the choice is based on familiarity, and this is the library I'm habituated to using. Once we have the libraries imported, calculating the ROUGE metric is as simple as a single call to the compute function of the rouge_score object we just created. The texts are passed to this function, along with a third parameter indicating whether you want to use word stems (lemmas) or whole words for the comparisons. It's common to compare only the stems so that words like "jump" and "jumped" are considered equal. Here are some lemmas examples: Jumping -> Jump.Running -> Run.Cats -> Cat. When using True for the steamer parameter, words with the same stem are considered equal, while using False treats them as distinct words. However, it's important to prepare the text a bit by adding line breaks at the beginning of each line and removing empty characters. To accomplish this, let's create the compute_rouge_score function: As you can see, this function modifies the texts received as parameters by adding line breaks to each sentence. Thereafter, it makes only a single call to the rouge_score.compute function. In this call, we are passing the generated text using both the T5 Base model and the fine-tuned T5 model. The result it returns is: {'rouge1': 0.47018752391886715, 'rouge2': 0.3209013209013209, 'rougeL': 0.34330271718331423, 'rougeLsum': 0.44692881745120555} We can see that the degree of similarity in ROUGE-1 is 47%, whereas in ROUGE-2, it's 32%. This clearly indicates that the summaries generated by both models are distinct. With some similarities but significant differences, suggesting that the fine-tuning process has influenced in how the model generates the summaries. However, we can't yet determine which model is better, as we haven't compared against any reference text.


Comparing with a reference text. Now we are going to use a second Dataset: the CNN_Dailymail, a well-known dataset that's part of the Datasets library. Apart from the articles, this dataset contains human-generated summaries that we are going to use as reference texts. We will compare the summaries generated by the two models with the reference summaries in the dataset. This will help us determine which model produces summaries that are more similar to the ones provided as reference texts. As in the first part of the article, we will load only a limited number of news. And it will help to keep the execution time under control. We obtain the maximum length of the selected summaries of the dataset, to indicate the maximum length that we want the response generated by the models to have. We now have all the necessary to generate the summaries. With the three summaries in their respective variables. Let's look at the content. The truth is that at a glance, it's really difficult to know which of the two models creates better summaries. Let's compute the ROUGE metrics for both models. summaries_t5_base: {'rouge1': 0.3050834824090638, 'rouge2': 0.07211128178870115,'rougeL': 0.2095520274299344, 'rougeLsum': 0.2662418008348241} summaries_t5_finetuned: {'rouge1': 0.31659149328289443, 'rouge2': 0.11065084340946411, 'rougeL': 0.22002036956205442, 'rougeLsum': 0.24877540132887144} With these results, I would say that the fine-tuned model makes better summaries than the T5-Base model. This is based on the fact that it achieves higher scores in all metrics, except for LSUM, where the difference is minimal. It's also important to note that ROUGE metrics are highly interpretable and do not provide an absolute truth. In other words, a model isn't necessarily better than another simply because its ROUGE scores are higher. These scores only indicate that the texts it generates have more similarity to the reference texts than those produced by another model. Both models have very similar results, but the fine-tuned model achieves higher scores in all metrics, especially in ROUGE-2 and ROUGE-L, except for ROUGE-LSUM. This implies that the Base model might generate texts that are more similar, while the fine-tuned model utilizes a vocabulary more similar to the reference texts. In any case, we cannot obtain clear conclusions since we have only analyzed a few summaries. To determine which model is the best, we would need to develop a different strategy. A nice approach can be: grouping the news by topic and examining whether there are significant differences in results.


Conclusions. Metrics are not enough, even metrics like ROUGE falls short for us, since it depends largely on the quality of the texts we take as reference. The human factor is still very important to check the suitability of the responses. This is why the Models that have been trained with human mediation are succeeding. At least we have one good news: it is quite easy to get metrics like ROUGE, which can help us in the decision process, and make better model judgments.


Inside AVIS: Google's New Visual Information Seeling LLM

Some Background The ideas behind Google's AVIS have their roots in recent areas of research. Recent explorations, such as Chameleon, ViperGPT, and MM-ReAct, have focused on enhancing LLMs with supplementary tools for multimodal inputs. These systems operate in two stages: planning, which involves deconstructing questions into structured instructions, and execution, wherein tools are employed to amass information. While this approach has exhibited success in rudimentary tasks, it often falters when confronted with intricate real-world scenarios. A growing interest has also been observed in utilizing LLMs as self-governing agents, as seen in WebGPT and ReAct. These agents interact with their environment, acclimatize based on real-time feedback, and accomplish objectives. However, these methodologies do not impose limitations on the assortment of tools that can be invoked at various stages, resulting in an extensive search space. Consequently, even the most advanced LLMs can succumb to infinite loops or propagate errors. The AVIS method addresses this issue through guided LLM application, influenced by human decisions drawn from a user study. Numerous visual queries within datasets like Infoseek and OK-VQA pose challenges even for human responders, often necessitating the aid of diverse tools and APIs. To gain an understanding of human decision-making when employing external tools, a user study was conducted. Participants were equipped with an identical set of tools as utilized in the AVIS method, including PALI, PaLM, and web search. They received input images, questions, cropped object images, and buttons linking to image search findings. These buttons provided assorted information about the cropped object images, such as entities within knowledge graphs, similar image captions, correlated product titles, and identical image descriptions. User actions and outputs were recorded and employed as a reference for the AVIS system in two significant ways. Primarily, a transition graph was constructed by analyzing the sequence of decisions made by users. This graph delineates discrete states and confines the available range of actions within each state. For instance, at the initial state, the system is limited to three actions: PALI caption, PALI VQA, or object detection. Secondly, examples of human decision-making were leveraged to steer the planner and reasoner within the AVIS system, imbuing them with pertinent contextual instances to heighten performance and efficacy.


Inside AVIS In AVIS, Google Research adopts a dynamic approach to decision-making specially tailored for addressing queries involving the quest for visual information. The methodology consists of three fundamental components within the system's architecture. 1) Initially, a planner takes the helm, deciphering the subsequent course of action, encompassing the appropriate API invocation and the corresponding query for processing. 2) Alongside, a working memory module is in place, diligently preserving data regarding outcomes stemming from API executions. 3) Lastly, a reasoner assumes a pivotal role by sifting through the results generated from API calls. This role encompasses assessing whether the acquired information holds enough merit to furnish a definitive response or if further data retrieval is imperative. The planner embarks on a sequence of steps whenever a verdict is to be rendered concerning the selection of tools and the corresponding queries to dispatch. Tailored to the current state, the planner unfolds an array of possible ensuing actions. The breadth of potential actions can become unwieldy, posing challenges in the search space. To tackle this quandary, the planner references a transition graph, expelling irrelevant actions from consideration. Actions previously executed and enshrined in the working memory are also excluded. Subsequently, the planner assembles an array of pertinent contextual instances, derived from the judgments earlier rendered by human participants during the user study. Armed with these instances and the reservoir of data encapsulated within the working memory, the planner crafts a prompt. This prompt is then channeled to the LLM, which furnishes a structured response. This output dictates the subsequent tool to activate and the correlated query to dispatch. This blueprint sanctions the planner to be triggered recurrently throughout the process, thereby effectuating dynamic decision-making that progressively navigates toward addressing the initial query. The reasoner steps into the fray to dissect the outcomes of tool executions. It gleans valuable insights and discerns the category into which the tool output aligns: informative, uninformative, or culminating response. The technique hinges on harnessing the LLM with suitable prompts and contextual instances for the reasoning task. If the reasoner ascertains readiness to yield a response, it duly issues the definitive reply, thereby drawing the task to a close. In instances where the tool output proves barren, the reasoner defers to the planner for the selection of an alternative action, grounded in the extant state. Should the tool output prove insightful, the reasoner orchestrates a state shift and reinstates the planner's authority, prompting a fresh decision premised on the novel state.


The Results As expected, Google evaluated AIVS on different visual information-seeking benchmarks such as Infoseek and OK-VQA datasets. The results were incredibly impressive, even achieving over 50% accuracy without fine-tuning. The AVIS research combines different ideas in a novel framework for visual information-seeking models. Don't be surprised if we see AVIS incorporated into the new wave of multimodal foundation models released by Google.


This AI newsletter is all you need #61

What happened this week in AI by Louie In recent months we have continued to see large language model (LLM) advancements and a gradual introduction of novel techniques but we haven't yet seen competition directly aiming to displace GPT-4 as the most advanced (and training compute-intensive) model. Google consolidated its AI efforts by merging Google Brain and Deepmind earlier this year and has also been rapidly scaling up its training compute resources. Its Gemini model is likely to be the first major new release from this new merged effort. Google has gradually been generating anticipation for Gemini by revealing information through interviews and controlled media releases. The company is likely preparing for Gemini to serve as its response to GPT-4, with ambitions for it to outperform the latter in certain capabilities. With the release now reportedly gearing up for "this fall," we are excited to see what new innovations and capabilities Gemini brings and how it stacks up against GPT-4. The effort is being led by Oriol Vinyals and Koray Kavukcuoglu, alongside Jeff Dean, who oversees hundreds of employees in Gemini's development. We have heard several things about the model, both via direct quotes from management and media leaks. "Gemini combines the strengths present in AlphaGo-type systems with the exceptional language capabilities inherent in large models," - Demis Hassabis. We also know that the model started training sometime before May; "We're already at work on Gemini - our next model created from the ground up to be multimodal, highly efficient at tool and API integrations, and built to enable future innovations, like memory and planning. Gemini is still in training, but it's already exhibiting multimodal capabilities never before seen in prior models." - Google CEO blog, May-23. The model is anticipated to be multimodal with full image generation capabilities like Midjourney. We have also heard Google is being careful about its training set and that it might have integrated Video and Audio data from YouTube into Gemini. The ongoing rivalry between Google and OpenAI is exciting to witness, and it will be intriguing to observe how these developments unfold in the future and particularly which techniques from AlphaGo Deepmind are integrating into LLMs. It looks like we are set to see even more exciting evolution in LLMs this year! - Louie Peters - Towards AI Co-founder and CEO This issue is brought to you thanks to OVHcloud: OVHcloud is offering GPUs at unbeatable prices to drive your AI needs. This includes a selection of NVIDIA cloud instances up to 60% off regular prices - while stocks last. Designed to accelerate data processing with the guarantee of complete data reversibility and resource flexibility, the OVHcloud AI portfolio also features bare metal servers and open-source ML solutions such as AI Notebooks, AI Training, and AI Deploy, all benefiting from OVHcloud's water-cooling technology to achieve the lowest energy consumption.


This AI newsletter is all you need #61

Hottest News Meta's Next Big Open Source AI Dump Will Reportedly Be a Code-Generating Bot Meta's next AI release will reportedly be a coding machine. This new model, dubbed "Code Llama," will be open-source and available free online. It could see a release as soon as next week. This is consistent with the company's strategy so far of releasing widely available AI software that makes developing new customizable AI models much easier. 2. OpenAI's Content Moderation OpenAI proposed the utilization of GPT-4 for the development of content policies and the facilitation of content moderation decisions on digital platforms. This approach aims to relieve humans of the burden involved, thereby enabling more uniform labeling of content and expediting feedback loops. 3. Microsoft Plans AI Service With Databricks That Could Hurt OpenAI Microsoft's recent Azure+Databricks offering provides Databricks users with the capability to employ any AI model, including open-source LLMs, for training their data on the Azure platform. This could potentially lead to a decrease in the number of companies procuring licenses for OpenAI models to fulfill similar use cases. 4. Google's Top AI Experts Plan on Setting Up Their Own Development Studio in Japan Google's top AI experts, whose research contributed to the development of AI bots like OpenAI's ChatGPT, Google Bard, Stability AI, Midjourney, and Dall-E, have intentions to establish their independent AI development studios. Sakana AI is spearheading efforts to create its proprietary generative AI model, which means software that can create text, images, code, and more. 5. Biased AI Needs Human Help To Avoid Harmful Climate Action, Say Researchers According to a new paper by researchers at the University of Cambridge, bias in the collection of data on which AI computer programs depend can limit the usefulness of climate scientists in predicting future scenarios and guiding global action. The paper concludes that human-guided technology remains instrumental in the development of socially responsible AI.


This AI newsletter is all you need #61

Five 5-minute reads/videos to keep you learning Anti-Hype LLM Reading List Navigating the current AI hype can be challenging, making it difficult to discern what is truly substantial. This compilation comprises a thoughtfully curated collection of foundational papers, interesting open questions, and a guide for gaining deeper insights into the space. 2. Why You (Probably) Don't Need To Fine-Tune an LLM This post elaborates on the reasons why fine-tuning might not be essential for your application. It delves into a comprehensive explanation of what fine-tuning entails and explores potential alternatives. The content is targeted toward those focused on building LLM applications. 3. Open Challenges in LLM Research The swift growth of AI has given rise to novel research directions. This article brings together the prevailing challenges within LLM research such as multimodality, alternatives to GPUs, innovative architectures, and more. 4. AI2 Dolma: 3 Trillion Token Open Corpus for Language Model Pretraining AI2 Dolma is a dataset of 3 trillion tokens drawn from a diverse array of sources including web content, academic publications, code repositories, books, and encyclopedic materials. Its primary objective is to provide researchers with the means to investigate the impact of data at scale. It is readily accessible for download on the HuggingFace Hub. 5. Tips for Writing NLP Papers This post shares how to build a standardized process for writing NLP papers. It presents essential components including content structuring, language precision, comprehensive literature reviews, accurate citations, and more. While certain pointers are tailored to NLP research, the principles laid out herein can be effectively employed across niches.


This AI newsletter is all you need #61

Papers & Repositories Self-Alignment with Instruction Backtranslation This paper presents a scalable method to build a high-quality instruction-following language model by automatically labeling human-written text with corresponding instructions. It starts with a language model fine-tuned on a small amount of seed data and a given web corpus. 2. Neuralangelo: High-Fidelity Neural Surface Reconstruction This paper presents Neuralangelo, which combines the representation power of multi-resolution 3D hash grids with neural surface rendering. The two key ingredients that enable this approach are numerical gradients for computing higher-order derivatives and coarse-to-fine optimization on the hash grids. 3. Consistent Collaborative Filtering via Tensor Decomposition This work developed Sliced Anti-symmetric Decomposition (SAD), a new model for collaborative filtering based on implicit feedback. SAD produces the most consistent personalized preferences while maintaining a top level of accuracy in personalized recommendations. 4. Efficient Guided Generation for Large Language Models This paper shows how the problem of neural text generation can be constructively reformulated in terms of transitions between the states of a finite-state machine. It guides text generation with regular expressions and context-free grammar by allowing the construction of an index over a language model's vocabulary. 5. neuml/txtai Txtai is an all-in-one open-source embedding database for semantic search, LLM orchestration, and language model workflows. It can be set up in minutes, runs locally, has a low footprint, and works with micromodels up to large language models. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #61

Meme of the week! Meme shared by rucha8062


This AI newsletter is all you need #61

Featured Community post from the Discord Marcklingen has recently introduced "langfuse," an open-source observability and analytics tool designed for LLM (Large Language Model) applications. This tool enables users to accelerate their application development process by providing a detailed perspective of precise execution traces, encompassing aspects like quality, cost, and latency. Currently, Langfuse Analytics is in a closed alpha phase, during which the core team collaborates with a user group to construct the most beneficial analytics platform for LLM applications. Check it out on GitHub and support a fellow community member. Share your feedback and questions in the thread here.


This AI newsletter is all you need #61

Article of the week How I Built A Video Recommendation System Using Large Language Models and Vector Database by Zoumana Keita In this article, you will learn how to leverage large language models, state-of-the-art text and speech analytics tools, and vector databases to build an end-to-end audio recommendation solution. This solution will suggest top videos based on users' interests.


This AI newsletter is all you need #61

Our must-read articles Metas New Text-to-Image Model - CM3leon Paper explained by Boris Meinardus Use LangChain to Extract a Dictionary from Text by Eugenia Anello Busy? This Is Your Quick Guide to Opening the Diffusion Models Black Box by Paul Iusztin If you want to publish with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #61

Job offers Senior Staff Writer (Tech Software) @Future Publishing (London, UK) Junior Software Engineer (React.js) - OP01114 @Dev.Pro (Remote) Solution Architect (Software Development) @Uni Systems (Warsaw, Poland) Software Engineer @Mention Me (Remote) Software Engineer, Full Stack @Basetwo (Remote) Senior Data Engineer @Hertility Health (Remote) Artificial Intelligence Engineer @Plain Concepts (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Meet SeamlessM4T: Meta AI's New Foundation Model for Speech Translation

SeamlessM4T SeamlessM4T (Massively Multilingual and multimodal Machine Translation) is an integrated platform encompassing ASR, T2TT, speech-to-text translation (S2TT), text-to-speech translation (T2ST), and S2ST functionalities. The model builds on a long history of Meta AI's breakthrough in the speech translation space. Notably, Meta AI introduced No Language Left Behind (NLLB) in the previous year - a text-to-text machine translation model engineered to encompass an impressive 200 languages. In the following months, Meta AI showcased the pioneering Universal Speech Translator. This groundbreaking system facilitated direct speech-to-speech translation for Hokkien, a language characterized by its absence of a widely adopted writing system. This endeavor also yielded the creation of SpeechMatrix, a monumental multilingual speech-to-speech translation dataset. This dataset, born from the innovation of SpeechLASER, marked a milestone in the realm of supervised representation learning. A subsequent stride materialized earlier in the current year with the unveiling of Massively Multilingual Speech. This comprehensive offering encompassed automatic speech recognition, language identification, and speech synthesis capabilities spanning an expansive array of over 1,100 languages. SeamlessM4T emerges, synthesizing insights gleaned from these diverse projects. The outcome is a transformative multilingual and multimodal translation experience, stemming from a singular model. This model is meticulously constructed, drawing from an extensive spectrum of spoken data sources, and culminating in state-of-the-art outcomes. To construct a unified model, Meta AI requires a lightweight sequence modeling toolkit that can seamlessly integrate with other modern PyTorch ecosystem libraries. To fulfill this need, Meta AI has reengineered fairseq, its original sequence modeling toolkit. By incorporating more efficient modeling and data loader APIs, fairseq2 now plays a pivotal role in driving the underlying modeling processes of SeamlessM4T. At the core of the model lies the multitask UnitY model architecture, designed to perform a range of functions, including generating translated text and speech. This architecture also facilitates automatic speech recognition, text-to-text translation, text-to-speech conversion, speech-to-text translation, and speech-to-speech translation - features that are already inherent in the vanilla UnitY model. The multitask UnitY model is structured around three primary sequential components. Text and speech encoders are entrusted with the task of recognizing speech input across nearly 100 languages. Subsequently, the text decoder transforms that meaning into various languages for textual content, followed by a text-to-unit model that decodes it into discrete acoustic units tailored for 36 speech languages. Through pre-training of the self-supervised encoder, speech-to-text, text-to-text translation components, and text-to-unit model, the quality of the model is enhanced, and its training stability is ensured. The resultant decoded discrete units are then transformed into speech using a multilingual HiFi-GAN unit vocoder. Meta AI employs a self-supervised speech encoder known as w2v-BERT 2.0 - an enhanced iteration of w2v-BERT distinguished by improved training stability and representation quality. This encoder is trained to discern structure and meaning within speech patterns, drawing insights from vast volumes of multilingual speech spanning millions of hours. Functionally, the encoder dissects the audio signal into smaller segments, constructing an internal representation of the spoken content. Given that spoken language comprises various sounds and characters, a length adaptor is employed to map these elements to corresponding words, albeit in an approximate manner. Similarly, Meta AI employs a text encoder grounded in the NLLB model. This text encoder is trained to comprehend textual content spanning nearly 100 languages, generating representations that prove valuable in translation tasks. Meta AI's text decoder is adept at processing encoded speech representations or textual representations. This capability is harnessed for tasks within the same language, including automatic speech recognition and multilingual translation endeavors. For instance, when a speaker utters the word "bonjour" in French, the corresponding translated text in Swahili, "habari," is seamlessly generated. Through multitask training, Meta AI leverages the prowess of a robust text-to-text translation model (NLLB) to guide the speech-to-text translation model via token-level knowledge distillation. In the context of speech production, Meta AI leverages acoustic units to represent the target speech. The text-to-unit (T2U) component within the UnitY model orchestrates the creation of discrete speech units based on the textual output. This component undergoes pre-training on ASR data prior to the UnitY fine-tuning phase. Subsequently, a multilingual HiFi-GAN unit vocoder is employed to convert these discrete units into audio waveforms. Data-driven models like SeamlessM4T derive significant benefits from substantial volumes of high-quality end-to-end data - specifically speech-to-text and speech-to-speech data. However, relying solely on human-transcribed and translated speech data is inadequate to address the complexities of speech translation for 100 languages. In response, Meta AI builds upon its pioneering work in text-to-text mining, employing a similarity measure in a unified embedding space, alongside initial explorations in speech mining, to generate additional resources for SeamlessM4T model training.


The Results With a singular model, Meta AI's SeamlessM4T attains cutting-edge outcomes across an impressive spectrum of nearly 100 languages. This accomplishment is augmented by its multitasking capabilities, spanning automatic speech recognition, speech-to-text, speech-to-speech, text-to-speech, and text-to-text translation functionalities. Notably, the system extends its advancements to encompass languages with low and mid-level resource availability, significantly enhancing their performance. This augmentation is accompanied by the system's unwavering excellence in delivering robust outcomes for high-resource languages. In the pursuit of accurate system evaluation, Meta AI introduces an extended metric, BLASER 2.0, that transcends text-based assessments. This evolved metric empowers the evaluation of both speech and text units with accuracy akin to its predecessor. Through rigorous testing for robustness, the system showcases exceptional resilience in speech-to-text tasks. Against the backdrop of background noises and variances in speaker characteristics, the system notches substantial enhancements - average improvements of 37% and 48%, respectively - outperforming the present state-of-the-art model. SeamlessM4T is certainly one of the most exciting foundation models in speech translation ever built. Hopefully, we will see it integrated into Meta AI's multimodal efforts.


Swipe Right for Your Career: Build A Tinder for Jobs

Architecture Overview The platform will have three main components:


1. Data Ingestion and Storage Resumes and job descriptions are collected from users and employers, respectively.They are preprocessed to clean and tokenize the text.AWS S3 is used to store and manage the data.


2. NLP and Matching Engine Resumes and job descriptions are encoded into dense vector representations using a language model such as GPT or a custom fine-tuned model.Similarity metrics (e.g., cosine similarity) are used to compare vectors and calculate match scores.A threshold is set to filter out low-scoring matches.


3. User Interface and Interaction A web-based user interface allows candidates to upload their resumes and employers to post job descriptions.AWS Lambda and API Gateway handle user interactions.


1. Data Ingestion and Storage: A Symphony in S3 Harmony We begin our masterpiece by curating the raw materials - the resumes and job descriptions. With the elegance of a conductor leading an orchestra, we utilize AWS S3 to store this treasure trove of textual data. The code orchestrates the upload and download processes, ensuring a seamless flow of information from users to the platform and vice versa.


2. NLP and Matching Engine: The AI Ballet Ah, NLP, the crown jewel of our ensemble! Picture a prima ballerina on a cosmic stage. Here, we enlist Hugging Face's Transformers library to turn mere text into waltzing vectors. The code elegantly orchestrates this transformation, crafting a melody of encodings that resonates with the very essence of resumes and job descriptions. And to measure compatibility? Cosine similarity sweeps in, casting a spotlight on the most harmonious pairings.


3. UI and Interaction: The Technological Tango Our grand stage wouldn't be complete without a dance floor for candidates and employers! The Flask framework orchestrates this grand affair. As candidates upload their resumes and employers set the scene with job descriptions in a React.js app, the AI spirits behind the scenes whirl in a data-driven waltz. Results are unveiled as a tapestry of potential matches, a visual testament to the power of technology.


Epilogue: Embrace the Future And there we stand, at the crossroads of technology and humanity, watching the magic unfold. As we bid adieu to our journey, remember this - AI is not just a tool; it's the masterstroke that paints the canvas of innovation. Our platform doesn't just match job seekers and employers; it's a testament to the brilliance that stems from the synergy of minds and machines.


Building Intuition on the Concepts behind LLMs like ChatGPT - Part 1- Neural Networks, Transformers, Pretraining, and Fine Tuning

Neural Networks LLMs like ChatGPT are trained on huge amounts of publicly accessible text data from the internet using artificial neural networks. Artificial neural networks are machine learning algorithms that are designed to mimic in an abstract way, our brain's structure and learning process. They are made up of layers of interconnected nodes or "neurons," and through repeated training iterations on massive amounts of text data, the network learns the patterns in the texts and the nuances of the language - enough to generate coherent words, sentences, or whole documents by itself. The artificial neural network is the main feature of a subset of machine learning called deep learning. It is very important in the field of AI due to its ability to capture intricate patterns and dependencies in data and generalize from these patterns to make predictions on new, unseen data. In the context of language modeling, this means predicting what word should come next given a sequence of a preceding word or words. Compared to conventional machine learning algorithms like linear regression, neural networks are able to represent and model non-linear relationships between different features present in large amounts of data through the use of nonlinear mathematical functions (the activation function) in the neurons of the network's hidden layers. Neural networks have produced consumer tech that you've probably interacted with (and they are not necessarily exclusive to language tasks) such as unlocking your phone using facial recognition, the augmented reality feature in your Pokemon game, or the show suggestions in Netflix home screens. Andrej Karpathy even argues that it can be a new and better way of writing software: For example, instead of hand coding the logic of a program (- if condition A is met, do x; if condition A is not met, do y) the neural network instead learns through examples from the training data that if it encounters 'condition A' in production it should do x. These conditions/logic are not defined by its creators, rather the neural network adjusts itself (by tweaking its billions or even trillions of parameters - the weights and biases) to conform to this desired behavior. Nobody knows what each individual weight and bias does specifically or how a single weight contributes to a specific change in the behavior of the artificial neural network. These parameters are changed en masse as a unit during training via gradient updates. (discussed in more detail later.) This is why you'll often hear machine learning models trained on neural networks as 'black boxes'. Their inputs and outputs can be observed, but the internal workings or how it does what it does is not easily understood. This is also the reason for discoveries of 'emergent' capabilities. As an LLM gets bigger and bigger (measured by its number of parameters), it starts coming out of training with unanticipated abilities. For example, GPT-2 was discovered to be good at language translation, GPT-3 was an excellent few-shot learner, and GPT-4 has shown sparks of artificial general intelligence or AGI. None of these were explicitly defined as a training goal - its main objective was to predict the next word in a sequence. Emergent behaviors are not unique to large neural networks. As a system gets bigger and more complex, the interaction between its individual components can lead to unexpected behaviors that cannot be fully explained by analyzing the properties of each individual component in isolation- a single ant is stupid, but a colony of ants can build very complex tunnel networks and wage war against other colonies. This phenomenon has been documented in systems like social insects (ants and bees), crowd behavior, and other biological ecosystems.


Pretraining Foundation Models The first step in creating something like ChatGPT is pretraining a base model or a foundation model. The goal of this step is to create a machine learning model that is able to autonomously generate a coherent word structure or generate human-like text (phrase, sentence, paragraph) by generating words in sequence based on its prediction of what word should come next given the preceding words. It's called pretraining as the output of this step - the base model is still a raw product that has limited practical applications and is usually only of interest to researchers. Base models are 'trained' further via the fine-tuning stages for specific tasks with a real-world utility like text translation, summarization, classification, etc. At the start of pretraining, the parameters of the neural network are set with random numerical values. The words in the massive internet text data are converted into numerical representations in the form of tokens (as an integer) and embeddings (as vectors)- before being fed to the neural network. Tokens and embeddings will be discussed in detail in the next part of this series but for now, think of a token as the unique ID of a word in the model's vocabulary and the embedding as the meaning of that word. The model is given a word or words and is asked to predict the next word based on those preceding word/s. Then, it is tested on unseen data and evaluated on the accuracy of its predictions based on the 'ground truth' next word from a hidden dataset previously unseen by the model. Consider the example sentence in the training dataset: "I have to go to the store". This sentence might be used as follows: The model is given "I have" and is expected to predict "to".Then it's given "I have to" and is expected to predict "go".Then "I have to go" and is expected to predict "to".Finally, "I have to go to the" and is expected to predict "store". Going through the whole corpus of the training dataset like this, the model will be able to learn which words tend to appear after different sets of words. It learns the dependencies between "I" and "have", "have" and "to", and so on. In the testing step, the process is similar, but the sentences or texts used are the ones that the model has not been trained on. It's a way to check how well the model generalizes its language understanding to unseen data. Let's consider the unseen sentence from the test set: "She needs to head to the ___" Even though this exact sentence was not part of the training dataset, the model can use its understanding of similar contexts it encountered to make an educated prediction. For example, it has been seen in the training sentence "I have to go to the store" that the phrase "to go to the" or "to head to the" is often followed by a location or a destination. Based on this, the model might predict "market", "store", "office", or other similar words, as they are common destinations in this kind of context. So, while the model was trained on "I have to go to the store" and variations of this text with similar meaning, it's able to generalize from that to understand that "She needs to head to the..." is likely to be followed by a similar type of word, even though this exact sentence was not part of its training data.


How Models 'Learn' At the start of pretraining, the model would usually output nonsensical sequences of words when asked to make a prediction as it hasn't 'learned' anything yet. In our example sentence earlier, it might generate the word 'apple' instead of the ground truth next word - 'store'. Since LLMs are probabilistic, 'wrong' in this context means the model is assigning a higher probability (to be selected) to the word 'apple' compared to the expected word - 'store'. The ultimate goal is to have the model output 'store' every time it's asked to predict the next word that comes next after the sequence "She needs to head to the ___". The difference in the actual and the expected or ground truth next word is calculated using a 'loss function' where the greater the difference, the higher the 'loss' value. The loss is a single number that 'averages' the loss or error of all the predictions you asked the model to make. Through several iterations of these steps, the aim is to minimize the value of this 'loss' through processes called backpropagation and gradient descent optimization. The model 'learns' or improves its prediction ability through these steps. You're probably wondering how you can 'calculate the difference between two words' to arrive at a loss value. Do note that what goes through the neural network are not actual texts (words, sentences) but numerical representations of these texts - their tokens and embeddings. The number representations of a word sequence are processed through the layers of the network where the output is a probability distribution over the vocabulary to determine what word comes next. An untrained model might assign a higher probability to the token id of the word 'apple' (say 0.8) compared to the token id of the ground truth next word - 'store' (at 0.3). The neural network will not encounter a single word or letter of any text. It works exclusively with numbers - basically a calculator with extra steps.  Through backpropagation, the degree of the error of the model (the loss value) is propagated backward through the neural network. It computes the derivative to the output of each individual weight and bias i.e. how sensitive the output is to changes in each specific parameter. For my people who didn't take on differential calculus in school (such as myself), think of the model parameters (weights/biases) as adjustable knobs. These knobs are arbitrary - in the sense that you can't tell in what specific way it governs the prediction ability of the model. The knobs, which can be rotated clockwise or counterclockwise have different effects on the behavior of the output. Knob A might increase the loss 3x when turned clockwise, knob B reduces the loss by 1/8 when turned counterclockwise (and so on). All these knobs are checked (all billions of them) and to get information on how sensitive the output is to adjustments of each knob - this numerical value is their derivative with respect to the output. Calculating these derivatives is called backpropagation. The output of backpropagation is a vector (a list of numbers) whose elements or dimensions consist of the parameters' individual derivatives. This vector is the gradient of the error with respect to the existing parameter values (or the current learnings) of the neural network. A vector has two properties: length or magnitude and direction. The gradient vector contains information on the direction in which the error or loss is increasing. The magnitude of the vector signifies the steepness or rate of increase. Think of the gradient vector as the map of a foggy hill you're descending from - gradient descent optimization is using the information about direction and steepness from the gradient vector to reach the bottom of the hill (the minimum loss value) as efficiently as possible by navigating to the path with the greatest downward incline (the opposite direction of the gradient vector). This involves iteratively adjusting the values of the weights and biases of the network (by subtracting small values to it i.e. the learning rate) en masse to reach this optimal state. After these steps, the hope is during the next training iteration, when the model is again asked to predict the next word for "She needs to head to the..." it should assign a higher probability to the word 'store'. This process is repeated several times until there is no significant change to the loss value meaning the model's learning has stabilized or has reached convergence. So the TL;DR on how neural networks learn to communicate in English (and other languages) is - math in serious amount. Like oodles. It boils down to reducing the value of a single number (the loss value) generated from complex computations within the neural network - where, as this number gets smaller, the more 'fluent' or 'coherent' the language model becomes. The millions or billions of mathematical operations applied between matrices and vectors in the inner layers of the network somehow coalesce into a geometric model of the language. To help with intuition, we've anthropomorphized the model by using words like 'understand', 'seen', and 'learn', but in truth, it has no capacity to do any of these things. It's just an algorithm that outputs the next best token of a sequence based on a probability distribution given a sampling method.


The Transformer The Transformer is the breakthrough in natural language processing (NLP) research that gave us ChatGPT. It is a type of neural network architecture that utilizes a unique self-attention mechanism. It's famously discussed in the paper 'Attention is All You Need' that came out in 2017. Almost all state-of-the-art LLMs (like BERT, GPT-1) that came out after this paper, were built on or using ideas from the transformer. It's hard to overstate the importance of this paper due to its impact on deep learning. It's now finding its way to vision tasks making it truly multi-modal and demonstrating its flexibility to handle other types of data. It also started the '...is all you need' memetic trend that even the Towards AI editorial team is unable to resist.  Prior to transformers, neural networks used in NLP that produced SOTA models, relied on architectures that utilize sequential data processing e.g. recurrent neural networks or RNNs - this means that during training, each word or token is processed by the network one after the other in sequence. Note that the order of words is important to preserve the context/meaning of a sequence - 'the cat ate the mouse' and 'the mouse ate the cat' are two sentences with two different meanings even though they are made up of the exact same words/tokens (albeit in a different order). One of the key innovations of the transformer is doing away with recurrence or sequential token processing. Instead of processing tokens sequentially, it encodes the position information of each word (i.e. in which order a word appears in the sequence being processed) into its embedding before being inputted in the network's inner layers. More importantly, transformers solved the issue of long-term dependencies that neural nets like RNNs struggled with. Given a long enough sequence of words (e.g. a very long paragraph), RNNs will 'forget' the context of the word it processed earlier in the sequence - this is called the vanishing gradient problem. RNNs store information on the relevance of words in a sequence up to that point in what's called the hidden state at each sequential or time step. As it processes a long sequence, gradients corresponding to earlier time steps can become very small during backpropagation. This makes it challenging for the RNN to learn from the early parts of the sequence and can lead to the 'loss' of information about words processed earlier. This is problematic for a next-word prediction model, especially if those 'forgotten' words are important to the context of the sequence currently being generated The transformer solves this limitation through the 'self-attention' mechanism. Same with positional encoding, each word, through its embedding is encoded with information on the degree or how much it should 'attend to' the rest of the words in the sequence - no matter the length of the sequence or the relative distance of the attended word in the sequence. This encoding is done simultaneously for all words in the sequence allowing the transformer to preserve the context of any sequence. The degree how which one word should attend to other words is a 'learned' trait stored in the model weights and encoded in the word embeddings via matrix multiplications. These 'learnings' get adjusted during each training iteration as the model learns more about the relationships between words in the training data. The final output of the self-attention layer (the Z matrix in our illustration above) is a matrix of the word embeddings that is encoded with information on the position of each word in the sequence (from the position encoding step) and how much each word should attend to all other words in the sequence. It is then fed to a traditional neural network like the one discussed earlier in the article (called the feed-forward neural network). These steps (attention + feed-forward - which makes up a transformer block) are repeated multiple times for each hidden layer of the transformer - 96 times for GPT3, for example. The transformation in each layer adds additional information to the 'knowledge' of the model on how to best predict the next word in the sequence. According to the LLM scaling laws published by OpenAI, to train better models, increasing the number of parameters is 3x more important than increasing the size of the training data. (Note: DeepMind has since published a paper with a differing view.) This translates to a significant increase in computational requirements, as handling a larger number of parameters demands more complex calculations. Parallelization, which is the process of dividing a single task into multiple sub-tasks that can be processed simultaneously across multiple compute resources, becomes essential in dealing with this problem. Parallelization is difficult to achieve with RNNs given their sequential nature. This is not an issue for transformers as it computes relationships between all elements in a sequence simultaneously, rather than sequentially. It also means that they work well with GPUs or video cards. Graphics rendering requires a large number of simple calculations happening concurrently. The numerous, small, and efficient processing cores that a GPU has, which are designed for simultaneous operations, make it a good fit for tasks such as matrix and vector operations that are central to deep learning. AI going 'mainstream' and the mad scramble to build larger and better models is a boon to GPU manufacturers. NVIDIA- specifically - whose stock price has grown 200% YTD as of this writing, has made them the highest-performing stock this year and pushed their market cap to USD 1 trillion. They join megacaps like Apple, Google, Microsoft, and Amazon in this exclusive club. The Transformer is a decidedly complex topic and the explanation above wholesale left out important concepts in order to be more digestible to a broader audience. If you want to know more, I found these gentle yet significantly more fleshed-out introductions to the topic: Jay Allamar's illustrated transformer, Lili Jiang's potion analogy, or if you want something more advanced - Karpathy's nanoGPT that babbles in Shakepear-ish.


Fine-tuning 'chat' models like ChatGPT The output of pretrainings are base models or foundation models. Examples of recently released text-generation foundation models are GPT-4, Bard, LLaMa 1 & 2, and Claude 1 & 2. Since base models already have extensive knowledge of the language from pretraining (the structure of sentences, relationships between words, etc.), you can leverage this knowledge to further train the model to do specific tasks - translation, summarization, or conversational assistants like ChatGPT. The underlying idea is that the model's general language understanding gained from pretraining can be used for a wide range of downstream tasks. This idea is called transfer learning. If you ask or prompt a base model a question, it will probably reply with another question. Remember that it's trained to complete a word sequence by predicting the word that should come next given the previous words in the sequence. Example: However, we can get a base model to answer questions by 'tricking' it into thinking that it's trying to complete a sequence: Using this idea, the model goes through another round of training using different sets of prompt/completion pairs in a question-and-answer format. Instead of 'learning English' from random texts found on the internet by predicting what words come next after a set of words, the model 'learns' that to complete a prompt in a 'question' form, the completion should be in an 'answer' form. This is the supervised fine-tuning (SFT) stage. Pretraining utilizes a type of machine learning called self-supervised learning where the model trains itself by creating the 'label' or ground truth word it's trying to predict from the training data itself. Here is our example from earlier: The model is given "I have" and is expected to predict "to".Then it's given "I have to" and is expected to predict "go". The labels or target words 'to' and 'go' are created by the model as it goes through the corpus. Note that the target/ground truth words are important as this is the basis of the loss value - i.e. how good the current model's prediction is versus the target- and the subsequent gradient updates. Compared to the pretraining phase, the training data preparation in the fine-tuning stages can be labor intensive. It requires human labelers and reviewers that will do careful annotation of the 'labels' or the target completions. However, since the model has already learned general features of the language, it can quickly adapt to the language task it's being fine-tuned for, even with the limited availability of task-specific training data. This is one of the benefits of transfer learning and the motivation behind pretraining. According to Karpathy, 99 percent of the compute power and training time and most of the data to train an LLM are utilized during the pretraining phase and only a fraction is used during the fine-tuning stages. Fine-tuning uses the same gradient update method outlined earlier but this time it's learning from a list of human-curated question/answer pairs that teaches it how to structure its completions i.e. 'what to say and how to say it'. It goes through other fine-tuning stages like reward modeling and reinforcement learning from human feedback (RLHF) to train the model to output completions that cater more to human preference. In this stage, the human labelers score the model's completions on attributes like truthfulness, helpfulness, harmlessness, and toxicity. The human-preferred completions get reinforced into the training i.e. it will have a higher probability to appear in completions of the fine-tuned version of the model. The output of these fine-tuning steps is the 'assistant' or 'chat' models like ChatGPT. These are the 'retail' versions of these foundation models and are what you interact with when you go to the ChatGPT website. The GPT-3 base model (davinci) can be accessed via an API. The GPT-4 base model has not been released as an API as of this writing and is unlikely to be released by OpenAI, given their recent statements about competition and LLM safety. These fine-tuning steps are generally the same for all available commercial and open-source fine-tuned models. End of Part 1 Note: Part 2 will talk about Embeddings which predates the LLM explosion but is equally as fascinating - how embedding models are trained, and how a sentence or document-level embeddings (used in RAG systems) are generated from word embeddings. We will also be discussing about Tokens and why it's needed. We have implied in this post that token = word to simplify things, but a real-world token can be an individual character or letter, a subword, a whole word, a series of words, or all of these types in a single model vocabulary! If I got anything wrong, I'm happy to be corrected in the comments! :)


Resources/References: 3blue1brown. What is a neural network? Geeks for Geeks. Artificial Neural Networks and its Applications Jay Alammar. The Illustrated Transformer Luis Serrano. What Are Transformer Models and How Do They Work? Andrej Karpathy. The State of GPT Andrej Karpathy. Let's build GPT: from scratch, in code, spelled out.


How To Read a Machine Learning Paper in 2023 for Beginners

Stay up to date with the hype Before we talk about how to really dive deep into a paper, we'll look at the first reason to read a paper. To stay up to date with the latest breakthroughs and hype. Here, you want the highest level of understanding. Even if you are a computer vision researcher, you should know about the greatest developments of Large Language Models. And I'm not trying to fuel your "FOMO", but I think it is important to be able to hold meaningful small talk with other researchers, or ideally get lucky and learn about a new technique that you can apply to your own research. Now, before you get to skimming the new paper, you first need to know about what the newest and hottest publication is. Therefore, I recommend simply following big AI Labs on Twitter, now called X, and LinkedIn, but also some AI news personalities like Lior S. or   Remember, we need to know what our goal of "reading" the paper is. In this case, we simply want a high-level understanding of what is happening. So, really, AI news personalities on Twitter, LinkedIn, and (for a bit delayed updates) on YouTube are really, really great!


Exploring ideas for your own research Again, before we get to the reason you probably want to hear the most, we'll briefly talk about the second reason for reading a paper: Exploring ideas for your own research. What are new techniques, datasets, or more efficient implementations that might be interesting for you to adopt? There were about 1,000 papers accepted to ICLR alone in 2022. There is absolutely no chance that you can read all of them thoroughly. Besides the hype papers, you will probably want to look at new ideas and insights in your research domain. And if you really break it down, many papers present one core new idea that often is really simple. E.g. if you are working on Language Models, the basic idea of Contrastive Decoding [1] pretty much just says "Have a powerful and weaker Language Model generate their output logits for the next token and subtract them element-wise for the new output logits. This will yield more diverse results". This process, as you might have thought, involves a lot of exploration. Mainly, either by simply googling a paper, going through the list of publications of a conference, or, even more naturally, by recursively looking at the references made in papers you are already reading. Eventually, we again want to read the Abstract and will hopefully find the main idea and its performance improvements, i.e., main results, always keeping in the back of our minds; how can this help me for my research? But since we are in 2023 and onwards, if you don't understand the main idea straight away, you can again ask AI tools to extract main contributions, novelty, and results and have it explain concepts in simple terms. In fact, in some cases, we can also directly ask to compare the new approach to another one!Another thing to look at is the architecture diagram, if there is one of course. This often allows you, at a glance, to get an overview of the flow of the model or pipeline.


Integrating into your own research - diving deep If you now feel like this paper, with its main idea and results, is so fitting to your own research, we finally get to the last reason of reading a paper. Really getting into the nitty gritty details to potentially integrate it into your own research and build on top of it. Straight up, if there is a video or Blog post on this paper, it is a jackpot! In my opinion, it's often best to start with a YouTube video or blog post and then read the paper. That way, you know what to expect and pay most attention to and already have a fundamental understanding. The paper lingo is very sophisticated and (arguably) unnecessarily tough. Now, we should look at the paper. Especially if there is no video or blog post, you will need to, if you haven't already, start with the Abstract and then go to the Results. If you want to build on top of this paper, it might be very useful to have a look at the datasets that they use for training and evaluation. If available, you might want to use those yourself! Now to the Methodology section. To really understand the model, I would recommend you start by looking at the model diagram. As mentioned, that will give you a feeling for the flow of data. Afterward, start reading the text explaining this model. On a fundamental level, a bit over-simplified, deep learning models are just a series of reshaping operations on some matrices or tensors. So if you look at individual blocks of the model, there will be an input with a certain shape and an output, with its respective shape. And in the text, there might be some maths formalizing this transformation. I can not emphasize this enough. You need to always understand how the data dimensions, i.e., its shape change. What I love to do is always write down the shapes after each operation and then the final block. If I do this step by step, it isn't as overwhelming as it looks in the end, and it helps me tremendously! And here is a PRO TIP. If there exists code to the paper: download it and step through it with the debugger in VSCode. This means either looking at the author's own implementation referenced in the paper, looking at third-party implementations on papers with code, or, in my opinion even easier, just importing and running a huggingface model and stepping through their very nice implementation.What you are looking for is to match the individual code operations or modules to the model diagram and maths formulas in the paper. And again, ALWAYS look at and understand the dimensions of the input and output. Now, when it comes to math formulas, they often look very scary but mean simple things! Especially if you look at the loss function. Very often, the authors will use a common loss, like the negative log-likelihood, Cross entropy, the Mean Square Error, and so on. But even if they formulate a completely new loss, it's just a mathematical formulation of a human idea. E.g., Train the discriminator to recognize real images as real and generated images as fake. Then, train the generator to produce images so that the discriminator thinks they are real. And rinse and repeat.You can, yet again, use AI to help you explain those formulas! We are really living in fascinating times! Using those different techniques, AI tools, VSCode debugging, simply looking at paper explanation videos or blogs, looking at the data shapes or dissecting the loss, whenever you find something you don't understand or whenever you have an "aha!"-moment, you should always highlight and take your own notes. Of course, there are really difficult papers, very theoretical or maths-heavy ones, or papers at the intersection of AI and Science that involve a lot of further domain knowledge. In the end, the solution to really understanding a paper is always the same. Keep going. If you don't understand an idea in the beginning when reading a paper, don't worry. Continue reading, and it will probably become clearer. If the paper you are currently reading is not giving enough explanations for referenced techniques, go to the papers they reference and read those. Another pro tip I like is to think of what and how the researchers might have talked about their ideas during a simple coffee break. I know... it might be annoying, but I promise, the beginning is always the hardest. This knowledge that you struggle with building up, in the beginning will help you in the future go through new papers with a lot less trouble! Again, I promise!


Summary of actionable tips So, to briefly summarise the actionable tips: I pretty much always start by reading the Abstract, then the Results, and then the Methodology section.Use other sources like YT or Blogs. The paper lingo is often very condensed and thus often tough.If there is code: Use the debugger to step through the code and always understand the shapes of input and outputAnd finally, don't hesitate to use AI tools like the SciSpace Copilot to help you summarize and extract main contributions, make the language more simple, and explain some maths formulas. I will be doing a lot more of those Paper Explanation blog posts (and YouTube videos!!). I hope that my summaries and explanations make it easier for you to follow along with the progress in AI, and if you don't want to miss future posts, don't forget to leave a follow!


References [1] Contrastive Decoding, X. L. Li et. al, 2023, https://arxiv.org/abs/2210.15097 [2] Generative Adversarial Networks, I. J. Goodfellow et. al, 2014, https://arxiv.org/abs/1406.2661


10 Cool Things You Can Do With Embeddings! [Part 1]

An Intuitive Explanation of Embeddings Embeddings are low-dimensional, learned continuous vector representations of discrete variables [1]. We can break up this definition and absorb the important points: lower dimensionality than input datacompressed representation of datacapture complex non-linear relationships learned by the model as a linear representationreduce dimensions by storing relevant information and discarding the noiseusually extracted from the final layers of the neural networks (just before the classifier) Let us intuitively understand the definition and the true potential of embeddings using an example. Suppose we are analyzing the performance of call centre operators based on their customer survey forms. The forms have been submitted by thousands of customers and contain millions of unique words. To perform any sort of text analysis, our input parameter set will be huge, and likely to result in poor performance. This is where embeddings come into play! Instead of taking the original input parameter set, we take all the unique words and express them as embeddings. This way, we reduce dimensions and bring down the input feature set to, let's say 9, which is more consumable by the model. Visualizing the embeddings for the operators, it is easy to spot that Operators B and C received similar customer feedback. Another example can be found here. [2]