Changing Data type in the source page -> To change the data type on the source page, go to the preview menu, click on the data type specified for each field, and choose the new data type.


Data Types in Tableau: Using & Cleaning; A Road to Tableau Desktop Specialist Certification

Changing Data type from Data Pane -> The data type can also be changed from the data pane, click on the data type specified for each field, and switch to a new data type.


Changing Data type in the View -> To change the data type in the view, right-click on the field, go to "Change Data Type" and choose a new data type.


Handling Mixed Data Types Sometimes, the columns contain a mix of data types. A column might constitute a mixture of date, date-time, strings, or blank spaces. When we try to connect the data file to Tableau, the mixed column is mapped to a single data type that infiltrates our data i.e. a column which is identified by Tableau as "Numerical data type" might also contain dates, strings, or blank spaces. Tableau considers the top 10000 rows for an Excel File and 1024 rows for CSV files i.e. if we ingest an xlsx file that has 10000 rows and if most of them are of string, then Tableau will regard the whole column as a string. When Tableau determines a data type for each field, and if the values don't match that respective data type. Tableau tries to handle that field by adding 'null' values to those records that don't match the specified data type. e.g. In a Boolean mapped column, if there are Text, Dates, or Numbers, those records will be treated as Null. In Numbers mapped columns, if we get Text it will be treated as Null but if we get a date, it will be converted to the difference of that date from 1/1/1900.


Sample Exam questions from this Topic By default how many rows does metadata shows? 1000500010010000 Solution: 100


References: [1] Tableau Help  Tableau Software [2] Personal Notes [3]Tableau Desktop Specialist Exam (New Pattern - 2021) - Apisero


Thanks for Reading! Feel free to give claps so I know how helpful this post was for you, and share it on your social networks, this would be very helpful for me. If you like this article and want to learn more about Machine Learning, Data Science, Python, BI. Please consider subscribing to my newsletter: Find me on Web: www.dakshtrehan.com Connect with me at LinkedIn: www.linkedin.com/in/dakshtrehan Read my Tech blogs: www.dakshtrehan.medium.com Connect with me at Instagram: www.instagram.com/_daksh_trehan


Want to learn more? How is YouTube using AI to recommend videos?Detecting COVID-19 Using Deep LearningThe Inescapable AI Algorithm: TikTokGPT-3 Explained to a 5-year old.Tinder+AI: A perfect Matchmaking?An insider's guide to Cartoonization using Machine LearningHow Google made "Hum to Search?"One-line Magical code to perform EDA!Give me 5-minutes, I'll give you a DeepFake!


This AI newsletter is all you need #58

What happened this week in AI by Louie This week we were excited to see two new developments in AI outside the realm of NLP. The latest development from Meta AI involves the unveiling of their Open Catalyst simulator application, which has just been released as a demo. By harnessing the power of AI, this application can predict catalyst material reactivity with remarkable speed, outperforming existing methods by nearly 1000 times. The team believes that this technology has the potential to significantly accelerate the discovery of cost-effective materials. While we think material science is a more complex subject for machine learning to tackle relative to proteins (and AlphaFold), we see a lot of potential for similar models to aid researchers screen for potentially interesting materials in space. In the future, we also expect to see models released to contribute to the search for superconductors, topical this week where a potential breakthrough has been in focus! In another exciting announcement, Google has introduced the Robotics Transformer 2 (RT-2) model, which they refer to as the world's first vision-language-action (VLA) model. The network undergoes training using text and images extracted from the web, allowing it to directly produce robotic actions as its output with a small amount of robot training data. The model exhibits a remarkable ability to comprehend complex commands, such as "throw away the trash." It achieves this by doing complex reasoning. For instance, recognizing that a banana peel becomes trash after being consumed and autonomously grasping the concept of discarding the trash, even without explicit training on the specific task. During the trials, the RT-2 model demonstrated a significant improvement in performance on unseen scenarios, achieving twice the effectiveness of its previous version. While surrounded by the constant stream of exciting news in NLP, it's refreshing to witness two equally thrilling advancements in other AI applications this week, specifically in the fields of robotics and materials science. We are pleased to see the recent breakthroughs and wave of investments in NLP begin accelerating progress in other areas. - Louie Peters - Towards AI Co-founder and CEO


This AI newsletter is all you need #58

Hottest News Meet FreeWilly, Our Large And Mighty Instruction Fine-Tuned Models Stability AI and its CarperAI lab have released Stable Beluga 1 and its successor, Stable Beluga 2 (formerly codenamed FreeWilly). Stable Beluga 1 leverages the original LLaMA 65B foundation model and has been fine-tuned using Supervised Fine-Tune (SFT) techniques. Similarly, Stable Beluga 2 leverages the LLaMA 2 70B foundation model. Both models are publicly available under a non-commercial license. 2. Stability AI Announces Stable Diffusion XL 1.0 Stability AI has announced the release of Stable Diffusion XL (SDXL) 1.0, the latest and most advanced version of its flagship text-to-image suite of models. SDXL is an open-access image model with a staggering 6.6 billion parameter model ensemble pipeline, demonstrating significant improvements in color, contrast, lighting, and shadow. 3. Stack Overflow announces OverflowAI Stack Overflow is integrating generative AI into its platform with OverflowAI. This includes semantic search and personalized results using a vector database. Additionally, they are enhancing search capabilities across different platforms and introducing an enterprise knowledge ingestion feature for Stack Overflow for Teams. 4. Introducing Bittensor Language Model - A State-of-the-Art 3B Parameter Model For Mobile and Edge Devices The Opentensor Foundation and Cerebras are pleased to announce Bittensor Language Model (BTLM), a new state-of-the-art 3 billion parameter language model that achieves breakthrough accuracy across a dozen AI benchmarks. It operates efficiently on mobile and edge devices with limited RAM, reducing the need for centralized cloud infrastructure. 5. OpenAI Scuttles AI-Written Text Detector Over 'Low Rate of Accuracy' OpenAI has decided to retire its AI classifier due to its low accuracy rate in detecting AI-generated text. The rapid development of large language models has made it challenging to effectively identify features or patterns.


This AI newsletter is all you need #58

Five 5-minute reads/videos to keep you learning The History of Open-Source LLMs: Better Base Models This article is the second part of a three-part series on the history of open-source LLMs. It covers topics such as the early days of Open-Source LLMs, the current revolution in building better base models, and the current and future trends in open-source LLMs. 2. Building Generative AI Applications with Gradio Hugging Face and DeepLearning.ai have launched a new short course on building generative AI applications with Gradio. The course focuses on creating user-friendly apps using open-source language models, with projects ranging from text summarization to image analysis and image generation. 3. Build an AI Chart Generator That Adapting to Any Dataset Type, in Only 50 Lines This is a tutorial for developing an automated chart generator. With this tutorial, developers can easily create AI chart generators using GPT-3.5 or GPT-4 with Langchain, requiring just 50 lines of code. 4. Automating Web Research This article explores the development of web research agents. The approach involves using an LLM to generate search queries, execute searches, scrape pages, index documents, and find the most relevant results for each query. 5. Creating an Automated Meeting Minutes Generator With Whisper and GPT-4 This guide explores the development of a meeting minutes generation tool that leverages Whisper and GPT-4 to efficiently summarize discussions, extract important details, and analyze sentiments.


This AI newsletter is all you need #58

Papers & Repositories karpathy/llama2.c: Inference Llama 2 in one file of pure C Andrew Karpathy has released an educational implementation of LLaMA 2 inference in pure C. This project enables you to train a LLaMA 2 LLM architecture in PyTorch and then load the weights into a single C file for efficient inference. 2. Universal and Transferable Attacks on Aligned Language Models A recent study explores the automatic construction of adversarial attacks on both open-source and closed-source language models, rendering them susceptible to harmful commands. These attacks also transfer to widely used chatbots, raising concerns about effectively patching these vulnerabilities. 3. FLASK: Fine-grained Language Model Evaluation based on Alignment Skill Sets This paper introduced FLASK, an evaluation protocol specifically designed for LLMs' performance assessment. It breaks down evaluations into 12 different skill sets, allowing for a detailed analysis of a model's performance based on specific skills such as logical robustness, factuality, and comprehension. 4. A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis WebAgent, an LLM-driven agent, utilizes Flan-U-PaLM and HTML-T5 to enhance autonomous web navigation and task completion on real websites. By breaking down instructions, summarizing HTML documents, and generating Python programs, it achieves a 50% increase in success rates compared to previous models. 5. WebArena: A Realistic Web Environment for Building Autonomous Agents WebArena is a realistic web environment that enables autonomous agents to develop their skills in tasks related to e-commerce, social forums, software development, and content management. It provides benchmarks for evaluating task completion and highlights the need for improved agents, as even advanced models like GPT-4 have a success rate of only 10.59%. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #58

AI4 2023: Industry's Leading AI Conference Just a Reminder to join us at Ai4 2023, the industry's leading AI conference, taking place in Las Vegas on August 79 at the MGM Grand. Read more about how the growth of Ai4 mirrors AI's adoption by the industry and join 2200+ AI leaders, 240 speakers, and 100 cutting-edge AI exhibits. Apply for a complimentary pass. Date: 7th-9th August 2023 (MGM Grand, Las Vegas)


This AI newsletter is all you need #58

Meme of the week! Meme shared by archiesnake


This AI newsletter is all you need #58

Featured Community post from the Discord Operand has shared its open-source Python library for agent integration, designed to complement existing libraries like HF Agent API and LangChain. The library enables you to connect agents with software systems and human users by defining actions, callbacks, and access policies, making it easy to integrate, monitor, and control your agents. Agency handles the communication details and allows for the discovery and invocation of actions across parties. Check it out on GitHub and support a fellow community member. Share your feedback and how you use it in the thread here.


This AI newsletter is all you need #58

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #58

Article of the week LangChain 101: Part 1. Building Simple Q&A App by Ivan Reznikov LangChain is a powerful framework for creating applications that generate text, answer questions, translate languages, and perform many more text-related tasks. This article marks the beginning of the LangChain 101 course. Starting with this article, the author discusses concepts, practices, and experiences by showing you how to build your own LangChain applications.


This AI newsletter is all you need #58

Our must-read articles Modern NLP: A Detailed Overview. Part 3: BERT by Abhijit Roy Forget 32K of GPT4: LongNet Has a Billion Token Context by Dr. Mandar Karhade, MD. Ph.D. Graph Attention Networks Paper Explained With Illustration and PyTorch Implementation by Ebrahim Pichka If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #58

Job offers Software Engineer III (Drupal) @Clarity Innovations, Inc. (Remote) Distributed Systems Software Engineer @INSHUR (Brighton, UK) Intern - Software Engineering Interns - ACI 01 @Activate Interactive Pte Ltd (Singapore) Machine Learning Engineer (Risk) @SHIELD (Singapore) Machine Learning Engineer @Robotec.ai sp. z o.o. (Warsaw, Poland/ Freelancer) Machine Learning Engineer, Fast Optimized Inference @Hugging Face (US Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Self-Supervised Learning and Transformers? - DINO Paper Explained

The DINO framework This is the training pipeline which is also part of the Self-distillation family. We once again have our original source image, apply two different sets of random augmentations, and end up with two different views, x1 and x2. We again have our online network, which is now called the student network, and our target, now called the teacher network, which once again is an exponential moving average of the student network.From here on, things start to look different. We have NO further projection layer NOR a prediction head! If there is no prediction head, how do we prevent (or rather reduce the likelihood of) representation collapse? Well, first of all, we have this centering right here, which can be seen as simply adding a bias c to the teacher predictions. This bias uses batch statistics to compute the mean of the batch and is updated similarly to the EMA. This hyperparameter m is pretty much the same as the 𝜏 parameter in the EMA equation. We update the bias parameter c slightly with every batch. This centering is probably the main operation to avoid collapse to a constant function! Centering prevents one dimension from dominating as a kind of pseudo-label but also encourages collapse to the uniform distribution! That is why the framework also somewhat relies on the sharpened softmax since it has the opposite effect. Imagine you have a bunch of pictures, and you want to teach the model to recognize different objects in those pictures. Now, if the model always pays attention to the same object because it gets its attention the most, like a cute dog, the model will only learn about dogs and not anything else. Centering is like trying to be fair and making sure that the AI learns about all the different objects equally. It's like saying, "Okay, let's divide the pictures into different groups, and each group should be looked at equally." This way, the model will learn about different objects in a balanced way. The model is encouraged to learn about a variety of features or different "pseudo-classes" rather than focusing on a single dominant feature/class. Sharpening, on the other hand, is a technique used to make sure that when wanting to "classify" an object, i.e., wanting to move the whole feature vector strongly along one feature dimension, the model doesn't get confused and always picks different objects randomly, i.e., simply always predicts a uniform distribution. It's like making sure that each time the model does tend to pick an object, it's confident and sure about its choice. After applying the far less sharpened softmax function to the student output, the final loss is the cross entropy loss. And why cross entropy and not the MSE like in BYOL? If we look at row 2, the MSE surprisingly does work with DINO but cross entropy (rows 1, 3, 4) simply seems to work better.DINO also works when adding a further prediction head to the student network (row 3)! But that doesn't seem to help much either.In fact, we don't even necessarily need the softmax operations if we use the MSE loss. But if we use cross entropy, we obviously do because that loss is defined over probability distributions, which, again, seems to work better! Okay... That was a lot to unpack and somewhat build intuition! But this hopefully shows how much experimentation is done to see what empirically works and what doesn't!


Loss function What also seems to work better is not simply using the cross entropy over all different view embeddings but using a very specific setup! Let's see what this actually means and return to our friendly little quokka! Let's again generate multiple views; this time, let's say 4. Change their color slightly, and crop and resize. Now, when cropping, we want to specifically have two cases, crops that contain 50% or more of the original image and crops with smaller cutouts. These large crops are called global crops, marked with the "g". When assigning our views to the two branches, for the teacher, we specifically want to only use our global views, and for the student, we'll use all views, i.e., the local and global crops. What we are now comparing is the teacher embedding of one of the global crops and all the embeddings of all samples from the student network, except for the embedding of the same global view. We can now compute the cross entropy of each student embedding with this one teacher embedding and repeat this whole process.Voila, we have our weird-looking cross-entropy loss that enforces local-to-global correspondence. We already talked about the two cropping cases in the SimCLR post, where we have two adjacent views and the here enforced case of local and global views. Apparently, the model learns better when learning to look at smaller parts of a bigger object and when trying to match it to its embeddings. The embedding of the global view has more information at hand and can thus better recognize the object.


Building intuition This assumption makes sense to me, as long as the teacher produces better embeddings than the student and is also made assumed in the BYOL paper, but since the authors of DINO went ham on experiments, ablation studies, and the quest of building intuition, they actually verified this assumption by looking at the accuracy of both student and teacher networks during training. The teacher does appear to be smarter than the student and whenever the student improves, the teacher does so as well, since it is a more stable version of the student. Keyword: exponential moving average. All that up to a certain point, where the teacher doesn't have anything to teach anymore, and both converge. But okay, cool. We just looked at the novel and improved self-distillation framework that, once again, is agnostic of the architecture. So, where are the transformers, and why are they specifically so special with this framework?! Let's simply look at a comparison between different Self-Supervised learning frameworks and plug in the classic ResNet and a Vision Transformer. When looking at the validation accuracy of the trained ResNets, we can see that the DINO framework yields the best performance but can rather be considered on par with the other baseline approaches. However, when replacing the ResNet architecture with a Vision Transformer, DINO unleashes its potential and outperforms the other baselines significantly! Especially in the case of k-NN classification. K-NN is one of the standard evaluation protocols for Self-Supervised learning in today's time but it wasn't back then when SimCLR and BYOL were developed and published. Sigh... Back then... As if that wasn't like 23 years ago... K-NN!Why is it so cool and how does it work? You don't need any fine-tuning! We still need labels, obviously, but we now simply project all our labeled data into our representation space to generate our class clusters. When wanting to classify a new data point, e.g., an image, we pass it through our Neural Network and project it into representation space. We then simply count the k nearest neighbors (in this case, 3) and do a majority vote. In this case, most of our neighbors are of class orange, so the new image is also classified as class orange. This k is a new hyperparameter (as if there weren't enough already) and, in the paper, is found to yield the best results when set to 11.


Results Okay, cool. Here is a table with super exciting results. DINO, of course, is the best when looking only at ResNets, only at ViTs, and across different architectures. But looking at same-sized architectures, it's still not beating a fully supervised model. Cool. But even cooler is again looking under the hood and visualizing the attention maps. When looking at those from the top 3 attention heads, we can see impressive properties! Similar to CNNs, where each kernel is responsible for extracting certain features, we can see that different attention heads pay attention to different semantic regions of an image! In the first example on the second row, one head pays attention to the clock face, one to the flag, and one to the tower itself! Or, one row below that example, we can again see how one head focuses on the collar, one on the head, and one on the white neck of the zebra! Isn't this so cool?! All of this is learned without any labels, without any specific segmentation maps!In fact, we could simply interpret one attention map as a segmentation output and compare supervised and Self-Supervised training using DINO! As already mentioned, when training in a self-supervised fashion, the model learns to focus more on the main object in the image than when training a network specifically for classification. The learning signal is stronger in the case of self-supervised learning! In classification, the model can learn to use shortcuts to solve its task. When trying to match the image of the bird to the bird class, the model can use the sky or the branches as important features.In the end, if the classification accuracy, or rather the loss that is optimized, is satisfied, the model doesn't really care, whether it actually only looks at the bird or also its surroundings. In Self-supervised training, on the other hand, we don't really have such a simple and straightforward optimization task. The model needs to learn to match color-augmented, local views to global views. It has to learn to produce embeddings while ignoring all augmentations applied to the original source image. The features it needs to learn to extract are much more specific. I hope that makes sense, even though your brain might be somewhat overwhelmed with all this information especially if you have just read all my previous posts on self-supervised learning!I mean... my brain was fried after doing all this research. And I probably didn't even get all the details! Self-supervised learning is really cool, powerful, and interesting! But all this was just scratching the surface! Further advancements, e.g., Masked Image Modeling, were used by the follow-up papers to DINO, iBOT [3], and DINOv2 [4]. The learned representations can be used for so many different downstream tasks! The training and architecture improvements of DINOv2 over iBOT are a list of many small, tuned knobs like the right patch size, teacher momentum, a better centering algorithm, and so on. Also, DINOv2 developed a very sophisticated data preprocessing pipeline to generate a much larger, but curated dataset.


References [1] Emerging Properties in Self-Supervised Vision Transformers, M. Caron et. al, https://arxiv.org/abs/2104.14294 [2] Bootstrap your own latent: A new approach to self-supervised Learning, J. B. Grill, F. Strub, F. Altche, C. Tallec, P. H. Richemond et al., https://arxiv.org/abs/2006.07733 [3] iBOT: Image BERT Pre-Training with Online Tokenizer, J. Zhou et al., https://arxiv.org/abs/2111.07832 [4] DINOv2: Learning Robust Visual Features without Supervision, M. Oquab, T. Darcet, T. Moutakanni et al., https://arxiv.org/abs/2304.07193 [5] A Cookbook of Self-Supervised Learning, R. Balestiero, M. Ibrahim et. al, https://arxiv.org/abs/2304.12210


How Not Be a Garbage Scientist

Googling: "What is Data Science?" Yes, Googling is very important all the time; even before kicking off something absolutely. It does not matter what subject you are about to learn. Learners need to align 3 things before kicking off their learning path: Whether the learner's background is the same or different.Understanding the prerequisites of the subjects before diving in.How to apply after acquiring the knowledge in the real world? It doesn't matter whether you search on Google or ask Bard or Chatgpt. Using these tools, you can navigate your learning journey.


How Not Be a Garbage Scientist

Knowing Why? This philosophy is very essential and mostly followed by every successful expert in every field in this world. Either you are about to start your career in the Data Science field or you are planning to switch from another background to this field. You should have a clear reason behind kicking it off, just buying a couple of courses or joining training classes. A couple of my friends and ex-colleagues changed in this domain from different fields of expertise. But, they did not become Data scientists; neither they wanted to. They learned analytics as a tool that serves their individual purposes.


How Not Be a Garbage Scientist

Tools are tools: Learning Python or R or any other language or ML tools won't make you a data scientist. These are just tools to be utilized by ML practitioners to solve the business problems that need to deliver at the customer's end or might be for personnel research.


How Not Be a Garbage Scientist

Domain Selection and making a roadmap Selecting the specific domain and creating a roadmap in that? If, you are an aerospace engineer and you are learning Data science and AI techniques. You can focus on what kind of equipment you are dealing with every day, and then you can learn only that part of Data science. Suppose you are learning DS to apply in a turbine power generation system. Simply learn the domain well where AI can add some value and then learn only the relevant tools and techniques to solve them. For example, you are trying to detect faults in advance in the turbine and about to forecast the power generation. Then, you need to learn Time series techniques and details of sensor data and dedicated forecasting and Anomaly detection algorithms only. You may not need to spend time in NLP at that point.


How Not Be a Garbage Scientist

Non-IT to IT trend: You won't have to jump into IT after learning DS or AI. You can apply this knowledge in any field. After, selecting the domain to apply and selecting tools and techniques to learn; it will be easier to create a complete solid roadmap. Nowadays, we see everyone choose to go to the IT field to mine 6 figures at the end of the month. But, this is simply camouflaged to squeeze the pocket of learners by Ed-etch companies and individuals.


How Not Be a Garbage Scientist

Zero Hype: If, everyone is learning LLM and Chatgpt and making millions overnight using Prompt engineering kinda stuff around you. You don't need to be reactive by jumping with them in the same. These are just noisy and hype trends. You will just end up with hype nothing else; which will barely help you to solve business cases in real-time during your work. Still, now, you can learn from this Old Gold: Link


How Not Be a Garbage Scientist

Consultation: Consult with Experts who have been working in the field for longer; instead of social media influencers. Many Social media Influencers have made AI and DS courses and are selling and earning million. On the other hand, learners are draining their pockets with zero output. Find someone who has been in this industry and contributed to many projects with his or her competency (timestamped evidence either on GitHub or LinkedIn or anywhere). Consult with them about learning strategy and what you are planning to achieve; even if you are learning from open source content. That's fine.


How Not Be a Garbage Scientist

Learning in a conventional way: Best way to do this is by learning the 3-way technique. Learning Statistics and ML: Complete theory with detailed concepts 2. Learning Tools as required (for example - Python, Rapidminer, Julia, etc.) 3. Learning representation: After developing a complete AI ML strategy. How will you represent that to the business stakeholders, partners and peers, and your boss? Representation of your work in Data science matters a lot. Building fancy models which can not be perceived by stakeholders or consumers ends up in a mess. I will always suggest learning in a conventional way. Instead of trying to learn from ticktock or Instagram influencers who does daily marketing vlog. Learning 20 topics in 20 minutes is nothing more than just garbage.


Modern NLP: A Detailed Overview. Part 4: The Latest Developments

Generating Long Sequences with Sparse Transformers In 2019, transformers started to get immensely popular and started to be used for numerous works. One issue emerged as the input context length grew in length. As transformers, use attention on the entire length of the input, the required time and memory requirements increase quadratically with the increase of the length of the input. The time complexity becomes O(N²) as for each token or word, the model attends to all the other words, so O(N) for each word, for all the words, it becomes N x O(N) = O(N²). This causes the learning to soon be intractable. OpenAI released this paper in 2019, to deal with this exploding time issue. The idea was to introduce a modeling change that will reduce the time complexity to N^(1+ 1/p) where p is the factorization element with a value >1, usually p=2, without compromising performance. So, finally, it becomes O(N*sqrt(N)). Observations: It was seen that, though the attention mechanism attends to all the pixels or words for a particular token, it is not always required as seen in the figure below. It was found that as layers were already learning the row and column-based sparse attention and data-dependent patterns were getting established. So, the authors concluded, sparse weighted kernels can be introduced for building an attention matrix without affecting the performance of the model too much. Idea: The authors broke down the full self-attention into 2 steps, with 2 different kernels based on observations of how attention is distributed. The authors found for attention to images, generally, attention is given to a certain pattern, but for non-image data like audio and text, there is no such fixed pattern for the kernel. The decided kernels are as shown in the image below. The first image (a) shows the actual transformer self-attention, where, to produce one output token, the model attends to all the steps or tokens occurring before the output position. The authors factorized the actual self-attention into two different attention-based kernels, as in (b) and (c ). (B) Strided Kernel: In this case, the model attends to the row and column of the output position. This kind was useful for image-based cases. (C ) Fixed Kernel: In this case, the model attends to a fixed column, and the elements after the latest column element. This kind was useful for text and audio. These factorized attention layers provided near-equal results but successfully reduced the operation time and memory consumption. The architecture of the sparse transformer: The authors introduced a residual architecture to combine factorized attention, against the original architecture of the transformer model. They proposed 3 approaches: (1) Each residual block will have 1 kind of factorized attention kernel, and we will arrange them in an interleaved manner (2) There will be 1 attention head, which will jointly attend the positions given by both the kernels and will function as a merged head. (3) There will be multiple attention heads as in the original architecture, multiple attention mechanisms will be done parallelly, and the results will be concatenated to get the final result. This architectural idea is also used in GPT 3. Paper link: https://arxiv.org/pdf/1904.10509.pdf Important Conclusion: By using factorized attention and interleaved residual structure, we can drastically reduce computation and time while maintaining efficiency.


Language Models are Few-Shot Learners The mentioned work was published by OpenAI in 2020. The model proposed stands as one of the largest models in the NLP world. The work done opened a lot of doors in the future for NLP, whose innovations we enjoy today. The authors realized 3 problems with the standing approach of the models. (1) There is a huge need for data to train the models, which are task-specific. But, it is sometimes not possible to find these supreme amounts of data. (2) The models are pre-trained on a large distribution of data as a generalized model, but after specific task fine-tuning, they are projected on a very a narrow-distribution, which makes the model skewed, and if the data is less, there is a chance of overfitting and bad performance. (3) As humans, we don't require too much data, to learn something, generally, we need a few demonstrations or even a prompt at times to learn, which is very different for machines. If we need models to behave similarly to humans, the data problem had to be solved. To solve these issues, the authors understood that a very generalized model needs to be trained. The only way was a concept called Meta-Learning.


Meta-Learning Meta-Learning is a developing field in machine learning, which teaches the models how to learn, so it is also called Learning to learn. It is understood that humans learn new things very quickly as we possess prior knowledge about most things from our regular experiences, which helps us to have an idea about how to learn, but when a model starts learning, it has no idea about the data or the target, which plays as the difference. This is often termed Conditioning, which is a property of any living being to learn from past experiences and stimuli. It is said that there are 3 main types of prior knowledge, Prior Knowledge of similarityPrior Knowledge of dataPrior Knowledge of learning Meta-learning plans to teach the model in such a way that it is possible to achieve what humans can do. It is often also used to perfect algorithms. There are certain possibilities seen with meta-learning: (A) Few-shot learning: Also called N-Way-K-Shot learning, where there are N classes and K samples from each class. The model is trained on the samples, from these data, which is called a support set, but the learning is validated on a separate set called the query set. Just note in this case, back-propagation happens on a very short support set. (B) One-Shot Learning: Only one sample from each N class are present in the support set, the model has to perform on the query set. Back-propagation is there. (C ) Zero-Shot Learning: No samples are present, the model directly performs on the query set. So, basically, a pre-trained data model can directly perform with less or no data for specific tasks. The pre-training phase is called the Meta-Learning phase and the specific learning phase is called the adaptation phase in the universe of meta-learning. To train a meta-learner, generally a plethora of different datasets and models are taken to make a generalized setting rather than a task-specific setting. There is a whole learning and algorithm stream of Meta-learning, which I am not going to go into, or else, it will be too long. I have just touched on the parts necessary to understand GPT3. To implement meta-learning in language models, authors used a method called in-context learning, which we will see in a minute. The idea was taken from the GPT-2 work, which proved that, if pre-trained on a huge corpus of different data, the model learns the skill of capturing several dependencies, and pattern recognitions which helps it to very quickly adapt to the new specific tasks with a very few demonstrations. Paper links: https://arxiv.org/pdf/2004.05439.pdf https://aclanthology.org/2022.acl-long.53.pdf Now, what is in-context learning?


In-Context Learning In-context learning means learning from an analogy. It is very similar to how we as humans learn. Usually, there are 3 cases with human learning: We are given a prompt, like, "Write a paragraph about a tree."We are given a prompt with a demonstration"Find the sum: 1+2=34+5= <prediction>"We are given a prompt with multiple demonstrations."Find the sum: 1+2=34+5= 96+7= <prediction>" In-context learning works in the same way. We input a prompt or a prompt with a demonstration in the context vector of the model, and the model outputs a prediction result. Now, in this case, there is no backpropagation or learning. The whole input with the prompt and the demonstration is input as a string, in the input vector of the model. It can be said its kind of prediction based on conditioning. Now, the authors have defined 4 cases to evaluate the model's performance: Fine-tuning as done in the previous cases, after pre-training, the model is trained on task-specific data.Few-Shot learning: In this case, on a pre-trained model, a prompt with multiple demonstrations was fed. The number of demonstrations ranges from 10 to 100 depending on how many can be fit in the context length of the input vector. The model predicts based on the demos. No back-propagation is there.One-Shot Learning: In this case, the prompt with only one demonstration is fed to the model. No back-propagation happensZero-Shot Learning: In this case, only a prompt is provided with no demonstration.


Training language models to follow instructions with human feedback: InstructGPT After releasing GPT3, researchers at OpenAI realized that though large language models can perform great, it does not necessarily mean they are operating according to user intent, on the contrary, the results generated by GPT3 can be untrue, toxic and not aligned enough to help people. To remedy the situation, the authors cut down GPT3 with 175B parameters, 100x to create a model with 1.3B parameters named GPT3.5 or InstructGPT. The model was fine-tuned with the help of human feedback to better align with human intent. The authors wanted the model to provide honest, helpful, and harmless answers. In order to make the model more human interaction friendly, the authors introduced a method from a previous work, which uses Reinforcement Learning with human feedback. The Proximal policy optimization or PPO reinforcement learning technique was used. In order to train the InstructGPT model, the authors have used 4 steps, the first step is plain old language modeling for pre-training followed by 3 steps. The authors started with a pre-trained GPT3 model after this step. The first step contained a short supervised fine-tuning step. The authors posted a pre-GPT3.5 portal, where users posted their prompts and demonstrations. Also, OpenAI hired 40 contractors after screening and made them write up some prompts and demonstrations. They made sure the tasks had enough diversity, including generation, question answering, dialog, summarization, extractions, and other natural language tasks, to avoid skewness in the fine-tuning step. They also eliminated the task prompts which were unclear in their motive. The pre-trained GPT3 model was fine-tuned on this data with a drop-out of 0.2 for 16 epochs.In step 2, we train a reward model for reinforcement learning, which will interact with the main model in the final PPO training phase. Now, in the case of a generative model, it might generate multiple outputs by feeding the same inputs. In this step, the fine-tuned model from the first step was fed the inputs, it gave multiple outputs and the contractors or labelers marked the preferred output. Thus, a dataset is created. The fine-tuned GPT3 model's final layer was removed and it was built to take in a prompt and an answer and it had to produce a scalar reward value. The dataset made by the labelers was used to train the reward model. To train the reward model, the size of the GPT3 model was reduced from 175B parameters to 6B parameters, as it was seen training such a big model was supremely time-consuming and also unstable at times. The scalar reward is formulated as the difference of log odds with the probability of the answer produced by the model to the preferred answer by the labeler. So, that is the human feedback mechanism is introduced.In the last and final step, the authors take the fine-tuned model from step 1 and reduce its dimension to 1.3B parameters, which is then fine-tuned again using a PPO reinforcement algorithm in a bandit environment. This basically means the model is given an arbitrary prompt from the user, and an answer is produced by the model. The reward model takes in the answer and provides a scalar reward. The trainee model tries to find an optimal policy to maximize the reward against the reward model. Findings: These were observed when InstructGPT was evaluated based on the target of human-requirement-aligned behavior and performance: Labelers significantly prefer InstructGPT outputs over outputs from GPT-3.InstructGPT models show improvements in truthfulness over GPT-3InstructGPT shows small improvements in toxicity over GPT-3, but no biasThe InstructGPT does not reflect on the language models on which it is trained when predicting public datasets.The model performs the generalization of the principles well when tested on data outside the fine-tuning domain.InstructGPT still makes simple mistakes. For example, InstructGPT can still fail to follow instructions, makeup facts, give long hedging answers to simple questions or fail to detect instructions with false premises. Paper link: https://arxiv.org/pdf/2203.02155.pdf


ChatGPT After the instant success of the InstructGPT model, OpenAI decided to fine-tune the InstructGPT model on more general conversational texts, to induce chatbot-based features in the model. This gave birth to chatGPT, which is more human-like and possesses better conversational skills. So, chatGPT is built for a specific task and is a much smaller version of the original GPT3 model.


Conclusion ChatGPT made a major impact on the world of technology as a tool that can revolutionize things. After the success, now OpenAI, with the backing of Microsoft, has released GPT-4 also, which extends the power of GPT to images also. The specs of the model have not been released due to security reasons. Google on the other hand also launched its venture in the world of chatbots with BARD, which is powered by the underlying LAMDA or Language Models for Dialog Applications models. The target of LAMDA is also very similar to chatGPT, building a chatbot that can be human-requirement aligned and can produce truthful, and nontoxic responses. Paper links: GPT4: https://arxiv.org/pdf/2303.08774.pdf LAMDA: https://arxiv.org/pdf/2201.08239.pdf Other than this also, there have been several improvements like XLNet and Ernie. Tech Giants are looking for constant improvements in the field with the hopes of a technically stronger tomorrow. Hope this helps. Happy Reading!!


Meet WebAgent: DeepMind's New LLM that Follows Instructions and Complete Tasks on Websites

Enter WebAgent WebAgent approaches the task by planning sub-instructions for each step, summarizing long HTML pages into relevant snippets based on these sub-instructions, and executing the task by grounding the sub-instructions and HTML snippets into executable Python codes. To build WebAgent, Google DeepMind combines two LLMs: "Flan-U-PaLM" for generating grounded code and "HTML-T5," a newly introduced domain-expert pre-trained language model responsible for task planning and conditional HTML summarization. HTML-T5, designed with an encoder-decoder architecture, excels in capturing the structure of lengthy HTML pages by utilizing local and global attention mechanisms, and it is self-supervised pre-trained on a vast corpus of HTML data synthesized from CommonCrawl. Existing LLM-driven agents typically handle decision-making tasks with a single LLM by prompting different examples per role. However, for more complex real-world tasks, this approach falls short. Google DeepMind's comprehensive evaluations demonstrate that WebAgent's combined method, integrating plug-in language models, significantly improves HTML understanding and grounding, leading to better generalization. WebAgent achieves over 50% increase in success rates for real-world web navigation, and detailed analysis reveals the critical role of coupling task planning with HTML summarization using specialized language models for successful task execution. Furthermore, WebAgent performs admirably on static website comprehension tasks, surpassing single LLMs in QA accuracy and performing competitively against strong baselines. Google DeepMind's WebAgent is an innovative composition of two distinct language models, HTML-T5 and Flan-U-PaLM, working together to enable efficient web automation tasks that involve navigating and processing HTML documents. HTML-T5, a domain-expert encoder-decoder language model, plays a crucial role in predicting sub-instructions for the next-step program and conditionally summarizing lengthy HTML documents. This specialized model strikes a balance between the general capabilities of language models like T5, Flan-T5, and Instruct-GPT, which exhibit superior web navigation with strong HTML comprehension, and the HTML-specific inductive biases found in prior transformer models like those proposed by Guo et al. HTML-T5 leverages local and global attention mechanisms in the encoder to effectively handle the hierarchical structure of HTML inputs. The local attention focuses on nearby tokens to the left and right of each element in the HTML, such as <input>, <label>, or <button>, while the transient global attention allows tokens to attend beyond the immediate neighborhood, by dividing the input sequence into blocks and computing global tokens through summation and normalization. This hierarchical approach aligns naturally with the structure of HTML documents, where elements are defined locally and iteratively integrated globally, such as <body>, <form>, or <div>. The model is pre-trained on a large-scale HTML corpus extracted from CommonCrawl, utilizing a mixture of long-span denoising objectives. In contrast to traditional dense attention used in natural language models, HTML-T5's adoption of local and global attention mechanisms better captures the explicit hierarchy present in HTML documents. Leaf elements in HTML (e.g., <input>, <label>, etc.) are effectively processed using local attention, while internal elements (e.g., <body>, <form>, etc.) are compressed into transient global attention. This design choice aligns seamlessly with the hierarchical syntax of HTML, enhancing the model's ability to comprehend the structure and attributes of HTML elements. To implement HTML-T5, Google DeepMind builds upon the base architecture of LongT5, which incorporates dense attention in the decoder. This combination of local and global attention mechanisms with the decoder's dense attention provides an effective and well-rounded solution for handling HTML documents in web automation tasks.


The Results To illustrate the capabilities of Google DeepMind's WebAgent, let's look at automation in the context of a real estate website. In this setting, WebAgent exemplifies a powerful flow encompassing planning, summarization, and grounded program synthesis. In this process, HTML-T5 takes the lead, employing an iterative approach to predict decomposed sub-instructions and task-relevant snippets, depicted in orange, by closely analyzing the HTML documents, instructions in yellow, and historical predictions in green. Simultaneously, Flan-U-PaLM, triggered by the sub-instructions and snippets in orange, deftly decodes Python programs represented in blue. The results achieved by WebAgent are truly remarkable, boasting an impressive 70% success rate on real websites. This significant advancement surpasses the performance of the single LLM approach by an impressive margin of over 50%. This achievement indicates that breaking down the task into a sequence of sub-problems, each tackled by different language models, can substantially enhance the overall task success. In addition, Google DeepMind presents an innovative and scalable recipe for crafting HTML-specialized language models. This approach involves training local and global attention mechanisms using a combination of long-span denoising objectives. The ultimate aim is to adeptly capture the hierarchical structures that underlie HTML documents, paving the way for improved comprehension and more effective handling of HTML-related tasks.


Meta's Llama 2: Revolutionizing Open Source Language Models for Commercial Use

I. Llama 2: Revolutionizing Commercial Use Unlike its predecessor Llama 1, which was limited to research use, Llama 2 represents a major advancement as an open-source commercial model. Businesses can now integrate Llama 2 into products to create AI-powered applications. Availability on Azure and AWS facilitates fine-tuning and adoption. However, restrictions apply to prevent exploitation. Companies with over 700 million active daily users cannot use Llama 2. Additionally, its output cannot be used to improve other language models.


II. Llama 2 Model Flavors Llama 2 is available in four different model sizes: 7 billion, 13 billion, 34 billion, and 70 billion parameters. While 7B, 13B, and 70B have already been released, the 34B model is still awaited. The pretrained variant, trained on a whopping 2 trillion tokens, boasts a context window of 4096 tokens, twice the size of its predecessor Llama 1. Meta also released a Llama 2 fine-tuned model for chat applications that was trained on over 1 million human annotations. Such extensive training comes at a cost, with the 70B model taking a staggering 1720320 GPU hours to train. The context window's length determines the amount of content the model can process at once, making Llama 2 a powerful language model in terms of scale and efficiency.


III. Safety Considerations: A Top Priority for Meta Meta's commitment to safety and alignment shines through in Llama 2's design. The model demonstrates exceptionally low AI safety violation percentages, surpassing even ChatGPT in safety benchmarks. Finding the right balance between helpfulness and safety when optimizing a model poses significant challenges. While a highly helpful model may be capable of answering any question, including sensitive ones like "How do I build a bomb?", it also raises concerns about potential misuse. Thus, striking the perfect equilibrium between providing useful information and ensuring safety is paramount. However, prioritizing safety to an extreme extent can lead to a model that struggles to effectively address a diverse range of questions. This limitation could hinder the model's practical applicability and user experience. Thus, achieving an optimum balance that allows the model to be both helpful and safe is of utmost importance. To strike the right balance between helpfulness and safety, Meta employed two reward models - one for helpfulness and another for safety - to optimize the model's responses. The 34B parameter model has reported higher safety violations than other variants, possibly contributing to the delay in its release.


IV. Helpfulness Comparison: Llama 2 Outperforms Competitors Llama 2 emerges as a strong contender in the open-source language model arena, outperforming its competitors in most categories. The 70B parameter model outperforms all other open-source models, while the 7B and 34B models outshine Falcon in all categories and MPT in all categories except coding. Despite being smaller, Llam a2's performance rivals that of Chat GPT 3.5, a significantly larger closed-source model. While GPT 4 and PalM-2-L, with their larger size, outperform Llama 2, this is expected due to their capacity for handling complex language tasks. Llama 2's impressive ability to compete with larger models highlights its efficiency and potential in the market. However, Llama 2 does face challenges in coding and math problems, where models like Chat GPT 4 excel, given their significantly larger size. Chat GPT 4 performed significantly better than Llama 2 for coding (HumanEval benchmark)and math problem tasks (GSM8k benchmark). Open-source AI technologies, like Llama 2, continue to advance, offering strong competition to closed-source models.


V. Ghost Attention: Enhancing Conversational Continuity One unique feature in Llama 2 is Ghost Attention, which ensures continuity in conversations. This means that even after multiple interactions, the model remembers its initial instructions, ensuring more coherent and consistent responses throughout the conversation. This feature significantly enhances the user experience and makes Llama 2 a more reliable language model for interactive applications. In the example below, on the left, it forgets to use an emoji after a few conversations. On the right, with Ghost Attention, even after having many conversations, it will remember the context and continue to use emojis in its response.


VI. Temporal Capability: A Leap in Information Organization Meta reported a groundbreaking temporal capability, where the model organizes information based on time relevance. Each question posed to the model is associated with a date, and it responds accordingly by considering the event date before which the question becomes irrelevant. For example, if you ask the question, "How long ago did Barack Obama become president?", its only relevant after 2008. This temporal awareness allows Llama 2 to deliver more contextually accurate responses, enriching the user experience further.


VII. Open Questions and Future Outlook Meta's open-sourcing of Llama 2 represents a seismic shift, now offering developers and researchers commercial access to a leading language model. With Llama 2 outperforming MosaicML's current MPT models, all eyes are on how Databricks will respond. Can MosaicML's next MPT iteration beat Llama 2? Is it worthwhile to compete with Llama 2 or join hands with the open-source community to make the open-source models better? Meanwhile, Microsoft's move to host Llama 2 on Azure despite having significant investment in ChatGPT raises interesting questions. Will users prefer the capabilities and transparency of an open-source model like Llama 2 over closed, proprietary options? The stakes are high, as Meta's bold democratization play stands to reshape preferences and partnerships in the AI space. One thing is certain - the era of open language model competition has begun.


VIII. Conclusion With the launch of Llama 2, Meta has achieved a landmark breakthrough in open-source language models, unleashing new potential through its commercial accessibility. Llama 2's formidable capabilities in natural language processing, along with robust safety protocols and temporal reasoning, set new benchmarks for the field. While select limitations around math and coding exist presently, Llama 2's strengths far outweigh its weaknesses. As Meta continues honing Llama technology, this latest innovation promises to be truly transformative. By open-sourcing such an advanced model, Meta is propelling democratization and proliferation of AI across industries. From healthcare to education and beyond, Llama 2 stands to shape the landscape by putting groundbreaking language modeling into the hands of all developers and researchers. The possibilities unlocked by this open-source approach signal a shift towards a more collaborative, creative AI future.


How To Stay Updated With Machine Learning and Computer Vision Advances In 2023?

Motivation Recent updates in machine learning (ML) and computer vision (CV) are a mouthful, from Stable Diffusion for generative artificial intelligence (AI) to Segment Anything as foundation models. Let's not forget large language models like Llama 2 and ChatGPT. It is getting increasingly difficult to stay up-to-date in the ML or CompVis community. In this article, I'll share with you some newsletters, youtube channels, podcasts and platforms that you can follow to stay buckled up on this hype train that we are in right now.


This article is organized as follows: NewslettersYouTube channelsPodcastsOther platforms Let's get started!


Newsletters Ground Truth: Currently one of my favorites right now! It gives you the latest and greatest breakthroughs happening in the computer vision space. Additionally, it also shares news on recent techniques and tools, as well as best practices. This can be really helpful in you're someone who is working in the industry. The Neuron: While this is not specifically for computer vision, it is a really good newsletter to follow, which gives you a relevant, interesting, and, more importantly, digestible compilation of what's going on in AI, both trends and tools. For me, it takes only about five minutes to read and it points to links that you can go through later if you want to know further details. The Batch: It is also a generic AI newsletter that shares recent news and insights happening in AI. At the beginning of the newsletter, Andrew Ng shares some of his opinions on the topics. Compared to the previous ones, this is a rather comprehensive newsletter. Import AI: It is a weekly newsletter that shares news on cutting-edge AI in research as well as in the industry. It also analyzes the implications it has on real world and talks about safety and ethical concerns in the field of AI. Davis Summarizes Papers: This is not exactly a newsletter, but the writer basically summarizes 10 to 20 ML research papers that come out on arXiv each week. I find this really helpful as it provides a compressed understanding of the papers all together. Of course, you can read the paper in detail if you find it really interesting. If you're a grad student or a researcher, you should definitely have a look!


YouTube Channels Yannic Kilcher: I guess you know his channel; if not, just subscribe already! I like to think of Yannic's channel as the BBC News of AI. As he mentions in about section: "I make videos about machine learning research papers, programming, and issues of the AI community, and the broader impact of AI in society." If you had only a YouTube channel to follow to keep track of AI stuff, this would be it. Two Minute Papers: For short summaries of AI research papers and news in general, this is a really good channel to follow. Think of Davis Summarizes Papers, but a video walkthrough. Some videos are longer than two minutes haha. Again, grad students and researchers take note! Abhishek Thakur: A new ML algorithm came out? Abhishek's channel makes tutorials to build them. Think of if this channel as not only keeping up-to-date with new ML algorithms but also learning how to implement those and build projects! This channel reminds me of my undergrad days (*nostalgic*) when I used to follow youtube tutorials for hobby projects. Some other awesome channels you could consider following are AssemblyAI, What's AI, and AI Coffee Break.


Podcasts Lex Fridman Podcast: As Lex says: "Conversations about nature of intelligence, consciousness, love, and power". My tops ones are this and this. The Robot Brains Podcast: Pieter Abbeel aims to discuss with leading experts in AI with a focus on robotics how to build robots with brains! This is my favorite one so far. Machine Learning Street Talk: Currently the top AI podcast on Spotify and is inspired by academic research. The podcast dives deep in AI technical analysis with pre-eminent figures in AI along with substantial scope and rigor by covering the main ideas in the field. TWIML AI Podcast: Through the podcast, they bring a diverse set of ML and AI researchers, practitioners, and innovators with the goal of making ML and AI more accessible to communities. It is targeted toward ML/AI researchers, data scientists, engineers, and tech-savvy business and IT leaders. Jay Shah Podcast: Jay interviews people in ML and AI from both industry and academia with the goal of getting advice from the interviewees on how to get started with it. Also, discuss their journey, insights, as well as latest research topics.


Other platforms Websites: Consider following paperswithcode and papers.labml.ai for recent papers in ML. You can sort according to date and time and also find papers by topic. GitHub: I like to follow 52CV to stay updated with papers and their code published in top vision conferences. For example, here are the papers along with code (mostly) for CVPR 2023, which is also categorized by specific topics like Object Detection and Continual Learning etc.. Twitter: I have used Twitter to stay up-to-date with ML/CV for a few years now. There are a bunch of people I follow who work on ML/CV in both industry and in academia. You can find them here.


Conclusion In this article, I talked about ways to stay updated with the machine learning and computer vision field for practitioners in academia or on the industry. I listed and described YouTube channels, newsletters, podcasts and other platforms which you could use to keep track of recent progress in ML/CV both in industry and academia. You can pick among the ones mentioned here and see what works for you instead of following all of them and getting overwhelmed!


About the author I am a Ph.D. candidate at Concordia University in Montreal, Canada, working on computer vision research. I am also an Applied ML Scientist at Decathlon, where I help build new ML systems that transform sports images and videos into actionable intelligence. If you're interested to learn more about me, please visit my webpage here.


Create Your Own Data Analyst Assistant With Langchain Agents

What kind of Agent are we going to create? We will create an incredibly powerful Agent that allows us to perform data analysis actions on any Excel sheet we provide. The best part is that despite its power, it is perhaps one of the simplest Agents to produce. So it's a great option as the first Agent of the course: Powerful and straightforward. The source code can be found in the GitHub repository of the Practical Course on Large Language Models. https://github.com/peremartra/Large-Language-Model-Notebooks-Course/blob/main/LangChain_Agent_create_Data_Scientist_Assistant.ipynb. As a model, we will use the OpenAI API, which allows us to choose between GPT-3.5 or GPT-4. It is important to say that the model used in an Agent should be the latest generation, capable of understanding text, making it, and making code and API calls. In other words, the more powerful the model, the better.


Starting our LangChain Agent. In this section, we will go through the code available in the Notebook. I recommend opening it and executing the commands in parallel. It has been prepared to work with a Dataset available on Kaggle, which can be found at https://www.kaggle.com/datasets/goyaladi/climate-insights-dataset. You can download the Excel file from the Dataset and use it to follow the exact same steps, or you can use any Excel file you have available. The notebook is set up, so you can upload an Excel file from your local machine to Colab. If you choose to use your own file, please note that the results of the queries will be different, and you may need to adapt the questions accordingly.


Install and load required libraries. As always, we need to install libraries that are not available in the Colab environment. In this case, there are four: langchain: A Python library that allows us to chain the model with different tools. We have seen its usage in some previous articles.openai: It will enable us to work with the API of the well-known AI company that owns ChatGPT. Through this API, we can access several of their models, including GPT-3.5 and GPT4.tabulate: Another Python library that simplifies the printing of data tables, which our agent may use.xformers: A recently created library maintained by Facebook that uses LangChain. It is necessary for the operation of our agent. Now, it's time to import the rest of the necessary libraries and set up our environment. Since we will be calling the OpenAI API, we will need an API key. If you don't have one, you can easily obtain it from: https://platform.openai.com/account/api-keys. As it is a paid API, it will ask for a credit card. Don't worry, it's not an expensive service. You pay based on usage, so if you don't use it, there's no cost. I have conducted numerous tests, not only for this article, as I write extensively about using the OpenAI API, and the total cost has been less than 1 euro the last month. It's crucial to keep your API KEY confidential. If someone gains access to it, they could use it, and you would be responsible for the charges incurred. I've set a monthly limit of 20 euros, just in case I make a mistake and accidentally upload it to GitHub or publish it on Kaggle. In my case, it's more likely that I will make some errors since I share the code on public platforms so that you have it available. Now, we can import the necessary libraries to create the Agent. Let's import three libraries: OpenAI: It allows us to interact with OpenAI's models.create_pandas_dataframe_agent: As the name suggests, this library is used to create our specialized agent, capable of handling data stored in a Pandas DataFrame.Pandas: The well-known library for working with tabular data.


Load the data and create the Agent. To load the data, I've prepared a function that allows you to upload an Excel file from your local disk. The crucial part is that the Excel file should be converted into a DataFrame named 'document'. I've used the Excel file from the climate insights dataset available on Kaggle. The process of creating an Agent is as simple as making a single call. As you can see, we are passing three parameters to create_pandas_dataframe_agent: The model: We obtain it by calling OpenAI, which we imported from langchain.llms. We are not specifying the name of the model to use; instead, we let it decide which model to return. The temperature parameter is set to 0, indicating that we want the model to be as deterministic as possible. The temperature value ranges from 0 to 2, and the higher it is, the more imaginative and random the model's response will be.The document to use: In this case, it's the DataFrame created by the read_csv function from the Pandas library.The 'Verbose' parameter: We set it to True because we want to see how the Agent thinks and what decisions it makes during the process. That's it! As I mentioned, it is one of the simplest Agents to create. We will explore other types of Agents later on. Now it's time to see how powerful it is.


Using the Agent. Let's see how it reacts to different questions about the data and how it proceeds with the steps. Understanding how to use our Agent is a crucial skill. Let's take a look at an example of the data used. After studying the data, we can start asking questions to our agent. To do that, we will use the run method of the newly created agent. The first question is going to be very general: we want it to look at the data and give us a brief summary of about 100 words. Let's analyze the response: If we study the trace, we can observe three fields: Thought: It shows us its thoughts, indicating what it plans to do and its immediate objective.Action: We see the actions it performs, usually calling Python functions that it has access to.Observation: The data returned by the actions, which it uses to elaborate on its next objective. Let's take a look at the first iteration. It starts by stating its objective: Then, it proceeds to define two actions. First, it loads a Python shell that it will use to execute Python commands. Then, it calls the describe function of the DataFrame created internally to view the shape of the data. The result is displayed in Observation. In the last iteration, it came to its conclusions. As we can see, it has understood the data perfectly and explains the content we can find. This a response that I would consider correct and really addresses the request we made. A perfect score for our agent. Now let's move on to the last task we give our agent in the notebook. A much more complex task, and I must say, it performs quite well. This task is more complex. We are asking the Agent to clean the data, choose an algorithm, and use it to make a forecast of the temperatures for the next 15 years. Finally, we ask for a chart displaying the predicted temperatures. As we can see, it is using various libraries to accomplish the mission we assigned. The Agent selects a Linear Regression algorithm and loads it from the SKlearn library. To generate the chart, it uses the Matplotlib library. If there is any criticism we can give, it is that the generated chart is not easy to read; we cannot clearly see whether there is an increase in temperatures or not. From my perspective, I believe it has performed very well in the assigned task, and I would rate it 7 out of 10. To improve the score, it would need to enhance the chart. Nevertheless, I was able to analyze the data and use a Machine Learning model to make a forecast of temperatures for the next 15 years.


Conclusions. As I mentioned at the beginning of the article, Agents based on Large Language Models are going to revolutionize the way we work. Not only in the field of data analytics, but I'm sure many jobs can benefit from the capabilities of these Agents. It's important to consider that we are at the early stages of this technology, and its capabilities will increase, not only in the power of the models but, most importantly, in the number of interfaces they will have available. In the coming months, we may witness these Agents breaking the barrier of the physical world and starting to control machines through APIs.


Why You Should Care About the EU AI Act Today

The European Union Artificial Intelligence Act The EU AI Act is a proposal that was adopted by the European Commission in 2021 - this past week it was passed by the European Parliament and the final version is expected to pass later in 2023. The rule will apply to anyone who is distributing AI systems in any of the 27 member states of the European Union.


Reasons and goals of the proposal The European Commission rightly points out that AI can bring massive benefits to many of the difficult challenges the European Union and the world are facing, such as climate, health, and mobility. However, the technology also comes with certain risks. Consider a not-so-distant future where deep fakes are so prevalent that the public is unable to differentiate between what is real and what is fake. The implications of this scenario are far-reaching, impacting not only individuals' social and privacy concerns but also posing a threat to the very fabric of democratic systems. The commission is striving for a balanced approach that will effectively manage the risks while maintaining and advancing the EU as a leader in developing positive AI technologies. Objectives of the EU AI Act: Ensure safety and respect existing laws - such as GDPR, copyright and other safety lawsEnsure legal certainty - uncertainty about future regulations can scare away investors and entrepreneursStrengthen governance and enforcement of laws on fundamental rights and safety for AI systemsCreate a single market for AI throughout the EU - prevent a scenario where each of the 27 member states has different rules and regulations for AI The proposal attempts to present a comprehensive and future-proof framework. It includes flexible mechanisms that enable it to be dynamically adapted as the technology evolves.


AI Risk Categories The rule defines 4 risk-based categories. The risk-based approach means that different models are treated differently, as opposed to a blanket or specific approach. This is considered a relatively forward-looking form of rule-making as it is not prescriptive to specific tools and it generally allows for innovation. Understanding which category your model/product fits into is key to understanding the requirements you will need to comply with.


Category 1: Unacceptable risk These are AI systems that: Deploy subliminal techniques in order to distort a person's behavior in a manner that is likely to cause physical or psychological harm;Exploit vulnerabilities of a specific group of persons due to their age, physical or mental disability, in order to materially distort the behavior of a person in a way that is likely to cause physical or psychological harm;Perform social scoring that can lead to unfavorable treatment of individuals or groups - this could mean an AI model that influences credit scores or hiring processes based on potential profiling of race, gender, religion, etc.Deploy 'real-time' remote biometric identification in publicly accessible spaces for the purpose of law enforcement, except in specific extreme situations. Restrictions: These systems pose an unacceptable risk and are prohibited.


Category 2: High Risk AI Systems are defined as high-risk if they are either of the following: Safety components of regulated products. For example, products that require a CE mark (machinery, toys, radio equipment), aircrafts, motor vehicles, medical devices, PPE, and more.Certain AI Systems for bio-identification, management of critical infrastructure, education and training, law enforcement, migration, asylum and border control, justice and democratic processes and more. Restrictions and requirements: Requires a risk management system with specific testing to identify risk management measuresData and data governance - data sets shall be subject to appropriate data governance and management practices concerning design choices, data collection, data annotation, processing, statistical properties and biases and more.Detailed and up-to-date technical documentationMust include automatic logging capabilities while the system is in useTransparency requirementsAppropriate and declared levels of accuracyRegistration of the AI system in an EU database for high-risk AI systems3rd party conformity assessment + CE mark


Category 3: AI Systems with transparency obligations These are AI systems that interact with humans, detect emotion, determine association with (social) categories based on biometric data, or generate or manipulate content (deep fakes). Restrictions: Information/transparency obligations such as disclosers to end-users.


Category 4: Low or minimal risk These are all other AI systems Restrictions: None.


Example AI Systems Example 1: An agricultural company is using a computer vision model to segment and identify the condition of recently picked tomatoes before shipping them out to distribution. This AI model performs a quality assurance task and it does not interact with humans. Therefore the model is considered Low or Medium Risk and does not have any regulatory requirements. Example 2: A national railway operator has decided to use AI to monitor and verify that tracks are free of debris or obstructions. Being an important safety component of the railway's operations and that products relating to rail are already regulated in the EU, this AI system would fall under the High-Risk Category and would need to comply with the requirements above. Example 3: In order to generate more viral traffic to a social network a Large-Language Model (LLM) is used as a chat-bot that encourages minors to engage in dangerous behavior in the guise of a fun or cool challenge or game. This type of use of an AI system goes against EU values and is strictly prohibited. Example 4: A satirical television show uses deep fakes of politicians and celebrities as part of its humorous content. This type of content would need to include a disclosure that the content is generated through automated means and is not real.


Supporting innovation The European Commission understands that compliance with regulation comes with a cost which can sometimes hinder innovation. To address that, the rule encourages EU member states to set up testing environments or sandboxes in which developers of AI systems can test their systems in a controlled manner. The full details of what this will look like haven't been published yet but, these are likely to be regulatory sandboxes (as opposed to infrastructure ones). A regulatory sandbox would grant certain companies exemptions from part of the regulation while under the supervision of regulators for a limited period of time. The European Commission will also consider the needs and limitations of SMEs and startups when setting fees relating to conformity assessments.


What you can do today Start by determining which category your AI system falls under so that you understand the level of scrutiny you can expect. There are some intricacies and details that need to be considered, so get your legal team involved (and don't assume that they are already following the changes). Your legal team can help you better understand the requirements that need to be considered for your specific AI system. Make sure you are employing ML development best practices and standards. These include data, code, annotations and model versioning, experiment tracking and more. Best practices will keep your projects organized, reduce technical debt and make you more efficient so you can focus on experimenting and deploying better-performing models. Data governance and management - the new rules require that datasets have appropriate statistical properties so that they properly represent the population of intended users and are not biased or do not discriminate against a certain population. This means that for each model or product, ML practitioners need to have a good understanding of the data that was used to train, test and validate the model - keeping track of the correct data lineage is critical and is almost impossible to do retroactively. Proper data management and governance are important to remain compliant with existing EU regulations (the AI Act reiterates the importance of compliance with existing rules) like GDPR and copyright rules. Tools like DagsHub's Data Engine would allow you to include metadata with information about copyright licenses or whether certain data includes personal information.


Conclusion The EU AI Act will affect every individual or company that is planning to offer AI solutions and products in the European Union. The overarching purpose of the regulation is positive - keep people safe. While some limitations and costs are required in order to comply with the rule, the rule itself is designed to be flexible and allow innovation. Understanding the regulation, involving your legal team and properly managing data and projects with industry best practices and tools is key to being prepared and reducing technical debt. Since the EU is first to publish major AI regulation it would be expected that similar frameworks be introduced in other countries as well. Disclaimer: The information provided in this post is not intended to be legal advice. It is intended for informational purposes only and should not be relied upon for legal advice. If you need legal advice, please consult a qualified legal professional.


Google's Symbol Tuning is a New Fine-Tuning Technique that In-Context Learning in LLMs

Symbol Tuning Google Research introduces "Symbol Tuning," a potent fine-tuning technique that addresses the limitations of conventional instruction tuning methods. While instruction tuning can enhance model performance and in-context understanding, it comes with a drawback: models might not be compelled to learn from examples since the tasks are redundantly defined through instructions and natural language labels. For instance, in sentiment analysis tasks, models can simply rely on the provided instructions, disregarding the examples altogether. Symbol tuning proves particularly beneficial for previously unseen in-context learning tasks, excelling where traditional methods falter due to underspecified prompts devoid of instructions or natural language labels. Furthermore, models tuned with symbols demonstrate exceptional prowess in algorithmic reasoning tasks. The most remarkable outcome is the substantial improvements in handling flipped-labels presented in-context. This achievement highlights the model's superior capacity to leverage in-context information, even surpassing pre-existing knowledge. Symbol tuning offers a remedy by fine-tuning models on examples devoid of instructions and replaced natural language labels with semantically-unrelated labels like "Foo," "Bar," etc. In this setup, the task becomes ambiguous without consulting the in-context examples. Reasoning over these examples becomes crucial for success. Consequently, symbol-tuned models exhibit improved performance on tasks that demand nuanced reasoning between in-context examples and their labels. To evaluate the effectiveness of symbol tuning, the researchers utilized 22 publicly-available natural language processing (NLP) datasets with classification-type tasks, considering discrete labels. Labels were remapped to random choices from a pool of approximately 30,000 arbitrary labels belonging to three categories: integers, character combinations, and words. The experiments involved symbol tuning on Flan-PaLM models, specifically Flan-PaLM-8B, Flan-PaLM-62B, and Flan-PaLM-540B. Additionally, Flan-cont-PaLM-62B (abbreviated as 62B-c) was tested, representing Flan-PaLM-62B at a scale of 1.3 trillion tokens instead of the usual 780 billion tokens. The symbol-tuning procedure necessitates models to engage in reasoning with in-context examples to perform tasks effectively, as prompts are designed to prevent learning solely from relevant labels or instructions. Symbol-tuned models excel in settings that demand intricate reasoning between in-context examples and labels. To explore these settings, four in-context learning scenarios were defined, varying the level of reasoning required between inputs and labels for learning the task (depending on the availability of instructions/relevant labels). The results demonstrated performance improvements across all settings for models 62B and larger, with modest enhancements in settings with relevant natural language labels (ranging from +0.8% to +4.2%), and substantial improvements in settings without such labels (ranging from +5.5% to +15.5%). Remarkably, when relevant labels were unavailable, symbol-tuned Flan-PaLM-8B surpassed Flan-PaLM-62B in performance, and symbol-tuned Flan-PaLM-62B outperformed Flan-PaLM-540B. This suggests that symbol tuning empowers smaller models to match the performance of larger models on these tasks, thereby significantly reducing inference compute requirements (approximately saving ∼10X in compute). In general, symbol tuning shows significant improvements in in-context learning tasks, particularly for underspecified prompts. The technique also shows stronger performance than traditional fine-tuning in reasoning tasks and are more able to use in-content information to override prior knowledge. Overrall, symbol tuning can become one of the most interesting fine-tuning techniques.


AI Systems: Unearthed Bias and the Compelling Quest for True Fairness

Fairness in AI Solutions Fairness is at the forefront of responsible AI, implying that AI systems must treat all individuals impartially, regardless of their demographics or backgrounds. Data Scientists and ML Engineers must design AI solutions to avoid biases based on age, gender, race, or any other characteristic. Data used to train these models should represent the population's diversity, preventing inadvertent discrimination or marginalization. Preventing bias seems like an easy job; after all, we are dealing with a computer, and how the heck can a machine be racist?


Algorithmic Bias AI fairness problems arise from Algorithmic bias, which is systematic errors in the model's output based on a particular person. Traditional software consists of algorithms, while machine learning models are a combination of algorithms, data, and parameters. It doesn't matter how good an algorithm is; a model with bad data is bad, and if the data is biased, the model will be biased. We introduce bias in a model in several ways:


Hidden biases We have biases; there's no question about that, stereotypes shape our view of the world, and if they leak into the data, they will shape the model's output. One such example of this occurrence is through language. While English is mainly gender-neutral, and the determiner "the" does not indicate gender, it feels natural to infer gender from "the doctor" or "the nurse." Natural Language models, such as translation models or Large language models, are particularly vulnerable to that and can have the results skewed if not appropriately treated. A few years ago, I heard a riddle that went like this. A boy was playing on the playground when he fell and was severely injured; the father took the child to the hospital, but upon arriving, the doctor said, "I cannot operate on this child; he is my son!" How can this be? The riddle was that the doctor was a woman, the child's mother. Now picture a nurse, a secretary, a teacher, a florist, and a receptionist; were they all women? Surely we know there are male nurses out there, and nothing keeps a man from being a florist, but it is not the first thing we think about. Just as our mind is affected by this bias, so is the machine's mind. Today, July 17, 2023, I asked Google Translate to translate some professions from English to Portuguese. Google's translation of occupations such as teacher, nurse, and seamstress, make use of the Portuguese feminine pronoun "A" indicating the professional is a woman ("A" professora, "A" enfermeira, "A" costureira, "A" secretaria). In contrast, occupations such as professor, doctor, programmer, mathematician, and engineer use the Portuguese masculine pronoun "O" indicating the professional is a man ("O" professor, "O" medico, "O" programador, "O" matemático, "O" engenheiro). While GPT-4 has made some improvements, and I could not replicate the same behavior with my short quick tests, I did reproduce it in GPT-3.5. While the examples presented don't pose much of a threat, it's easy to think of potentially dire consequences of models with the same technology. Consider a CV analyzer that reads a resume and uses AI to determine if the applicant is suitable for the job. It would undoubtedly be irrational and immoral, and in some places, illegal to disregard the applicant for a programmer position because her name is Jennifer.


Unbalanced classes in training data Is 90% accuracy good? How about 99% accuracy? If we predict a rare disease that only occurs in 1% of people, a model with 99% accuracy is no better than giving a negative prediction to everyone, completely ignoring the features. Now, imagine if our model is not detecting diseases but people. By skewing the data toward a group, a model may have issues detecting a misrepresented group or even ignore it entirely. This is what happened to Joy Buolamwini. In the documentary Coded Bias, MIT computer scientist Joy Buolamwini exposed how many facial recognition systems would not detect her face unless she wore a white mask. The model's struggles are a clear symptom that the dataset heavily underrepresents some ethnic groups, which is unsurprising, as the datasets used to train these models are highly skewed, as demonstrated by FairFace [1]. The misrepresentation of the group proportions can lead the model to ignore essential features of misrepresented classes. While FairFace [1] balanced its dataset among the different ethnicities, it's easy to see that important datasets in the industry, such as LFWA+, CelebA, COCO, IMDM-Wiki, and VGG2 are composed of about 80% to 90% of white people, this is a distribution that is hard to see even in the whitest of countries [2] and, as demonstrated by FairFace [1], can significantly degrade models performance and generalization. While facial recognition may allow your friend to unlock your iPhone [3], we may face worse consequences from different datasets. The US judicial system systematically targets African Americans [4]. Suppose we create a dataset of arrested people in the US. In that case, we will skew the data toward African Americans, and a model trained on this data may reflect this bias to classify Black Americans as dangerous. This happened to COMPAS, an AI system to create a risk score of a criminal rescind in crime, exposed by ProPublica in 2016 for systematically targeting black people [5].


Data leakage In 1896, in the case of Plessy vs. Ferguson, the US solidified racial segregation. In the National Housing Act of 1934, the US federal government would only back neighborhood building projects that were explicitly segregated [6]. This is one of the many reasons why race and address are highly correlated. Consider now an electricity company creating a model to aid bad debt collection. As a data-conscious company, they decided not to include name, gender, or personally identifiable information on their training data and to balance the datasets to avoid bias. They instead aggregate the clients based on their neighborhood. Despite the efforts, the company has also introduced bias. By using a variable so correlated to race as an address, the model will learn to discriminate against race, as both variables could be interchanged. This is an example of data leakage, where a model indirectly learns to discriminate against undesired features. Navigating a world of systemic prejudice can be challenging; bias will sneak into the data in the most unexpected ways, and we should be overly critical of the variables we include in our model.


Detecting Fairness problems There is no clear consensus on what fairness means, but there are a few metrics that can help. When designing an ML model to solve a problem, the team must agree on the fairness criteria based on the potential fairness-related issues they may face. Once the criteria are defined, the team should track the appropriate fairness metric during training, testing, validation, and after deployment to detect and address fairness-related problems in the model and address them accordingly. Microsoft offers a great checklist to ensure fairness is prioritized in the project [7]. Consider dividing people into two groups of sensitive attributes A, a group a with some protected attributes, and a group b without those attributes; we may define some fairness metrics as follows: Demographic Parity: This metric asks if the probability of a positive prediction for someone from a protected group is the same as for someone from an unprotected group. For example, the likelihood of classifying an insurance claim as fraudulent is the same regardless of the person's race, gender, or religion. For a given predicted outcome R this metric is defined by: Predictive Parity: This metric is all about the accuracy of positive predictions. In other words, if our AI system says something will happen, how often does it happen for different groups? For example, suppose a hiring algorithm predicts a candidate will perform well in a job; the proportion of predicted candidates who actually do well should be the same across all demographic groups. If the system is less accurate for one group, it could be unfairly advantaging or disadvantaging them. For a given realized outcome Y we can define this metric as: False Positive Error Rate balance: Also known as equal opportunity, this metric is about the balance of false alarms. If the AI system makes a prediction, how often does it wrongly predict a positive outcome for different groups? For instance, when an admissions office rejects applicants to a university, how often are good suitable candidates rejected in each group? We can define this metric as: Equalized odds: This metric is about balancing both true positives and false positives across all groups. For a medical diagnostic tool, for example, the rate of correct diagnoses (true positives) and misdiagnoses (false positives) should be the same regardless of the patient's gender, race, or other demographic characteristics. In essence, it combines the demands of Predictive Parity and False Positive Error Rate Balance and can be defined as follows: Treatment equality: This metric examines the distribution of mistakes across different groups. Are the costs of these mistakes the same for other groups? For instance, in a predictive policing context, if two people - one from a protected group and one from an unprotected group - both don't commit a crime, they should have the same likelihood of being mistakenly predicted as potential criminals. Given the false positives FP and false negatives FN of a model, this metric can be defined as follows: At least in classification problems, computing the fairness criteria can be easily done using a confusion matrix. Still, Microsoft's fairlearn provides a suite of tools [8] to compute those metrics, preprocess data, and post-process predictions to comply with a fairness constraint.


Addressing fairness While fairness must be in the mind of every Data Scientist throughout the entirety of the project, we can apply the following practices to avoid problems: Data collection and preparation: Ensure your dataset is representative of the diverse demographics you wish to serve. Various techniques can address bias at this stage, such as oversampling, undersampling, or generating synthetic data for underrepresented groups.Model design and testing: It is crucial to test the model with various demographic groups to uncover any biases in its predictions. Tools like Microsoft's Fairlearn can help quantify and mitigate fairness-related harms.Post-deployment monitoring: Even after deployment, we should continuously evaluate our model to ensure it remains fair as it encounters new data and establishes feedback loops to allow users to report instances of perceived bias. For a more complete set of practices, one can refer to the previously mentioned checklist [7].


In conclusion Making AI fair isn't easy, but it's important. It is even harder when we can't agree on what fair even is. We should ensure everyone is treated equally and no one is discriminated against by our model. This will become harder as AI grows in complexity and presence in daily life. Our job is to ensure the data is balanced, biases are questioned, and every variable in our model is scrutinized. We must define a fairness criterion and adhere closely to it, being always vigilant, especially after deployment. AI is a great technology at the foundation of our modern data-driven world, but let's ensure it is great for all.


References [1] FairFace: Face Attribute Dataset for Balanced Race, Gender, and Age [2] https://en.wikipedia.org/wiki/White_people [3] https://www.mirror.co.uk/tech/apple-accused-racism-after-face-11735152 [4] https://www.healthaffairs.org/doi/10.1377/hlthaff.2021.01394 [5] https://www.propublica.org/article/machine-bias-risk-assessments-in-criminal-sentencing [6] https://en.wikipedia.org/wiki/Racial_segregation_in_the_United_States [7] AI Fairness Checklist - Microsoft Research [8] https://fairlearn.org/


How to Use the Bisection Method for Numerical Computing

Bisection method The bisection method in mathematics is a simple approach for locating numerical roots to an equation with a single unknown. The bisection method is the simplest numerical method for resolving the transcendental problem. We shall go into great detail on the bisection approach with solved issues in this article. To determine the roots of a polynomial problem, utilize the bisection method. It splits and separates the interval in which the equation's root is located. The continuous functions intermediate theorem serves as the method's guiding premise. It operates by reducing the distance between the positive and negative intervals until it approaches the right response. By averaging the positive and negative intervals, this technique reduces the distance between the variables. Although it is a straightforward process, it moves slowly. The bisection method is sometimes referred to as the dichotomy method, binary search method, interval halving method, and root-finding method.


Bisection Method Algorithm The bisection method is an approximation method to find the roots of the given equation by repeatedly dividing the interval. This method will divide the interval until the resulting interval is found, which is extremely small. CODE: Let's take a simple function. F(x) = x * x - 3 Let's write a very basic code to create a simple pipeline for bisection method code. The above is very basic but a basic code which is representing the bisection method at a brute-force level.


Let's see its output The above output is of playing with a and b for 50 iterations. The system will get in trouble in processing numbers greater than 16 digits due to limitations. But we can see that the values of roots are coming closer and closer. Substituting these latest values in the equation will get you, Very close to 0 !!! Now we know this logic is working. let's optimize the code and visualize the output so that we can understand it better. I modified this code such that it will take inputs and it will display the graph of roots. In the above fig, we can see our starting points and their F(x) values. the red highlighted line is what we want to achieve with these initial roots. In the above figure, I have plotted the outputs of the last 10 points out of 50. You can see the values converging towards 0 and the final few points are almost 0. It is a sign that we got out roots. Iterations are the limit you can put to control the whole execution. the parameter could be the threshold value for the Function. So either you have to stop after completing a certain number of iterations or if you achieved a certain threshold value. You can change the function and use this code to find roots for your own equation, or you can directly use the below link, which will land you on the bisection method online calculator.


Limitations of the Bisection method: To overcome these limitations, we have many other methods which are on the advanced level for root finding with different mechanisms. We will cover those in the next series of blogs.