GPT-4 a New Era of AI

Introduction In this article, I would like to give you a short overview of the most important findings and GPT-4 insights from the "Sparks of Artificial General Intelligence" paper (2023 Bubeck et al.). What are the main improvements compared to chatGPT? Why will GPT-4 and the following algorithm change our daily live drastically? A group of Microsoft engineers had a good head start of around five months on experimenting with the latest GPT-4 algorithm. They have some amazing findings, which I want to summarize here quickly. Thanks to Sebastien Bubeck et al. for the great work.


GPT-4 a New Era of AI

Table of Content IntroductionSeven Main Findings- Output Images from Text-Prompt- Produce Quality Code- Math- Ability to Use Tools - Interacting With the World- Understanding Humans- Weakness & Social InfluencesSummary


GPT-4 a New Era of AI

1 - Output Images from Text-Prompts GPT4 can draw images from a text-prompt. The images are not perfect but can be used with, e.g., Stable Diffusion to polish them up. Knowing the results from Stable Diffusion, might not sound very impressive, but here image generation is integrated into the discussion, making it more dynamic. Like any intense discussion, I will profit from a quick sketch on the whiteboard. Further, this can be seen as a great primer for Stable Diffusion, where you first get an Image where things are placed where you need it and only later graphically enhanced.


GPT-4 a New Era of AI

2 - Produce Quality Code GPT-4 is able to pass technical interviews on LeetCode. The results suggest (actually, the authors do) that GPT4 could be hired as a software engineer in its current state. They tested different difficulty levels and compared them to human levels. For the comparison, they left out all answers of humans who did not answer any correct answer to compare to at least a base knowledge of coding. The results are astonishing, especially in the hard bracket (See table below). Next to plain code, GPT-4 can also produce running 3D-games with a Zero-shot approach. The games are not trivial and work right from the get-go. ChatGpt (GPT-3.5) cannot do this.


GPT-4 a New Era of AI

3 - Math Next, to be able to use a calculator as a tool, GPT-4 was tested in the International Mathematics Olympiad (IMO) and could produce proofs for the given Math problems. Those problems are not simply calculus but require a higher level of understanding and creativity to solve the given problems. I would not even try to solve those problems. This is a remarkable step into general problem-solving capabilities. Being able to solve non-trivial problems by following a non-linear strategy is a key capability for an autonomous "thinking" system we all are longing for (Not the world-domination-type, but rather the do-the-dishes-for-me type)


GPT-4 a New Era of AI

4 - Ability to Use Tools A very important finding is the ability to use tools correctly without demonstrations and only limited instructions. The example they use is a calculator. GPT-4 just uses a calculator to solve simple calculus problems. Beforehand, ChatGPT (GPT3.5) would get the answer from estimating the next token, in this case, a number/result. ChatGPT (GPT3.5) was not able to use any tools. GPT-4 increases the reliability of its answers by using the right tool for the right job. On top of that, the algorithm can also use multiple tools to solve more complex problems like penetration testing of a security system.Further, GPT-4 can use tools, understand the output and use the results to advance solving a problem. As an example, it uses APIs to access calendars, browse the internet for information, communicate the findings via email, set an appointment, and finally inform all participants about the appointment in an email. The assistant we were all waiting for desperately is knocking on the door.


GPT-4 a New Era of AI

5 - Interacting With the World Also interesting, GPT-4 can produce map-visualizations from text prompt while navigating through them without getting lost. This could be interesting for autonomous navigation in the near future. A nice example would be cleaning robots which will receive audio commands, translate that into text prompts and create their internal map based on this. This map will be explored and optimized without any further input. We might not notice a large difference in the improved orientation skills. However, this is also an important step toward autonomous acting assistance.


GPT-4 a New Era of AI

6 - Understanding Humans This is really remarkable paragraph in the paper. The researchers tested several algorithms on the "theory of mind". The theory of mind describes the asset of being able to understand the cause of an opponent's behavior. This is important to understand the frame of a situation based on feelings, beliefs, desires, intentions, and also knowledge of the opponents. This goes even further by understanding thirds, fourths, etc. beliefs, emotions, etc., influencing the situation. GPT-4 passes advanced theory of mind tests used to assert the understanding of its opponent for its surrounding. GPT-4, in contrast to chatGPT, can even understand the root causes for misunderstanding in a given realistic situation/discussion. Asked about a possible solution for a stuck conversation, it offered several sensible solutions. While not fully tested in all regards, GPT-4 passes not only tests used on children but even more complex, nuanced, and realistic scenarios regarding the theory of mind. This further increases the capability of the algorithm to explain their reasoning, but as the theory of mind is also in part connected with empathy, this, most important, becomes another huge step towards GAI.


GPT-4 a New Era of AI

7 - Weakness & Social Influences GPT-4 is brilliant in incremental tasks such as Math problems, where one solution leads to the next problem. However, GPT-4 is an autoregressive model. This means that the model does not know what it will produce. This might explain the lack of generating complex music compositions, as for a good composition, you must know what the parts of your composition are, to be able to build up towards them and interconnect them properly. GPT-4 cannot work backward. Another problem is Misinformation. GPT-4 is capable of producing fake news and propaganda on a large scale using images, links, tools, ... . This might have huge implications for future use of GPT-4 as a manipulation or scamming tool.


GPT-4 a New Era of AI

Summary For me, using tools and interacting as a personal assistant is the most exciting, as it will impact us most direct in daily life. However, understanding the human counterpart is the biggest step to next-level AI in general. What time to be alive? WOW!


The AI Revolution: How Auto-GPT Unleashes a New Era of Automation and Creativity

Diving into Auto-GPT: AutoGPT is an innovative, open-source application that leverages the power of GPT-4, OpenAI's advanced generative pre-trained transformer technology. It is designed to autonomously develop and oversee a wide array of tasks. The self-prompting system employed by AutoGPT allows it to plan, justify decisions, and create detailed tasks. The application combines GPT-4's text generation capabilities with internet access for data retrieval and storage, as well as speech generation using the ElevenLabs API. Although AutoGPT is still in its experimental stage, it showcases the potential of AI-driven agents for a variety of tasks, including content creation, information gathering, and advanced memory management. Auto-GPT is capable of solving complex problems that require long-term planning using GPT-4's reasoning ability. It can create task lists and complete them one by one, even researching and learning to develop a viable business idea. To put it in simple terms, AutoGPT is designed to break down complex tasks into smaller sub-tasks that can be assigned and executed independently. This process continues recursively until the sub-task becomes small enough to be executed directly. By reliably completing larger and more complex sub-tasks, the system can batch and repeat them while keeping a good handle on the types of sub-tasks it can and can't execute successfully. Of course, there is still much work to be done to fine-tune and improve the system's performance on various levels.


Exploring Auto-GPT's New Abilities: AutoGPT utilizes GPT-4 to generate, prioritize, and execute tasks, and it uses plug-ins to access the internet and other resources. The system relies on external memory to keep track of its actions and provide context, allowing it to assess its situation, self-correct, and add new tasks to the queue. The system then prioritizes these tasks based on their urgency and importance, allowing it to work efficiently and effectively. Access to the internet: Auto-GPT can access the internet and gather real-time information from various sources. This keeps it up-to-date with the latest trends and allows it to access specialized and domain-specific sites to find important information for specific tasks.Effective memory management: Auto-GPT has effective long-term and short-term memory management. This allows it to save, read, and process information more effectively.GPT-4 text generation: Auto-GPT uses GPT-4 for text generation. This allows it to provide highly user-friendly and context-sensitive outputs.Parallel processing: Auto-GPT can spawn multiple instances of the agent working in parallel. This allows it to achieve goals more quickly.Access to various platforms: Auto-GPT has access to various platforms, such as ElevenLabs (for speech capabilities). This allows it to interact with different resources and services.Improved file storage and summarization: Auto-GPT's file storage and summarization abilities are improved with GPT 3.5. This allows it to store important documents and files while summarizing their content effectively.Continuous mode: Auto-GPT can run in continuous mode, making it 100% automated. This allows it to complete tasks without human intervention.


Simple example Let's take an example of how it works. Say your goal is to increase Twitter followers. The first thing AutoGPT will do is to get the higher level tasks to accomplish this goal. Content CreationEngage with Your AudienceConnect with InfluencersOptimize Your ProfilePromote Your Account Next, for each of the items above, it will spawn instances to try to get the corresponding tasks from there. For illustration, let's take content creation. AutoGPT will define the tasks to achieve for content creation as Share engaging content that adds valueUse high-quality visualsUse relevant hashtags to expand reach Next is to drill down into each of the above tasks.... Here is the task list for "Share engaging content that adds value" Identify your target audience and their interestsConduct research to stay up-to-date on industry news and trendsCreate original content that offers value to your followersUse visually appealing images or videos to accompany your postsPost consistently to maintain engagement with your followersUse relevant hashtags to reach a wider audienceInteract with your followers by responding to comments and messagesAnalyze your content performance to optimize future posts. Its a recursive process where every leaf in the tree is addressed. This process continues recursively till all the tasks are completed (forever). Every time it creates these tasks, it will justify and explain why it is doing this. Remember, AutoGPT may experience challenges such as getting sidetracked or confused, becoming stuck in a loop, leaving tasks incomplete, and lacking robustness as an agent. These are some of the potential issues that the system may encounter. Over time, this model will be improved. The crucial point to note is that AutoGPT is capable of automatically creating, analyzing, and prioritizing tasks, and it recursively generates new tasks until all tasks are completed. Here are some additional examples of goals you can ask AutoGPT, and it will create tasks and sub-tasks recursively to achieve those goals. Some examples include: Fitness: Completing a 10K race in under an hourFinancial: Saving $10,000 for a house down paymentSocial Media: Gaining 10,000 Instagram followers within six monthsLearning: Achieving fluency in a new language within a yearCooking: Mastering five gourmet recipes and hosting a dinner party for friends and family For each of these, AutoGPT will recursively spawn tasks, justify which ones it wants to pursue, and create new tasks, and this process continues till all the tasks are completed.


Other AI models Auto-GPT can even be accessed directly from the browser without having to install any Python scripts, known as Agent-GPT. Other projects like Baby AGI and Jarvis Hugging GPT aim to integrate language models and other functions to automate complex tasks. In these systems, the language model serves as a controller, utilizing other language or expert models and tools to achieve goals as autonomously as possible. You can read about Hugging GPT here. You can read about building a simple 2-step AutoGPT style bot here.


Conclusion As we continue to explore the capabilities of self-prompting AI models like Auto-GPT, it's hard not to feel a sense of wonder at the possibilities they present. From running entire companies to brainstorming innovative ideas, these AI models are poised to revolutionize the way we live and work. What implications does this hold for our future with AI and Auto-GPT? The potential applications are extensive, and the possibilities appear almost limitless. Any research tasks that require recursive analysis of multiple permutations and combinations are excellent candidates for AutoGPT. Here are a few exciting ideas to spark your imagination Medical advancements: Auto-GPT could revolutionize the medical field by aiding researchers in discovering new treatments and cures for diseases, potentially saving countless lives in the process. Sustainable living: AI models could help us develop innovative ways to reduce our carbon footprint and live more sustainably. They could analyze and optimize energy consumption, waste management, and transportation systems, leading to a greener and more eco-friendly world. As we forge ahead into the future, AI models like Auto-GPT will undoubtedly play a significant role in shaping the world around us. The key is to use these incredible advancements responsibly, ensuring that we harness their potential for good while mitigating potential risks. Together, we can work towards a future where AI technology enhances our lives and helps us tackle the world's most pressing challenges, making the impossible possible. So, buckle up and get ready for an exciting ride into the world of AI and Auto-GPT!


Flutter and ChatGPT: Revolutionising App Development while Ensuring IP Compliance

Overview As a developer, I'm always on the lookout for new tools and technologies that can help me be more productive and efficient. Recently, I started exploring the possibilities of AI-assisted development with Flutter, a popular cross-platform framework for building mobile and web applications. In this article, I'll share my experience and learnings from building a sample app with Flutter and ChatGPT. Before we dive into it, let's talk about why I chose to use AI tools for this project. As a practice head in a large organization, I'm always looking for ways to improve our development processes and make them more efficient. AI-assisted development is one such way that can help us save time and effort by automating repetitive tasks and suggesting code snippets based on context. In this project, I wanted to explore how AI tools can help me speed up development with Flutter. Flutter, a cross-platform framework that enables developers to build high-quality mobile and web applications with a single codebase that can run on Android, iOS, Web, Mac, Windows, and Linux. In this article, we will discuss how AI-assisted development with Flutter can benefit developers and organizations.


Cross-Platform Capabilities Flutter's cross-platform capabilities enable developers to create one codebase that powers applications across multiple platforms. This feature significantly reduces development time and costs and ensures that the application works seamlessly across different devices.


Faster Development Cycles AI-assisted development with Flutter provides a faster development cycle. Developers can use pre-built widgets and tools to create visually appealing and responsive applications quickly. This feature saves time and allows developers to focus on other aspects of the application, such as user experience and functionality.


Improved Quality and Performance Flutter's pre-built widgets and tools provide a consistent and robust user experience across different platforms. Additionally, Flutter's fast development cycles enable developers to quickly test and debug their applications, ensuring high-quality and high-performance applications.


Customization Flutter allows developers to customize their applications according to their specific needs. This feature enables developers to create unique and visually appealing applications that stand out from the competition.


Considerations for Choosing AI Tools in an IP-Sensitive Organisation While AI-assisted development with Flutter provides several benefits, it is essential to consider certain factors when choosing an AI tool for an IP-sensitive organization. Some of these considerations include:


Security and Privacy The AI tool should have robust security features to protect sensitive data and code from unauthorized access, modification, or theft.


IP Ownership Ensure that the AI tool's terms of service and licensing agreement align with the organization's IP ownership policies. The tool should not claim ownership or control over any IP generated using it.


Compliance Verify that the AI tool complies with relevant regulations and industry standards, such as GDPR, HIPAA, and ISO 27001.


Transparency Look for AI tools that provide clear explanations of their algorithms and decision-making processes, as well as any limitations or biases in their outputs.


Scalability Consider the scalability of the AI tool, both in terms of its ability to handle large amounts of data and its compatibility with the organization's existing tech stack.


Vendor Support Evaluate the vendor's reputation, experience, and customer support capabilities, as well as their ability to provide timely updates and bug fixes.


Customisability Determine the degree to which the AI tool can be customized and integrated into the organization's existing workflows and development processes.


Cost Consider the cost-benefit analysis of using the AI tool, factoring in the potential benefits of increased efficiency, reduced development time, and improved outcomes, as well as any licensing or subscription fees.


Demo - Flutter with ChatGPT To better illustrate the benefits of AI-assisted development with Flutter, let's create code from ChatGPT that displays a list of articles. The application will have a home screen that shows a pre-populated list of articles and a search feature that allows users to search for articles by keyword. So, when I asked ChatGPT to share code for the above prompt, it quickly generated a complete snippet of the Home Screen with a sample pre-populated list of articles. Snippet 1 - Home Screen with search icon button on the app bar and a list This code has literally everything we need to get started on the page. I have just augmented it a bit to implement the navigation to the ArticleDetailsPage and fix some lint issues like the use of const, etc. Snippet 2 - SearchDelegate for the articles list Snippet 3 - ArticleDetailsPage I am amazed by how great this details page turned out! It flawlessly adheres to the design guidelines for font weights and colors for all screen elements, giving it a professional and polished look that makes it seem almost ready for deployment.


Output This would have taken me a couple of hours to build on my own, but with the help of ChatGPT, I was able to create it in less than 10 minutes! The code generation was so fast, it was even quicker than setting up the Flutter project and testing it on the Simulator! Incredible! Although this was a simpler use case, ChatGPT does wonders with other complicated aspects of the development as well. On another tryout, I was able to create a fully functional Firebase Mobile Authentication flow through it. I will soon share findings from that exploration as well in my next articles!


Conclusion Overall, I found that using AI tools such as ChatGPT was a great way to speed up the development process and reduce the amount of boilerplate code that I had to write. However, it's important to keep in mind that AI tools are not a replacement for human developers and should be used as a complement to their skills. In conclusion, I hope this article has given you a glimpse into the possibilities of AI-assisted development with Flutter. By using AI tools to automate repetitive tasks and suggest code snippets, we can save time and effort and focus on building high-quality, user-friendly applications. AI-assisted development with Flutter will completely be transforming the way we build mobile and web applications.


This AI newsletter is all you need #42

What happened this week in AI by Louie AI agents were in the spotlight in the AI and LLM community this week because of projects like Auto-GPT. The concept is that language models such as GPT-4 can "self-prompt" or "auto-prompt" - that is, they can generate and execute their own prompts based on initial input. Through this approach, multiple instances of these models can be assigned different roles and work collaboratively towards achieving a common goal, such as executing a series of tasks, reviewing and fixing mistakes, and even developing and debugging their own code. While current AI Agent projects are still in the early stages and require further refinement, we see tremendous potential for AI Agents. Furthermore, by granting language models access to other tools, such as web search engines, mathematical engines, and information retrieval systems, we believe that many new and emergent capabilities can be unlocked. After witnessing the rapid pace of progress and continuous newsflow of new models in NLP, we were delighted to see some of these concepts being applied to advance Computer Vision, as demonstrated by Meta's new Segment Anything Model (SAM). The open-source release of this project is particularly noteworthy, and we hope it will help to reduce the costs and barriers to entry for many computer vision tasks. The Segment Anything Model serves as a promptable foundation model for image segmentation, allowing it to learn a general notion of objects and generate masks for any object in images or videos. Moreover, it can be linked to other tasks and other generative models. For example, it can be connected to stable diffusion to build amazing image-editing end-to-end models like Grounded Segment Anything recently shared on GitHub. We believe this is just the first of many to integrate SAM into a larger and "real-world applicable" project. - Louie Peters - Towards AI Co-founder and CEO


This AI newsletter is all you need #42

Hottest News Anthropic's $5B, 4-year plan to take on OpenAI According to documents obtained by TechCrunch, Anthropic aims to raise up to $5 billion in the next two years to challenge competitor OpenAI using its new AI model, Claude-Next. The company also plans to enter more than a dozen major industries. 2. Meta introduced Segment Anything Meta has introduced Segment Anything, which aims to democratize segmentation. The Segment Anything Model is a versatile, promptable model trained on a diverse dataset. It is available under a permissive open-source license (Apache 2.0). 3. Defamed by ChatGPT: My Own Bizarre Experience with Artificiality of "Artificial Intelligence" The AI program ChatGPT was found to have generated a false claim of sexual harassment without a factual basis. This incident highlights the biases and dangers of using AI to tackle disinformation. ChatGPT relied on a cited article that was never written and quoted a statement that was never made by the newspaper. 4. Poe's AI chatbot app now lets you make your bots using prompts Poe, created by the Q&A site Quora, enables users to experiment with cutting-edge AI technologies through a mobile interface. With Poe, users can build their own chatbots using prompts and an existing bot as the foundation. Additionally, users have the option to pay for access to more advanced bots powered by new language models, such as GPT-4 and Claude+. 5. Thoughts on a Crazy Week in AI News Last week's topic was the concerns surrounding recent advancements in systems such as ChatGPT and GPT-4. With significant AI news, including the letter and the Times Magazine post, there is a growing need for greater transparency on the inner workings of these large AI systems, including their training and evaluation processes.


This AI newsletter is all you need #42

Three 5-minute reads/videos to keep you learning StackLLaMA: A hands-on guide to train LLaMA with RLHF This blog post demonstrates the various steps involved in training a LlaMa model to answer questions on Stack Exchange using RLHF. The approach combines Supervised Fine-tuning (SFT), Reward/preference modeling (RM), and Reinforcement Learning from Human Feedback (RLHF). By employing these techniques, HuggingFace has released the StackLLaMA model. 2. A Recipe for Training Large Models This report offers a valuable compilation of practical tips to consider when training a larger machine learning model. It discusses crucial topics such as initialization and experimentation strategies and offers various examples of useful tools that may come in handy. 3. AI might not steal your job, but it could change it AI has the potential to impact around 300 million jobs worldwide. The legal industry is among the fields where AI shows promise in automating language-based jobs, but human guidance remains essential. This article presents an unbiased analysis of how AI is likely to impact various industries. 4. ML papers of the week This is a thoughtfully curated and high-quality list of machine learning (ML) papers that one can regularly refer to. With so much hype around ML in online discussions, it is important to go back to the source and understand the current trends. 5. 5 Lessons Learned from Adding ChatGPT to a Mature Product Mito shared the concrete lessons they learned from launching their AI feature. The biggest lesson was that adding a chatbot that generates code is easy, but actually figuring out how that code interacts with your existing product is much harder.


This AI newsletter is all you need #42

Papers & Repositories Torantulino/Auto-GPT This is an experimental open-source project aimed at making GPT-4 fully autonomous. Auto-GPT is an app showcasing the capabilities of GPT-4 in autonomously managing businesses to increase their net worth. 2. mayooear/gpt4-pdf-chatbot-langchain This is a chatbot powered by GPT-4 and LangChain designed for handling large PDF documents. With this technology, users can create a question-answering chatbot that can process several large PDF files. The tech stack used includes LangChain, Pinecone, Typescript, Openai, and Next.js. 3. HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in HuggingFace HuggingGPT is a framework that utilizes LLMs to connect various AI models in machine learning communities to solve AI tasks. It employs ChatGPT to plan tasks upon receiving a user request, select models based on their available function descriptions in Hugging Face, execute each subtask using the chosen AI model and summarize the response according to the execution results. 4. Summary of ChatGPT/GPT-4 Research This paper offers a comprehensive survey of ChatGPT and GPT-4 and their potential applications across various domains. The study aims to provide insights into ChatGPT's capabilities, potential implications, and ethical concerns while also offering direction for future advancements in this field. 5. ChatGPT Survey: Performance on NLP datasets This is a survey of 141 arXiv papers that compared ChatGPT to other NLP models. It revealed that ChatGPT's performance was lower than anticipated. The reasons for this included suboptimal utilization, biased results, and incomplete multilingual evaluation. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #42

Announcing Our Discord Job Board We have launched a new job board on the Learn AI Discord community that allows any company to share their job openings directly with our community, and you can apply within Discord. Simply scroll up the channel to view all the job offers. This job board is powered by our friends at @Freeflow, and once you share a job, you can also share it with other partnered servers to reach more potential candidates. We hope you find this resource helpful. If you have a job opening to share and are looking for candidates, please take a look at the job board here and consider sharing it with our community.


This AI newsletter is all you need #42

Weekly AI Podcast In this week's What's AI podcast, Louis Bouchard interviewed Jeremy Cohen, the founder, and CEO of Think Autonomous. This interview delves into the self-driving car industry and the different machine learning-related roles within it, including a discussion on how to become a self-driving car engineer. The conversation also covers topics such as LiDAR technology and its functionality, the comparison between using LiDAR versus cameras on self-driving cars, and demystifying industry myths like the $200,000 salary for a computer vision engineer. Listen on YouTube, Spotify, or Apple Podcasts!


This AI newsletter is all you need #42

Meme of the week! Meme shared by DrDub#0108


This AI newsletter is all you need #42

Featured Community post from the Discord rmarquet#2048 has shared a GitHub repository containing a collection of artificial intelligence projects, that features detailed documentation and code. It covers various topics and techniques in the field of AI, including computer vision and reinforcement learning. Check it out here and support a fellow community member. Share your feedback in the thread here!


This AI newsletter is all you need #42

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #42

Article of the week Unpacking the Differences: Machine Learning in Research versus Production Environments by Youssef Hosni This article examines the disparities between machine learning in research environments and production environments. The implementation of machine learning differs significantly depending on the context in which it is used. Furthermore, it delves into the components of production machine learning and the challenges it poses, to provide a better understanding of the complexities involved.


This AI newsletter is all you need #42

Our must-read articles The Complete Guide to Spiking Neural Networks by Ali Moezzi Cosine Similarity for 1 Trillion Pairs of Vectors by Rodrigo Agundez If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #42

Job offers Senior/Principal Software Engineer - Conversational AI @NinjaTech AI (Palo Alto, California) Lead Site Reliability Engineer @ASAPP (Remote) Member of Technical Staff, Data Acquisition @Cohere (Remote) Full stack engineer (Frontend Focus) @Cresta (Remote) Senior Python Backend Engineer @Chattermill (Remote) Manager, Data Science @Angi (Remote) Senior Machine Learning Engineer - Computer Vision @BenchSci (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


From Pre-RNNs to GPT-4: How Large Language Models are Changing NLP

Introduction Large Language Models (LLMs) are currently a popular topic of discussion across various social media platforms such as LinkedIn and Twitter. Their prevalence is evident from the fact that ChatGPT has already made it to the "Trending" section on Twitter. These models are being utilized in various forms of content, floating all over. Not long ago, I received a video on WhatsApp that showcased a husband's remarkable ability to reconcile with his angry wife via text. What made this situation particularly noteworthy was the fact that the husband employed the use of ChatGPT to help him navigate the conversation and persuade his wife to forgive him One more recent incident that went viral was this. Staging the Moon Landing (made with Midjourney) (source). This is just one among the many! A few weeks ago, my mom called me, and she asked me, "Hitesh, what is this ChatGt....ChatT....ChatGPT". I said, yes, Mom, you mean ChatGPT. Initially, she was totally impressed with the things. We went on a video call and shared my screen, where I showed her some prompts and the answers that it gave. Her first question after seeing the prompt was, "How is it doing?". I told her, why don't you try asking ChatGPT itself how is it doing all this? , and then our conversation ended. Fast forwarding a few weeks later, we were again talking, and she asked, "Is there some new version of ChatGPT already?". I asked her curiously, are you really following some blogs, or it's through your amazing WhatsApp messages/people around? She mentioned that people create images by writing the text. The text translates to an image, and that image is shared everywhere. I told her about GPT-4 and, in short, explained to her the multimodal concept. She was initially impressed, but when we spoke, she said, "Hitesh, I feel this is not right. I don't know where the world is heading and how would a person remain creative with such things getting developed?" Immediately next she said, "Are these models safe, what are the jobs that will be impacted, can you share with me some negative impact, etc.?" I could relate to her, however, I then explained her a bit about "Responsible AI" assuring her that it is essential to balance advancements in Generative AI with responsible implementation. She was convinced that we need to move forward with caution and responsibility to ensure that AI is used for the betterment of society and that "Generative AI-Responsible AI should go hand-in-hand". You all might be wondering why I am narrating this here. Let me tell you the reason behind this. It was in the year 2013/2014 when I started working on classical NLP. This included n-grams, meta/dense features, and Topic Modelling on millions of documents. During this time, Regular Expressions (Regex) were significantly popular. People used to do courses on Regex 101 website and many other places. I spent almost a month mastering regex in those days. I remember working on a project where Regex was just one part of it. I had a task to extract the Top 2 pages out of a 1000+ pager report and then perform further analysis and topic modeling on top of it. Let me tell you something, extracting those 2 pages was not an easy task for me. I had to stratify and sample several types of reports and then build an ensemble of regex with several conditions. By 'stratify and sample', I mean I was manually scanning through reports with more than 700+ reports being scanned in 2 months. The extraction accuracy started with as low as 30% of page/content getting extracted to reach 85% as the maximum. Finally, when I had to deploy this application in production, it turned out to be ~175 expressions stacked together to reach this level of extraction. It was one of my finest experiences working on this project. Fast forwarding to today, the work that used to happen in several months of the timeframe in technology is happening in minutes today. This is today: I asked the model to write a regular expression and here we are. And most importantly, it will not just give you the expression but also the explanation for it. People who have worked in a regex will realize how important is the explanation in regex if we find one from the internet. I remember spending my hours mastering and making the project successful, and today here we are. Hence, in today's blog, we will see how technology has evolved with time and where we have reached today. It's very important for all of us to know the limitations of every phase/model and "WHY" it has led to further innovations. I would like to introduce you to Pragyan Prakash, who has contributed to writing the content in the following sections of this blog.


History of Language Models and Their Evolution with Time In this section, we will provide you with an overview of the complete NLP ecosystem, including its evolution and current state. We will discuss the introduction of each phase and the models that were developed during these phases. Our goal is to share our learnings with you. As the number of models is infinite, we will limit our explanation of each model to its "what", "why", and "how", along with future developments. So get ready for a lot of studying and storytelling! Let's explore the evolution of Large Language Models (LLMs) and the various phases that have emerged over time. In this section, we will examine the limitations of each phase and the subsequent need for further development.


Early Phase (Phase before Recurrent Neural Networks) The phase before the introduction of recurrent neural networks (RNNs) in natural language processing (NLP) was primarily characterized by the use of rule-based methods, such as n-gram models and Hidden Markov Models (HMMs). These methods had several limitations, including: 1) Limited context understanding: Rule-based methods were limited in their ability to understand and model complex language structures and dependencies, particularly in cases where context played a significant role. 2) Difficulty handling long sequences: These methods struggled to handle long sequences of text, leading to poor performance on tasks like language generation and machine translation. 3) Handcrafted features: These methods relied on human experts to handcraft features, making them less adaptable and scalable to new applications and domains. 4) Limited use of deep learning: These methods had limited ability to leverage the power of deep learning techniques, which had been shown to be highly effective in other domains.


Phase 1 Recurrent Neural Networks (RNNs) have gained popularity because they were able to overcome some of the limitations of traditional feedforward neural networks, which could only process fixed-length input sequences. Here are some of the ways that RNNs overcame these limitations and gained popularity: 1) Handling Variable-Length Input Sequences: RNNs are designed to process sequential data, which means that they can handle input sequences of variable length. This is because RNNs use a recurrent connection that allows information to flow from one time step to the next. This means that they can handle sequences of any length, which makes them well-suited for tasks like speech recognition, language modeling, and machine translation. 2) Capturing Temporal Dependencies: Because RNNs are designed to process sequential data, they are able to capture temporal dependencies between elements of the input sequence. This allows them to model long-term dependencies and capture the context of each element in the sequence. This is important for many NLP tasks, where the meaning of a word or sentence can depend on the words that came before it. 3) Efficient Parameter Sharing: RNNs use the same set of weights for each time step in the sequence, which means that they are able to share parameters across time. This makes RNNs more efficient to train and allows them to generalize better to unseen data. 4) Flexibility and Adaptability: RNNs can be used for a wide range of NLP tasks, from language modeling and speech recognition to sentiment analysis and machine translation. They are also highly adaptable and can be extended and modified in many ways, such as with the introduction of gating mechanisms like LSTMs and GRUs. Overall, RNNs gained popularity because they were able to overcome the limitations of traditional feedforward neural networks and provide a more flexible and adaptable approach to sequence modeling in NLP.


Phase 1.1 Long Short-Term Memory (LSTM) models overcame some of the limitations of traditional Recurrent Neural Networks (RNNs) and gained popularity because they were able to better capture long-term dependencies in sequential data. Here are some of the ways that LSTMs overcame the limitations of RNNs and gained popularity: 1) Handling Vanishing Gradient Problem: One of the main limitations of RNNs is the vanishing gradient problem, which occurs when the gradients in the network become too small to update the parameters effectively. LSTMs were designed to overcome this problem by using a gating mechanism that allows them to selectively remember or forget information from previous time steps. This gating mechanism helps LSTMs to maintain long-term dependencies over many time steps. 2) Capturing Long-Term Dependencies: LSTMs are able to capture long-term dependencies by using a memory cell that allows information to flow through the network over many time steps. The memory cell is controlled by three gates - the input gate, output gate, and forget gate - which control the flow of information into and out of the cell. This allows LSTMs to capture long-term dependencies in the input sequence without being affected by short-term fluctuations. 3) Flexibility and Adaptability: Like RNNs, LSTMs can be used for a wide range of NLP tasks, such as language modeling, speech recognition, and machine translation. They are also highly adaptable and can be extended and modified in many ways, such as with the introduction of additional gates or with the use of multi-layer LSTMs. 4) Improved Accuracy: LSTMs have been shown to achieve better accuracy than traditional RNNs on a wide range of NLP tasks. This is because they are able to better capture long-term dependencies in the input sequence, which is important for tasks like language modeling and machine translation. Overall, LSTMs overcame the limitations of RNNs and gained popularity because they were able to better capture long-term dependencies in sequential data, which is important for many NLP tasks. Their use of gating mechanisms and memory cells allowed them to overcome the vanishing gradient problem and achieve higher accuracy than traditional RNNs. There was also a phase where Gated Recurrent Units (GRUs) were introduced. To understand in short, they were introduced as a simpler alternative to Long Short-Term Memory (LSTM) models. While LSTMs are known for their ability to capture long-term dependencies in sequential data, they have a more complex architecture with more parameters than GRUs. This complexity can make LSTMs slower to train and more prone to overfitting. GRUs were designed to overcome some of these limitations by having a simpler architecture with fewer parameters than LSTMs, while still being able to capture long-term dependencies in the input sequence. This simplicity makes GRUs faster to train and less prone to overfitting while also making them easier to understand and interpret. GRUs achieve this simplicity by using only two gates - the reset gate and the update gate - compared to the three gates in LSTMs. The reset gate controls how much of the previous hidden state should be forgotten, while the update gate controls how much of the new input should be added to the new hidden state. By adjusting these gates, GRUs are able to selectively update and forget information from the previous time step, which allows them to capture long-term dependencies in the input sequence.


Phase 2 Transformers overcame the limitations of LSTMs and gained popularity for several reasons: 1) Parallel Processing LSTMs process input sequences sequentially, which makes them slow and difficult to parallelize. Transformers, on the other hand, process the entire input sequence in parallel, which makes them much faster and easier to parallelize. This is achieved through a self-attention mechanism that allows the model to attend to all input tokens at once. 2) Attention Mechanism LSTMs rely on a fixed-length hidden state to encode the entire input sequence. This means that they can struggle with long input sequences, where relevant information may be spread out over many time steps. Transformers, on the other hand, use a self-attention mechanism that allows the model to attend to all input tokens at once. This allows them to capture long-range dependencies more effectively, making them better suited for tasks that involve long input sequences. 3) Scalability LSTMs are not very scalable, as their memory requirements grow linearly with the length of the input sequence. Transformers, on the other hand, have a constant memory requirement, which makes them much more scalable. This makes it possible to train larger models with more parameters, which can lead to better performance on complex NLP tasks. 4) Transfer Learning Transformers are well-suited to transfer learning, where a pre-trained model is fine-tuned on a specific task. This is because the self-attention mechanism allows the model to capture general patterns and structures in the input sequence, which can be transferred to new tasks with minimal modification. This makes it possible to achieve state-of-the-art performance on a wide range of NLP tasks with relatively little data. Overall, transformers overcame the limitations of LSTMs and gained popularity because they are faster, more scalable, and better suited to tasks that involve long input sequences. Their use of a self-attention mechanism allows them to capture long-range dependencies more effectively, and their constant memory requirement makes them more scalable. Additionally, transformers are well-suited to transfer learning, which makes it possible to achieve state-of-the-art performance on a wide range of NLP tasks with relatively little data.


Phase 3 Finally comes the Generative-Pretaining(GPT) phase that has been built on the strengths of Transformer models. The following are its key benefits: 1) Large-scale pretraining Generative pretraining models were trained on massive amounts of data, sometimes on the order of trillions of words. This allowed them to learn a vast amount of knowledge about the structure and patterns of language, which they can then use to perform a wide variety of language tasks. 2) Few-shot learning Generative pretraining models can perform well on new tasks with very little additional training data, thanks to their ability to generalize from a large pretraining corpus. This is known as few-shot learning, and it allows the model to quickly adapt to new tasks with only a small amount of fine-tuning. 3) Zero-shot learning In some cases, generative pretraining models can even perform well on tasks that they have not been explicitly trained on, using only a natural language prompt as input. This is known as zero-shot learning, and it is made possible by the model's ability to reason about language and generate coherent responses. 4) Multi-task learning Generative pretraining models can perform well on a wide range of language tasks, from language modeling to question answering to language translation. This is achieved by training the model on multiple tasks simultaneously, allowing it to learn a more generalized representation of language that can be applied to a variety of tasks. Overall, generative pretraining models overcame the limitations of transformer models by leveraging large-scale pretraining, few-shot and zero-shot learning, and multi-task learning. This has made them some of the most powerful models in NLP, with the ability to perform a wide range of language tasks with remarkable accuracy and flexibility. That concludes our discussion of the phases of Large Language Models (LLMs). We hope you now have a good understanding of why and how these developments have occurred. In the upcoming sections, we will also provide an overview of the most prominent LLMs available today, along with concise explanations of each. For your reference, we have compiled an Excel sheet (shown below is the sample screenshot of the Excel sheet) that includes valuable information such as: Model NameModel CategoryYear it got introducedModel ParametersModel creatorsLimitations of each modelTraining TimeModel definition For the complete list of LLMs, please refer here (We made every effort to ensure the accuracy of the information in our compilation by conducting extensive research and verifying it through academic papers. However, if you happen to come across any errors in the data, please do not hesitate to contact us. We will promptly correct any mistakes and ensure that our content remains as reliable and up-to-date as possible) So that's it from our side, folks. In conclusion, we hope that this overview of the NLP ecosystem and the evolution of Large Language Models has been informative and helpful for you. We recognize that the field of NLP is constantly evolving and that there are always new developments and breakthroughs to keep up with. That's why we will continue to share our knowledge and insights with you, as well as keep learning ourselves. We appreciate your interest in this exciting field and thank you for taking the time to read our content. Please do not hesitate to reach out to us with any questions, comments, or feedback. We look forward to continuing the series of blogs. In the future, you will also see blogs on Azure Open AI (ChatGPT, GPT-4, and other models). Just before I go, I wanted to mention that I received a message from ChatGPT. It's possible that the bot was overwhelmed with all the appreciation. Here you see what it says!


Signing off, Hitesh Hinduja & Pragyan Prakash Hitesh Hinduja  LinkedIn Pragyan Prakash  LinkedIn


A Data Scientist's Guide to Prompt Engineering

Four benefits of prompt engineering Improved accuracy: Prompt engineering can improve the accuracy of AI models by providing them with more specific and relevant information. This can lead to more effective outcomes and better decision-making.Increased efficiency: Prompt engineering can increase AI models' efficiency by reducing the time and effort required to train them.Customizability: It can improve the flexibility of AI models by allowing them to be used for a broader range of tasks.Increased user satisfaction: Prompt engineering can increase user satisfaction with AI models by making them more user-friendly and intuitive.


Industry-specific use cases of prompt engineering Healthcare: Prompt engineering is being used to improve the accuracy of medical diagnosis, develop new treatments, and personalize healthcare. For example, prompt engineering can summarize a patient's medical history and symptoms, identify potential drug interactions, and create personalized treatment plans. Finance: In finance, prompt engineering is used to create intelligent assistants that can provide personalized investment advice or help customers with financial planning. These assistants can provide more effective and relevant guidance by customizing prompts based on a customer's financial goals and risk tolerance. Education: Prompt engineering personalizes learning, provides feedback on assignments, and creates engaging learning experiences. For example, prompt engineering can generate a personalized learning plan for each student, provide feedback on essays and code, and create interactive stories and games.


Tips and tricks to generate effective prompts 1. Understand the use case in detail to add "keywords" while generating prompts. Instead of the prompt "Write a blog on the life of a data scientist", you can instead phrase it as, "Act like a data scientist and write a blog on the life of a data scientist covering various aspects of their life such as the career journey, what to expect in the job and challenges." 2. Test and refine the prompts based on the output. If the desired output is not received in the first iteration, try to add more keywords or details to modify the response. 3. Reduce information that doesn't serve any purpose in the prompt. For example, instead of saying write a short, precise, and exciting description of XYZ product, use "Use 35 sentences to write the product description."


Limitations and challenges Bias in prompt generations: Prompt engineering can introduce bias into the training data if the prompts are not carefully designed. This can lead to biased models against certain groups of people or produce inaccurate or misleading results. Difficulty in generating effective prompts: Generating effective prompts can be challenging, requiring domain expertise and a deep understanding of the underlying task. Poorly designed prompts can lead to inaccurate or irrelevant model output. Limited flexibility: Prompt engineering can limit the flexibility of language models, as they are trained to produce output based on a specific prompt. This can make adapting the model to new tasks or scenarios challenging. Time-consuming: Prompt engineering can be time-consuming, requiring significant manual effort to design and test prompts. This can make scaling prompt engineering to larger datasets or more complex tasks challenging. Lack of generalization: Language models trained using prompts may not generalize well to new and unseen data. This can limit the usefulness of the model in real-world applications. Data privacy concerns: Prompt engineering requires access to large amounts of training data, which may contain sensitive information about individuals. This raises concerns about data privacy and the potential misuse of personal information. Ethical considerations: Prompt engineering raises ethical concerns around the responsible use of language models, particularly in healthcare, finance, and education. There is a need for guidelines and regulations to ensure that language models are used responsibly and ethically. By carefully crafting effective prompts, data scientists can ensure that the model is trained on high-quality data that accurately reflects the underlying task. This, in turn, can lead to more accurate and relevant model outputs, resulting in improved performance on various tasks. As such, prompt engineering has emerged as a crucial area of research in natural language processing and machine learning. For more such content or to provide feedback, you can follow me on LinkedIn Thanks to the team at Symbl.ai for all the support and for reviewing this article.


Large Language Models and GPT-4: Architecture and OpenAI API

Introduction Large language models have revolutionized the field of natural language processing in recent years. These models are trained on massive amounts of text data and can generate human-like language, answer questions, summarize text, and perform many other language-related tasks. One of the most highly anticipated models in this field is the upcoming GPT-4, which is rumored to have a staggering trillion parameters. In this article, we will explore the impact of large language models on natural language processing and how they are changing the way we interact with machines.  DONATE/TIP If you like this Article  Watch Full YouTube video with Python Code Implementation with OpenAI API and Learn about Large Language Models and GPT-4 Architecture and Internal Working.


Impact of GPT-4 on NLP The sheer scale of GPT-4, if true, would make it the largest language model ever created, and its potential impact on natural language processing is immense. With such a massive model, we can expect unprecedented levels of accuracy and sophistication in language understanding and generation, as well as the ability to process vast amounts of text data quickly and efficiently.


What are Large Language Models? Large language models are a type of artificial intelligence (AI) model that are specifically designed to understand natural language. They are capable of processing and generating text, and can be used for a wide range of applications, including language translation, summarization, and question-answering. These models are typically composed of a large number of interconnected nodes, which allow them to make predictions about what words are likely to come next in a sentence, based on the context in which they are used. One of the most well-known large language models is GPT-3, which has 175 billion parameters. In GPT-4, Which is even more powerful than GPT-3 has 1 Trillion Parameters. It's awesome and scary at the same time. These parameters essentially represent the "knowledge" that the model has acquired during its training. The more parameters a model has, the more accurate its predictions are likely to be, since it has access to a wider range of contextual information. Large language models are typically trained on massive amounts of text data, which allows them to learn the patterns and relationships between words and phrases.


How do Large Language Models Work? Large language models work by predicting the probability of a sequence of words given a context. To accomplish this, large language models use a technique called self-attention. Self-attention allows the model to understand the context of the input sequence by giving more weight to certain words based on their relevance to the sequence. Each word in the input sequence is first transformed into a vector representation called an embedding. These vectors represent the meaning of the word in the context of the sequence. The model calculates three types of vectors for each word: the query vector, the key vector, and the value vector. These vectors are used to calculate the attention score for each word in the sequence. The attention score reflects how important each word is to the context of the input sequence. The attention scores are then used to weigh the value vectors for each word. The weighted value vectors are then summed to produce a context vector, which represents the context of the input sequence. The context vector is then used to predict the probability of the next word in the sequence. The output of the model is a probability distribution over the vocabulary, which can be sampled to generate new text.


Coding Example using OpenAI API GPT-4 has made it easier to access large language models by providing an API that developers can use to generate natural language text. In this section, we will explore how to use the GPT-4 API for various natural language processing tasks. Example 1: Generating Python code One useful application of the OpenAI API is generating code based on a given prompt. Let's say we want to generate Python code that takes in an array of lists and then Finds the Odd and Even in it. We can use the OpenAI API to generate the code for us. Here's an example: Example 2: Sentiment analysis Another application of the OpenAI API is sentiment analysis. Let's say we want to analyze the sentiment of a given piece of text. We can use the OpenAI API to do this. Here's an example: Example 3: Translation Finally, we can use the OpenAI API for translation. Let's say we want to translate a piece of text from English to French. We can use the OpenAI API to do this. Here's an example: Example 4: Building a ChatBot Application using OpenAI API. It can talk with you like a Human. Given perfect prompt to the OpenAI API will give more perfect responses. Here's an example: For more Explanation and detail, Check the below video that explain Architecture and Working of Large Language Models in Interactive Way:


Final Thoughts Large language models such as GPT-4 have revolutionized the field of natural language processing by allowing computers to understand and generate human-like language. These models use self-attention techniques and vector embeddings to produce context vectors that allow for accurate prediction of the next word in a sequence. As these models continue to improve and become even more powerful, we can expect to see even more exciting applications of natural language processing in the future.


Introduction to Deep Learning Part 2: RNNs and LTSM

Recurrent Neural Networks Recurrent Neural Networks (RNNs) are a powerful type of artificial neural network that excels at handling sequential data. This makes them particularly useful for a variety of applications, such as natural language processing, speech recognition, and time series prediction. In this beginner-friendly explanation, we'll explore the inner workings of RNNs, their architecture, and how they can be built from scratch. Let's start by discussing what makes sequential data unique. Unlike other types of data, sequential data consists of a series of elements arranged in a specific order. Examples include text, speech signals, and time series data. RNNs have been specifically designed to handle this kind of data by processing each element in the sequence one at a time, while keeping track of previous elements in the sequence through a hidden state. The architecture of RNNs is what sets them apart from other types of neural networks. They consist of a series of interconnected nodes, with each node responsible for processing one element in the sequence. These nodes are organized in a chain-like structure, allowing information to flow from one node to the next. The hidden state is the key feature of RNNs, as it captures information from previous nodes in the chain and uses it to influence the processing of future elements in the sequence. See the structure below. As the information flows through the network, the nodes process the data and update the hidden state. This hidden state then influences how the next node processes its input.


Training So how do we train RNNs? The solution is the backpropagation through time (BPTT) algorithm. BPTT is a modification of the standard backpropagation algorithm, see previous post, designed to handle the unique structure of RNNs. The main difference is that BPTT takes into account the temporal dependencies between the nodes in the network, allowing the error to be propagated back through the entire sequence.


Fixing vanish gradients: LSTM Long Short-Term Memory (LSTM) networks are a type of Recurrent Neural Network (RNN) specifically designed to tackle the problem of vanishing gradients. The vanishing gradient problem occurs when gradients become smaller and smaller as they propagate backward through the network during backpropagation, resulting in slower learning or even preventing the network from learning altogether. LSTM networks tackle this issue by introducing a memory cell, which allows the network to selectively remember or forget information over time. The memory cell is controlled by three gates: the input gate, the forget gate, and the output gate. These gates are responsible for controlling the flow of information into, within, and out of the memory cell. They work together to decide what information to store, update, or discard at each time step. See the memory cell below. Specifically, the input gate controls whether or not new information should be added to the memory cell. The forget gate controls whether or not old information should be removed from the memory cell. And the output gate controls how much of the memory cell should be output to the next layer of the network. In the diagram, ht-1 is the previous state, ht is the new state and xt is the input at the current step.


Applications in real life Now we understand the structure of RNNs, let's check out what they're used for. They have numerous applications in real life, particularly in domains involving sequential data such as time series, natural language, speech, and audio. Some examples of real-life applications of RNNs are: Language Modeling: RNNs are used for language modeling tasks such as speech recognition, machine translation, and text generation. Time Series Analysis: RNNs can be used for time series forecasting, where the goal is to predict future values based on past observations.Image Captioning: RNNs can be used for image captioning, where the goal is to generate a natural language description of an image. And that's it! In the next article, we will look at Attention Transformers and the new craze around natural language processing. Is it worth the hype?


LLaMA-GPT4All: Simplified Local ChatGPT

Introduce GPT4All GPT4All is a large language model (LLM) chatbot developed by Nomic AI, the world's first information cartography company. It was fine-tuned from LLaMA 7B model, the leaked large language model from Meta (aka Facebook). GPT4All is trained on a massive dataset of text and code, and it can generate text, translate languages, write different kinds of creative content, and answer your questions in an informative way. GPT4All is available to the public on GitHub. LLaMA is available for commercial use under the GPL-3.0 license - while the LLaMA code is available for commercial use, the WEIGHTS are not. This effectively puts it in the same license class as GPT4All. Nomic is working on a GPT-J-based version of GPT4All with an open commercial license. GPT4All is not going to have a subscription fee ever. GPT4All is Free4All. Although GPT4All is still in its early stages, it has already left a notable mark on the AI landscape. Its popularity and capabilities are expected to expand further in the future.


LLaMA-GPT4All: Simplified Local ChatGPT

How to Run GPT4All Locally GPT4All Readme provides some details about its usage. Here will briefly demonstrate to run GPT4All locally on M1 CPU Mac. Download gpt4all-lora-quantized.bin from the-eye.Clone this repository, navigate to chat, and place the downloaded file there. Simply run the following command for M1 Mac: Now, it's ready to run locally. Please see a few snapshots below: Similar to ChatGPT, GPT4All has the ability to comprehend Chinese, a feature that Bard lacks. If you want to interact with GPT4All programmatically, you can install the nomic client as follows. Install the nomic client using pip install nomic.Use the following Python script to interact with GPT4All:


LLaMA-GPT4All: Simplified Local ChatGPT

Chat4All Demystified GPT4All aims to provide a cost-effective and fine-tuned model for high-quality LLM results. The GPT4All model was fine-tuned using an instance of LLaMA 7B with LoRA on 437,605 post-processed examples for 4 epochs. Detailed model hyperparameters and training codes can be found in the GitHub repository. GPT4All developers collected about 1 million prompt responses using the GPT-3.5-Turbo OpenAI API from various publicly available datasets. After an extensive data preparation process, they narrowed the dataset down to a final subset of 437,605 high-quality prompt-response pairs. Developing GPT4All took approximately four days and incurred $800 in GPU expenses and $500 in OpenAI API fees. The final gpt4all-lora model can be trained on a Lambda Labs DGX A100 8x 80GB in about 8 hours, with a total cost of $100. A preliminary evaluation of GPT4All compared its perplexity with the best publicly known alpaca-lora model. Results showed that the fine-tuned GPT4All models exhibited lower perplexity in the self-instruct evaluation compared to Alpaca. However, this assessment was not exhaustive due to encouraging users to run the model on local CPUs to gain qualitative insights into its capabilities.


LLaMA-GPT4All: Simplified Local ChatGPT

TL;DR Considering the expensive LLMs in training and serving, Meta LLaMA is a foundation for accelerating LLM open-source community. Stanford's Alpaca, based on LLaMA, offers an optimized smaller model with enhanced performance. Now, GPT4All, also built on LLaMA, enables local execution. Generative AI is evolving rapidly every day. Thanks to Brandon Duderstadt for reviewing this article.


This AI newsletter is all you need #41

What happened this week in AI by Louie This week, there was a focus on AI safety, privacy, and regulation. Although the emergence of the next generation of AI models has many advantages, democratization, accessibility, and affordability of generative AI tools, as well as the increased capabilities of LLMs, have created a significant potential for misuse and misinformation, either produced by users or the systems themselves. The current discrepancy between the pace of growth and the development of safety measures and regulations makes the discussion of AI safety even more pressing. Last week, one of the radical steps toward AI safety was taken with an open letter from The Future of Life Institute (FLI). The letter, signed by over 50,000 people, urges all AI labs to pause the training of AI systems that are more powerful than GPT-4 for at least six months. The letter highlights the risks of AI, criticizing the "out-of-control race to develop and deploy ever more powerful digital minds that no one - not even their creators - can understand, predict, or reliably control." It also emphasizes the lack of appropriate planning and management for this potentially highly disruptive technology. The FLI letter has sparked a wide range of opinions on whether LLMs and AI should be regulated, and what the actual risks are. Many responses have been generated, both for and against, with individuals highlighting their own perceived risks. Eliezer Yudkowsky, a researcher at the Machine Intelligence Research Institute, wrote an article expressing his support for the letter's intentions but explaining why he did not sign it. He believes that the letter understates the severity of the situation and does not request enough action to address it. In contrast, Yann LeCun has been dismissive of the need for regulation, responding with an "Okay doomer..." attitude. Another recent development towards AI safety is Italy's ban on the use of ChatGPT due to privacy concerns, citing non-compliance with EU data protection regulations (GDPR) by OpenAI. Consequently, OpenAI has been compelled to implement a complete geo-block on the usage of ChatGPT within Italy. At Towards AI, we see several potential risks associated with AI, including 1) the amplification of misinformation through automated propaganda or deepfakes and other tools to empower bad actors, 2) over-reliance and overconfidence in systems that still make mistakes, 3) the misalignment of existing laws and regulations such as copyright and GDPR, 4) social and economic disruption resulting from rapid AI adoption and its impact on jobs and existing industries, and 5) existential risks from superintelligence or misaligned AGI. While the open letter has valid points regarding AI risks and the need for regulation, we don't believe that a pause in AI development would be effective or desirable, as it's difficult to ensure that other countries like China wouldn't continue to develop these models. AI progress and adoption are inevitable, and countries that limit its use are likely to fall behind. However, we do believe that more thought, care, and investment should be put into optimizing the odds of positive AI outcomes while minimizing risks. We also think that AI should be regulated, and governments should establish new departments, policies, and internal expertise to pre-empt and manage some of these risks. Although it's difficult to determine the likelihood of misaligned AGI risks and their timescale, given what's at stake, it makes sense to invest heavily in researching and managing these risks, even if the odds are small. - Louie Peters - Towards AI Co-founder and CEO


This AI newsletter is all you need #41

Hottest News Twitter's Recommendation Algorithm Twitter has made its tweet recommendation algorithm code available on GitHub, which provides insight into the factors that determine whether a tweet appears on a user's timeline. The blog post that accompanies the code release serves as an introduction to how the algorithm selects tweets for a user's timeline. 2. The Only Way to Deal With the Threat From AI? Shut It Down Time magazine published an opinion piece by Eliezer Yudkowsky in response to the letter by FLI. He stated, "I refrained from signing because I think the letter is understating the seriousness of the situation and asking for too little to solve it." According to him, AI poses an existential threat, and we are not adequately prepared to deal with it. Therefore, he argued that it is necessary to "shut it all down." 3. ChatGPT gets "eyes and ears" with plugins that can interface AI with the world OpenAI's plugins expand ChatGPT's capabilities to interact with the Internet, enabling functions like flight booking, grocery ordering, web browsing, and more. These plugins are small pieces of code that instruct ChatGPT on how to utilize external online resources. However, some AI researchers worry that giving AI models access to external systems can lead to harm, without the need for consciousness or sentience. 4. What We Still Don't Know About How A.I. Is Trained One of the prominent aspects of GPT-4 is its ability to respond to queries with confidence. However, this is both a feature and a bug. The developers of GPT-4 acknowledge in a technical report that it can sometimes make basic reasoning mistakes that are inconsistent with its proficiency across numerous domains. 5. The Italian government has banned ChatGPT The Italian data protection agency has ordered OpenAI to block ChatGPT in Italy, citing unlawful data gathering. Their main concern is privacy violations, arguing that OpenAI is non-compliant with EU data protection regulations (GDPR). OpenAI has complied with the order by disabling ChatGPT for users in Italy.


This AI newsletter is all you need #41

Three 5-minute reads/videos to keep you learning Superhuman: What can AI do in 30 minutes? In an experiment, AI was used to generate a comprehensive marketing campaign in just 30 minutes for a new educational game launch. The AI conducted market research, developed a website and social media campaign, and more. This post explores the potential and disruptive power of AI in marketing. 2. Malleable software in the age of LLMs This article explores the significant changes that LLMs may enable in the creation and distribution of software, as well as in how people interact with software. It answers various questions surrounding topics, such as interaction models, software customization, intent specification, and more. 3. Everything you need to know about prompting This is an interview with Sander Schulhoff, the creator of learnprompting.org, the largest online resource for prompting. It explores the exciting skill of prompting, which can lead to various opportunities and enhance productivity. It discusses the significance of learning this skill and provides tips for improving it. 4. From Deep to Long Learning? This article provides a summary of recent work from Stanford aimed at significantly increasing the context window for language models. By enabling longer prompts and outputs, this advancement may lead to new possibilities in tasks such as summarizing entire books, editing entire repositories of code, and generating multimodal videos. 5. Five years of GPT progress This article offers an overview of the history of Generative Pre-trained Transformer (GPT) research, emphasizing the latest state-of-the-art models and their distinctions. It showcases how the current GPT research is leading to significant advancements in the field.


This AI newsletter is all you need #41

Papers & Repositories Vicuna: An Open-Source Chatbot with Quality Similar to ChatGPT Vicuna-13B is an open-source chatbot trained by fine-tuning LLaMA on user-shared conversations collected from ShareGPT. Preliminary evaluation using GPT-4 as a judge shows that Vicuna-13B achieves over 90% quality of OpenAI ChatGPT and Google Bard while outperforming other models like LLaMA and Stanford Alpaca in over 90% of cases. 2. Meet in the Middle: A New Pre-training Paradigm This paper introduces a new pre-training paradigm that improves both the training data efficiency and capabilities of LMs in the infilling task. The effectiveness of this paradigm is demonstrated through extensive experiments on both programming and natural language models, where it outperforms strong baselines. 3. LLaMA-Adapter: Efﬁcient Fine-tuning of Language Models LLaMA-Adapter is an effective method for fine-tuning models into instruction-following ones by using learnable prompts. With multi-modal inputs, it produces high-quality responses and achieves excellent reasoning capabilities. With the help of 52K self-instruct demonstrations, LLaMA-Adapter introduces only 1.2M learnable parameters to the frozen LlaMa 7B model and takes less than one hour to fine-tune on 8 A100 GPUs. 4. ChatGPT Outperforms Crowd-Workers for Text Annotation Tasks This paper investigates the potential of large language models (LLMs) for text annotation tasks, specifically focusing on ChatGPT. The paper shows that ChatGPT zero-shot classifications, without any additional training, outperform MTurk annotations and achieve this at a significantly lower cost. 5. A Comprehensive Survey of AI-Generated Content This paper provides a survey of Artificial Intelligence Generated Content (AIGC), highlighting recent advancements in complex modeling and large datasets, and exploring new ways to integrate technologies such as reinforcement learning. It also offers a comprehensive review of the history of generative models, covering both unimodal and multimodal interaction. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #41

Weekly AI Podcast Louis Bouchard has launched a weekly podcast aimed at demystifying the various roles in the AI industry and discussing interesting AI topics with expert guests. The podcast, available on YouTube, Spotify, and Apple Podcasts, features interviews with industry experts. In the latest episode, Louis interviews Sander Schulhoff, the creator of Learn Prompting, the most comprehensive guide on prompt engineering. As shared in our learning section above, the interview is all about prompting, demystifying it, and condensing it into a one-hour discussion. This is the goal of this weekly podcast: to demystify something about AI every week with the help of an expert. Each episode features a specific topic, sub-field, or different roles related to AI, with the aim of teaching and sharing knowledge from experts who have worked hard to gather it. A small teaser for the next episode: it will be about self-driving cars!


This AI newsletter is all you need #41

Meme of the week! Meme shared by neuralink#7014


This AI newsletter is all you need #41

Featured Community post from the Discord Oliver Z#1100 has created a Chrome extension called TwOp that can generate AI-powered social media posts by entering topics, keywords, themes, and desired tones. The extension is open-source and available for download on the Chrome Web Store and GitHub. Check it out and support a fellow community member. Share your feedback or questions in the thread here.


This AI newsletter is all you need #41

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #41

Article of the week Google's New AI Model PaLM-E Explained by Louis Bouchard In this article, the author provides an in-depth tour of PaLM-E, Google's latest publication, which is described as an embodied multimodal language model. This means that it can comprehend various types of data, including text and images, from the ViT and PaLM models mentioned earlier.


This AI newsletter is all you need #41

Our must-read articles Foundation Models: Scaling Large Language Models by Luhui Hu How to Build a Low-Code Sales Dashboard with Python and Deepnote by Asish Biswas StyleGAN2: Improve the Quality of StyleGAN by Albert Nguyen If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work in our network if it meets our editorial policies and standards.


This AI newsletter is all you need #41

Job offers Senior Software Security Engineer, Cloud Security @Anthropic (San Francisco, USA/Hybrid) Director, Product - Data Platform @Biobot Analytics (Remote) Data Scientist (H/F/X) @Shadow (Paris, France/Hybrid) Large Language Model (LLM) Developer @Asimov (Boston, MA, USA) Machine Learning Engineer/Scientist @Unlearn.AI (San Francisco, USA/Hybrid) AI Research Assistant @Modern Intelligence (Austin, TX, USA) Data Engineer - Jr/Mid Level @pulseData (Remote) AI/ML Engineer @career (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Unveiling and Addressing Bias in Natural Language Processing

Identifying Bias in Word Embeddings The first step in addressing bias in NLP systems is to identify and measure the presence of biases in word embeddings. In the field of psychology, the Implicit Association Test (IAT) has been widely used to measure implicit biases in humans by examining the latency in reaction times to presented pairs of words. For example, the IAT might measure the response times when associating "man" with "math" and "woman" with "art." The differences in response times can reveal underlying biases in how we perceive and associate these concepts. In a similar vein, we can apply the concept of the IAT to word embeddings. Word embeddings, such as Word2Vec and GloVe, are trained on large text corpora and encode semantic relationships between words as vectors. These vectors can then be used to perform analogy tests that reveal potential biases in the embeddings. For example, if the analogy "man is to computer programmer as woman is to homemaker" holds for a given word embedding, the model encodes a gender stereotype. Researchers have developed the Word Embedding Association Test (WEAT) to detect biases in word embeddings systematically. WEAT compares two sets of target words (e.g., male and female names) and two sets of attribute words (e.g., career and family-related words) and tests the null hypothesis that there is no difference in the semantic similarity between the target sets and attribute sets. By permuting all combinations of target sets and attribute sets, WEAT calculates a test statistic that quantifies the degree of association between target and attribute word sets. If there is no bias present, the difference in association scores should not be significant. On the other hand, if a bias is detected, we can identify which pairs of input and target words contribute to the observed bias. WEAT has successfully identified biases in various dimensions, such as gender, race, and age, in popular word embeddings like Word2Vec and GloVe. The results of these tests have shed light on how biases present in the training data can inadvertently become embedded in the word vectors, leading to the potential reinforcement of stereotypes and discriminatory behavior in AI systems built on top of these word embeddings.


Gender Bias in Word Embeddings A particularly pervasive form of bias in word embeddings is gender bias, which can have significant implications for AI fairness and the perpetuation of stereotypes. Using the Word Embedding Association Test (WEAT), researchers have uncovered gender biases in popular word embeddings, such as Word2Vec and GloVe, revealing that women are more associated with family-related words, while men are more associated with career-related words. These findings highlight how cultural history and societal norms can become embedded in the usage and associations of words in a language. As word embeddings are trained on large text corpora, essentially reflections of human language and cultural history, it is unsurprising that they may inadvertently capture and perpetuate existing biases. One notable implication of these findings is that AI systems built on biased word embeddings can inadvertently reinforce stereotypes and contribute to biased behavior. For example, a machine translation system trained on biased embeddings might translate a sentence about a doctor to use a male pronoun in one language and a female pronoun in another, thus perpetuating gender stereotypes associated with certain professions.


Detecting Bias in Sentence Embeddings While word embeddings have shown evidence of bias, examining whether these biases are also present in sentence embeddings is crucial. Sentence embeddings are capable of capturing the context and semantics of an entire sentence. Researchers have developed the Sentence Encoder Association Test (SEAT) to detect potential bias in these embeddings. SEAT is an extension of the WEAT and is applied to simple sentence templates where the target word is inserted. For example, given the template "This is <word>", it could compare embeddings of sentences like "This is Adam" and "This is Jasmine" to measure potential gender bias. Researchers can identify any inherent biases within the model by analyzing these sentence embeddings. In addition to examining gender bias, SEAT has been employed to uncover other types of biases. One such bias is the "angry black woman" stereotype, where black women are portrayed as loud, angry, and imposing. Another example is the "double bind" women face in professional settings. This bias states that if women succeed in male-dominated professions, they are often perceived as less likable and more hostile than their male counterparts who perform at a similar level. It is essential to note that the results of SEAT may not generalize beyond the specific words and sentences used in the dataset. Additionally, cosine similarity, a common metric used to measure representational similarity in word embeddings, may not be suitable for newer models like BERT. Despite these limitations, SEAT provides valuable insights into potential biases present in sentence embeddings and helps us understand how these biases may impact the performance and fairness of NLP systems.


Debiasing Techniques and Their Limitations In our pursuit of creating fairer NLP systems, various debiasing techniques have emerged to address the biases present in word embeddings. In this section, we will discuss two primary methods for debiasing word embeddings and explore their limitations. Method 1: Post-processing Debiasing One approach to debiasing is to process the word embeddings after they have been trained. By defining gender bias as the projection of a word on the "gender" direction, we can zero out the gender component for each word. This method aims to eliminate gender bias by effectively removing gender-related information from the embeddings. However, this technique has its limitations. As we will see later in this section, biases often remain even after post-processing. Method 2: Modifying the Loss Function During Training Another approach involves training debiased word embeddings from scratch by modifying the loss function of the training algorithm, such as GloVe. In this method, we attempt to concentrate gender information in the last coordinate of the word embeddings by splitting the embeddings into gendered and non-gendered components. Neutral gender words are made orthogonal to the gendered components. When using these word embeddings, we can ignore the last coordinate, effectively removing gender information. However, this method has also been shown to be insufficient in completely eradicating bias. Limitations of Debiasing Techniques: Although these debiasing methods show some promise, they have several limitations that hinder their effectiveness. Bias by Neighbors: A word that is biased toward a gender may have neighboring words that are also biased toward that gender. Simply debiasing the relationship between a word and gender doesn't eliminate the bias from its neighboring words. For example, if "nurse" is associated with females, its neighboring words like "receptionist" or "caregiver" might also be associated with females. As a result, even after debiasing the direction from the word to the gender, the bias may still persist due to the relationships between neighboring words and gender.Incomplete Bias Removal: Studies have shown that with both debiasing methods, some bias still remains. Most word pairs maintain their previous similarity, which indicates that the biases have not been entirely removed. To further investigate this issue, researchers have clustered the most biased words in the vocabulary before and after debiasing. The analysis of these clusters revealed that there was still a strong association with gender even after the debiasing process.Classifier Performance: Another way to evaluate the efficacy of debiasing techniques is to train a classifier to predict the gender of a word based on its embeddings. Using the 5,000 most biased gendered words, researchers trained a support vector machine (SVM) classifier on 1,000 random samples and predicted the gender for the remaining 4,000 words. The results showed that the prediction accuracy dropped only marginally with both debiasing methods, suggesting that the debiased embeddings still carry significant gender information. These limitations highlight the challenges faced in addressing biases in NLP systems. Despite the progress made in developing debiasing techniques, they have not yet proven to be completely effective in eliminating biases from word embeddings. It is essential for researchers and AI practitioners to continue exploring new methods and refining existing techniques to address bias in NLP better and build fairer AI systems.


Towards Fairer NLP Systems As we have explored the presence of bias in word and sentence embeddings, it becomes clear that addressing these biases is crucial in developing fair and unbiased AI systems. To this end, we must consider various strategies to mitigate bias and promote ethical AI development. One of the primary approaches to reducing bias is incorporating diverse training data. By including a wide range of text sources that represent different genders, races, cultures, and perspectives, we can create a more balanced foundation for AI models to learn from. This can lead to more representative word and sentence embeddings that better reflect the true diversity of human language. In addition to diversifying training data, ethical AI frameworks can guide developers in designing and evaluating their systems. By adhering to guidelines and best practices that prioritize fairness, accountability, and transparency, AI practitioners can create more robust and unbiased models. Collaborating with ethicists and social scientists can provide valuable insights into the ethical implications of AI systems and help identify potential pitfalls in their development. Interdisciplinary collaboration can also play a significant role in mitigating biases. By combining the expertise of linguists, computer scientists, psychologists, and sociologists, we can develop a more comprehensive understanding of the cultural and societal factors that contribute to bias in NLP systems. This collaborative approach can lead to novel debiasing techniques and more effective measurements of bias in AI models. In conclusion, addressing bias in natural language processing systems is paramount for ensuring fairness and ethical AI applications. As we have seen, biases in word embeddings and sentence encodings can reinforce existing stereotypes and perpetuate harmful beliefs. While researchers have made progress in detecting and mitigating these biases, existing debiasing techniques are not yet perfect and may still leave some residual bias in the system. As AI practitioners, it is our responsibility to be aware of our work's biases and seek ways to minimize their impact actively. By incorporating diverse training data, adopting ethical AI frameworks, and fostering interdisciplinary collaboration, we can move closer to creating fair and unbiased NLP systems. Ultimately, our efforts to tackle bias in AI will result in more accurate and useful technologies and contribute to a more equitable and just society.


References [1] Caliskan, Aylin, Joanna J. Bryson, and Arvind Narayanan. Semantics derived automatically from language corpora contain human-like biases. (2017) Science 356.6334: 183186. [2] Bolukbasi, Tolga, et al. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. (2016) Advances in neural information processing systems 29. [3] May, Chandler, et al. On measuring social biases in sentence encoders. (2019) arXiv preprint arXiv:1903.10561. [4] Gonen, Hila, and Yoav Goldberg. (2019) Lipstick on a pig: Debiasing methods cover up systematic gender biases in word embeddings but do not remove them. arXiv preprint arXiv:1903.03862.


Automating Zero-Shot Classification Generating Model Labels with GPT-3

End Note In this article, we discussed how to implement a Zero-shot text classification algorithm automating the label generation process using GPT-3 (or GPT4, even better!). For this implementation, I am assuming that the corpus we are labeling belongs to a given domain (therefore, the labels produced by GPT-3 will be relevant) I appreciate any feedback and constructive criticism! My email is carbor100@gmail.com


Build and Train Vision Transformer from Scratch

Overview Let's have a quick theory overview before we proceed to the practical part of the tutorial.


Transformer & self-attention The history of transformers began with the Attention Is All You Need work. Initially, they were used for machine translation but later expanded to solve various tasks. Jay Alammar explains transformers in his pretty detailed article, Illustrated Transformer. In the diagram below, you may see the architecture of the transformer network for the machine translation task. Transformer has Encoder and Decoder blocks. We only need encoders for the vision transformer model.Encoder (so also a Decoder) is based on a mechanism called self-attention. Multi-Head Attention block calculates the element importance (or attention) score for each element in a sequence. For example, let's take a sentence The animal didn't cross the street because it was too tired. [5]. One attention score vector for the element `it` may look like Encoder and Decoder blocks are identical except for a tiny difference. Encoder can attend to all elements in a sequence to calculate attention scores. You may see encoder attention in figure 3. BERT model is an example of encoders-only architecture. Decoders, on the contrary, can attend only to the previous elements in a sequence during the calculation of attention scores. For instance, GPT model is a decoder-only model.


Vision Transformer As already mentioned above, we can use transformers for image classification tasks. The main difference between Vision Transformer and an NLP transformer is that we should apply a special embedding operation to the images. The image embedding begins with image preprocessing. The image should be split into 2D patches as shown in figure 4. The resulting number of patches N can be calculated as where H is height, W is width and P is patch size. After we've got image patches, we flatten each patch to a 1D vector. The size of the flattened patch vector can be calculated as where C is the amount of color channels (C = 3). After patch transformation is done, we have an image represented as a matrix of size N x M. This matrix is the input tensor that is fed to the model. The input tensor then goes through a linear projection which is then concatenated with [class] token parameter and summed with learnable position embedding. The authors of the original paper discuss position embedding in Appendix D.4 of the original paper. After processing the input tensor with embeddings, there is a standard set of encoders with a classification head at the end. Let's move to the code .


Installation As was said above, vision_transformer repository contains the full working project, but the installation of dependencies may be tricky.


CPU or MPS If you plan to train the network on CPU or you're using Apple computers, you may proceed with the standard installation flow, i.e., install packages from requirements.txt file:


CUDA If you're going to train the network on GPU, you should install PyTorch with the command from the official website. Then you have to install the remaining packages from requirements-cuda.txt. The installation instructions may look like this:


Dataset We use CIFAR10 dataset, which consists of 60,000 images of 10 classes (50k for training and 10k for validation/testing). The size of a single image is 32 x 32 with 3 RGB color channels. Full code that manages datasets can be found at src/dataset.py. The first thing we should do is import the required objects and define constants: PyTorch already has CIFAR10 dataset implemented in its child package torchvision. So, we use the one directly from torchvision.datasets.


Lightning Data Module We create CIFAR10 dataset in pytorch_lightning.LightningDataModule. Data Module simplifies the usage of datasets and, especially, data loaders during the training phase. Let's see the full code of the data module. Then we skim through it step by step. In the code above, we create a classCIFAR10DataModule that inherits LightningDataModule. The main intent of the data module is to create data loaders, not datasets. This is why we pass batch sizes in the constructor. LightningDataModule has several methods to override: prepare_data method is called within a single CPU process, meaning that your data will not be corrupt. It is called before the training, so we use it to download CIFAR10 data into the local directory;setup method is called after prepare_data. Here we instantiate train and validation datasets;train_dataloader returns data loader for training;val_dataloader returns data loader for validation;classes is just a property that returns a number of classes.


Transforms Transform is an operation to apply to an image before we pass it to the model. We define two sets of transforms: for training and validation. Training transform looks like this: CIFAR10 has only 50,000 training images which is a relatively small dataset for neural networks, so we use data augmentation to expand it. The first four transforms do it: RandomHorizontalFlip reflects image horizontally with 50% probability (by default);RandomResizedCrop crops a random portion of an image and resize it to the given size (in our case 32 x 32);RandomRotation rotates an image by a given degree from a range (in our case a range from -30 to 30 degrees);AutoAugment is a pre-trained set of auto-augmentation policies. It is described in AutoAugment: Learning Augmentation Strategies from Data. We use pre-trained auto-augmentation for CIFAR10. Then we convert our image to a tensor with size 3 x 32 x 32, normalize it with CIFAR10 mean and std values, and split the image tensor on patches.