Deploying Machine Learning Projects as Serverless APIs

Introduction When done right, Machine Learning and Data Science have the potential to help improve business processes, today millions of people have taken the rise of AI as a call to action to create more value for themselves and the world. However, even the most well-built model cannot provide value when left statically in a Jupiter Notebook. One way to ensure your models are valuable to the business is to make them constantly available, for this, we can expose our models as APIs, these APIs can feed into a larger system i.e a recommendation model feeding into an E-commerce application, or your APIs could feed into a dedicated system or application i.e a segmentation model can feed a web-based dashboard for marketing decision making, and a document extraction model can feed a dedicated internal application for document extraction. In this article, we would build a simple Azure function that re-trains and applies a K-means model on a retail dataset whenever it's called. This article would follow this structure: Standard APIs Vs Serverless APIs: Here we explain what Serverless is.Introducing the Model: We introduce the model we would deploy.Functions In Azure: Demonstrating the process of configuring functions in VS Code.Testing: Then we discuss testing the function locally.Creating Azure Resources for Functions and Deploying: We then demonstrate how to create the resources needed to run the function on Azure.


Standard APIs Vs Serverless APIs Serverless software refers to any software that abstracts out the hassle of configuring and managing your own infrastructure, you just bring your code and run it, plug and play. The best part; you only pay for what you need. This is in contrast to Traditional APIs, where you have to configure and manage a server for your code, and this server is always running costs even when the API is idle. Serverless APIs are a no-brainer in many ways and offer several cost and flexibility advantages over Traditional APIs. So why then do we still use regular APIs? Well, here are some drawbacks of Serverless Computing you should consider when developing a Machine Learning System of your own: Serverless Systems usually have a cold start problem. This means it may take longer than regular API to start and execute a task.They are usually built for short-running tasks that take about 510 minutes to run. With some providers allowing up to an hour.


Introducing the Model Enough theory, in this section we will begin building up to the final cloud function, but first, we have to briefly introduce the model we would be building. Here we would be using a Segmentation Model, we would assume that a data scientist has already figured out what the optimal parameters would be and all we have to do is to set up the retraining pipeline. To learn more about developing a clustering model from scratch, check out the references. The code for retraining has been modularized into four scripts namely: get_data.py: Retrieves the data through an SQL query. Here we use pandas' read_sql function to run a query on a given database engine that will be configured in app.py. preprocess.py: Preprocesses the data, performs feature engineering, and removes unwanted data. train.py: Retrains the model with new data. app.py: Ties all the previous modules together, performs clustering on the data and returns a dataframe with the clusters as a new column. It also contains optimal init parameters. Assess.py (Bonus): Contains a handy set of custom-defined functions that help assess the performance of the clustering model.


Functions In Azure Now that we have introduced the model and its code. We can begin to talk about converting said code into an Azure function, but first, we have to configure the function. There are several ways to do this, for me the most convenient way would be by using a VS Code extension for function apps. Alternatively, you can also use Azure Command-Line and the Azure Portal. To follow along, you will need: An Azure Subscription, you can get a free one hereVisual Studio Code installed on your systemA Python Runtime We can set up our function quickly with the following steps: Download Azure Function Extensions: Go to the extension tab and search for functions, you will see Azure functions come out just click install. (Fig 1.1) 2. Go to the Azure Icon (Fig 1.2), under the resources section chose the option to sign in to your Azure account. 3. While still in the Azure tab, go to the workspaces section and choose "Create New Function". (Lighting Icon in Fig 1.2) 4. After you click the icon to create a new function, a box will pop up on top of your screen, asking you what folder to create the function project. Chose an empty folder that will store all of your function's code. 5. After choosing a folder, you will be prompted to pick a language for your function's runtime, here we pick Python, cause our function's code is in Python. 6. Next we have to select a Python interpreter, Azure function uses the runtime of the Python interpreter to build a virtual environment for the function. As such, choosing an interpreter from a custom environment you built is best (i.e a Virtual Environment). You can do this by adding the full path to the Python environment. 7. Next we select the trigger type for our function. Triggers are the switch that makes our function run. They can be HTTP-based, Timer based or very much. 8. Next we create a name for our function and choose an authorization level, for learning purposes you can just choose 'Anonymous'. After this, we are done with the configuration, the function will take a few minutes to complete.


Testing the Function. Now, our function is ready to be tested. To run the function locally we first activate the virtual environment for our project, navigate into our project folder and then run the following prompt in the terminal: The above prompt runs the function, and if it runs successfully your terminal should look something like this: Now the function is running, but still waiting for responses. To actually test the function's logic we need to call a GET/POST request. We can do this with the help of the Python request function, as seen in the following code segment. I call consume.py:


Deployment Now that the function has been fully tested, it's time to deploy it. We can also do this straight from VS Code, however, we need to ensure we have these two components ready: A requirements file: To make sure that all the packages used to run the function locally are available and in their correct versions when deployed, we use a requirement file. To create a requirement file run the code below in the terminal while your virtual machine is activated: 2. Function App resource in Azure: To deploy the project, we first need to create a function app resource in Azure, you can think of this as a container that holds the environment needed to run the function's code. We can do this straight from the VS Code IDE by navigating to the 'Resources' section of the Azure tab and clicking on the plus icon then following these steps: A. Select Subscription: You'll be prompted to choose an Azure Subscription, if you have only one active subscription under your Azure app this step will be skipped. B. Enter a globally unique name for the function app: Choose a name that can make up a valid URL path. This name would be validated to ensure that it is unique. C. Select a runtime stack: Choose the language version that you used to run the function locally. D. Select a location for new resources: Choose a location where your resource will run from, for the best performance, choose a location close to where your function will be called from. Once our resources are ready, we can deploy the function code by navigating to the Azure tab, in the workspace section select the deploy button(Fig 2.1). After the deployment is complete, you can click on "View output" to view deployment results. Now, we have officially built and deployed our first Azure Function.


Conclusions and Next Steps In this article, we learned how to create and deploy Azure functions using VS Code, but you can also build it straight from the Azure Portal and with other tools such as IntelliJ, Eclipse, and the Azure Functions Core Tools.Here, we discussed building a Serverless API, which is an Azure function with an HTTP trigger, however, Functions can be triggered in a variety of ways. Including Timers, Queues, and even changes in blob storage. Read more here.Before deploying to Production, you need to consider certain concepts such as Logging, Security and Monitoring. All of these are streamlined by Azure. You can read more in the documentation here.If you encounter any problems running the instructions in this tutorial you can reach out to me on Linkedin. References [1] Aurelien Geron, Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2019.[2] Official Documentation for Azure Functions.


Introduction to Speech Recognition With TensorFlow

History of Speech Recognition The development of speech recognition technology dates back to the 1940s when it was used for military communication and air traffic control systems. In the 1950s, researchers developed the first commercial speech recognition system to recognize digits spoken into a telephone. This system was limited to identifying only numbers, not full words or sentences. In the 1960s, researchers developed more advanced speech recognition systems to recognize isolated words and short phrases. This marked a significant advancement in speech recognition, enabling machines to understand basic speech commands. In the 1970s, researchers developed the first large-vocabulary speech recognition systems. These systems were capable of recognizing connected Speech and could handle large vocabularies. At the same time, the development of artificial neural networks and deep learning began to revolutionize the field of speech recognition. This led to the development of more accurate speech recognition systems in the 1980s. In the 1990s, the first commercial speech recognition products were released. These products were based on deep learning models and could be used in various applications, such as dictation software, voice-controlled user interfaces, and speech-to-text transcription services. In the 2000s, the development of speech recognition technology continued to advance with the development of more accurate models and the incorporation of acoustic models. This led to the development of virtual assistant devices such as Google Home and Amazon Alexa. In the 2010s, the development of deep learning algorithms further improved the accuracy of speech recognition models. In 2020, individuals utilized speech recognition technology for various purposes, ranging from customer service to healthcare and entertainment."


What are the Problems and Challenges One of speech recognition's main challenges is dealing with human speech variability. People may have different accents, pronunciations, and speech patterns, and the model must recognize this variability accurately. Additionally, background noises and other environmental factors can interfere with the model's accuracy. Another challenge is dealing with words that sound similar. For example, the words "to" and "too" may sound similar but have different meanings. The model must distinguish between these words to generate an accurate output. Similarly, words with multiple meanings can be difficult for the model to interpret accurately. The quality of the audio signal also affects the accuracy of speech recognition. Poorly recorded audio signals can make it difficult for the model to recognize the Speech. Additionally, the model must be trained on a large dataset of audio samples to achieve a high level of accuracy. Finally, the model must be able to recognize and process multiple languages. Different languages have different phonetic and grammatical rules, and the model must recognize these differences to generate an accurate output. All of these challenges make speech recognition a difficult task. However, with the advancement of deep learning and natural language processing, speech recognition has become more accurate and efficient. With the right techniques and data, it is possible to create a high-quality speech recognition model.


Techniques in speech recognition: The development of machine learning has significantly improved the accuracy of speech recognition. Machine learning algorithms are used to recognize complex speech patterns, understand natural language, and distinguish between different languages. Deep learning is one of the most popular machine learning techniques in speech recognition. Deep learning uses artificial neural networks to learn from large datasets and can be used to recognize complex patterns. It has been used to develop virtual assistant devices such as Google Home and Amazon Alexa and speech-to-text transcription services. Other machine learning techniques used in speech recognition include Hidden Markov Models (HMM), Dynamic Time Warping (DTW), and phonetic-based approaches. HMM is a statistical approach for modeling time series data and is used for recognizing speech patterns. DTW is a technique used for comparing two temporal sequences and is used for recognizing similar speech patterns. Phonetic-based approaches recognize Speech based on their phonetic similarity. Additionally, there are techniques for improving the accuracy of speech recognition models. Beamforming is a technique that reduces background noise by focusing on the sound source. Noise cancellation is a technique used to reduce background noise by subtracting it from the audio signal. Both of these techniques can improve the accuracy of speech recognition models. Until 2018, the most common technique in Speech Recognition was Deep Neural Networks with LSTM, and everything changed when transformers were released. When Transformers were released, they significantly impacted the field of speech recognition. The Transformers are a type of neural network used for natural language processing tasks and to recognize complex patterns in the input audio. They are beneficial for tasks such as speech recognition because they can model long-term dependencies in the data. The introduction of Transformers has allowed for more accurate speech recognition models. You can use them to recognize different languages, understand natural language, and distinguish between similar words. The increased accuracy of these models has enabled the development of virtual assistant devices, voice-controlled user interfaces, and speech-to-text transcription services.


Implementation: In this tutorial, I will demonstrate how to combine a 2D convolutional neural network (CNN), recurrent neural network (RNN), and a Connectionist Temporal Classification (CTC) loss to build an automatic speech recognition (ASR) model. This tutorial will utilize the LJSpeech dataset, which features brief audio recordings of a solitary speaker reciting passages from seven non-fiction books. To gauge the effectiveness of our model, we'll employ the Word Error Rate (WER) and Character Error Rate (CER) evaluation metrics. These metrics calculate the discrepancy between the recognized words/characters and the original spoken words/characters. WER is determined by summing up the number of substitutions, insertions, and deletions that occur in the sequence of recognized words and dividing the result by the total number of initially spoken words. CER follows the same principle but on a character level.


Prerequisites: Before we begin, you will need to have the following software installed: Python 3;TensorFlow (We will be using version 2.10 in this tutorial);mltu==0.1.6


The LJSpeech Dataset: We'll begin by downloading the LJSpeech Dataset. This dataset contains 13000 audio files in a ".wav" format. All the actual labels are also given to us in the annotation file. To simplify this for us a little, I wrote a short script that we'll use to download this dataset: Likely a slower method to download this dataset, but you don't need to do anything manually. Now we have our dataset downloaded, and we need to preprocess it before moving forward to another step. Preprocessing consists of several steps. All the preprocessing we can do with the following code: First, we are loading raw audio data with librosa.load(audio_path) from the Python library librosa. It loads an audio file specified by the audio_path parameter and returns a tuple of two objects: The raw audio signal as a NumPy array, representing the audio samples and the sample rate (number of samples per second) of the audio signal, typically defined as an integer. So, we are iterating through dataset metadata and preprocessing the 'wav' audio data with actual transcription. It looks as follows: Then we preprocess this rad audio data further to receive a spectrogram that we will use to train our model. An audio spectrogram is a visual representation of the frequency content of an audio signal over time. It displays the signal's power spectral density (PSD), which gives a measure of the strength of different frequency components of the signal. The spectrogram is usually represented as an image. The X-axis represents time, the Y-axis represents frequency, and the color or brightness represents the magnitude of the frequency components at each time frame. The brighter the color or higher the brightness, the higher the magnitude of the corresponding frequency component. Spectrograms are commonly used in audio analysis and processing, as they provide a clear representation of the frequency content of a signal and can reveal important information such as pitch, harmonics, and transient events. They are also useful for identifying different types of sounds and for performing tasks such as noise reduction, pitch correction, and audio compression. An example of an audio spectrogram would look following: Keep in mind that when we were iterating through actual transcription, we replaced all capital letters with lower ones and removed all unusual alphabet letters. When we prepare our dataset, we can create our TensorFlow data provider, which will provide us with data through all the training processes; we won't need to hold all the data on RAM. As before, I will use my "mltu" package: You may notice that I am using the WavReader as the data preprocessor and SpectrogramPadding, LabelIndexer, and LabelPadding as transformers. In the given code, the purpose of each component is as follows: WavReader: This class reads audio files (in WAV format) and converts them into spectrograms. It uses the parameters frame_length, frame_step, and fft_length to determine how the audio signals should be split into frames and transformed into spectrograms.SpectrogramPadding: This class is used to pad spectrograms to a consistent length so that all spectrograms in a batch have the same shape. It uses the parameter max_spectrogram_length to determine the length to which the spectrograms should be padded and the padding_value to determine the value used for padding.LabelIndexer: This class converts text labels into numerical representations, for example, transforming words into integers. It uses the vocab parameter, a dictionary of all the words in the vocabulary, to determine how to map words to integers.LabelPadding: This class is used to pad text labels to a consistent length so that all text labels in a batch have the same length. It uses the parameter max_word_length to determine the length to which the text labels should be padded and the padding_value to determine the value used for padding. These components are used together to preprocess the data before it is fed into a machine-learning model. By preprocessing the data this way, it becomes easier to train a model on the data and ensure that the model receives a consistent input format. When training the model, we can't rely on training loss. For this purpose, we'll split the dataset into the training 90% and validation 10%:


The model architecture: CNNs for speech recognition: Convolutional Neural Networks are a type of machine learning architecture that is mostly used for analyzing visual datasets. They are good at analyzing images because they can pick up on the spatial and temporal relationships between the pixels in the image. The convolution layer examines the essential features of the input data, and the subsampling layer compresses these features into a more straightforward form. For speech recognition, CNNs take a spectrogram of the speech signal, which is represented as an image, and use these features to recognize speech. RNNs for speech recognition: Recurrent Neural Networks are a type of deep learning architecture that can handle large sequential inputs. The key idea behind RNNs is that they use the current information and previous inputs to produce the current output. This makes them well-suited for sequential data tasks, such as natural language processing and speech recognition. RNNs are the preferred deep learning architecture for speech recognition because they are good at modeling sequential data. They can capture the long-term dependencies between the features in the input dataset and produce outputs based on past observations. This is particularly useful for speech recognition tasks because the output of a speech frame depends on previous frames of observations. RNNs and their improved version, Long-Short Term Memory (LSTM) RNNs, have the best performance (Except Transformers) for speech recognition tasks among all deep learning architectures and are the preferred choice. So we'll define our model: For a deeper understanding of the architecture, we can open our model.py file, where we create the TensorFlow sequential model step-by-step: Great, now we have the model that we need to compile. Let's do it: As you might notice, I am using CTCloss and custom CER and WER metrics. CER and WER I introduced in my previous tutorials, but these also are one of the most common metrics to tell how accurate our predictions are from actual transcription. CTC is the most common loss when training stuff related to language recognition and extraction. Now, we can define our callbacks (introduced in previous tutorials) and start the training process:


Training process: To track the training process, we added the TensorBoard metric, there we can check what our curves of loss, CER, and WER metrics were. Here is the loss curve: We can see that while training, our loss was constantly decreasing; that's what we expected to see. But we might see that validation loss was falling since the 48 training step, and then it increased. This means our mode might be overfitting. We may see similar scenarios in our CER and WER curves. Let's take a look at them. Here is the CER curve: I was wrong; the CER of validation was constantly improving until step 100. But we can see here a huge gap between training and validation CERs. This is because we are not using any augmentation techniques for our audio data. Let's look at the WER curve: It looks very similar to CER; that's what I was expecting. But overall, our CER is only 1.7%, and our WER is 7%. This means that our model performs well on this dataset!


Test model inference: Our model is trained, and it gave us pretty satisfying results. How can we test it out on single inference? I wrote a script that iterates through validation data from our training data: If you want to test this on your recording, remove the iterative loop and link our audio 'wav' recording, it should handle it! The trained model can be downloaded from this link.


Conclusion: Speech recognition is a field of AI with a rich history of advancements dating back to the 1940s. With deep learning and natural language processing integration, speech recognition has become more accurate and efficient. The main challenges in speech recognition include the following: Dealing with human speech variability;Recognizing similar words;The quality of the audio signal. Several techniques are used in speech recognition, including Deep learning, Hidden Markov Models, Dynamic Time Warping, and phonetic-based approaches. Additionally, beamforming and noise cancellation techniques can be used to improve the accuracy of speech recognition models. The introduction of transformers has significantly impacted speech recognition, enabling more accurate models for tasks such as speech recognition, natural language processing, and virtual assistant devices. This tutorial demonstrated how to build a basic speech recognition model using TensorFlow by combining a 2D CNN, RNN, and CTC loss. With the right techniques and data, speech recognition can be a powerful tool for many industries. The trained model used in this tutorial can be downloaded from this link. Complete tutorial code on GitHub.


Theory of Mind AI: The Power of Empathy

Navigation The Theory of Mind (ToM)What is Empathy?How to Measure Emotions?How to Measure Motivational Parameters?What is an Experience?How to Get Experiences?How Does Experiencing Events Work?How Does Empathy Work When Observing an Individual?Limitations and challengesKey takeaway


Theory of Mind AI: The Power of Empathy

The Theory of Mind (ToM) Research on ToM and empathy in AI is still in its early stages, but there are several studies and papers that provide valuable insights into the topic. For example, a recent paper by researchers at the University of Cambridge explored the potential for ToM in AI to improve social interactions and communication between humans and machines [1][2][3]. Another study by researchers at the University of Southern California looked at how machine learning algorithms can be trained to recognize and respond to human emotions [4].


Theory of Mind AI: The Power of Empathy

What is Empathy? Empathy is the ability to understand and connect with the emotions of others [5]. It's a powerful tool that allows us to form deeper relationships with people and plays a crucial role in human interactions. With the advancements in Artificial Intelligence, machines are now capable of demonstrating empathy by predicting the emotional state and future actions of individuals. By empathizing with others and putting ourselves in their shoes, we can imagine how we would react in similar situations. For example, if we were an antelope encountering a hungry and strong lion, we would instinctively feel the need to run away. On the other hand, if we were the lion, we would see the antelope as easy prey and choose to attack. Empathy helps us understand the actions of others and how they affect our own emotional and motivational state. Furthermore, empathy provides us with significant information. It gives us a sense of how others might feel about an unfamiliar object even before we interact with it. If we see others expressing fear towards a snake, we would probably avoid playing with it.


Theory of Mind AI: The Power of Empathy

How to Measure Emotions? One's emotions are greatly affected by the prevailing factor in their surroundings that catches their attention. Observing the emotions of those in our vicinity allows us to understand their innermost sentiments. As our actions are directly linked to our emotions, it's crucial to remain mindful of them. We are aware that our emotions have an impact on our behavior. However, the reverse is also true. By observing someone's behavior, we can perceive their emotions. By observing an individual's body language and facial expressions [5], we can gain insight into their emotions. A person who smiles with an open body posture is likely to be feeling happy. Conversely, if someone is frowning with closed body language, they are likely feeling angry or sad. How a person walks can often be a revealing indicator of their emotional state. Walking speed, posture, and facial expressions are common cues that convey feelings. For example, if someone is walking briskly with a frown, they may be angry or upset. On the other hand, if someone is strolling leisurely with a smile, they are likely feeling content or happy. Researchers have developed a skeleton recognition program [6] to track the angular velocity of limb movements, providing a way to determine a person's emotional state more accurately. The tone of voice and the words used can also provide clues about a speaker's emotional state. The speed of speech can indicate excitement, while slower speech may suggest sadness. Analytical techniques, such as FFT analysis and sentiment analysis, along with machine learning algorithms, can be used to identify emotions based on spoken words. More about artificial emotions can be found in the following articles:


Theory of Mind AI: The Power of Empathy

How to Measure Motivational Parameters? We can assess a person's motivation by examining their actions and objectives, as these reflect their underlying motivational factors. AI agents can interpret the outcome of actions in a variety of ways. Event recognition involves analyzing the movement of an object using a skeleton recognition program. By analyzing the speed and direction of the subject's limbs, we can draw conclusions about what they are doing. For example, if someone is turning their head, they may be searching for something. Alternatively, if someone is quickly approaching an object, it could be interpreted as an attack. We can then determine the level of motivation for each individual involved. Intention recognition is the process of identifying what someone wants or is attempting to do, particularly in situations where communication is ineffective. It can be challenging to determine someone's intentions, but by observing their actions and taking the context into account, we can make informed assumptions. If someone is holding a present, they are probably intending to give it to someone. Understanding others' intentions is critical in developing effective communication and building strong relationships. For more information, check out the article:


Theory of Mind AI: The Power of Empathy

What is an Experience? Our experiences [7] encompass a collection of events that have impacted our lives, both positively and negatively. These events vary greatly and can include interactions with others, personal achievements, and even challenging circumstances we have faced. It's important to note that events are not random occurrences but rather the result of actions taken by either ourselves or others. By taking note of the events that have transpired and understanding the sequence of actions that led to them, we gain valuable experience. Analyzing the patterns and consequences of past events can provide valuable insights into how to approach future situations and make more informed decisions. It's no surprise that experience is often regarded as a valuable asset, as it can inform and improve our future actions and outcomes. At its core, the experience can be viewed as a list of actions and their causes and consequences, providing us with a roadmap for navigating future challenges.


Theory of Mind AI: The Power of Empathy

How to Get Experiences? Artificial intelligence can gain experiences in two ways - either directly through its own sensors and processing or indirectly by observing the actions and data of humans or other machines, or by reviewing literature. Direct experience means processing and analyzing data first-hand through its sensors such as vision, audio, and touch. For instance, an AI designed to navigate through a maze will gain direct experience by using its sensors to understand the environment and move through it. On the other hand, indirect experience involves analyzing data collected from others such as studying the behavior of humans or other AI systems or reviewing data generated by them. Experiences are essential for the development and growth of artificial intelligence. It is crucial to understand how experiences can be gained and to choose the most effective methods to improve AI's capabilities. Developers use various techniques, such as supervised and unsupervised learning to train the AI system during its creation. Once the AI system is created, it starts gaining experiences through its operation, using various algorithms and techniques to process data and make decisions. As it performs tasks, it learns from the outcomes, and these experiences shape its future behavior. For example, a chatbot used in customer service gains experience as it interacts with customers, answering their questions and resolving their issues. As it communicates with more customers, it learns how to respond to various inquiries and improves its ability to provide accurate and helpful responses. Experiences are crucial for AI systems to learn, adapt, and improve. The more experience an AI system gains, the better it becomes at performing tasks and making decisions. LLMs are one of the possible ways to gather and process experiences. An LLM can act as a storyteller, creating stories from the sequence of events on the input. These stories can be later processed by other LLMs for different assessments and predictions.


Theory of Mind AI: The Power of Empathy

How Does Experiencing Events Work? Experiencing is essentially the evaluation of the impact of past, present, and future events on our own motivational parameters. In the case of an AI agent, the interface for various sources of information can be streamlined so that the central processor does not differentiate between stories that come directly from the text or from the "storytelling" module that builds stories based on its own observations. Past stories can be read from memory, and potential future stories can be obtained from the "generative module." To process the story, the "event module" is used first to dissect the story into events. The "evaluator" module then processes individual events, making an assessment of their impact on motivational parameters. The list of events is later sorted by their assessed value. Based on these assessments, the best event to proceed with is selected from a range of possible future events. The worst possible future event is, of course, avoided. To execute or prevent an event, a set of possible actions is read from memory. An action is chosen that has the optimal impact on motivational parameters. Experiencing events is therefore, the processing of the stream of events, evaluating their impact on us (our motivational parameters), and seeking optimal actions to minimize or maximize the chances of a particular event occurring. Motivational parameters are, therefore, at the center of experiencing events. Their value depends on the events experienced and is assessed based on individual experiences. The actions that the AI agent will take depend on the value of these parameters.


Theory of Mind AI: The Power of Empathy

How Does Empathy Work When Observing an Individual? Empathy, at its core, involves experiencing what the observed individual is going through. When observing someone, we evaluate their motivational parameters and the emotions they display. Gathering information on the individual's motivational parameters is a continuous process that leads to a better understanding of that person. During the process of empathy, we temporarily set aside our own motivational parameters and emotions and load those of the observed individual into our memory. Based on these loaded parameters and emotions, along with our own experiences, we create a list of possible actions to take. We filter these actions based on their impact on the observed individual's loaded motivators and choose the action with the optimal impact. This approach helps us determine the action that the observed individual is most likely to take at that moment, as we would if we were in their shoes. After completing the empathy process, we reload our own motivational parameters and emotions regarding the observed individual. We evaluate the impact of the predicted action on our own motivational parameters. If the impact is positive, we will encourage the individual to take that action, but if it is negative, we will try to prevent them from doing so. Using empathy, an AI agent can predict the desires and actions of users or third parties in their environment. This prediction is of great importance in various fields, such as robotics and automation, healthcare, marketing and advertising, education, and customer service.


Theory of Mind AI: The Power of Empathy

Limitations and challenges The Theory of Mind (ToM) and empathy are complex cognitive abilities that are challenging to incorporate into AI systems. Some of the limitations and challenges associated with integrating ToM and empathy into AI systems include: Lack of understanding of human behavior: There are still a lot of unknowns about human behavior. This makes it challenging to accurately model human behavior in AI systems.Interpretation of emotions: Emotions are complex and difficult to interpret. AI systems may struggle to accurately interpret human emotions and use them to guide decision-making.Contextual understanding: This can be challenging to achieve in AI systems, especially in complex and dynamic environments.Ethics and privacy concerns: Empathy is the collection of feelings and emotions of personal data, which raises ethical and privacy concerns. For example, AI systems that use empathy to predict human behavior could be used for nefarious purposes, such as manipulating people's behavior.


Theory of Mind AI: The Power of Empathy

Key takeaway The key takeaway from the text is that experiencing events involves evaluating their impact on our motivational parameters and seeking optimal actions to either minimize or maximize their occurrence. Empathy involves temporarily setting aside our own motivational parameters and emotions to evaluate those of the observed individual and make predictions about their actions. An AI agent can use empathy to predict the desires and actions of users or third parties, which is important in various fields such as robotics, healthcare, marketing, education, and customer service.


5 Reasons Why You Need Synthetic Data

Real data collection and labeling aren't scalable For small datasets, it is usually possible to collect and manually label data; however, many complex machine-learning tasks require massive datasets for training. For example, models trained for autonomous vehicle applications need large amounts of data collected from sensors attached to cars or drones. This data collection process is slow and can take months or even years. Once the raw data is collected, it must then be manually annotated by human beings, which is also expensive and time-consuming. Furthermore, there is no guarantee that the labeled data that comes back will be beneficial as training data since it may not contain examples that inform the model's current gaps in knowledge. Labeling this data often involves humans hand-drawing labels on top of sensor data. This is very costly as high-paid ML teams often spend a huge portion of their time making sure labels are correct and sending mistakes back to the labelers. A major strength of synthetic data is that you can generate as much perfectly labeled data as you like. All you need is a way to generate quality synthetic data. Open source software to generate synthetic data: Kubric (multi-object videos with segmentation masks, depth maps, and optical flow) and SDV (tabular, relational, and time series data). Some (of many) companies that sell products or build platforms that can generate synthetic data include Gretel.ai (synthetic data sets that ensure the privacy of real data), NVIDIA (omniverse), and Parallel Domain (autonomous vehicles). For more, see the 2022 list of synthetic data companies.


5 Reasons Why You Need Synthetic Data

Manually labeling real data can sometimes be impossible There is some data that humans can't fully interpret and label. Below are some use cases where synthetic data is the only option: Accurate estimation of depth and optical flow from single imagesSelf-driving applications that utilize radar data that isn't visible to the human eyeGenerating deep fakes that can be used to test face recognition systems


5 Reasons Why You Need Synthetic Data

Real data has privacy and safety issues Synthetic data is highly useful for applications in domains where you can't easily get real data. This includes some types of car accident data and most types of health data that have privacy restrictions (e.g., electronic health records). In recent years, healthcare researchers have been interested in predicting atrial fibrillation (irregular heart rhythm) using ECG and PPG signals. Developing an arrhythmia detector is not only challenging since annotation of these signals is tedious and costly but also because of privacy restrictions. This is one reason why there is research in simulating these signals. It is important to emphasize that collecting real data doesn't just take time and energy but can actually be dangerous. One of the core problems with robotic applications like self-driving cars is that they are physical applications of machine learning. You can't deploy an unsafe model in the real world and have a crash due to a lack of relevant data. Augmenting a dataset with synthetic data can help models avoid these problems. The following are some companies using synthetic data to improve application safety: Toyota, Waymo, and Cruise.


5 Reasons Why You Need Synthetic Data

Real data is not programmable Autonomous vehicle applications often deal with relatively "uncommon" (relative to normal driving conditions) events like pedestrians at night or bicyclists riding in the middle of the road. Models often need hundreds of thousands or even millions of examples to learn a scenario. One major problem is that the real-world data collected might not be what you are looking for in terms of quality, diversity (e.g., class imbalance, weather conditions, location), and quantity. Another problem is that for self-driving cars and robots, you don't always know what data you need, unlike traditional machine learning tasks with fixed datasets and fixed benchmarks. While some data augmentation techniques that systematically or randomly alter images are helpful, these techniques can introduce their own problems. This is where synthetic data comes in. Synthetic data generation APIs allow you to engineer datasets. These APIs can save you a lot of money as it is very expensive to build robots and collect data in the real world. It is much better and faster to try to generate data and figure out the engineering principles using synthetic dataset generation. The following are examples that highlight how programmable synthetic data helps models learn: better cyclist detection (Parallel Domain), prevention of fraudulent transactions (American Express), and surgery analysis and review (Hutom.io).


5 Reasons Why You Need Synthetic Data

A model trained exclusively on real data is not performant enough (e.g., slow development velocity) In industry, there are a lot of factors that affect the viability/performance of a machine learning project in both development and production (e.g., data acquisition, annotation, model training, scaling, deployment, monitoring, model retraining, and development velocity). Recently, 18 machine learning engineers took part in an interview study that had the goal of understanding common MLOps practices and challenges across organizations and applications (e.g., autonomous vehicles, computer hardware, retail, ads, recommender systems, etc.). One of the conclusions of the study was the importance of development velocity which can be roughly defined as the ability to rapidly prototype and iterate on ideas. One factor affecting development velocity is the need to have data to do the initial model training and evaluation as well as frequent model retraining due to model performance decaying over time due to data drift, concept drift, or even train training-serving skew. The study also reported that this need led some organizations to set up a team to label live data frequently. This is expensive, time-consuming, and limits an organization's ability to retrain models frequently. Synthetic data has the potential to be used with real-world data in the machine learning life cycle (pictured above) to help organizations keep their models performant longer.


5 Reasons Why You Need Synthetic Data

Conclusion Synthetic data generation is becoming more and more commonplace in machine learning workflows. In fact, Gartner predicts that by 2030, synthetic data will be used much more than real-world data to train machine learning models. If you have any questions or thoughts on this post, please reach out in the comments below or through Twitter.


This AI newsletter is all you need #35

What happened this week in AI by Louis While ChatGPT continues to create excitement in AI, this week we saw lots of active discussion and argument around the model's inner workings and why it is so successful in producing meaningful text. The opinion is divided on just how smart ChatGPT is, how it works, and how significant its release is to the future of AI. We find both sides of these arguments valuable. Different perspectives and thoughtful new ways of describing the workings of large language models and transformers such as ChatGPT can help to continue to improve these models and build on their shortcomings. An article in The New Yorker described ChatGPT as a "Blurry JPEG of the Web" because of its ability to retain much of the information on the Web. However, it's important to note that if you're looking for an exact sequence of bits, you won't find it. Instead, all you will ever get is an approximation, or worse, a hallucination (ChatGPT confidently giving you false information it invented). Unlike a blurry JPEG, ChatGPT is revolutionary because the approximation is presented as grammatical text, which ChatGPT excels at creating. Though it may not be perfect, the text produced by ChatGPT is usually acceptable. In essence, you're still looking at a blurry JPEG, but the blurriness occurs in a way that doesn't make the picture as a whole look less sharp. Stephen Wolfram's article provides another interesting and very in-depth take on how ChatGPT works. He runs through the surprising contrast between ChatGPT's relatively basic concept, simple NN elements, and basic operations behind it - and its remarkable results. While it doesn't always "globally make sense" because it's just saying things that "sound right" based on its training set. Wolfram also discusses the similarities and differences between ChatGPT's underlying neural-net structure and the human brain - and some of its shortcomings, such as lack of computational capability. Overall, he concludes; ChatGPT is a great example of the remarkable things large amounts of simple computational elements can do and also provides great impetus to better understand human language and the processes of thinking behind it. While ChatGPT has made significant strides in making AI more accessible, the next frontier in the field could be building trust in AI. Especially building trustable AI that does not fail confidently and can explain its results in ways humans can understand and judge. Demonstrating the reliability and trustworthiness of an AI system could become a competitive advantage that outweighs having the largest or fastest repository of answers.


This AI newsletter is all you need #35

Hottest News 1.Microsoft to demo its new ChatGPT-like AI in Word, PowerPoint, and Outlook soon Microsoft plans to expand it to core productivity apps such as Word, PowerPoint, and Outlook by integrating OpenAI's language AI technology and Prometheus Model in the coming weeks. 2. ChatGPT Burns Millions Every Day. Can Computer Scientists Make AI One Million Times More Efficient? Training and running a large language model like ChatGPT is expensive. While our brains are a million times more efficient than the GPUs, CPUs, and memory that make up ChatGPT's cloud hardware, neuromorphic computing researchers are working to make the miracles that big server farms in the clouds can do today much simpler and cheaper, bringing them to small devices. This approach is similar to the way the brain works, with hardware, software, and algorithms blended in an intertwined way. 3. Audiobook Narrators Fear Apple Used Their Voices to Train AI After facing backlash, Spotify has paused an arrangement that allowed Apple to use some audiobook files for training machine learning models. The dispute arose after several narrators learned of a clause in contracts between authors and Findaway Voices, a leading audiobook distributor, which gave Apple the right to "use audiobook files for machine learning training and models." 4. Honest Lying: Why Scaling Generative AI Responsibly is Not a Technology Dilemma in as Much as a People Problem In recent instances, AI models confidently provided responses containing incorrect historical, scientific, or physical information. The expected widespread adoption of large language models may increase the production of false or erroneous memories without the intent to deceive, which is defined as "confabulating" or "honest lying" by ClinMedjournal. 5. How should AI systems behave, and who should decide? OpenAI shares information on how ChatGPT's behavior is shaped, as well as their plans for improving its behavior, allowing for more user customization, and increasing public input into the decision-making process in these areas.


This AI newsletter is all you need #35

Three 5-minute reads/videos to keep you learning 1.How Not to Test GPT-3 Doing psychology on large language models is harder than you might think. A recent study by a business professor at Stanford on the Theory of Mind has been one of the major news in the AI world recently. This article explores whether GPT-3 really mastered the theory of mind (ToM). The truth, however, is that GPT often fails in problems involving the theory of mind. 2. How to find a job in Generative AI, and what is it like? In this video, Louis Bouchard dives into a conversation with Or Gorodissky, VP of R&D at D-ID about how to find a job in Generative AI and what the day-to-day is like. The interview covers a wide range of topics related to Generative AI such as the ideal education to get into this field, the job interview process and what's it like to work in a startup in such a fast-evolving industry. 3. PEFT: Parameter-Efficient Fine-Tuning of Billion-Scale Models on Low-Resource Hardware PEFT approaches enable comparable performance to full fine-tuning with few trainable parameters. HuggingFace has introduced a PEFT library that provides the latest Parameter-Efficient Fine-tuning techniques, seamlessly integrated with HuggingFace Transformers and Accelerate. This enables the use of the most popular and performant models from Transformers, coupled with the simplicity and scalability of Accelerate. 4. Create consistent characters in Midjourney In this Twitter thread, @nickfloats shares the best way he has found to create consistent characters in Midjourney. He provides a step-by-step guide for the process, including prompting techniques and setting up the entire workflow. 5. Could Stable Diffusion Solve a Gap in Medical Imaging Data? The Stanford AIMI scholars have found a way to generate synthetic chest X-rays by fine-tuning the open-source Stable Diffusion foundation model. This breakthrough is promising, as it could lead to more extensive research, a better understanding of rare diseases, and even the development of new treatment protocols. This article provides a walk-through of the process.


This AI newsletter is all you need #35

Papers & Repositories 1.LangChain: Building applications with LLMs through composability The LangChain library aims to assist in the development of applications that can leverage the power of LLMs with other sources of computation or knowledge. It is designed to help with six main areas: LLMs and Prompts, Chains, Data-Augmented Generation, Agents, Memory, and Evaluation. 2. Describe, Explain, Plan, and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents This paper investigates the planning problem in Minecraft, an open-ended and challenging environment for developing multi-task embodied agents. The DEPS approach offers better error correction through feedback during long-haul planning and provides a sense of proximity through a learnable module called the Goal Selector. The experiments also show the first multi-task agent capable of accomplishing over 70 Minecraft tasks and almost doubling overall performance. 3. Level Generation Through Large Language Models The paper explores the utilization of LLMs to create levels for the game Sokoban, and concludes that LLMs are capable of generating such levels. It also reveals that the performance of LLMs is significantly related to the size of the dataset. Additionally, the paper presents initial tests on regulating LLM-level generators and discusses the potential research areas. 4. Augmented Language Models: A Survey This survey reviews the works in which language models (LMs) are augmented with reasoning skills and the ability to use tools. After reviewing the current advances in ALMs, the work concludes that this new research direction has the potential to address common limitations of traditional LMs, such as interpretability, consistency, and scalability issues. 5. Google Research, 2022 & beyond: Algorithmic advances This article is part of a series of posts covering different research areas at Google. It highlights Google's progress in scaling up machine learning (ML) solutions, ensuring privacy in ML, developing market algorithms, and advancing the algorithmic foundations of large-scale ML deployment. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #35

Upcoming Community Events The Learn AI Together Discord community hosts weekly AI seminars to help the community learn from industry experts, ask questions, and get a deeper insight into the latest research in AI. Join us for free, interactive video sessions hosted live on Discord weekly by attending our upcoming events. Graph Neural Networks (NN Architecture Seminar #7.1) This week's session in the (free) nine-part Neural Networks Architectures series will be led by Pablo Duboue (DrDub) and focus on Graph Neural Networks. This is the first half of the 7th lecture. During this session, he will explore topics such as Graph Processing Architectures, Local vs. global, GNNs, DGCNNs, GCN, and MPNN. Find the link to the seminar here or add it to your calendar here. Date & Time: 21st February, 11 pm EST 2. Graph Neural Networks (NN Architecture Seminar #7.2) This is the second half of the nine-part Neural Networks Architectures series will be led by Pablo Duboue (DrDub) and focuses on Graph Neural Networks. Find the link to the seminar here or add it to your calendar here. Date & Time: 23rd February, 6:30 pm EST If you missed the first part of the series, find last week's event recordings here. 3. LAIT's Reading Group Learn AI Together's weekly reading group offers informative presentations and discussions on the latest developments in AI. It is a great (free) event to learn, ask questions, and interact with community members. Join the upcoming reading group discussion here. Date & Time: 25th February, 10 pm EST Add our Google calendar to see all our free AI events!


This AI newsletter is all you need #35

Meme of the week! Meme shared by neuralink#7014


This AI newsletter is all you need #35

Featured Community post from the Discord Daemonz#2594 has created an open-source tool for exploring GitHub data using GPT-powered querying. The tool generates SQL queries and presents the results visually, allowing users to ask questions in natural language and chat with the 5 billion rows of GitHub data. Check out the tool here and support a fellow community member. Share your feedback in the thread here.


This AI newsletter is all you need #35

AI poll of the week! Join the discussion on Discord.


This AI newsletter is all you need #35

Article of the week Traffic Forecasting: The Power of Graph Convolutional Networks on Time Series by Barak Or, PhD The Graph Convolutional Network (GCN) represents a groundbreaking development in deep learning, demonstrating its versatility and potential for addressing real-world problems. Traffic prediction is a critical issue in transportation, and the capacity to apply GCN algorithms for this purpose holds immense promise and has the potential to significantly impact the transportation industry.


This AI newsletter is all you need #35

Our must-read articles How to Use Hugging Face Pipelines? by Tirendaz AI Understanding Machine Learning Performance Metrics by Pranay Rishith Creating our first optimized DCGAN by Pere Martra If you want to publish with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #35

Job offers Senior Data Scientist @Link Financial Technologies, Inc (Remote) Senior ML Operations Engineer @Overjet (Remote) Senior Full Stack Engineer @Labelbox (Remote) AI/ML Wireless Communications Engineer @Anduril Industries (Costa Mesa, CA, USA) Senior Software Engineer @CCRi (Charlottesville, NC, USA) ML Research Scientist @Genesis Therapeutics (Burlingame, CA, USA) Junior - Mid-level Data Scientist @Anagenex ( Boston, MA, USA) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net. If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Learnings From My Data Science Career So Far

1. Everything you do has an opportunity cost But don't give it more attention than it deserves Let me be honest - my mind is busy assessing the opportunity cost of writing this article. I could have read the long pending paper on prompting, implemented some algorithm for revision, done Leetcode, etc (this "etc" stands for at least 5 other tasks). But I realized over time there is an opportunity cost to "worrying about opportunity cost". And let me tell you - that's significant to the extent that you end up doing nothing. This is for the planning maniacs trying to optimize every single action they take in the hope that their next project will finally move the world. I have spent countless hours planning the next:- best next Data Science project to do- best book on Machine Learning to read- a best note-taking tool to go for But guess what, Takeaway - Instead of waiting for the next best project idea, just START, however small the project is. 10 small projects are worth more than a single "best" one. Quantity is a prerequisite to Quality. The small implementations build foundations for the big ones and compound, once again, rules. The ideas from different projects slowly compound, helping you make and connect the dots. You can't simply wake up one day with a novel Transformer based architecture without understanding what Attention truly means. Lower your bar, go small, and learn new ideas along the way, and you will realize that:


2. Learn to learn from less Informational Minimalism is what we need I wanted to study Transformers, and I ended up downloading 5 different books. Do I really need 5 books? No.Then why this madness? I believe it was the fear of missing out on information. What if there are better explanations in another book? What if this book is missing some important equation that could have made my understanding better? What should you do? Pick one book, make sure it's almost the best (we would try to be optimal here), and just be fine with it for foundational understanding. Just read enough so as to be able to apply what you learned. Don't pick up another resource until needed. Otherwise, the Law of Diminishing Returns will start kicking your a**. The marginal gain in knowledge (if at all) due to overlapping content won't be worthy of your time. Instead, spend time applying knowledge that you already have. Fail at it and then learn the missing parts to finish the task.


3. Refer to the same resource for revising concepts: Reciprocal Anchoring Don't fire up your favorite browser every single time you revise KNN. Refer to the notes that should have already been made. Or if not, try to use the same reference. This is because Let's break this down. Learning happens in different modalities, namely via visuals, text, sound, etc. While learning about, let's say Voronoi diagrams for KNN in a book, your subconscious mind also registers metadata such as the position of the image on the page, colors in the image, the approximate position of the page in the book, and the preceding and succeeding context around the content. This metadata acts as "Anchor points" that help you put your learning in a context (visual), and it is this context that lets you recall the concepts you learned not only quickly but accurately as well. For example, I refer:1. Linear Algebra by Gilbert Strang for revising LA2. Speech and Language Processing by Dan Jurafksy, each time I feel rusty in foundational NLP concepts3. And so on After all this, it might still be tempting to refer to a different source because who knows if you missed anything. And this is when you need to scroll up a bit and re-read the previous section.


4. Sort out your note-taking workflow sooner than later I have mentioned this in my earlier posts here and here, and it's worth mentioning again. Think of your notes as a second you, someone with whom you can park your entire knowledge base. You would want him/her to be: Quick, giving you information as quickly as possibleAccessible from anywhereAccurateAnd most importantly, should be able to connect things as closely as your brain does Give your second-you the time it deserves, and it will take care of you forever.


5. Do this if you are looking for unique project ideas Just peep into your lives deep enough, and you won't ever run out of them. Moreover, what could be a greater motivation to learn data science than to be able to apply it to your own life? I wanted to develop the habit of reading research papers, so I created a paper annotation tool that would automatically highlight important pieces in a research paper and organize the new information in the form of a knowledge graph. In case you are interested, here is the Github link for the project. You can make your projects novel either: by inventing a novel Algorithm and apply to a common use caseby applying a common Algorithm to a novel use case Looking at the rate at which ML papers are published each day, I find 2nd approach to be more feasible. Just find the problems you face, which we should already be good at, and apply some simple algorithm to solve them. If it works, move on to a more complex one.


6. Reading papers is not only for Researchers I wish I had started this earlier. Few instant advantages: Staying updated with state-of-the-artReal-world applications of algorithms in different contextsIt helps you in developing deeper intuitions about algorithms, as the papers discuss their pros and cons, what worked and didn't work, etc. Reading research papers is the fastest way to develop mental But what papers to read? Well, that depends on what you want to achieve: Is it just for fun and exploration?Explore Alpha Signal. It's an aggregator of the best ML papers that delivers via a weekly newsletter.Is it targeted to a specific topic or related to something you are attempting at your job?Start with Survey Papers. That's a good way to jump-start your understanding of the new concept, as survey papers put everything into context, putting all the approaches at a glance. Make a habit of reading papers, and you will thank yourself for doing so.


7. Strive to be a Producer Our brain hurts when it comes to creating something, and that's precisely why you should do it. It forces you to first convert your subconscious thoughts to dots and then connect them in a way that you otherwise wouldn't have. Reading books, doing courses, and watching youtube tutorials, are passive activities which trick your brain into believing that it learned something. Let me tell you - it didn't. Don't fall for this trap. Instead of reading 10 books on a topic, just read one and create information using your learnings. Maybe: Write a blog describing your unique perspective on the topicImplement the concept and push it to GitHubOr at least take notes in your favorite app (mine is Obsidian, by the way) To give my own example, just to share my learnings, I created a blog on Blogger.com And then I moved on to GitHub as I felt getting some stars would feel awesome. I didn't stop there and moved to Substack, where I am planning to stay for a while: What you just saw is an example of how overanalyzing opportunity costs (blogging platforms in this case) and the fear of missing out truly take over you. Anyways, the point here is to BUILD and CREATE stuff.


8. Plan for a Marathon, but Execute like a Sprint There have been numerous instances when I decided to Leetcode for interview preparation, I was able to maintain a few days worth of streak but eventually fell out of the loop. Back to square one - this is what happens when the "space" in the "spaced-repetition technique" is put to the test. Things die out if not attempted aggressively (in terms of time taken to implement). Plan long-term, break the goal into pieces, and target each one with as much force as possible.


Will ChatGPT replace Data Scientist???

Before we try to ask Chat GPT if this is possible, let us understand what Chat GPT actually is: Chat GPT is a language model created by Open AI. Language models use various statistical and probabilistic techniques to determine the probability of a given sequence of words occurring in a sentence. Language models analyze bodies of text data to provide a basis for their word predictions. It is the reason that machines can understand qualitative information. Now, this is just a simple explanation. The model built by Open AI is a state-of-the-art, world-class language model.


Will ChatGPT replace Data Scientist???

Before we ask ChatGPT, I will tell you what I think and will try to explain my thought process When I first read an article on AI some 1015 years back, the author spoke about the evolution of AI. He explained this as the path for AI's rise Artificial Intelligence (AI)Artificial General Intelligence (AGI)Artificial Super Intelligence (ASI)


Will ChatGPT replace Data Scientist???

Artificial Intelligence:- This is where we are right now. Artificial intelligence is one that focuses primarily on one single narrow task with a limited range of abilities.


Will ChatGPT replace Data Scientist???

Artificial General Intelligence:- AGI is the phase where AI would be able to carry out tasks from different aspects of life. Ex:- We as humans are multi-talented; we can sing, dance, read, write, etc... Basically, we can perform various tasks that are not limited to one field or aspect of life. So AGI would be on the level of a human mind. Due to this fact, I feel we are still far away from reaching AGI, as we still don't know all there is to know about the human brain itself. Forget recreating an AGI similar to human capability at this current point.


Will ChatGPT replace Data Scientist???

Artificial Super Intelligence:- ASI - Now, this is scary stuff. ASI refers to AI technology that will match and then surpass the human mind. To be classed as an ASI, the technology would have to be more capable than a human in every single way possible. Not only could these AI things carry out tasks, but they would even be capable of having emotions and relationships. Let me warn you here. The first time I read articles on ASI some 1015 years ago, those articles mentioned that we wouldn't even understand what ASI is trying to do. The human brain would be like an ants brain when compared to ASI's. It wouldn't stop here, ASI's intelligence will grow leaps and bounds as they keep updating themselves. I felt like, what is the use of me studying/working etc... Kind of like I lost my willingness to live. It took me some days to come out of that phase. The estimates in those articles were such that we would have reached AGI by 2020 & by 2025, or max by 2030 we would reach ASI. Scary right? It would be an existential crisis, isn't it? But here we were in 2022, still wondering if Chat GPT would take our jobs??? Now I am a Data Scientist, I myself have used chat GPT to write code and lots of other stuff. What Chat GPT does is gives you an answer to a specific question we have asked. That is, it can perform a task not a narrow one but just one task of question-answering. We still need humans to ask the right questions, for ideation, to make decisions, etc...


Will ChatGPT replace Data Scientist???

Don't Trust me Still?? Let's hear it from the horse's mouth, then, shall we?? This is what Chat GPT says. Long answer short -AI models like Chat GPT can be a valuable tool for data scientists, but they cannot replace the important role that data scientists play in various industries. This is true for most of the roles. I don't think Chat GPT will replace all the roles, but yes, using Chat GPT and many new versions of such tools, Roles will require humans to learn to collaborate with Technology to adapt better and optimize our work or Life, for that matter. Here is a fun video from AiyooShraddha that has become viral. Quoting her here "AI cannot take away human paranoia and human curiosity, AI can only do things that humans have been doing. Only humans can do what no human has ever done before" I really feel Technology will always be an enabler, not a replacer Hope you liked this article. If yes, do consider following me for more informative posts like these.


ChatGP...Me? Building a Telegram bot "clone" with OpenAI's GPT

Overall Strategy The plan was to fine-tune one of the models from the GPT-3 family with my own Telegram chats. For this exercise, I selected just one chat with a friend whose conversations with me lay on the tamer end of the spectrum yet hopefully contained enough messages (48,000) to sufficiently teach the model my speaking style (and life story). For the fine-tuning process, I broadly followed the strategy laid out in OpenAI's customer support chatbot case study, with some modifications. The fine-tuning was done in Python. For serving, I spun up a Firebase application, written in NodeJS, to 1) store conversation history and 2) serve the model via Telegram by responding to Telegram HTTP webhooks. More detailed steps follow, and you can get both fine-tuning and serving code at this repository.


Fine-Tuning The concepts behind fine-tuning GPT models are covered in detail here. In a nutshell, we need to show the GPT models a number of examples, where each example contains a prompt (what is fed into the model), and its corresponding completion (what the model returns). The challenge is how to turn our Telegram conversation history into a series of such prompt-completion pairs, where the prompts and completions are engineered to meet our goal.


Getting the conversation history from Telegram Pretty trivial, just follow Telegram's instructions and download the history in JSON format. Read the data to get a list of messages.


Turning the messages into meaningful prompts-completions pairs OpenAI's customer chatbot fine-tuning example suggests designing prompts as follows: In English, this means that this (simulated) conversation with an agent... ...would yield 3 prompt-completion pairs. The first two are as follows: Prompt 1: Completion 1: Prompt 2: Completion 2: This prompt-completion schema is neat but still results in the single-response-per-message behavior that we want to avoid. To address this, we modify the schema slightly. Let's say we have this Telegram conversation: This will yield 2 examples (the first set of messages, initiated by me, doesn't count as a completion since the final bot will not initiate randomly, unlike its irl counterpart) Prompt 1: Completion 1: Note that the first message in the completion does not come pre-pended with "me". This is because it's already included in the prompt. By doing so, we force GPT to respond as "me". The customer service chatbot examples do this as well. Also, I added an <END> token, to signal the end of a series of replies. This teaches GPT to also emit this signal at the end of its replies when being used live. Prompt 2: Completion 2: With this mechanism, GPT eventually learns to respond as "me", and is also able to understand and reply in sets of messages, as opposed to just one message per turn. The last idea to introduce is how to segment the message history into conversations. As seen by the prompts above, we are passing in earlier messages earlier on in the conversation so that GPT is able to maintain conversational context. However, in the context of Telegram chats, it wouldn't make sense to do this indefinitely up the message history since a message history is segmented into multiple, distinct conversations, which should be mostly independent of each other. We will need some way of programmatically breaking up the message history into such conversations. I decided to flag the start of a new conversation each time at least 1 hour had passed without further messages from me. With all that theory out of the way, here is the code for creating the conversations. With the conversations properly segmented, we then create the prompt-completion pairs.


Initiate fine-tuning With the training examples ready, we now install the OpenAI command-line interface (CLI). We are ready to prepare the fine-tuning JSONL file that OpenAI expects. Finally, we call the fine-tuning endpoint via the CLI. Here, we opt to fine-tune the default model, Curie. Curie is the 2nd best of the 4 OpenAI text models currently available. Performance would likely be improved by using Davinci, the best one, but it also costs 10x more. As it was, the fine-tuning on Curie cost me $47 USD, so Davinci would have been a bit too costly to stomach, even in the name of science.


Testing the fine-tuned model in Python Before serving the model via Telegram, we test it out on Python. Initial testing showed the bot repeating itself very often. I adjusted the hyperparameters frequency_penalty and presence_penalty to reduce repetitions while playing around with temperature to adjust the bot's imaginativeness. Eventually, this ensued: Deep. It was now time to serve the model up in Telegram!


Serving We now have a model that takes in n ≥1 prior Telegram message as a prompt, and spits out m ≥ 0 messages in response (yes the bot theoretically could just ignore you). The broad strategy behind serving this model as a Telegram bot is to: Store all messages from users and from the bot in a databaseDevise some mechanism or rule by which the model is triggered (remember, we want to avoid triggering it upon every message - that would make the conversation really unnatural and weird!).Write some logic to respond to the aforementioned trigger by:a. compiling all prior messages in the current conversationb. generating the prompt, sending it to the model's API endpoint, c. sending the returned completion back to the user via Telegram's APIDeploy the necessary logic behind an HTTP endpoint and set the Telegram bot's webhook. The main point of this article is on the fine-tuning and prompt engineering process, so I'll cover the serving aspects in lesser detail. Do refer to the serving repo for more detailed instructions!


Architecture To achieve all this, I used Google Cloud's Firebase platform. Messages would be stored in Cloud Firestore, while an HTTP-triggered Cloud Function would listen to Telegram's webhooks and execute the necessary logic.


Triggering Mechanism After racking my brain over how to design a stateless, serverless system that would only trigger the model if at least x seconds had elapsed since the last message, I decided to skip all this complexity and instead having the user trigger the bot's response by sending in an "X". Simplicity ftw!


Cloud Function With the above simplification, just one function, written in NodeJS, was enough to handle everything! This function runs every time a Telegram message is sent to the bot. It does the following: The full function can be found here. Do refer to the readme on how to deploy the whole architecture onto Google Cloud!


Telegram Stuff To get a bot up and running, we have to first create a Telegram bot via the one-and-only Botfather (remember to save your bot token). Once the function above has been deployed to Firebase, you need to set the bot's webhook to point to the HTTP endpoint that Google just created for you. This can be done via Postman or simply curl. And there you have it! A Telegram bot that hopefully talks like you.


Observations After letting a few closer friends and colleagues play around with the bot (christened ChatBings), here are some of what I've learned about it (and them). It picked up my linguistic style and quirks pretty well... It's terrible at small talk (probably learned from me), and often blows people off (hopefully not learned from me...) While it pretty much mimics (and amps up) my style, I can't say the same for content. It's basically a fantasist with a lurid imagination that has clearly gone way beyond the scope of the conversation history it was fine-tuned on. In addition, it's inconsistent across conversations, making up things as it goes along. Sometimes I'm single, sometimes attached, sometimes I like girls, sometimes guys - you get the drift. It's especially imaginative and chatty when talking about relationships and love. I hypothesize it's because the internet is full of such writing because it certainly didn't pick this up from the fine-tuning data... It comes up with the occasional meme-able quote. My friends have a repressed urge to enquire about my love life.I use "HAHAHA" and  wayyy too much in chats. HAHAHA. Jokes aside, the outcome really surpassed my expectations. It's passable as me, at least until one starts to delve a little deeper.


Final thoughts and conclusions All in all, the whole process of building this bot was really fun. It taught me a ton about prompt engineering and GPT's hyperparameters, and the end product was surprisingly legit, even without using the costlier Davinci model. It clearly was fun for my friends who interacted with it, too (at the expense of my OpenAI and GCP credits). And now I've got something to show off at social gatherings! That said, while being an excellent gimmick, the bot is still (at least according to those who tried it), inferior as a chatting companion to the real deal. I guess I (and we?) should take heart from that...? Also, I'm not convinced of the utility of doing something like this in a business context. The model's ability to hallucinate beyond what it was fine-tuned on is just too dangerous, especially for critical domains. Perhaps with future enhancements and/or with a better fine-tuning / prompt engineering strategy, this might yet change. Still, even today, something like this could potentially be used for certain niche situations. One possibility I've been mulling is that of bringing loved ones back to life on Telegram - hollow as the chat might be, it might still bring a sense of comforting familiarity. On the flip side, I'm also quite concerned about something like this being used for love scams at an industrial scale, given the bot's clear penchant for romance. And with that, that's the end of this missive! Have ideas on how else this could be used or improved? Or want to have a chat with the real me? Do hit me up on LinkedIn!


Demystifying AI for everyone: Part 1 -NLP Basics

But hey, why do we even care about learning NLP?? That's because, knowingly or unknowingly, we all use NLP in our day-to-day lives Have you ever wondered how we get those auto-correction suggestions while typing messages or how does google lens read the words written in an image? Everything is powered by NLP. So let's see a few use cases


Natural Language Processing (NLP) use cases: Sentiment Analysis: This is the process of understanding the sentiment of the person speaking/writing. Ex:- Analysis of tweets/reviews of customers to understand what they feel about a company's products. Document Summarization: This is used to summarize huge blocks of texts Ex:- Book summary or Summary of customer feedback etc... Language Translation: Translate from one language to another Ex:- English to Japanese or vice versa. Speech-to-text & Text-to-speech:- These are used to transcribe an audio or text or vice versa. The transcribed text can then be fed to the computers for further processing. Ex:- Amazon Alexa There are many other use cases, I hope you guys get a gist of a few So in this article, let's touch upon how machines understand text data:- Computers understand only binary information. 1 or 0, in short, numerical information. Hence, we need to first convert text data to numerical format so that we can feed it into various NLP machine learning models for the above-mentioned use cases. But even before we convert text to numbers. We need to work on the text data to clean it and structure it in the proper format. Following are the steps that are generally used in the text preprocessing pipeline (some steps can be omitted based on the context of the problem):- Remove white spaces (extra spaces in the text, these are present due to formatting issues)Remove punctuationsRemove numbersRemove stop words (common words which won't give much information as they are present in all documents Ex:- a, an, of, the, etc...)Remove symbols (Ex:- @, <, $, %, etc...)Lowercase all wordsPerform stemming/lemmatization on all words (Ex:- Runs, Running, Run all become run) As I mentioned earlier, this is just an example of a standard general preprocessing pipeline, this should be customized project to a project basis. Post this, we need to Tokenise the documents - Tokenisation is a process of breaking up text documents into chunks of words So now our input data would look something like this - Every word becomes one column, and every document (sentence) is a row Now this input is then used for Vectorization Vectorization is nothing but converting words into vector formats so that computers can understand them And Voila, you have understood the basics, I might say, the core of NLP. There are many Vectorization techniques:- Bag of Words (BOW)TFIDFWord Embeddings This is a topic that will require a whole article, so I will cover this in the next article. Hope you enjoyed this post; I have tried to explain it in a very simple manner. All the above-mentioned steps are taken care of by libraries, and you don't need to code anything on your own. I remember when I first started learning NLP, I had a fear of everything. But when I actually started taking an interest, it was very easy. Just try to keep learning and take small steps towards NLP. I promise nothing is difficult if you are willing to apply yourself. All the best in your journey. Onwards and Upwards people...


This AI newsletter is all you need #34

What happened this week in AI by Louis This week was rather chaotic in the world of large language models (LLMs) and "Generative AI" as large tech companies scrambled to display their technology in the wake of ChatGPT's success. Microsoft announced an AI-powered version of the Bing search engine that incorporates OpenAI's ChatGPT technology into its Edge browser. In response, Alphabet announced their alternative to ChatGPT, named Bard. However, promotional material for Bard contained inaccurate information raising questions as to whether Google rushed out its release. Meanwhile, Chinese web giant Baidu is preparing to launch a generative AI chatbot, ERNIE, later this year. What people call "Generative AI" is increasingly looking to be the next major platform for founders and startups to use to build new products. The barriers to entry to starting a business have now been reduced. You can rapidly and affordably create a prototype or minimum viable product on the back of prompting or fine-tuning APIs of LLMs like ChatGPT. But it is difficult to know how the ecosystem will play out and what capabilities and products will be built into the LLMs and owned by the likes of OpenAI, Microsoft, and Google and which will be performed by the surrounding startup ecosystem. This week we published a new blog Learn Prompting 101: Prompt Engineering Course & Challenges as a summary of Prompt Engineering and how to talk to LLMs and get the most out of them. This forms an introduction to the comprehensive open-source Learn Prompting course that we have contributed to.


This AI newsletter is all you need #34

Hottest News Alphabet Stock Plunge Erases $100 Billion After New AI Chatbot Gives Wrong Answer In Ad Questions were raised about whether Alphabet rushed the release of its new Bard LLM after promotional material contained inaccurate information. This comes as Microsoft begins to integrate ChatGPT into its Bing search engine and opens a debate about how generative AI will change the way people browse and interact with the internet going forward. 2. Generative AI: The Next Consumer Platform We've entered the age of generative AI. It could be the next major platform upon which founders build category-defining products. This article explores the main consumer categories with opportunities like search and product discovery, education, dating, coaching, e-commerce, and more. 3. Hands-on with the new Bing: Microsoft's step beyond ChatGPT Microsoft announced a new AI-powered version of the Bing search engine using the same technology behind ChatGPT. The way Microsoft has integrated these chatbot powers into its Edge browser is different from ChatGPT. This version allows you to ask questions about real-time news and events unfolding by integrating chatbot capabilities into its Edge browser. 4. China's Baidu reveals generative AI chatbot based on language model bigger than GPT-3 The Chinese web giant, Baidu, has made AI the focus of its hyperscale cloud and is set to launch a generative AI chatbot later this year. According to a Baidu spokesperson, the company plans to complete internal testing in March before making the chatbot available to the public. The spokesperson added that what sets ERNIE apart from other language models is its exceptional understanding and generation capabilities, thanks to its ability to integrate extensive knowledge with massive data. 5. Announcing the launch of the Medical AI Research Center (MedARC) Medical AI Research Center (MedARC) announced a new open and collaborative research center dedicated to advancing the field of AI in healthcare. MedARC aims to develop large AI models, also known as foundation models, for use in medicine and to build interdisciplinary teams that can address clinical needs.


This AI newsletter is all you need #34

Three 5-minute reads/videos to keep you learning Understanding Large Language Models - A Transformative Reading List In just five years, large language models (transformers) have revolutionized the field of natural language processing. To help researchers and practitioners get started with these models, this article provides a chronological reading list of academic research papers. The list covers the main architecture and tasks, scaling laws, improving efficiency, and steering large language models to intended goals and interests. 2. Machines Learn Better if We Teach Them the Basics Although AI agents have shown impressive performance in certain tasks, they often struggle to generalize to new environments and lack the abstract skills necessary to succeed in diverse contexts. This limitation arises from their limited foundation of concepts and the vast space of possibilities they must explore. To overcome this limitation, computer scientists are developing new techniques to teach machines foundational concepts before unleashing them into the wild. This article delves into the details of these emerging approaches and their potential impact on AI development. 3. Boltus, The God of AI - A four-episode series of learning to use AI with funny production This Twitter series is a four-episode guide to using AI, covering a range of topics. These include deploying diffusion models at scale, building text-to-image generators, integrating Stable Diffusion into a Slack workspace, and improving the speed of serving Stable Diffusion by 3x. 4. Solving a machine-learning mystery Researchers are studying a new concept called in-context learning, where a large language model can learn a new task after seeing only a few examples, without updating its parameters. This phenomenon could be explained by smaller, simpler linear models embedded in the larger model that can be trained to complete the new task using only existing information. This research sheds light on the learning algorithms that large models can use and could help models complete new tasks without costly retraining. Scientists from MIT, Google Research, and Stanford University are working to unravel this mystery. 5. The Most Important Job Skill of This Century A product race is underway in the world of artificial intelligence. AI evangelists believe generative AI will become the overlay for search engines, as well as creative work, memo writing, research, homework, sketching, outlining, storyboarding, and teaching. This means that the future of work could depend on how well people can talk to AI and the skill required to do so: prompt engineering. This article explains why. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #34

Upcoming Community Events The Learn AI Together Discord community hosts weekly AI seminars to help the community learn from industry experts, ask questions, and get a deeper insight into the latest research in AI. Join us for free, interactive video sessions hosted live on Discord weekly by attending our upcoming events. The Neural Network Architecture Seminar (#6) This week's session in the (free) nine-part Neural Networks Architectures series will be led by Pablo Duboue (DrDub) and focuses on Transformer Networks. During this session, he will explore topics such as transformers, BERT, GPT, T5, pretraining, transfer learning, zero-shot, and few-shot learning. Find the link to the seminar here or add it to your calendar here. Date & Time: 14th February, 11 pm EST 2. LAIT's Reading Group Learn AI Together's weekly reading group offers informative presentations and discussions on the latest developments in AI. It is a great (free) event to learn, ask questions, and interact with community members. Join the upcoming reading group discussion here. Date & Time: 18th February, 10 pm EST Add our Google calendar to see all our free AI events!


This AI newsletter is all you need #34

Meme of the week! Meme shared by neuralink#7014


This AI newsletter is all you need #34

Featured Community post from the Discord Doomlaser#5687 created a post about how he integrated the OpenAI API into a roguelike FPS game, which would generate randomized and emergent character dialogue with live text-to-speech. Instead of manually writing the dialogue, each character would receive prompts based on their background and the situation they are in, leading to unique and dynamic conversations. Check out the full post here and support a fellow community member. Share your feedback in the thread here.


This AI newsletter is all you need #34

AI poll of the week! Find out the answer on Discord!


This AI newsletter is all you need #34

Article of the week Understand the Fundamentals of an Artificial Neural Network by Janik Tinz ANNs are typically implemented using frameworks like TensorFlow, Keras, or PyTorch, which are suitable for very complex ANNs. However, data scientists need to have a fundamental understanding of ANNs. This article aims to help you understand how a neural network works by covering the introduction to the basics of ANNs, followed by an in-depth explanation of the basic concepts of ANNs.


This AI newsletter is all you need #34

Our must-read articles Six Amazing Unknown Python Libraries by Dhilip Subramanian An Intuitive Explanation of Policy Gradient by Renu Khandelwal If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #34

Job offers Machine Learning Infrastructure Engineer (Generative AI) @PhotoRoom (Remote) Senior Data Engineer @Clarity AI (Remote) Machine Learning Engineer - Central Platform Team (MLOps / MLInfra) @Canva (Remote) Senior Data Scientist I @Signifyd (Remote) Staff Data Scientist @Zapier (Remote) Data Lead @Acin (Remote) Data Analyst @Pocket Worlds (Remote) Data Scientist @MyFitnessPal (Remote) If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


Improving Accuracy of Text Extraction with Simple Techniques

Part1:Simple text extraction We will use pytesseract for text extraction because it is most commonly used for text extraction applications. Now we will read the image. I purposefully took the harder image with a lot of noise because we wanted to see the performance of simple text extraction on the complex images. Let's start text extraction. If you downloaded pytesseract, then the above one should be the location to store it. It will help you execute and use all language models effectively. In the text variable, we are storing all texts that are extracted by pytesseract model. The psm mode is very important in extraction. psm stands for "page segmentation mode" how to scan an image to extract text is done by psm mode. If you want to read more on psm then go through the below blog. After using psm mode 8, we got the following output If you see this, then it is irrelevant and does not give any information from a text extraction perspective. Let's change psm mode to 4. This output is better than psm 8. So we will take this mode to refine our text extraction model. Many times there is a case where we get output from text extraction, but it is somewhat distorted than the original word. Ex. the original word is KITCHEN, and the model is giving kitc, kitche, tchen, KITC, kitchen like any combination of words. So for this possibility, we will use one technique with one library. the library difflib is used to calculate the difference between the original and the predicted word. Working on the difflib is very simple. You have to provide a tentative dictionary of words that may come into your images for extraction. Then this library asks for the threshold to match. Any possibility which is matching with our words in the dictionary with the given threshold will get replaced with the original word from the dictionary. importing and working with Difflib If you use the above code and print emptydict then the following is the output. It is much more accurate and usable than the output that we previously had without any preprocessing. What if we want a specific special character and want to eliminate any other present in extraction? We have a solution for that. A RegEx, or Regular Expression, is a library that we can use to solve this problem. After concatenating every extraction in one dataframe, we can use the above code to replace desired signs.


Better than the previous one!! The better option for such cases is to change the library and go for higher-level libraries like EasyOCR, Google vision, etc. but these small changes can be useful when we are doing any kind of text extraction. Above mentioned technique you can use when you are sure about data keywords that we want to extract, and you can update your dictionary of mapping at any stage for new words.


Other ways to improve the accuracy of the model: This reduces wrong interpretations of characters and patterns and helps to reduce noise. Like in Tesseract, you can select a language package. Most OCR engines work well with horizontal and vertical alignment. It gets harder when there is skewness or angled rotation in the text. Some OCR engines have PSM (page segmentation mode), where we can select the orientation of scanning. Brightness and contrast are the two things that enhance the readability and explicitness of features for text extraction. there are many apps that can help to adjust the contrast and brightness of an image. The proper lighting condition can improve the results of OCR. Compressions, resizing, and other image manipulations tend to lose image information. .JPG format tends to lose more data after such operations where. PNG and . TIFF doesn't lose data to such an extent. Choosing image format wisely can improve OCR's efficiency. Below mentioned python code can be useful to convert images into desired DPI. Dots per inch or DPI is the main factor when considering the quality of the image. A resolution of less than 300 DPI can make the image unclear. There are many DPI conversion tools online that can get you the required DPI. So try to use a higher DPI image. For more understanding of concepts like DPI, Resolution, PNG, JPG & other image formats, PPI, Lossless formats, etc.


Useful materials  I found one pretty good series to understand OCR by Python Tutorials for Digital Humanities . This is a link for the video series on OCR.One interesting GitHub account I found was of Kba for OCR. Here is the link: kba /awesome OCR . You will get to know many things as well as interesting projects with source code from this account.I found one excellent book "OCR with Tesseract, OpenCV, and Python" by Dr.Adrian Rosebrock .link for the book: Book link So these are all things that can help you to solve your text extraction-related problem statements at basic levels. 4. A project for "OCR models reading Captchas" you can refer to this video.


If you have found this article insightful Give article claps if you liked this article! If you found this article insightful, follow me on Linkedin and medium. you can also subscribe to get notified when I publish articles. Let's create a community! Thanks for your support!


How to Use Hugging Face Pipelines?

What is NLP? NLP is a subfield of AI that allows computers to interpret, manipulate and understand human language. The goal of NLP tasks is to analyze text and voice data like emails, social media newsfeeds, video, audio, and more. With the NLP techniques, you can handle various tasks such as text classification, generating text content, extracting an answer from a text, etc. NLP doesn't just deal with written text. It also overcomes complex challenges in speech recognition and computer vision, such as creating a transcript of a sound sample or a description of an image. Cool, we learned what NLP is in this section. Let's go ahead and have a look at what the Transformers library is.