Hidden States visualization: Let's take a look at the embeddings before training a classifier to get 2D visualization. For simplicity, we will apply it only for the cls_embeddings it would be the same for the other types of embeddings. We have to reduce the hidden states to 2D, many efficient models can be used to reduce the dimensions of the embeddings: UMAP, PCA, T-SNE We will use the PCA algorithm:1. Data normalization: Standardize the embeddings using StandardScaler of scikit learn2. 2D dimension reduction: fit a PCA model on the embeddings and extract the two first components3. Hexbin Visualization: for each target class, we will visualize the bins of each score (from 1 to 5 with step = 0.5) Let's take a look at the Vocabulary class plot: PS: This visualization section technique is very inspired by this HuggingFace NLP GitHub example Some patterns can be identified from this plot, : for most of them, extreme scores are well separated, 2.5 scores seem to be scattered around all places, and for the others, there are clear overlaps. PS: Do not forget that these embeddings are generated by a model pre-trained on predicting masked words in sentences, not to classify scores.


Transformers for Multi-Regression - [PART1]

Multi-Regressor Head Training Let's train a multi-regression model on our embeddings: I chose a gradient boosting-based model: Xgboost In our case: multi-class regression, we will be using the MultiOutputRegressor estimator of scikit-learn. I will let you check the original and further implementation from @SWIMMY excellent notebook of different tree-based models + stacking with a meta-model. To see which pooling has the best separation representation, we will use the cross-validation evaluation for each pooling embedding. The global metric consists of averaging the RMSEs of the 6 of the target columns: this metric is called MCRMSE (mean column-wise root mean squared error) Let's define the evaluation metric: Now launch the CV training: During the CV training, for each embedding type, a fold is kept apart at each iteration, and the model is trained on the remaining fold. Then we predict the unseen fold. Hence we obtain out-of-fold predictions (OOF predictions), which means that every prediction has been accomplished on unseen data. Then we evaluate the RMSE on each class and the global MCRMSE on the OOF predictions for each embedding type: we obtain the following performances: It seems like the MeanMax pooling gives the best performance, but the Mean pooling is veeery close. As the mean max pooling takes twice the volume as the mean pooling, we will choose the mean encoding to fine-tune the transformerThe least efficient pooling method on our corpus is the Max poolingFor all of the representations of analysis metrics, vocabulary is the easiest target to estimate (lowest RMSE), and cohesion is the hardest one (highest RMSE) To infer new predictions, a friend of mine, Mathurin Ache, who is also a great data scientist and a Kaggle master, taught me two methods : Use the CV models to predict the new data: averaging the predictions allows for reduce variance however increases the prediction per sample time, and if we have to deploy this approach, we have to save all the CV modelsTrain the retained model on the whole training data once, then predict the new predictions, which can increase a little the generalization error a, but it is recommended in case of deployment (prediction time and single model to monitor) As long as we are in Kaggle competition context we opted for the first method, using the Mean-Max Pooling:


Credits: My work in this part was inspired by these excellent resources, do not hesitate to consult them: @swimmy notebook: an excellent recap for gradient boosting models blending, I borrowed the XGBoost part that I applied to the transformer embedding@rhtsingh notebook: the different ways to explore transformer representations@Y.NAKAMA notebook: the loss function and the multilabel - Stratified cross-validationHuggingface NLP GitHub: the dimension reduction and the transformer embedding visualization


Conclusion: Thanks for Reading my post I hope it will be useful! As a reminder, all my work can be found here. In this post, we saw how to use a pre-trained transformer to extract context-capturing embeddings and use it to train a multi-regressor (Xgboost in our case) to model analysis metrics of the student's essays. In the next part, I will address the same problem but by fine-tuning the transformer, and updating all its encoder stack, this time. Furthermore, I will show you how to use Weights & Biases great platform to track model performances and create model artifacts. Transformers for Multi-Regression task


META's PEER: A Collaborative Language Model

If you have found it interesting: You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support! Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more. Or feel free to check out some of my other articles on Medium: https://medium.com/mlearning-ai/generate-a-piano-cover-with-ai-f4178bc9cb30


A Step-by-Step Approach To Building a Text Summarization Webapp in Python From Scratch

1. Identify and utilize a text summarization model from Hugging Face For our project, we will be using the model facebook/bart-large-cnn provided by Lewis et al. [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension]To use the model's Inference API, select the model from the Hugging Face library and then clickInference APIunder the Deploy button.On clicking that, we will see a python script that can be utilized for the inference, as shown below. An Access Token is needed to get the API_URL and headers.Create a profile in Hugging Face and create a new access token by following this path Profile -> Settings -> Access Tokens Tab.The newly created access token can be used in place of Bearer and headersNow let's move to the next part of our project to create a web application.


2. Build the front end with HTML and CSS. The code for the front end is written in two parts  - A static file - A HTML file The static file consists of customizations to improvise the look of our front end. As the name suggests, the contents of a static file are not going to be changed according to the user's input or actions. A static file can include anything such as images, videos, cascading style sheets(CSS), flash files, etc., and these are not dynamically generated by the web server like a typical HTML response because they remain static. In the desired location, create two folders named staticand templatesIn the static folder, create a new file named main.css, where CSS is the cascading style sheet that is used to style the HTML.main.css consists of tags present in the HTML code with their corresponding styling and formatting. For a more detailed tutorial on how CSS is used, please refer to this link.The code for the static file main.css is given below, followed by an explanation. As we can see, the above CSS file consists of appearance-based formatting and customization for most of the tags used in our HTML code. Code Explanation Lines 2 to 9 - consist of styling and formatting aimed at the tag header.Lines 11 to 25 - consist of formatting for tag h1.Lines 28 to 34 - have the styling for body tag, along with a background image named image.gifLines 35 to 39 - formatting for containerLines 41 to 47 - styling for div tag, which specifies the division of some contentLines 49 to 64 - formatting for tags h2 and h3Lines 68 to 77 - consist of formatting aimed at placing two text boxes side by side with the help of tags parent and child. Let's move on to creating the front end with our HTML code. The HTML is built with the objective of providing a structured pleasant user interface, a space for users to provide input and a space for showing the summary. In the already created folder named templates, create a new file named index.html.The code for index.html is given below, followed by the explanation. Code Explanation Lines 4 to 11- Header of the HTMLLine 9 - We have added the source of the static file (main.css).Line 10 - We have given a title for our webpage named Summarization Application.Lines 13 to 42 - Body of the HTMLLines 14 to 19 - Contains the headertag, which is used to provide the introductory part of a web page. In our case, we have given "My Summarization App"Lines 23 and 34 - Two text boxes are created and placed side by sideLines 25 to 31 - The form created for getting the input data and sending it to the back end. We have utilizedform action to specify that the submitted data has to be sent toget_summary the method written in the back end for processing.Line 27 - The text box for input is created and is named with the variable nameinput_text which is used in the back end for processing input data.Lines 29 and 30 - Buttons Submit and Clear are created.Line 38 - A text box for displaying the output is created with the name result which is used in the back end to store the summarized text.Having built the front end for the web app, let us move on to build the back end.


3. Build the back end with Python Flask and include the summarization task. The job of the back end is to get the input from the front end,utilize the Hugging Face's Inference API by sending the input data to the summarization model and gathering its output(summary)send the summary to the front end Let's name our backend python file asapp.py, kindly refer to the code given below, followed by the explanations. We have to remember that the front end and the back end should be connected with common variable names.The inputs to the summarization model are  - Text to be summarized (obtained from the front end) - Minimum length of the summary (initialized) - Maximum length of the summary (initialized) Code Explanation To begin with, let's install Flask using the command pip install flaskLine 6 - includes the path for the static fileLines 7 to 9 - index method to render the index.html fileLines 12 to 38 - get_summary methodLines 15 and 16 - includes the URL for the Inference API and the authentication tokens, respectively. Make sure to paste your authentication token in the Bearer part.Line 25 - specify the input from the front end using the variable input_textLines 26 and 27 - specify the maximum and minimum lengths for the summary with variables min_len and max_lenLines 30 to 33 - send the query to the model, including the input variables. The result is stored in output variable.Line 36 - the summary from the model stored inoutput["summary_text"] is sent to the front end using the result variable.As we can see, the common variable names used for connecting the front end and back end are input_text, get_summary, result.The full code for the project is given in this GitHub link.


Summarization App On running the app.py python code, the front end (created using main.css and index.html files) is rendered in the local host http://127.0.0.1:5000.Opening the local host link in a browser will give us the web application.A screenshot of the output is given below. We can see Summarization Application as our page title, MY SUMMARIZATION APP as the header content, two text boxes side by side with titles Input Text and Summarized Text, two buttons Submit and Clear.If an input text is given in the left side box and Submit button is pressed, the get_summary method is invoked and the summarization model is utilized.In the following screenshot, we can see that the input text is pasted and Submit button is clicked. As a result the get_summary method is invoked and the change in the address bar can be noted in the following screenshot (http://127.0.0.1:5000/get_summary). When the get_summary method is invoked, the inference API of the Summarization model is utilized, and the output is shown in the Summarized Texttext box. Hurray!!! We have successfully created a Web application for the most popular Summarization task in Natural Language Processing.


Summary We had an introduction on summarization, how we are going to build the web app for extracting the summary of a text, and a gist of all the elements to be used. We proceeded to understand the simple 3 step process of developing the web app, starting with selecting a model from Hugging Face for the task. Next, we understood the usage of HTML and CSS with a code example for building the front end. Followed by that, we inferred the usage of Flask with a code example for building the back end. Finally, we saw the result of our hard work, the rewarding Summarization Web Application. The same procedure can be extended to build any other kind of Natural Language Processing task using a model from Hugging Face. Proceed with your task and succeed!!!!


This AI newsletter is all you need #20

What happened this week in AI by Louis eDiffi, DALLE and stable diffusion. This is, once again, what happened this week in AI. There's nothing new about stable diffusion, but is it here to stay? Will there be open-source competitors or even proprietary-worthy competitors? eDiffi, as we cover below, is a new proprietary competitor by NVIDIA. They defend that it achieves better results, more fidelity, and allows more control to the user. Yet, it is not open-source. On the same idea, OpenAI just opened its beta to the public. That means anyone can pay to use it or their API. Are you interested in such models if you cannot access their code? What if they build an amazing and super user-friendly tool at a relatively cheap price? Isn't it worth it? Personally, I think so. But I also think it slows down progress, and I'd much rather see code and available checkpoints for other researchers (at least) to implement and build upon. I am convinced the next few conferences like CVPR will have tons of papers based on diffusion models and improving them without re-training or quick fine-tuning, simply thanks to stable diffusion. Still, I'd love to have your take on this, and this is what this week's poll is about. See more below! *Reminder* Activeloop is giving away two NeurIPS tickets for our community (in-person & virtual! Sorry for the error last week)! You can visit the NeurIPS conference, where you can attend amazing talks, workshops, tutorials, meet with the best researchers, and many interesting brands and tools in our industry. Just join our Discord community and send me a private message (@Louis B#1408) to have a chance to win a ticket to the in-person event!


This AI newsletter is all you need #20

Hottest News A weekly newsletter you should follow!The newsletter I am referring to is AlphaSignal. If you like my daily newsletter, you'll love Lior's weekly recap of the top 1% of papers, tweets, GitHub repos, and more. He developed an algorithm to figure out the most interesting papers, repos and tweets from the research community and manually curates the last few to ensure quality. I recently discovered it and it is awesome. You will love it and the quality is simply out of this world! Have a look at his newsletter and let me know what you think!Direct Data Access - a new way to interface with your dataDagsHub is launching Direct Data Access - a new and improved way to interact with your data. Providing an intuitive interface to stream and upload data for any ML project. It doesn't require any adaptation, and lets you keep all the benefits of a versioned and shareable dataset based on open-source tools. If you want to dive into the docs, you can go there now. Let me know if you find this useful!FastDup  A tool for gaining insights from a large image collectionFastDup is a tool for gaining insights from a large image collection. It can find anomalies, duplicate and near duplicate images, clusters of similarity, learn the normal behavior and temporal interactions between images. It can be used for smart subsampling of a higher quality dataset, outlier removal, novelty detection of new information to be sent for tagging.


This AI newsletter is all you need #20

Most interesting papers of the week eDiffi: Text-to-Image Diffusion Models with Ensemble of Expert DenoiserseDiffi is a new generation of generative AI content creation tool that offers unprecedented text-to-image synthesis with instant style transfer and intuitive painting with words capabilities. Learn more in our article!UPainting: Unified Text-to-Image Diffusion Generation with Cross-modal GuidanceThey propose an effective approach to unify simple and complex scene image generation combining the power of large-scale Transformer language model in understanding language and image-text matching model in capturing cross-modal semantics and style, is effective to improve sample fidelity and image-text alignment of image generation.POP2PIANO : POP AUDIO-BASED PIANO COVER GENERATIONPop2Piano, a Transformer network that generates piano covers given waveforms of pop music! Generate a piano cover of any pop music! Here is the code. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #20

Meme of the week! This is the first step to real AGI (and we are dreaming about it). Meme credit to AgressiveDisco#4516.


This AI newsletter is all you need #20

Featured Community post from the Discord A member of the community, tomi.in.ai#8822, has made a Foundational Research Paper Summary video on the 2015 paper that changed a lot of things: U-Net. It's a popular algorithm widely used for image segmentation tasks. Watch his video here and give him a like for us! P.S.: the idea of his channel is to make high-level summaries of most of the popular computer vision algorithms out there.


This AI newsletter is all you need #20

Article of the week 5 Types of ML Accelerators by Luhui Hu Recent years have seen exponential growth in the scale of deep learning models. It becomes increasingly challenging as deep learning training and serving evolve. Scalability and efficiency are two major challenges for training and serving due to the growth of deep learning models. Are deep learning systems stuck in a rut? No! The author summarizes five primary types of ML accelerators or accelerating areas.


This AI newsletter is all you need #20

Our must-read article Jupyter Extensions to Improve your Data Workflow by Cornellius Yudha Wijaya If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #20

Ethical Take on the Blueprint for an AI Bill of Rights by Lauren I'm late to this analysis train, but the Blueprint for an AI Bill of Rights was (somewhat) recently released! It details a set of 5 guidelines that are intended to shape how AI is deployed, and I highly recommend reading through it. Many have taken it to be one of the greatest steps in moving toward applying AI ethics in a more practical and authoritative context, especially as we've seen failures of self-regulation like the recent firing of Twitter's entire ethical AI team. When the AI Bill of Rights was announced, I was really excited because I had hoped this set of guidelines would act as a precursor to more enforceable legislation, which has since been confirmed. The Blueprint for an AI Bill of Rights also serves as a statement that the U.S. is working to play catchup in our AI ethics shortcomings, as we're far behind other regions such as the E.U. in this regard. However, we haven't cracked the code. Others have pointed out that this set of rules fails to address important issues, including this piece from The Conversation about the AI Bill of Rights by Professor Christopher Dancy. I usually advocate for more generalist approaches to be able to specialize in more detailed applications, but Dancy's approach highlighted that by not addressing specific routes of oppression - namely systemic racism - the Blueprint fails to adequately address the issue, falling short of where we need to land for an application of AI ethics to really be effective against some of the worst cases of harm and bias. This is a clear case in which a specific address is quite necessary. This may be rectified in future legislation, which I am very much hoping it does. As grateful as I am to have the U.S. government taking a stance and planning action to shape where AI needs to go, I can't help but draw the parallel with other AI ethics issues that need to be explicitly addressed, such as neurodivergent users being exploited by attention-cultivating algorithms. Not every issue can be named, but the big ones ought to be, and a failure to do so may result in the generality being exploited beyond its intended protections. I will be keeping a very close eye on what comes of these guiding principles and how legislation is advancing in response!


This AI newsletter is all you need #20

Job offers Data Scientist @ Alethea Group (Remote, US) Machine Learning Engineer, Infrastructure @ Earnin (Remote) AI Content Fellowship @ Deepgram (Remote) Machine Learning Engineer, Copilot Model Improvements @ Github (Remote, US) Machine Learning Engineering Manager @ Verana Health (Remote) AI Implementation Manager (Healthcare) @ ClosedLoop (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net.


Cancer Research Needs Better Data

Why do we need it, and what are the problems? We still have many unanswered questions: Many cancers do not have a clear etiology; we do not know what genetic and environmental factors predispose to the onset of cancer.Second, are there universal factors necessary for the development of cancer? Some researchers talk about the possibility that there is a set of necessary conditions for it to develop.Also, it is unclear why some patients with the same cancer respond to treatment, and others do not.Another question is how much diet affects the response to therapy: there are many conflicting studies (low-fat versus high-fat, obesity role).Although we know that the leading cause of cancer death is metastasis, how it forms and the mechanisms behind metastasis are still unclear. In addition, we do not know why some cancers prefer to metastasize to some tissues more than others (the most accepted hypothesis is called "seed and soil").Many of the treatments that look promising in research fail in clinical trials (85% of clinical trials despite years of research).Not to mention that many researchers wonder how to treat a disease that continually evolves.the tumor microenvironment facilitates the tumor, might it be possible to reprogram it against the tumor?Many of the therapies today are specific to a particular type of tumor, but researchers wonder if there is a possible drug for all tumors (e.g., whether all tumors are vulnerable to the same type of pathway).Finally, there are researchers wondering whether our knowledge of cancer is sufficient to find a cure. Not to mention that as was recently noted by a recent article in Nature, this is an issue of great social importance: As I discussed in a previous article, data quality heavily influences the results of machine learning models. Even the most sophisticated artificial intelligence model will not produce good results in the presence of a bad dataset (garbage in, garbage out) The first problem is that cancer is a complex disease, and cancer cells evolve in response to therapy. For this, we need quality data that allow us to eliminate potential confounding factors and have reliable models. There are cancer registries (even the first ones were established in the early 1900s), and they generally collect information on demographics, diagnosis, tumor histology, treatment, and outcome. There are both general registries and registries focused on a special type of cancer, initiatives at the hospital level and at the national level. Often, however, these registries are not standardized. In addition, more and more information has been collected over the years, but errors and missing entries have also accumulated. The second problem is that consent must be obtained. Collecting data and information requires complex bureaucracy and permission from various entities. The collection of samples and data is expensive and laborious organization. Not always both physicians and researchers are willing to take on the burden, nor is the funding always there. In addition, a patient will see several specialists or may go through several hospitals (perhaps for a second opinion or therapy present at a particular hospital). Fragmented healthcare is another obstacle to data collection. Even if these data are collected, they must then be aggregated and made homogeneous.


Cancer Research Needs Better Data

How we solve it? TGCA is an example of a project that sought to characterize thousands of patients. The result is 2.5 petabytes of data covering more than ten thousand patients and 33 tumor types. This dataset has been used in more than a thousand studies (both bioinformatics but also artificial intelligence). In addition, it has been useful in understanding the benefit of new therapies. There are other projects like the cancer dependency map to bring new insights into new therapies, and others are under study. multimodal learning is now a field that is seeing rapid growth; more and more artificial intelligence models are capable of learning from heterogeneous data (images, medical notes, genomic data, tabular data, etc.). So we need similar datasets to be able to train such models. Some initiatives are being studied in England (UK biobank) and the U.S. (Count Me), but they are still few. In addition, on the privacy profile, there are new stricter regulations but also new technologies to anonymize patients (blockchain, new artificial intelligence models for anonymization). Countries such as England and Denmark also have been looking at how they can centralize health systems and thus also patient data. On the other hand, both institutions and universities have realized the importance of having to maintain both databases and the codes that have been developed (either by providing courses to researchers or by hiring specialized figures). The pandemic itself has brought increased collaboration among various institutions. For example, more than 75 institutions have worked together in the National COVID Cohort Collaborative, with the aim of collecting clinical data from more than 6.5 million people with covid in the US.


Cancer Research Needs Better Data

Parting thoughts Research needs quality data. Most clinical trials fail after years of research and billions in investment. One of the causes is our incomplete knowledge of the mechanisms of cancer onset and resistance. This is why data collection is essential, but this alone is not enough we need it to be curated, centralized, and available to the community. Large studies such as TGCA have shown value to the community of scientists. They have been used to test hypotheses and develop increasingly accurate models and new therapies. Medicine and biology are increasingly entangled in data science, is one of the fundamental principles of data science is the quality of the source data.


Cancer Research Needs Better Data

If you have found it interesting: You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support! Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more. Or feel free to check out some of my other articles on Medium:


10 AI Websites That Will Excite You to The Core! Part:2

1] OpenAI playground What if a website can write code for you? a blog for you? resignation letter for you? love letter for your girlfriend/boyfriend or loved ones? Yes!!! all things are possible by OpenAI playground! Almost any activity that includes understanding or producing natural language or code can be used with the OpenAI API. We provide a range of models with varying degrees of power appropriate for various activities, as well as the option to fine-tune your own unique models. Everything from content creation to semantic search and classification may be done using these models. Model: based on GPT-3 models called Davinci, Curie, Babbage, and Ada. You can choose any of the above according to your use case.


what are deepfakes? Deepfakes are synthetic media in which a person in an existing image or video is replaced with someone else's likeness. The term "deepfakes" is a combination of "deep learning" and "fake". Deepfakes use potent machine learning and artificial intelligence techniques to edit or synthesize visual and audio information that can more readily fool, even though the act of producing fake content is not new.


3] inferkit Using a cutting-edge neural network, the text generation tool from InferKit reads the text that you supply and generates what it believes will happen next. It can produce any amount of text about essentially any topic and is customizable.


Who is this for? Creative and fun uses of the network include writing stories or poetry. Other use cases might be marketing or auto-completion.


4]tensorflow playground TensorFlow playground is a website where you can visualize the training process with live changes in outputs with the help of numbers and actual diagrams. You can experiment with many things like changing activation, epochs, regularization, features, hidden layers, and many more... The deep playground is an interactive visualization of neural networks written in TypeScript using d3.js. they use GitHub issues for tracking new requests and bugs. You can also contribute to their repository.


5]Lalal.ai The stem separation problem has also been solved brilliantly by LALAL.AI if you need to preserve the instrumental backing track of a song but take out the vocals. One of the most powerful tools in AI. It makes use of a neural network known as Cassiopeia, which was developed over numerous iterations using 20TB of training data. It can extract vocals, drums, guitar, piano, bass, synthesizer, and general accompaniment and is really easy to use. The outcomes are remarkable and merit a try for yourself. LALAL.AI, thankfully, lets you input 10 minutes of music and provides a detailed preview of the extractions. Although you won't be able to download the extracted stems, it's still worth subscribing to the service if the results are good enough for your needs. You do not require a subscription, in contrast to other AI technologies. Instead, depending on your preference, LALAL.AI offers one-time packages for 90 or 300 minutes of extraction. So stretch your music senses to make your favorite music piece!


6] Replika.com Feeling alone? want to chat with anyone? or a replica of you who thinks like you? visit this website. An AI companion who is eager to learn and would love to see the world through your eyes. Replika is always ready to chat when you need an empathetic friend. Even though talking to Replika feels like talking to a human being, rest assured - it's 100% artificial intelligence. Your Replika is unique to you and wants to know what your world is like. Replika combines a sophisticated neural network machine learning model and scripted dialogue content. It has been trained on a large dataset to generate its own unique responses.


7] Quick, draw! Quick, Draw! is an online game developed by Google that challenges players to draw a picture of an object or idea and then uses a neural network artificial intelligence to guess what the drawings represent. The AI learns from each drawing, improving its ability to guess correctly in the future. Quick Draw is a new social space lottery game with drawings every five minutes that display winning numbers on monitors in-store. For a minimum of $1, players choose how many numbers (spots) they'd like to play, which determines a player's odds and potential prizes.


8] Hotpot.ai Hotpot uses AI and user-friendly tools to assist creators. Many people, especially those in developing countries, lack access to professional graphics, images, and text. This can be altered with more intelligent software. Our goal is to make video production and graphic design 10x faster and more accessible. their objective for professional designers is to augment the creative process by automating repetitive chores. The objective is to make design and image creation as simple as texting a friend for non-designers. text-to-image AI empowers anyone to create attractive paintings, illustrations, and images. Describe what you want, and watch Hotpot bring it to life. You can create your own images and NFTs with a license. For NFT traders, this will be a life changer.


11]AI + LEARNING There are many AI-related websites by google under tag experiments with google. You can find a much more interesting website on the below link.


If you have found this article insightful If you found this article insightful, follow me on Linkedin and medium. you can also subscribe to get notified when I publish articles. Let's create a community! Thanks for your support!


If you want to support me : As Your following and clapping is the most important thing, but you can also support me by buying coffee. COFFEE.


All Types of ML Accelerators

AI Engineering The past decade has been the era of deep learning. We are thrilled with unstopping milestones from AlphaGo to DELL-E 2 and more. And we cannot count how many AI-powered things have happened in our daily lives, including Alexa devices, Ads recommendations, warehouse robots, self-driving cars, and more. Recent years have seen exponential growth in the scale of deep learning models. It is not news that Wu Dao 2.0 model contains 1.75 trillion parameters, and it takes about 25 days to train GPT-3 on 240 ml.p4d.24xlarge instances of the SageMaker training platform. But it becomes increasingly challenging as deep learning training and serving evolve. Scalability and efficiency are two major challenges for training and serving due to the growth of deep learning models. Are deep learning systems stuck in a rut? No! I introduced distributed parallel training for scaling out training in my earlier two articles: model parallelism and distributed parallel training. I shared ML compilers for accelerating training and serving. Are these all solutions for the umbrella of deep learning? No! Here I'll summarize five primary types of ML accelerators or accelerating areas.


All Types of ML Accelerators

Understand ML Lifecycle in AI Engineering Before fully covering ML accelerators, let's first visit the ML lifecycle. ML lifecycle is a lifecycle of data and models. Data is food for ML and can determine model quality. Every area in the lifecycle is full of opportunities for acceleration. MLOps can automate the process of ML model deployment and serving. But it is limited to the horizontal process of AI workflow and cannot improve training and serving fundamentally due to the nature of operations. AI engineering, far beyond MLOps, can holistically (both horizontally and vertically) engineer the process of ML workflow and the architecture of training and serving. Furthermore, it can accelerate serving and training through effective orchestration for the entire ML lifecycle. Based on the holistic ML lifecycle with AI engineering, there are five primary types of ML accelerators (or accelerating areas): hardware accelerators, AI computing platforms, AI frameworks, ML compilers, and cloud services. Please see their relationship diagram below. We can see hardware accelerators and AI frameworks are the mainstream of acceleration. But recently, ML compilers, AI computing platforms, and ML cloud services have become increasingly important. Let's take a closer look at all five AI accelerators below.


All Types of ML Accelerators

1. AI Frameworks We cannot skip choosing the right AI framework when talking about accelerating ML training and serving. Sadly, there is no perfect or best AI framework for all use cases. Three AI frameworks widely used in research and production are TensorFlow, PyTorch, and JAX. They lead from different perspectives, such as ease of use, production maturity, and scalability. TensorFlow: TensorFlow is the flagship AI framework. TensorFlow has dominated the deep learning open-source community since the beginning. TensorFlow Serving is a well-defined, mature platform. TensorFlow.js and TensorFlow Lite are also ripe for the web and IoT. But due to the limitations of deep learning early exploration, TensorFlow 1.x was all about building static graphs in a very non-Pythonic way. This became a barrier for instant evaluation using the "eager" mode, which allowed PyTorch to ramp up quickly in research. TensorFlow 2.x tried to catch up, but unfortunately, upgrading from TensorFlow 1.x to 2.x has to be brutal. TensorFlow also introduced Keras for easier use from the high level and XLA (Accelerated Linear Algebra) optimizing compiler to improve low-level speed. PyTorch: With its eager mode and Pythonic approach, PyTorch is a significant force in today's deep learning world, from research to production. In addition to TorchServe, PyTorch integrates with framework-agnostic platforms such as Kubeflow. Also, PyTorch's popularity was tied to the success of Hugging Face's Transformers library in the first place. JAX: Based on device-accelerated NumPy and JIT (Just-In-Time), Google rolled out JAX. It is a more native framework for deep learning rapidly gaining traction in research, as PyTorch did a few years ago. But it's not an "official" Google product yet, as Google claims.


All Types of ML Accelerators

2. Hardware Accelerators We can have a lengthy article on hardware accelerators. Undoubtedly, NVIDIA's GPUs ignited to speed up DL training, though it was initially intended for video cards. The popularity of graphics cards for neural network training exploded after the advent of general-purpose GPUs. These GP-GPUs could execute arbitrary code, not just rendering subroutines. NVIDIA's CUDA programming language provided a way to write this arbitrary code in a C-like language. With their relatively convenient programming model, massive parallelism, and high memory bandwidth, GP-GPUs now oﬀer an ideal platform for neural network programming. Today, NVIDIA supports a range of GPUs from desktop to mobile, workstations, mobile workstations, consoles, and data centers. With the success of NVIDIA's GPUs, there is no lack of successors along the way, such as AMD's GPUs, Google's TPU ASIC, etc.


All Types of ML Accelerators

3. AI Computing Platforms As described above, the speed of ML training and serving significantly depends on hardware (e.g., GPU and TPU). These drivers (that is, AI computing platforms) become critical for performance. There are two well-known ones: CUDA and OpenCL. CUDA: CUDA (Compute Unified Device Architecture) is a parallel programming paradigm released in 2007 by NVIDIA. It is designed for graphic processors and a vast array of general-purpose applications for GPUs. CUDA is a proprietary API only supporting NVIDIA's GPUs for Tesla Architecture. The CUDA-supported graphics cards include the GeForce 8 series, Tesla and Quadro. OpenCL: OpenCL (Open Computing Language) was initially developed by Apple and is maintained by the Khronos group for heterogeneous computing, including CPUs, GPUs, DSPs, and other types of processors. This portable language is adaptable enough to allow each hardware platform to achieve high performance, including NVIDIA's GPUs. NVIDIA is now an OpenCL 3.0 conformant and is available on R465 and later drivers. Using the OpenCL API, people can launch compute kernels written using a limited subset of the C programming language on a GPU.


All Types of ML Accelerators

4. ML Compilers ML compilers play a vital role in accelerating training and serving. ML compilers can significantly improve the efficiency of large-scale model serving. There are many popular compilers, such as Apache TVM, LLVM, Google MLIR, TensorFlow XLA, Meta Glow, PyTorch nvFuser, and Intel PlaidML. Please refer to ML Compilers for more details.


All Types of ML Accelerators

5. ML Cloud Services ML cloud platforms and services manage ML platforms in the cloud. They can optimize to improve efficiency in several ways. Take Amazon SageMaker, for example. It is a leading ML cloud platform service. SageMaker provides extensive features for the ML lifecycle, from preparing to building, training/tuning, and deploying/managing. It optimizes many ways for training and serving efficiency, for instance, multi-model endpoints on GPU, cost-effective training using heterogeneous clusters, and proprietary Graviton processors suited for CPU-based ML inference.


All Types of ML Accelerators

Final Remark It becomes more and more challenging as DL training and serving scale. Improving DL training and service efficiency is sophisticated. Based on the ML lifecycle, there are five areas to accelerate ML training and serving: AI frameworks, hardware accelerators, compute platforms, ML compilers, and cloud services. AI engineering can orchestrate all these together for comprehensive efficiency with engineering principles.


Code Reproducibility Crisis in Science And AI

Do not dare to ask The use of software and codes in medicine, biology, and even psychology has exploded in recent years. Almost no scientific article now does not feature the use of code or has amassed a dataset. On the other hand, biology and computer science have elegantly merged into a new science called bioinformatics. In addition, many scientific articles use machine learning models (from simple regression to sophisticated convolutional neural networks ). The first problem is that often, even when authors have developed a new algorithm, it is not published in any repository. Not to mention that often the details of the algorithm are deliberately described in a vague way that hinders anyone who wants to reproduce it. In general, a very small percentage of journals require that both the data and the code be published (research showed that only 7 percent of surgical journals required this). So often, the result is to read in the methods: data not available. On the other hand, journals that require in their guidelines that data be shared make do with a vague statement such as "available upon request" or, my favorite, " data and code available upon reasonable request." Apparently, many of the requests are not reasonable because, very often no response is obtained. Victoria Stodden tested this system by writing to authors of 200 articles published in science to obtain raw data and code. They were only able to get responses to a minimal extent, and I could only replicate the results in 26% of the cases. The methods are also uninformative, as shown by a Mayo Clinic study that analyzed 225 articles noted that only in 12 articles was there enough detail to reproduce how the statistical tests were conducted. Not to mention, the articles where more sophisticated algorithms are used and the model is described in succinct terms (often stuff like "we used a convolutional neural network to...). In addition, another reason why researchers are reluctant to publish datasets is that they do not get citations. In a world where their career depends on the number of citations, better to keep it to themselves and use it for another paper.


The library of forgotten codes Scientific research is based on open-source software. In 2019, an image of a black hole was published that surprised and fascinated the world. Few know that among the libraries used for that image is Matplotlib. Data science and other research fields rely on open-source libraries in Python or R. Underlying many of these libraries are students, Ph.D. students, and postdocs. Often, the code is developed by students who have not received adequate training and do not know the best documentation and testing practices. Researchers often spend hours and hours trying to fix bugs or compatibility issues without finding adequate documentation or answers to their questions. In addition, many of these codes were written with abandoned libraries and deprecated functions (called "dependency hell"). The code is then published by a Ph.D. student or postdoc who is at the end of their contract and abandoned to its fate. As Nelle Varoquaux co-developer of scikit-learn acknowledges, the field is evolving rapidly. The needs of a library are evolving, datasets are getting larger and larger and for example, cannot fit in a RAM. As mentioned often the codes are poorly documented. Many researchers prefer to publish code on their personal GitHub pages, rather than in repositories such as (CRAN and Bioconductor for R, or PyPi for python). In addition, another problem is that research often lacks funding. Agencies often do not value the importance of maintaining code, and the funds that are available are few. In requests for funds, research groups omit the funding request for a software engineer, knowing that it is unlikely to be approved. For example, soon after the black hole image was published, funds were requested to support the scipy ecosystem that made it possible and the grant was rejected.


The artificial intelligent being who could not reproduce itself Does data science fare any better? One would think that those who make code their business should know the importance of reproducibility and best practices for documentation. In fact, the situation is not rosy in data science either. In fact, not all articles published the code or the details of their algorithm. Sometimes, when developing a new algorithm, it is difficult to be able to compare it with a benchmark or the state-of-the-art. As one Ph.D. student at the University of Montreal recounts, they tried to reproduce the code of another algorithm to compare it with their own: We tried for 2 months and we couldn't get anywhere close." Forty-three percent of articles accepted in major conferences cannot be reproduced. This is the result of a survey conducted by Open Review, and technically these articles have passed a double-blind review. Moreover, State-of-AI reported that only 15% of the scientific article reports the code. Sometimes code is not present, methods are obfuscated on purpose, or only pseudocode is presented. Not to mention, that without the correct choice of hyperparameters, the original dataset even with the code would be difficult to be able to reproduce the model. it has been pointed out, that the same random seed changes the performance of the model quite a bit at times. An example, one of the most widely used models is YOLO which is now up to the seventh version, the code is on GitHub but just scroll through read.me (the main page) to find: the excuses for not sharing models are a wide variety: the code is based on unpublished code, the code is protected by interests, and the algorithm is too dangerous (OpenAI and Google's favorite excuse). In fact, Google Health published an article in 2020 where it claimed that its AI was capable of detecting breast cancer lesions in images. The article presented so little information about the code that 31 irritated scientists published a letter in Nature in which they practically branded Google's article not as a scientific article but as an advertisement for a product. In the letter, they practically stated that artificial intelligence is facing a reproducibility crisis. This is no small matter, as the various AI models have real impact and applications. If researchers cannot explore code and models, they cannot be assessed for robustness, bias, and safety. In any case, being able to replicate a model is no small task. Especially when it comes to large models, the computational and knowledge requirements to be able to replicate a model from methods are quite difficult. In fact, Facebook itself had difficulty replicating the results of AlphaGO. The team claimed that succeeding without the available code was an exhausting task (they literally wrote in the article that the task was "very difficult, if not impossible, to reproduce, study, improve upon, and extend"). And if it is difficult for a multinational company with a huge budget, imagine independent laboratories. The data science community does not like to share datasets either. Not least because this could expose them to embarrassing errors: for example, the models used to generate the images were trained on a dataset called LAION, but this is far from perfect. First, it was obtained by massively downloading from the web without asking for any permission. Meanwhile, most are images of artists, and photographers who would not have imagined that their works would be used to train AI models. Inside they even found medical images that should not have been there, photoshopped celebrity porn, stolen nonconsensual porn, and even images of executions done by terrorists.


The legacy to whom comes after us In general, the idea of many researchers is that if you haven't made the most of a dataset, you don't release it. Without Iris datasets, though, who among us would have learned to program? For this reason, NASA grants a period of exclusivity to Hubble Space Telescope users before the data are published in a public data repository and the whole community can benefit (for Webb Space telescope the window is one year). Today, many journals also require genomics data to be published in special repositories (dbGAP, GEO Dataset, and so on). Access to the data is protected by a formal procedure since it is patient medical data and potentially privacy-threatening. Even funding agencies are realizing the need for data to be shared and that curation is important. National Institutes of Health (NIH) in 2020 required that grant applications that had a data component had to specify their data management and sharing plan. Then there are Gordon and Betty Moore Foundation, the Alfred P. Sloan Foundation, and the Chan Zuckerberg Initiative that funds open-source software (for example, the last one funds scikit-image, NumPy, the ImageJ, and Fiji platforms). In addition, recently Schmidt Futures (an organization founded by former Google chief executive Eric Schmidt) decided to fund a network of engineers dedicated to maintaining scientific software (including models related to global warming, and ecological changes). There are also journals that specialize in describing and sharing datasets (such as Scientific Data). Not to mention that repositories such as Zenodo allow you to associate a DOI with code and data and make them citable as well. What about the code? Meanwhile, there are projects that aim to solve data illiteracy. In fact, there are organizations such as Software Carpentry and the eScience Institute (University of Washington) that organize boot camps. Or the Netherlands eScience Center, which has made available guidelines. Universities are beginning to realize that models and code written by students and postdocs may be good for publication but is not maintainable. Until now, libraries were often maintained on a voluntary basis, but it is not sustainable. Therefore, English universities have created the position of a research software engineer who is responsible for working on software that is important for academic research. Also, we recommend using containers (such as docker) for all code that is published so that it can run without all the compatibility issues. There are also repositories for containers specifically for data and biological code (such as bioflow, RNASeq pipeline, and pipeline in R or python). Papers such as Nature Machine intelligence require that the code be available on Code Ocean or Google Colab so that during peer review you can run the code. Leading scientific conferences are also paying more attention to the presence of code, to better control the methods. Many are encouraging that there be tabs on what data was used, in the description of the code. NeurIPS also started linking code when this is present. Not to mention that not only does the community often try to replicate published models, but there is also an initiative called ReScience that is responsible for publishing AI models that have been replicated.


Parting thoughts The founding principle of science is the ability to replicate results. Today science is increasingly based on the use of large datasets, analysis software, and machine learning models. Both data science and science need to advance data and models that are both available and reusable. Also, the main libraries that you import into your code are (scikit-learn, NumPy, matplotilib to give an example). They are developed and maintained by researchers and volunteers and this is not always sustainable. Algorithms have and will have more and more real-life applications and impacts, so it is critical that they are tested by independent labs. Not to mention that the training dataset has a huge impact on the potential bias, and this should also be discussed (and once a model is trained it is hard to make it forget what it should not have known). When a model is an open source, it allows it to be reused. AlphaFold2 would not have had the same impact if it had not been made open source and its predictions made available to the community. For this same reason, we will see many more applications based on stable diffusion than on DALL-E. The good news is that researchers, universities, and funding agencies have realized this.


If you have found it interesting: You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support! Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more. Or feel free to check out some of my other articles on Medium:


How will Generative AI Change the World Down the Road?

1. Vast Digital Content in the Metaverse and NFT Space The Metaverse and NFT space requires vast digital content. In the Metaverse, you can buy and sell existing digital assets, from your favorite video game character to your favorite comic book hero. You may even be able to create a unique NFT version of yourself and sell it. NFTs are not just limited to owning physical objects - they can also be used to create a digital version of anything. The possibilities are endless: you could own an NFT version of any entity or concept in existence - from a marble statue of Donald Trump's head on top of Mount Rushmore (which is already on sale) all the way down to an individual grain of sand from any beach in the world (also currently available). Generative AI makes all these possible. It can instantly create new relevant digital assets and also build unique NFTs based on your descriptions in no time.


2. Music and Songwriting Generation AI can create new music and songs now. This process is already being used today by companies like Spotify, which allows users to choose different genres based on their preferences (e.g., pop rock). However, these choices are still limited since they only allow users access through pre-created libraries available at those websites instead of enabling them to create something unique from scratch. With generative AI, it is possible that you wouldn't need any human intervention when creating your music or writing lyrics for your songs. All you will have to do is give instructions about what type of instrumentation and style you want for your piece; then, the machine will take care of everything else automatically.


3. Autonomous Vehicles Autonomous vehicles (AV) will have a tremendous impact on society, with the ability to change how we travel and live. They'll be able to drive in more challenging weather conditions and traffic conditions - like snow or heavy rain - that would challenge any driver. They can also make decisions faster than humans can, so they won't get stuck at intersections waiting for red lights or stop signs that may never turn green if there's no traffic coming from any direction. AI is the foundation for self-driving autonomous vehicles. Generative AI will be the brain to tackle the above subtle problems. It will extend AV to be a human device (far beyond smartphones) to communicate and learn incrementally with the passengers. Generative AI will eventually change the way we travel.


4. Malicious Hacking Generative AI is a double-edged sword for malicious hacking. Malicious hacking is a cyber-attack where someone gains unauthorized access to a computer system for their gain. In this case, unauthorized access is made by infecting the system with malware (malicious software). The use of generative AI in malicious hacking has been around. Hackers have used the technology to create new malware that can evade current antivirus software and anti-malware solutions. This new generation of malware uses advanced encryption, making it virtually impossible to detect or remove without damaging your computer or system. On the other hand, generative AI can defend against malicious hacking based on continuous learning.


5. Drug Discovery Drug discovery is a long and expensive process. Generative AI has the potential to help speed up this process by finding new drug candidates, discovering new uses for existing drugs, and discovering new drugs for rare diseases. But this is not the process we are familiar with about text-to-image or text-to-video. It will be DrugA-to-DrugB and requirements-to-drug.


6. New Materials and Chemical Synthesis Generative AI will profoundly impact the field of materials science and chemical synthesis. With the capability of generative AI, scientists can design new materials that can solve some of the world's biggest problems. For example, scientists could use this technology to design new types of batteries with longer-lasting capacity than we currently have. The technology could also create better solar panels or superconductors, reducing our reliance on fossil fuels to power our devices and machines. AI-generating advances will likely have enormous impacts on chemical synthesis. For example, if we could develop a material that's 50% more efficient than graphene at conducting heat but is still light enough and flexible enough for use in mobile phones (which are currently limited by their poor thermal conductivity), then this would mean that our phones would become significantly faster and less prone to overheating during intensive tasks such as gaming or video editing.


7. Automatic Electronics Design Generative AI can accelerate automatic electronics design. Automatic electronics design is a computer-aided electronic design that uses a generative adversarial network (GAN) to generate electronic schematics. GANs were first used in 2014, and one study found them to be able to create schematic diagrams with an accuracy exceeding 90%. The recent transformer architecture accelerates this design.


8. AGI Renaissance What is the most meaningful impact of GAI? This opens the door to Artificial General Intelligence (AGI). AGI is strong AI or full AI. It can understand or learn any intellectual task that a human being can. This is our ultimate goal in developing artificial intelligence. According to OpenAI and DeepMind, generative AI (GAI) is a significant milestone in the development of AGI. The success of GAI will ignite the AGI renaissance. It is just beginning, but it will be a giant leap in AI evolution and human history.


Final Remark In a nutshell, we can see that Generative AI is a technology that will significantly impact our lives. The benefits are clear. It will make things easier for us and save us time by creating music and writing books for us or improving our health through better drugs and medicines. But there are also downsides - for example, malicious hackers could use this to do bad things like taking over control of other computers without anyone knowing what happened or even detecting when someone has been hacked! It will be interesting to see how both sides work out over time...


This AI newsletter is all you need #18

What happened this week in AI by Louis You must've heard about Stability AI and its recent $101 million funding for open-sourced AI, which is such good news for all of us in AI. As exciting as it can be, this shows how much power open-source can have and how promising a company based around that is. I hope this kind of news will propel the open-source way for our field and I deeply hope the big companies will follow this path. I believe giving knowledge away (for free) is much more valuable to a company in the end than selling a product that you are the only one contributing to, where the open-source community ends up catching up to you, forcing you to iterate and compete constantly. In today's world, you can get help from anyone, anywhere, so why not do so? Why not help science progress and get your products even better by having thousands of hands getting into your work and improving it as Stability AI did with Stable Diffusion? Look what it did. Stable Diffusion is even better and more popular than DALLE now. Speaking of which, what is even more exciting than open-source getting funding and support right now is diffusion-based models. There are new papers every day using such diffusion models to generate images, generate 3D models, generate videos, edit images from sketches or from text and more. It is just incredible and has completely replaced GANs or transformers, but are they scalable? I'd love to know what you guys think. Is diffusion here to stay and build upon, or is it just another step towards a better approach to come within a few months?


This AI newsletter is all you need #18

Hottest News Google's new AI can hear a snippet of song - and then keep on playingThis new technique is based on a paper and framework called AudioLM, used for high-quality audio generation with long-term consistency. It maps the input audio to a sequence of discrete tokens and casts audio generation as a language modeling task in this representation space!A dataset of 5,85 billion CLIP-filtered image-text pairs!The dataset LAION-5B is 14x bigger than LAION-400M, previously the biggest openly accessible image-text dataset in the world.Deloitte's fifth edition of its State of AI in the Enterprise research report is out!Deloitte "surveyed more than 2,600 global executives on how businesses and industries are deploying and scaling artificial intelligence (AI) projects. Most notably, the Deloitte report found that while AI continues to move tantalizingly closer to the core of the enterprise - 94% of business leaders agree that AI is critical to success over the next five years - for some, outcomes seem to be lagging."


This AI newsletter is all you need #18

Most interesting papers of the week Compressed Vision for Efficient Video Understanding"We propose a framework enabling research on hour-long videos with the same hardware that can now process second-long videos."TOKEN MERGING: YOUR VIT BUT FASTERA simple method to increase the throughput of existing ViT (visison transformer) models without needing to train.Museformer: Transformer with Fine- and Coarse-Grained Attention for Music GenerationA Transformer with a novel fine- and coarse-grained attention for music generation. Transformer with a novel fine- and coarse-grained attention for music generation. Enjoy these papers and news summaries? Get a daily recap in your inbox!


This AI newsletter is all you need #18

Featured Community post from the Discord Shubham Trivedi just shared a notebook on market segmentation and is looking for your feedback! If you have a few minutes available, please have a look and let him know what you think (Shubham.Trivedi#1648 on discord)!


This AI newsletter is all you need #18

Towards AI Tutorials At Towards AI, we are big fans of Tutorial & Practical content with code and examples to help learn, understand and implement AI and Machine Learning models and applications. In pursuit of this goal, we have commissioned a series of blog posts (with accompanying Python code and Google Colab files) on various Machine Learning algorithms and topics from our Technical Editor Pratik Shukla. The first (of many) mini series focuses on the Gradient Descent algorithm, from fundamental principles through to a comparison of the variants with elaborated code examples in Python: The Gradient Descent AlgorithmMathematical Intuition behind the Gradient Descent AlgorithmThe Gradient Descent Algorithm & its Variants We are also establishing a new Tutorials & Practicals page on our website, where we will host this type of content, both published by our own editorial team and submitted from our contributors, and work to improve its organization and accessibility. We also want to provide a better forum for authors and readers to interact on this content and for readers to ask questions or help as they work through the project. As such, we are also establishing a new Towards AI Tutorials and Practicals forum in our Learn AI Together Community. So please join if you have any questions, let's learn together!


This AI newsletter is all you need #18

Article of the week Gradient Descent Optimization by Rohan Jagtap For another perspective on Gradient Descent, by chance this week one of our contributors published another Tutorial together with implementation of a gradient descent optimizer in TensorFlow.


This AI newsletter is all you need #18

Our must-read articles GELU: Gaussian Error Linear Unit Code (Python, TF, Torch) by Poulinakis Kon Overview of Computer Vision Tasks & Applications by Youssef Hosni Why Should Euclidean Distance Not Be the Default Distance Measure? by Harjot Kaur If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #18

Job offers Machine Learning Engineer, Copilot Model Improvements @ Github (Remote, US) Sr. Data Scientist, Marketplace and Merchandising @ Splic.com (Remote, US) Machine Learning Engineering Manager @ Verana Health (Remote) AI Implementation Manager (Healthcare) @ ClosedLoop (Remote) Machine Learning Engineer, Exploratory Projects, Information Abstraction @ Cohere (Flexible) Machine Learning Engineer @ Weights and Biases (Remote) Senior/Staff Machine Learning Engineer, Infrastructure & Earnin (Remote) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net or post the opportunity in our #hiring channel on discord! If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


4 Cognitive Biases In AI/ML Systems

Selection Bias Selection bias refers to the selection of training/testing data that is not representative of the entire population.Example: An engineer chooses the first 100 volunteers who responded to his email as his training dataThe problem: The first 100 respondents may be more enthusiastic about a product or study than the last 100 respondents. By explicitly choosing the first 100 respondents, the engineer introduces unfairness in his data collection method.The solution: select a random sample of 100 users from your pool of email respondents instead of the first 100.


4 Cognitive Biases In AI/ML Systems

Reporting Bias Reporting bias refers to people's (conscious and unconscious) tendencies to suppress the information they report.Example: Many amazon products have more 5-star and 1-star reviews than 2, 3, or 4-star reviews because people who have extreme experiences (either positive or negative) are more likely to post a review than those who have neutral experiences.The problem: An engineer that uses online reviews as their primary data source may create an AI model that is great at detecting extreme sentiments but not so much at detecting more neutral, subtle sentiments.The solution: Consider broadening the data collection scope to account for the underrepresented data.


4 Cognitive Biases In AI/ML Systems

Implicit Bias Implicit bias refers to people's unconscious tendencies to make assumptions or associate stereotypes with others.Example: An engineer who is often bitten by dogs believes that dogs are more aggressive than cats, even though that may not be scientifically true.The problem: The engineer would believe that the ground truth is "dogs = aggressive" and thus fine-tuned her AI model to label dogs as being more aggressive than cats.The solution: As implicit bias is unconscious to an individual, having multiple engineers code the AI and establishing proper peer review procedures would reduce the occurrence of such biases.


4 Cognitive Biases In AI/ML Systems

Framing Bias Framing bias refers to people's tendency to be influenced based on how information is presented.Example: An engineer who sees a dark and dull website for a product believes that the product must have poor sales, ignoring the actual positive sales number of the product.The problem: In designing the AI algorithm, the engineer may take into account subjective variables, such as the color of a website, instead of focusing on objective metrics.The solution: Avoid subjective (usually qualitative) data and prioritize objective, factual data instead.


4 Cognitive Biases In AI/ML Systems

Summary and Best Practices A quick recap of the 4 cognitive biases in AI/ML systems Selection Bias (selecting data not representative of the entire population)Reporting Bias (people's tendency to underreport information)Implicit Bias (people's unconscious tendencies to assume)Framing Bias (people's tendency to be affected by how information is presented) The 5 best practices to enhance objectivity in AI/ML systems Always select a random sample (instead of the first or last hundred data points)Verify your data sources through comparison with other data sourcesAssign more (diverse) engineers to develop the AI/ML systemEstablish proper peer review procedures to cross-examine logic and unconscious biasPrioritize objective and factual data over subjective (usually qualitative) data.


10 AI Websites That Will Excite You to The Core!

1. VERSE BY VERSE Are you struggling to write a nice poem for your girlfriend/boyfriend or your loved ones? Don't worry !! Verse by Verse will completely help you to compose poems for yourself. With the help of Verse by Verse, you may create a poem using ideas from some of America's most well-known poets, including Dickinson, Whitman, Poe, Wheatley, Longfellow, and others. They trained AI systems to function as your muse while you write a poem of your own, providing recommendations in the manner of each individual poet to make this possible. Here is the link! enjoy and stretch your creativity with the help of legendary poets!


2. DALL·E 2 If there is a discussion about amazing websites in AI, then how can anyone forget DALL·E 2? Do you want to make a portrait or drawing or create an image of your complicated thoughts? But is it hard for you to draw it on paper? What if you can just describe your thoughts in words/texts and create images/drawings? Don't worry, DALL·E 2 will help you here in the same fashion! According to openAI blog post, OpenAI's DALL·E 2 is a 12-billion parameter version of GPT-3 trained to generate images from text descriptions, using a dataset of textimage pairs. We've found that it has a diverse set of capabilities, including creating anthropomorphized versions of animals and objects, combining unrelated concepts in plausible ways, rendering text, and applying transformations to existing images. Previously it had a waitlist. After registering, you have to wait for your turn, but recently it is free from the waitlist, and you can directly log in and create images from texts. You will get 50 credit points after registration and 15 points for each month. Each search will be equal to 1 point.


3. Nightcafe It works on a stable diffusion model, which will be released in 2022. Although it can be used for various tasks, including inpainting, outpainting, and creating image-to-image translations directed by text prompts, its primary usage is to generate detailed visuals conditioned on text descriptions. How does it work? According to the stability.ai blog, The model itself builds upon the work of the team at CompVis and Runway in their widely used latent diffusion model combined with insights from the conditional diffusion models by our lead generative AI developer Katherine Crowson, Dall-E 2 by Open AI, Imagen by Google Brain and many others. We are delighted that AI media generation is a cooperative field and hope it can continue this way to bring the gift of creativity to all.


4. Let's Enhance Want to make your distorted and blurry photos beautiful? you can use let's enhance! It makes low-res images into clear and sharp pics to be proud of, using cutting-edge image processing algorithms.


5. QuillBot AI If you are unable to phrase your words and struggling to write, you can use the Paraphrasing tool of QuillBot AI. the difference between other rephrasing tools is it uses state-of-the-art AI to predict, phrase, and rewrite your thoughts. It also gives us many options for your texts to a phrase, including synonyms analysis, grammar check, etc. It is very useful for new writers who want to write but don't know what perfect phrasing can be! https://quillbot.com/


6. Talk to books Talk to the book is one of the most amazing websites that I have found! you can talk to a book like in the same way as we talk to humans. you can ask anything on the dialog box, and it will return the extracted semantic pieces of answers that fit best to your question. How does it work? As research google blog says, the approach was to use billions of lines of dialogue to teach an AI how real human conversations flow. Once the AI has learned from that data, it is then able to predict how likely one statement would follow another as a response. In these demos, the AI is simply considering what you type to be an opening statement and looking across a pool of many possible responses to find the ones that would most likely follow.


7. Teachable Machine Is model training quite expensive right? AI-ML-DL models require training which must have been faster and less expensive. what if we get an online website that will assist you with the training image, sound & pose data? that's actually cool!


8. The Person Does Not Exist It is so cool that AI can generate images of people that are not alive and have never existed in the world. It uses GAN ( Generative Adversarial Network) generated real "FAKE PEOPLE". check the site and refresh the page. It generates a fake image of no-existing people in 5 seconds.


9.Resemble.ai Resemble's AI voice generator lets you create human-like voiceovers in seconds. EMOTIONS-Add an infinite amount of emotions to your voice without any new data. Happy, sad, angry, all preloaded, out of the box. SPEECH-TO-SPEECH Transform your voice into the target voice with real-time speech-to-speech. Granular control over every inflection and intonation. 10.FACE SWAPPER If you want to see how swapping faces will look then this is an amazing website for it. You can swap faces with celebrities as well as you can upload your own images and check for swapping. How does it work? according to the Face swapper blog, Face Swapper uses artificial intelligence to detect faces on photos and replaces them with other faces of your choice. For replacements, you can use photos of your own or choose any photo you like from our gallery.


If you have found this article insightful If you found this article insightful, follow me on Linkedin and medium. you can also subscribe to get notified when I publish articles. Let's create a community! Thanks for your support!


If you want to support me : As Your following and clapping is the most important thing, but you can also support me by buying coffee. COFFEE.


You can also read my blogs related to [image by author created by Dall. E]


Google UniTune: Text-driven Image Editing

If you have found it interesting: You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support! Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more. Or feel free to check out some of my other articles on Medium:


Deep Learning-Based Deepfake Detection in a Nutshell

1. What Is Deepfake Detection The deepfake creation pipeline is not a perfectly monitored process. Hence, most Deepfake generators leave fingerprints in Deepfakes that are specific to the relevant Deepfake creation architecture or the individual generator. These variations in Deepfake videos can be classified as spatial inconsistencies: incompatibilities that occur within individual frames of the video and temporal inconsistencies: the incompatible features that occur across the sequence of frames of the video [1]. Spatial inconsistencies include facial region incompatibilities with the background of the video frames, resolution variations, and partially rendered organs and skin textures (all the human features of the face might not be rendered correctly). Most common Deepfake generators fail to render characteristics such as eye blinking and teeth. Sometimes, the Deepfake generation uses white strips instead of teeth that are even visible to the naked eye on still frames [Figure 2]. Temporal inconsistencies include abnormal eye blinking, head poses, facial movements, and variations in luminance in the frame sequence across the video. Fortunately, both spatial and temporal fingerprints left by deepfake generators could be identified by deepfake detectors made of Deep Neural Networks (DNNs), i.e., the principle behind the deepfake detection process. Nevertheless, the extensive application of Generative Adversarial Networks (GANs) in Deepfake generators has challenged the balance between deepfake detection and creation.


2. Deepfake Detection Pipeline Deepfake detectors are binary classification systems that output whether the input digital media is real or fake. Deepfake detection is not carried out by one single black box-like module, but it comprises several other modules and steps which function together to deliver the detection result. The common steps in the Deepfake detection pipeline are as follows [2]. The input of the Deepfake digital media.The preprocessing includes face detection and augmentation.The feature extraction of the processed frames.The Classification/Detection.Output the authenticity of the image. Generally, a typical DL-based Deepfake detector comprises 03 major components to perform the above tasks. A preprocessing module.Feature extraction module.Evaluator module (deep learning classifier model). In the next 03 chapters, the major steps: Data preprocessing, feature extraction, and detection/classification processes are explained in detail.


3. Data Preprocessing After the data collection phase, data should be preprocessed before being used in the training and testing steps of the Deepfake detection pipeline. The data preprocessing is done automatically using libraries available such as OpenCV python library, Multi-task Cascaded Convolutional Networks (MTCNN), and You Only Look Once (YOLO) algorithm, etc. Preparation of the data set to train the model also plays a vital role in the performance of a Deepfake detector. Enhancing techniques such as rescaling (stretching), shear mapping, zooming augmentation, rotating, brightness change, and horizontal/vertical flipping in suitable ranges, can be applied in order to increase the generalization of the data set [3]. The first step in data preprocessing involves extracting individual frames from the video clips. After the frames are extracted, the next steps involve the detection of faces from the extracted video frames. Since the abnormalities frequently occur in facial regions, selecting only the facial regions helps the feature extraction model to focus only on the Regions of Interest (ROI), saving the computation cost that was to be used on full-frame scanning. Once the facial regions are detected, they are cropped from the rest of the background of the frame and follow a series of steps to make them be usable in the model training and testing. Another reason to crop the facial regions is to make all the input images to the model at the same size.


4. Feature Extraction The preprocessed frames are then fed to the feature extractor. Most of the feature extractors are Convolutional Neural Networks (CNN)-based. It is a new trend that some of the recent research has been carried out to prove the improved effectiveness and efficiency in the application of Capsule Networks on the feature extraction process. The feature extractor extracts the spatial features available on the preprocessed video frames. The feature extract is capable of extracting vision features, local features/ facial landmarks such as the position of the eye, nose, mouth, dynamics of the mouth shape, biological features such as eye blinking, etc. The extracted feature vectors are then sent to the classifier network to output the decision.


5. Classification The deep learning model used for classification is often called the backbone of the Deepfake detector. As the name suggests, the classification network is responsible for the most serious task of the Deepfake detection pipeline: i.e., classifying and determining the probability of being the input video is Deepfake or not. Most classifiers are binary classifiers where output is (0) for Deepfakes and (1) for pristine frames. The classifier is again another convolution layer (a CNN) or different deep learning architecture such as Long Short-Term Memory (LSTM) networks and Vision Transformers (ViTs). The actual functionality of the classification model varies from DNN used. As an example, the eye-blinking features extracted in the feature extractor module can be used by the LSTM module in the classification module in order to determine the temporal inconsistencies of the eye-blinking patterns of the frames and, based on that, decide whether the input is a Deepfake or not [3]. In most cases, a fully connected layer can be observed in the Deepfake detector networks. Since the output from convolution layers denote high-level features of the data, those outputs can be flattened and connected to a single output layer to result in the ultimate decision.


6. Summary There has been a significant competitive development in both Deepfake creation and detection in the past few years. Research related to Deepfake detection with deep learning techniques is heavily influenced due to the accuracy of the results compared to non-DL approaches. Deep Neural Network architectures like CNNs, RNNs, ViTs, and Capsule Networks are widely employed in the implementation of Deepfake detectors. The common Deepfake detection pipeline consists of a data preprocessing module, a CNN-based feature extractor, and the backbone classification module. Further, there is a significant dependency of Deepfake detection on the fingerprints left by Deepfake generators on the Deepfakes. As the present GAN-based Deepfake generators are capable of synthesizing more realistic Deepfakes with minimum inconsistencies, new DL-based approaches must be developed to optimize Deepfake detection. Deepfake detection approaches based on Deep-Ensemble Learning techniques can be identified as such modern and comprehensive approaches to combat Deepfakes [4]. Nonetheless, the vacancy for an effective and efficient Deepfake detector is still open.


7. References [1] Ruben Tolosana, Ruben Vera-Rodriguez, Julian Fierrez, Aythami Morales, Javier Ortega-Garcia, DeepFakes and Beyond: A Survey of Face Manipulation and Fake Detection (2020), Information Fusion, 2020. [2] Nguyen, Thanh & Nguyen, Cuong M. & Nguyen, Tien & Nguyen, Duc & Nahavandi, Saeid. Deep Learning for Deepfakes Creation and Detection: A Survey (2019). [3] P. K. P. K. Sawinder Kaur, Deepfakes: temporal sequential analysis to detect face-swapped video clips using convolutional long short-term memory (2020), Journal of Electronic Imaging, p. 29(3), 2020. [4] M. S. Rana and A. H. Sung, DeepfakeStack: A Deep Ensemble-based Learning Technique for Deepfake Detection (2020), 7th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2020 6th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom), 2020.


This AI newsletter is all you need #17

What happened this week in AI by Louis Designer was announced this week. A DALLE-powered product by Microsoft for creating designs with AI's help. It's a similar platform to Canva but uses even more AI to generate custom images rather than pre-available ones. I believe it is just the first of many to use generative AI in a product, just like all the GPT-3-based companies that were created. The gist of Designer is to spam for generations until you get the right one, as the results are not perfect, especially not on the first run, which is one approach of AI design companies. The other option would be to wait for the perfect results before using it, so the user doesn't have to work. I'd love to know about your preferred approach for creating AI-based content. Would you mind having to click refresh a few times before getting something great, or would you rather wait indefinitely for a model that provides good-looking results every time? Another question is: would you rather have an AI you can spam for generations, or get only generations "approved" by a real/trustable designer/artist to give you confidence that the generation does indeed look great? I know that some of us (like me) have a pathetic eye for design quality.


This AI newsletter is all you need #17

Hottest News An AI Music Generator based on Diffusion!StabilityAI, the company behind Stable Diffusion, announced an "AI Music Generator" called Harmonai based on the "Dance Diffusion" Model, which is basically a stable diffusion model for generating sound instead of images.Designer, a Microsoft AI design app powered by OpenAI's DALL-E 2Yes, Microsoft is opening up access to a Canva-like web app for free to users on a waitlist. Just type in a text description of the image you want, and the AI will generate examples that they can add to your design. This is pretty scary news for Canva and Adobe!A podcast entirely generated by artificial intelligence"Every week, we explore a new topic in depth, and listeners can suggest topics or even guests and hosts for future episodes. Whether you're a machine learning enthusiast, just want to hear your favorite topics covered in a new way or even just want to listen to voices from the past brought back to life, this is the podcast for you."


This AI newsletter is all you need #17

Most interesting papers of the week EVA3D: COMPOSITIONAL 3D HUMAN GENERATION FROM 2D IMAGE COLLECTIONS"EVA3D, an unconditional 3D human generative model learned from 2D image collections only able to sample 3D humans with detailed geometry and render high-quality images (up to 512×256) without bells and whistles (e.g. super-resolution)."UNDERSTANDING HTML WITH LARGE LANGUAGE MODELS"We show that LLMs pretrained on standard natural language corpora transfer remarkably well to HTML understanding tasks."Reconstructing Personalized Semantic Facial NeRF Models From Monocular VideoA novel semantic model for the human head defined with a neural radiance field. The 3D-consistent head model consists of a set of disentangled and interpretable bases, and can be driven by low-dimensional expression coefficients. Enjoy these papers and news summaries? Get a daily recap in your inbox! This week we would like to highlight an exciting new AI application from our friends at Kinnu: In a collaboration with Deepmind's head of product, a new startup is using artificial intelligence to increase human intelligence. A new learning startup called Kinnu is implementing AI to find "cognitive shortcuts" and increase the effectiveness of human learning. They see the future of accelerated learning as a melding of neuroscience and AI. They're testing the approach with a free course on AI, co-authored with Deepmind's very own head of product. Kinnu is also experimenting with entirely AI-generated learning material (check out the French Revolution course to see what that looks like). Is this the future of education, like Dall-E for learning content?


This AI newsletter is all you need #17

Featured Community post from the Discord One of our members just started a personal project I definitely support: writing! The user in question, friscons93#3360 (or Davide), just published his first article ever:"I just wanted to share with you my first data science article on medium: it's about aqueous solubility prediction of drugs by using Machine Learning and PCA." Read Davide's article!


This AI newsletter is all you need #17

Articles of the week Outline a Smaller Class With the Custom Loss Function by Konstantin Pluzhnikov The accuracy metric may not always be adequate to produce the desired results. We could want to lower the rate of false positives or false negatives. When a dataset is unbalanced, and the answer we're looking for falls into a narrower class, it can be useful. The author gives an overview of how to maximize classification while working with unbalanced datasets.


This AI newsletter is all you need #17

Our must-read articles How To Create a Siamese Network With Keras to Compare Images by Pere Martra A Simple Adjustment Improves Out-of-Distribution Detection for Any Classifier by Ulyana Tkachenko DeepMind's AlphaTensor: Deepmind's Alphatensor: The AI That Is Reinventing Math by Salvatore Raieli If you are interested in publishing with Towards AI, check our guidelines and sign up. We will publish your work to our network if it meets our editorial policies and standards.


This AI newsletter is all you need #17

Ethical Take on AI's role in the labor market by Lauren We often mention how new AI technologies can be disruptive to workers, such as image generation tools potentially disrupting the work of visual arts workers such as graphic designers, photographers or painters. But it's important to note that many labor markets have historically adapted along with new AI advancements (speaking from a U.S. perspective). Many thought that Photoshop was the end for graphic design, and then graphic designers became the experts on it. If you're an engineer that works with designers and artists, you know how invaluable the artwork is in bringing code to life through beauty and storytelling. As an editor, Grammarly certainly hasn't put me out of a job! The good news is that there's a whole area of AI ethics that studies the impact of AI on various labor markets, and this gives us incredible insight into the jobs that will change, the jobs that will be created, and the economic and social impacts due to novel AI.


This AI newsletter is all you need #17

Job offers Machine Learning Engineer @ Weights and Biases (Remote) Senior/Staff Machine Learning Engineer, Infrastructure & Earnin (Remote) Research Scientist in ML for Climate Modeling @ The Allen Institute for AI (Hybrid Remote) Senior AI Software Engineer @ Spot AI (Remote) Senior Machine Learning Researcher for Copilot @ Github (Remote) Senior Software Engineer @ Captur (Remote, +/- 2 hours UK time) Interested in sharing a job opportunity here? Contact sponsors@towardsai.net or post the opportunity in our #hiring channel on discord! If you are preparing your next machine learning interview, don't hesitate to check out our leading interview preparation website, confetti!


3D Models from Text!

Watch the video We've seen models able to take a sentence and generate images. Then, other approaches to manipulate the generated images by learning specific concepts like an object or particular style. Last week Meta published the Make-A-Video model that I covered, which allows you to generate a short video also from a text sentence. The results aren't perfect yet, but the progress we've made in the field since last year is just incredible. This week we make another step forward. Here's DreamFusion, a new Google Research model that can understand a sentence enough to generate a 3D model of it. You can see this as a DALLE or Stable Diffusion but in 3D. How cool is that?! We can't really make it much cooler. But what's even more fascinating is how it works. Let's dive into it. If you've been following my work, DreamFusion is quite simple. It basically uses two models I already covered: NeRFs and one of the text-to-image models. In their case, it is the Imagen model, but any would do, like stable diffusion. As you know, if you've been a good student and watched the previous videos, NeRFs are a kind of model used to render 3D scenes by generating neural radiance fields out of one or more images of an object. But then, how can you generate a 3D render from text if the NeRF model only works with images? Well, we use Imagen, the other AI, to generate image variation from the wanted text! And why do we do that instead of directly generating 3D models from text? Because it would require huge datasets of 3D data along with their associated captions for our model to be trained on, which would be very difficult to have. Instead, we use a pre-trained text-to-image model with much less complex data to gather, and we adapt it to 3D! So it doesn't require any 3D data to be trained on, only a pre-existing AI for generating images! It's really cool how we can reuse powerful technologies for new tasks like this when interpreting the problem differently. So if we start from the beginning: we have a NeRF model. As I explained in previous videos, this type of model takes images to predict the pixels in each novel view, creating a 3D model by learning from image pairs of the same object with different viewpoints. In our case, we do not start with images directly. We start with the text and sample a random view orientation we want to generate. Basically, we are trying to create a 3D model by generating images of all possible angles a camera could cover, looking around the object, and guessing the pixels' colors, densities, light reflections, etc. Everything needed to make it look realistic. Thus, we start with a caption and add a small tweak to it depending on the random camera viewpoint we want to generate. For example, we may want to generate a front view so that we would append "front view" to the caption. On the other side, we use the same angle and camera parameters for our initial, not trained, NeRF model to predict the first rendering. Then, we generate an image version guided by our caption and initial rendering with added noise using Imagen, our pre-trained text-to-image model, which I further explained in my Imagen video if you are curious to see how it does that. So our Imagen model will be guided by the text input as well as the current rendering of the object with added noise. Here, we add noise because this is what the Imagen model can take as input. It needs to be part of the noise distribution it understands. Then, we use the model to generate a higher-quality image. Add the image used to generate it, and remove the noise we manually added to use this result to guide and improve our NeRF model for the next step. We do all that to better understand where in the image the NeRF model should focus its attention to produce better results in the next step. And we repeat all that until the 3D model is satisfying enough! You can then export this model to mesh and use it in a scene of your choice! And before some of you ask, no, you do not have to re-train the image generator model - as they say so well in the paper: it just acts as a frozen critic that predicts image-space edits. And voilà! This is how DreamFusion generates 3D renderings from text inputs. If you'd like to have a deeper understanding of the approach, have a look at my videos covering NeRFs and Imagen. I also invite you to read their paper for more details on this specific method. Thank you for reading the whole article. I invite you to also watch the video at the top of the article to see more examples! I will see you next week with another amazing paper!


3D Models from Text!

References ►Read the full article: https://www.louisbouchard.ai/dreamfusion/ ►Poole, B., Jain, A., Barron, J.T. and Mildenhall, B., 2022. DreamFusion: Text-to-3D using 2D Diffusion. arXiv preprint arXiv:2209.14988. ►Project website: https://dreamfusion3d.github.io/ ►My Newsletter (A new AI application explained weekly to your emails!): https://www.louisbouchard.ai/newsletter/


Google's Audiolm: Generating Music by Hearing a Song's Snippet

If you have found it interesting: You can look for my other articles, you can also subscribe to get notified when I publish articles, and you can also connect or reach me on LinkedIn. Thanks for your support! Here is the link to my GitHub repository, where I am planning to collect code and many resources related to machine learning, artificial intelligence, and more. Or feel free to check out some of my other articles on Medium:


A Simple Adjustment Improves Out-of-Distribution Detection for Any Classifier

Background Identifying outliers in test data that do not stem from the distribution of the training data is critical for deploying reliable machine learning models. While many special (e.g., generative) models have been proposed for this task of Out-of-Distribution (a.k.a. anomaly/novelty) detection¹ ² ³, these are often specific to particular data types and nontrivial to implement. Instead, simpler methods for OOD detection that use an already-trained classifier on data with class labels have become quite popular⁴ ⁶. Methods like KNN distance⁵ ⁷ and Mahalanobis Distance³ ⁴ leverage a trained neural network's intermediate feature representations to identify OOD examples. An even simpler approach is to only use the predicted class probabilities output by the trained classifier and quantify their uncertainty as a measure of outlyingness. Two particularly popular OOD methods are Maximum Softmax Probability (MSP)⁶ or Entropy⁴ ⁵. Compared to most other methods, MSP and Entropy need less information from the model and require less computing to identify outliers. Here we introduce a simple improvement to these baseline methods to improve their effectiveness.


Baseline prediction-based OOD detection methods Consider an image x and classifier model p = h(x) where p is the model's predicted probability vector that this image belongs to each class k ∈ {1,...,K}. Based on p, one can compute two simple OOD scores for x. Maximum Softmax Probability (MSP) - quantifies how confident the model is in the most likely class it predicts: Entropy - quantifies how evenly spread the model's probabilistic predictions are amongst all K classes: These scores have been shown to work surprisingly well for detecting OOD images⁶, despite the fact that they do not explicitly estimate epistemic uncertainty⁴.