What Does ChatGPT Say About Machine Learning Trend and How Can We Prepare For It?
Updated: Feb 11, 2023
ChatGPT has become a recent hype in Artificial Intelligence and Machine Learning, as this chatbot gives fast responses and unexpectedly human-like conversations with high accuracy. As an AI and ML tool itself, what does it say about the current trends of Machine Learning? Here are the top 3 answers given by ChatGPT:
In short answer: 1. Automated Machine Learning (AutoML); 2. Deep Learning; 3. Explainable AI (XAI)
Machine learning, especially practical ML, has evolved from mastering foundational skills such as math and statistics, to the growing trend in expanding and integrating with various disciplines, such as data engineering, software engineering, operations, ethics and security etc.
Since enterprises are continuously scaling ML projects and applying it to critical decision making processes, I believe that ChatGPT has given us the reasonable answers of the increasing trend in AutoML, Deep Learning and Explainable AI.
Each field is a very broad knowledge domain that requires years of learning to master the skill. In this article, I would like to share some learning materials that may help to get your hands dirty in the field quickly. These materials are generally a combination of code implementation + case study + brief theory and background.
For a video walkthrough, check out the complementary YouTube video.
AutoML and MLOps
Automated Machine Learning (AutoML) falls within the broader discipline of machine learning operations (MLOps), which integrates machine learning (ML) workflow with software development operations (DevOps). MLOps includes the entire workflow from data collection, to data preprocessing, to model building, to model deployment and maintenance. Whereas AutoML primarily focuses on automating the model building phases within this entire lifecycle and it typically handles feature engineering, hyper parameter tuning, model selection etc. Personally I think Machine Learning Operations (MLOps) is a more trending and also more comprehensive domain than AutoML. This is because even though machine learning has been introduced to enterprises and institutes for decades, a high percentage of ML projects still fail to be deployed into production. Therefore, there are more efforts putting into MLOps and ML in production in recent years. The Google Trend also indicates an outstanding growing speed of MLOps topic in the last 5 years.
How Can We Get Started in MLOps?
1. GitHub Repository - “Awesome MLOps”
It is indeed an awesome GitHub repository that collects a large amount of blogs, books, documentations related to MLOps. This collection surrounds the topics of model testing, deployment, monitoring; data engineering, software engineering; workflow management, process management and more.
If you are getting started with MLOps, I found this a great resource to gain an overview of various subtopics in this field by reading one or two short articles on each subtopic for a basic understanding.
Some articles mentioned in this repository that I recommend as a MLOps beginner:
2. Book - “Building Machine Learning Pipelines” by Hannes Hapke and Catherine Nelson
This book primarily introduces the MLOps framework - TensorFlow Extended (TFX) that allows bringing machine learning pipelines into production. The first part of the book covers implementing the typical ML workflow using TFX which typically starts from data ingestion, to data validation, to data preprocessing, to model training, to model analysis and validation, to model deployment and maintenance. The second part of the book focuses on orchestrating TFX ML pipelines using Apache Beam, Apache Airflow and Kubeflow. The authors explains this part with both code snippets and screenshots of the interactive user interfaces.
Overall, the authors organize MLOps concepts in a logical structure and demystify productionizing ML pipeline with practical examples, which makes the concepts easy to follow. In addition to this book, the TensorFlow Extended (TFX) YouTube playlist serves as a perfect complement of this book. I recommend watching the videos first and using the book for digging deeper into specific topics.
3. Book - “Practical MLOps” by Noah Gift and Alfredo Deza
The book covers a wide range of fundamental concepts and toolings in MLOps. Especially the first chapter “Introduction to MLOps” talks about the hierarchical need of machine learning engineering, and provides a roadmap of different skillsets required for mastering MLOps. According to the authors, MLOps is composed of 25% DevOps, 25% Data, 25% Model, 25% Business, and it is the mixture of these domains that makes a scalable and efficient MLOps pipeline in production.
The following chapters, the book dives into the topics, such as bash and command line, containerization, AutoML and continuous integration(CI)/continuous delivery(CD) etc. Additionally, it covers how to implement MLOps through three largest public cloud providers - AWS, GCP and Azure.
The author Noah Gift also hosts a YouTube channel “PragmaticAI”, where he walkthroughs each chapter in this book with code demos and examples. I highly suggest checking this book or series out if you would like to have a comprehensive understanding of various toolings involved in the MLOps process.
Deep Learning
There is no doubt that deep learning has became a hot topic for recent decades. As you can see from Google Trend, the topic has remained at top search volume for at least 5 years already.
Specifically, computer vision and natural language processing (NLP) are two deep learning fields that yield major progress recently. To start with these fields, I do think that it is crucial to have a fundamental understanding of basic machine learning algorithms first, then generalize and extend the knowledge to deep learning. Here I have curated some resources that may help to reduce the learning curve.
How Can We Get Started in Deep Learning?
1. Book - “Hands-on Machine Learning with Scikit-Learn, Keras and Tensorflow” by Aurelien Geron
This book begins by walking through popular machine learning algorithms using scikit-learn, such as classification models (e.g. decision tree) , regression models (e.g. linear regression), ensemble learning (e.g. random forest), unsupervised learning algorithms (e.g. clustering). Then neural networks and deep learning are introduced in the second half of the book, primarily utilizing Tensorflow framework and Keras API. The book covers a wide range of topics, including:
Computer Vision using Convolution Neural Network (CNN)
Natural Language Processing (NLP) using Recurrent Neural Network (RNN)
Autoencoder and Generative Adversarial Network (GAN) for data representation
Reinforcement learning
This book has successfully coupled intuitive visual illustrations with practical use cases. The author does not discuss these topics in depth, which makes it a perfect fit for deep learning beginners. However, if you would like to dive deeper into certain deep learning topics, I recommend using this book as a starting point and branches out to more specialized areas.
2. Online Course - “Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning” by DeepLearning.AI
This course is created by DeepLearning.AI which is a platform founded by Andrew Ng and covers a wide range of deep learning specializations (mostly also available on Coursera).
The course starts with an overview of deep neural network structure and basic components, such as tensor, activation functions, multi-layers neural networks. Afterwards, it primarily focuses on computer vision and convolution neural network (CNN), introducing different layers in CNN - filtering layer, pooling layer, dense layer etc.
With the support of the course material, you get your hands on creating your own deep learning models in a relatively rapid speed.
Please note that this is a beginner friendly course for those who have just stepped into deep learning and computer vision in particular. It does not involve cloud computing and most coding exercises are implemented on Google Colab. If you are interested in advanced topics and cloud based practices, I recommend checking out other courses in their “Deep Learning Specialization”.
Explainable Artificial Intelligence (XAI)
Explainable Artificial Intelligence (XAI) is a reasonably young field but catches a lot of attention lately, as more black box models are emerging. These black box models are increasingly involved in high-stake decision making (e.g. medical diagnosis, self-driving car) or embedded in the daily life of a wider population (e.g. Siri, ChatGPT). Therefore, it is unavoidable that we have to consider and establish best practices of ethical and responsible AI. Consequently, Explainable AI is in increasingly high demand to bring more transparency and visibility to the model internal processes.
Below is the Google Trend of search term “Explainable Artificial Intelligence” in the last 5 years.
How Can We Get Started in XAI?
1. Book - “Interpretable Machine Learning with Python” by Serg Masis
The book is a comprehensive guide of ML interpretation mechanisms, which can be broken down into three sections. Firstly, it summaries the importance of interpretability and explainability in machine learning practices. Then the second part covers different types of techniques along with their Python implementation, such as global interpretation methods (e.g. SHAP), local interpretation methods (e.g. LIME), CNN visualizers (e.g. Saliency Map) and counterfactual explanations.
The last section gives some practical advices on tuning ML models for better explainability. Instead of building black box models and interpreting the output afterwards, it is also a common practice to build interpretable model in the first instance through model tuning techniques, such as feature engineering and selection, regularization, implement model constraints etc.
2. Video Tutorial - “Explainable AI” YouTube Playlist by DeepFindr
This video series created by “DeepFindr” provides nice visual illustration of model interpretation techniques and explain the theory behind them in a simple way. Each video typically ends with the practical implementation so that it provides the audience with a more concrete understanding of the concept. The introduction video of this series is very valuable, in terms of providing an overview and differentiating explanation techniques into various categories:
model agnostic vs. model specific
global explanation vs. local explanation
explanations of different data types: image vs. text vs. tabular data
Hope you found this article helpful. If you’d like to support my work and see more articles like this, treat me a coffee ☕️ by signing up Premium Membership with $10 one-off purchase.
Take-Home Message
According to ChatGPT’s answer of current trends in practical machine learning, the top 3 on the list are Automated Machine Learning and Deep Learning and Explainable Artificial Intelligence. To equip ourselves with these skillsets, I have recommend some resources that help us get started and hands on these fields quickly.
AutoML / MLOps:
GitHub Repository - “Awesome MLOps”
Book - “Building Machine Learning Pipelines”
Book - “Practical MLOps”
Deep Learning:
Book - “Hands-on Machine Learning with Scikit-Learn, Keras and Tensorflow”
Online Course - “Introduction to TensorFlow for Artificial Intelligence, Machine Learning, and Deep Learning”.
Explainable Artificial Intelligence:
Book - “Interpretable Machine Learning with Python”
Video Tutorial - “Explainable AI” YouTube playlists