Last week the annual AWS technology conference re:Invent kicked off virtually. Typically a week-long physical conference held in Las Vegas this year, with the ongoing COVID-19 pandemic, the conference moved to a virtual 3-week event. If you have been paying attention to re:Invent, the past couple of years, Machine Learning has always taken center stage during re:Invent keynotes, so much that it has overshadowed other releases. This year AWS decided, and rightly so, that Machine Learning deserved its own dedicated keynote.

Why should you care? While most machine learning experiments might start locally, you eventually end up in the cloud once you start doing machine learning at a production scale. Besides, when you look at these production workloads, they are overwhelmingly composed of algorithms in TensorFlow, PyTorch, and MXNet. And finally, over 90% of cloud machine learning based on TensorFlow and PyTorch runs on AWS.

Let start with the hardware.

AWS Trainium

A machine learning chip custom-designed specifically for training machine learning models in the cloud. This is the second silicon from AWS after AWS Inferentia. It shares the same AWS Neuron SDK for developers. Cost-effective and high performance for your deep learning training on the cloud.

EC2 Instances Powered by Habana Gaudi

EC2 instances powered by Habana Gaudi accelerators. Also available via SageMaker, AWS ECS, and AWS EKS.

Why it matters? Up to 40% better price-performance over the current GPU-based EC2 instances. This, in turn, means more cost-effective scaling for your deep learning training.

Amazon SageMaker Updates

Amazon SageMaker is a fully managed machine learning service from AWS that enables you to build, train, and deploy machine learning. It is also the most robust machine learning service in the cloud today. Here are some of the SageMaker releases from re:Invent.

SageMaker Data Wrangler — Enables you to process and visualize the data needed for machine learning with minimal code. Why it matters? Data preparation undifferentiated heavy lifting that consumes most of the time in machine learning. Data Wrangler makes it easy to clean and preprocess your data with over 300 built-in transformations while giving you visibility into what happens behind the scenes. You can also author custom transformations in PySpark, SQL, and Pandas.

SageMaker Feature Store — A purpose-built repository enables you to store, update, retrieve, and share features. Why it matters? Provides an easy way to reuse and share enriched features and avoid duplication of work within teams. Your team can check-in and check out features similar to how they work with code repositories. Offline and online options make it easy to consistently achieve consistency during training and inference when you need low latency.

SageMaker Pipelines — A CI/CD service built for machine learning workloads. Why it matters? You could stitch together a pipeline from a combination of AWS Lambda, Step Functions, and other services in the past. With pipelines, you can create workflow straight from SageMaker studio. Pipelines have built-in templates that enable you to get started quickly, and you also have the ability to create your own templates using cloud formation. You can define custom stages and build both automatic and manual approvals. Tip: Before trying out pipelines, you need, or your administrator needs to enable project templates in SageMaker studio.

SageMaker Profiler For Debugger — Enables you to identify bottlenecks and maximize resource utilization during training with a single parameter. Profiler is an additional capability in Sagemaker debugger, and you can enable it with just a few additional parameters without any significant code changes. Why it matters? In the past, you had to write custom scripts and combine dashboard from CloudWatch to monitor resource utilization while training was in progress. Profiler makes it easy to identify under or over-utilized resources and take remedial actions. The profiling results can be visualized from within SageMaker Studio and also available via the SageMaker debugger API.

SageMaker Clarify — A new feature that enables you to detect bias and explain model behavior. Why it matters? Bias in your dataset and models can lead to inferior predictions. With Clarify, you can detect both pre-training biases, that is, the bias already part of your data, and post-training bias that is part of your trained model. In addition to bias detection, Clarify also explains why the model the decisions it did. Clarify uses SHAP values to explain the contribution each input feature makes to the final decision.

SageMaker Distributed Training — While distributed training has existed in SageMaker, the new releases enable data and model parallelism with a few lines of code. Why it matters? New Distributed Training on Amazon SageMaker makes it possible to train large, complex deep learning models up to two times faster than current approaches. Distributed training with Amazon SageMaker’s Model Parallelism engine can efficiently split large, complex models with billions of parameters across multiple GPUs by automatically profiling and identifying the best way to partition models.

SageMaker Edge Manager — Allows developers to deploy, monitor, and maintain machine learning models on a fleet of edge devices. Why it matters? Enables you to deploy models on a fleet of edge devices and continuously monitor the model fleet to detect deterioration. It applies specific hardware optimizations that make performances up to 30x better compared to hand-tuning the models.

SageMaker Jumpstart — Provides a set of solutions and models for common use cases, such as fraud detection, predictive maintenance, and demand forecasting, that can be deployed readily with just a few clicks. Why it matters? Developers that are new to machine learning find it hard to get started. Even experienced practitioners sometimes find it confusing to scale to meet production demands. With JumpStart, you can quickly find relevant information and solutions for your machine learning use cases.

Machine Learning for Databases

Apart from releases within the AWS machine learning stack, re:Invent also saw ML expand across other AWS services like Redshift ML and Neptune ML. This enables database developers with limited machine learning skills and seasoned ML practitioners to create, train, and run machine learning models using familiar SQL code. Why it matters? Developers can directly get inference results from their data in RedShift into their applications. With Neptune ML, this enables common use cases like knowledge graphs and recommendation systems.

Amazon Q for QuickSight

A feature powered by machine learning that uses natural language processing to answer your business questions. Why it matters? Build natural language query capabilities within your QuickSight dashboards with a few clicks. Simply put, ask questions in plain language and get answers instantly.

Amazon Lookout for Metrics

An AI service that uses machine learning to automatically detect and diagnose anomalies in business and operational time series data. You can connect to multiple sources like S3, RDS, and third-party SAAS providers like Salesforce. Why it matters? Enables you to detect anomalies in virtually any time series data. In addition to detection, it also helps identify the root cause of the anomaly. You can use this via the console in a few clicks or programmatically integrate it with other applications via the API.

Machine Learning for Industrial Services

Machine Learning for industrial services received significant attention during this year’s re:Invent releases.

Amazon Monitron — A machine learning solution to detect abnormal behavior and enable preventive maintenance for industrial machinery. Monitron has two main components: Monitron sensors that attach to the machines to capture vibrations and temperature. The Monitron gateway collects the data from the sensors and sends it to the AWS cloud where we can apply machine learning. The final component is a mobile app where you can receive alerts when abnormal behavior is detected. Why it matters? An end to end solution for anomaly detection on industrial machinery, including all the hardware, software, and cloud infrastructure. This makes it easy to deploy a turnkey solution in a matter of days.

Lookout for Equipment — Works similar to Monitron but allows you to use your own sensors and hardware.

Lookout for Vision — Expands the anomaly detection capabilities to detect anomalies within a set of images with built-in sophistication to handle camera angle lighting variations.

Panorama Appliance — If you have existing smart cameras, you can use the Panorama appliance to process feeds directly and send them to the AWS cloud for further analysis. The appliance comes with pre-built models in the appliance, which are optimized for different industry domains.

How these matter? Leverage your existing investments in industrial hardware and extend them with machine learning capabilities quickly.

DevOps Guru

Another AI service that enables you to detect application behaviors that deviate from normal operating patterns and identify operational issues long before they impact customers. Why it matters? Deploy a machine learning powered solution that automatically anticipates DevOps issues before they appear with no manual setup or machine learning expertise required.

Amazon HealthLake

A service that enables you to store and transform health data using specialized machine learning models to identify trends and make predictions. Why it matters? A HIPPA eligible service that organizes data chronologically can be used to build machine learning models in SageMaker. The services also enables you to share data with other providers using standard file formats.

That was a lot of machine learning and artificial intelligence releases. AWS re:Invent is still underway, and you can register for free and watch any of these or any of the other deep dive sessions. AWS also offers a generous free tier and if you do not already have an AWS account, I highly recommend signing up for the free tier and trying out these services.

Happy building!